Eclair

Back to dev (#3339)

After the v0.14.1 release.

Prevent `channel_id` collisions (#3337)

* Reject channel if initial DB write fails

If the first DB write of a channel fails, we want to abort the channel
immediately, otherwise our state may be unusable. If for some reason
we already have a channel (for example because of a partial restore
from a DB backup), we want to avoid overwriting this state as well.

* Add global `channel_id`s collision resistance

We add a global, concurrent map that keeps track of every temporary
and final `channel_id` being used. We always add to this map before
creating channel actors or transitioning to a final `channel_id`
(and thus before storing channel data in the DB).

We only remove from that map when a channel actor dies: no other
channel can be associated with the corresponding temporary or
final `channel_id` since the channel was created after registering
the IDs.

Accept Bolt12 invoices with reply path (#3325)

We previously rejected Bolt12 invoices when a reply path was provided
in the onion message. This wasn't in the spec, and it could make sense
to provide a specific reply path with an invoice in case the payer
considers the invoice invalid and wants to send back an invoice_error
to inform the recipient.

This created compatibility issues with LDK.

Fixes #3322

Back to dev (#3313)

After the 0.14.0 release.

Use official feature bit for `option_simple_taproot` (#3144)

Use the official feature bit and name for taproot channels and the
corresponding channel types. Activate taproot channels support by
default (without support for announcing such channels yet).

See https://github.com/lightning/bolts/pull/995

Fix flaky zero-conf integration test (#3291)

There's a race condition between Bob and Carol's router states, which
aren't updated exactly at the same time. Retrying the condition should
fix this race condition.

Add fuzz tests for onion, route blinding and lightning message codecs (#3282)

- Expand LightningMessageCodecs fuzz tests to cover all message types
- Add fuzz tests for payment and message onion payload codecs
- Add fuzz tests for route blinding codecs
- Add fuzz tests for onion failure message codecs

Signed-off-by: Nishant Bansal <nishant.bansal.282003@gmail.com>

Fix flaky `PeerStatsTracker` test (#3280)

If the test runs too quickly and the message is processed in less than
1 millisecond, there is a race condition since reading from the database
excludes the upper timestamp bound (`timestamp < now`).

We also remove our usage of the `BETWEEN` keyword for postgres.
The `BETWEEN` keyword is inclusive of both bounds, while we want to
exclude the upper bound to allow pagination.

Add per-peer profit scoring (#3247)

We create a new set of actors that keep track of payment statistics
across our peers and rank them to identify the top profit earners.
Based on those statistics, the actors issue recommendations to:

- allocate more liquidity towards nodes that are generating revenue and
may run out of liquidity in the next few days
- reclaim liquidity from inactive channels
- change our relay fees to optimize increases or decreases in outgoing
flow and volume

Fix race condition in `Postman` causing flaky `OfferPayment` tests (#3270)

The integration test "send blinded multi-part payment a->b->c (single
channel a->b)" in `OfferPaymentSpec` fails intermittently. The root
cause is a race condition in `Postman` where the subscription for an
onion message reply is registered after the message is sent, allowing
the reply to arrive and be silently dropped before the subscription
exists.

In integration tests where all 3 nodes run on the same JVM, the onion
message round-trip (Alice -> Bob -> Carol -> creates invoice -> Carol
-> Bob -> Alice) can complete in just a few milliseconds - fast enough
to beat the `Subscribe` message to the `Postman`'s mailbox.

This explains why the test is flaky: it usually works (round-trip slower
than subscribe), but occasionally fails (round-trip faster than
subscribe, reply dropped).

We simply move the subscription registration to *before* the message is
sent to the network.

Add bitcoin rpc call to check if an address belongs to our wallet (#3267)

We use the "ismine" property that is returned by the "getaddressinfo" RPC call.

Improve support for plugin-defined features (#3264)

We allow injecting non-official features via plugins, by extending the
`CustomFeaturePlugin` trait. However, we previously added those features
to a set of unknown features: we can do better and add them to the known
features map, which provides pretty-printing in logs and simplifies
debugging.

We also had an issue where `hasFeature` would always return `false` for
plugin features, even when they were supported by both nodes, because we
only looked at our `activated` features map which only contains official
features.

We get rid of the `UnknownFeature` class entirely and keep the encoded
feature bits after deserializing them. We take those feature bits into
account in `hasFeature`, which fixes the issue.

This creates a slightly weird asymmetry because our local features,
which are created based on our `eclair.conf`, will not have this field
set while features that are read from network messages or the DB will
have this field set. A better architecture would be to remove the
`activated` map entirely and directly wrap the feature bits, while
providing helper functions to interact with features without having
to directly read a `BitVector`. This is a larger refactoring though,
because we'd need to add support for filtering `init` / `node` features
using bitmasks, which isn't entirely trivial to handle and requires
careful management of plugin features.

Fix flaky onion message test (#3266)

The "automatically connect to known nodes" test was flaky, because there
is a race with our code that lazily removes node announcements from our
DB after channels close (in `handleChannelSpent` in `Validation.scala`).

We now explicitly wait for gossip to ensure that we cannot run into this
race condition, which should fix the flakyness during CI.

Reject offers with amount set to `0` (#3265)

This doesn't make any sense, the field should be omitted if any amount
is acceptable: setting it to `0` is confusing.

See https://github.com/lightning/bolts/pull/1316

Remove support for zlib encoding for channel queries (#3263)

We removed support for sending compressed data when it was removed from
the spec (see https://github.com/ACINQ/eclair/pull/2244). We meant to
remove support for receiving compressed data as well as a follow-up, but
it slipped for a long time! There's no reason to keep supporting it now.

Add `ChannelFundingCreated` event (#3256)

We add a `ChannelFundingCreated` event that is emitted when the funding
transaction or a splice transaction has been signed and can now be
published, either by us or by our peer (depending on who funds).

This can be handy to detect peers that are using black-listed inputs and
immediately close the channel before it confirms and can be used.

Fix flaky wallet funding tests (#3253)

We fix a few flaky tests in our channel FSM that test scenarios where
errors are received while creating funding transactions. The issue was
that we used a dummy wallet that could complete its call before our
calls to `awaitCond(alice.stateName == WAIT_FOR_FUNDING_INTERNAL)`.

We use a new dummy on-chain wallet that never responds to transaction
funding calls to allow tests to inject failure events.

Improve channel and payment events (#3246)

We improve the following events:

- `TransactionPublished` includes more details about mining fees and
an optional liquidity purchase
- all channel events include the latest `channel_type`
- `PaymentRelayed` exposes the `relayFee` earned

Include the `node_id` of channel peers in payment events (#3243)

When scoring peers, we don't really care about individual channels but
rather about the peer itself, who is identified by its `node_id`. We
update payment events to include the `node_id` of our channel peer, to
make it easier to compute statistics about each of our peers.

We also rework the events in `PaymentEvents.scala` to be more consistent
now that we've added support for trampoline, splicing, liquidity ads and
on-the-fly funding, which are all the features we had planned that can
impact peer scoring.

Note that we don't yet update the schema of the `AuditDb`, which means
that some of the data isn't stored and is currently filled with a dummy
value. We will include *all* updates to the DB schema in a future PR and
will fix this.

More tests for accountability (#3240)

These tests are also added to lightning-kmp (https://github.com/ACINQ/lightning-kmp/pull/846) to check compatibility.

CI: fix test with latest bitcoind (#3239)

Add flag to build without multiprocess architecture which we don't use and requires and additional dependency (capnproto).

Rework channel lifecyle events (#3237)

We emit several events during the channel lifecycle, which have become
a bit of a mess over the years, especially with the addition of 0-conf,
splicing and on-the-fly funding.

We now use the following events:

- `ChannelCreated` once the funding transaction is created
- `ChannelFundingConfirmed` once the funding transaction is confirmed,
which is also emitted for splice transactions
- `ChannelReadyForPayments` once the channel is ready for payments,
after exchanging `channel_ready` for the channel creation or
`splice_locked` for splice transactions

The order between `ChannelFundingConfirmed` and `ChannelReadyForPayments`
depends on whether 0-conf is used or not.

We remove `ChannelOpened`, which was actually a subset of the existing
`ChannelReadyForPayments` event (which was added afterwards).

We add a few fields to existing channel events, which we don't yet
store in the DB to avoid modifying it, but will store later when we
modify the schema of the `AuditDb`.

We now store an entry in the `AuditDb` whenever a splice transaction
confirms, which allows tracking the full history of a channel's changes.

Accountable HTLCs (#3217)

Add accountability signal for HTLCs, it replaces endorsement.
See https://github.com/lightning/bolts/pull/1280

Add `CommitSigBatch` codec (#3205)

While we send `commit_sig` messages individually on the wire, we need
a codec for the batch object when using the cluster mode, when peer
connection actors live on remote machines.

Add more tolerance in tests (#3199)

DER encoding size can vary by +- 3 bytes, not +- 2.