Eclair

Reclaim liquidity from idle channels (#3269)

We more aggressively reclaim liquidity from idle channels. We start by
identifying channels where the liquidity doesn't seem to be moving in
either direction. Then we lower our relay fees until we reach our
minimum configured value. Once we've reached that minimum fee, if the
payment volume doesn't increase, we eventually close those channels
(but we always keep one channel with those peers, we only close the
redundant channels).

Use bitcoin-lib 0.47 (#3268)

* Use bitcoin-lib 0.47

* Add checksums for secp256k1-kmp 0.23.0

* der2compact() and compact2der() methods have been removed, new signInputCompact() method has been added

* Use Transaction.encodeWitnessEcdsaSig() instead of low-level calls

* Replace local der() methode with bitcoin-lib's encodeWitnessEcdsaSig()

Improve data stored in `AuditDb` (#3245)

* Improve `channel_events` in the `AuditDb`

We improve the `channel_events` table in the `AuditDb` by:

- adding the `funding_txid` from each event
- adding the `channel_type` from each event
- using hex instead of blobs in sqlite
- providing APIs to list these events
- adding indexes on `channel_id` and `remote_node_id`

The added data isn't available in past events. We decide to simply
rename the older tables and start fresh: previous data will not be
available from the API, but can still be queried directly in SQL if
necessary.

* Improve transaction events in the `AuditDb`

We improve the `transactions_published` and `transactions_confirmed`
tables in the `AuditDb` by:

- adding more details about mining fees and feerates
- adding input and output counts
- using hex instead of blobs in sqlite
- providing APIs to list transactions by `remote_node_id`
- adding indexes for listing APIs

The added data isn't available in past events. We decide to simply
rename the older tables and start fresh: previous data will not be
available from the API, but can still be queried directly in SQL if
necessary.

* Improve payment sent and received events in the `AuditDb`

We improve the `sent` and `received` payment tables in the `AuditDb` by:

- adding `start_at` / `settled_at` timestamps to measure duration
- adding `remote_node_id` for every channel used
- using hex instead of blobs in sqlite
- reordering columns and renaming them wherever useful

The added data isn't available in past events. We decide to simply
rename the older tables and start fresh: previous data will not be
available from the API, but can still be queried directly in SQL if
necessary.

* Improve payment relayed events in the `AuditDb`

We improve the `relayed` and `relayed_trampoline` payment tables in the
`AuditDb` by:

- unifying the way we handle multi-part payments, by having one row per
payment, whether incoming or outgoing, and reconciling based on type
and direction when listing events
- adding `remote_node_id` for every channel used
- using hex instead of blobs in sqlite
- renaming columns for better clarify

The added data isn't available in past events. We decide to simply
rename the older tables and start fresh: previous data will not be
available from the API, but can still be queried directly in SQL if
necessary.

* Improve relay statistics computation

We change the relay statistics we compute, to more accurately reflect
nodes that are good or bad peers and inform liquidity allocation
decisions.

* Add liquidity purchase fees in transaction events

We now track liquidity purchases as well in transaction events, as their
fees must be taken into account when evaluating the overall fees earned
from a node.

We also add the number of on-chain transactions made with a given peer
to its relay stats.

Close connection when receiving malformed messages (#3273)

When receiving a message that we're supposed to be able to decode, for
which decoding fails, we must send a warning and close the connection.
This ensures that the connection state cannot become corrupted, which
may otherwise happen if we fail to deserialize a `commit_sig` message
(for example) and simply ignored it while continuing to process the
next messages.

Fixes #3271

Update Bitcoin Core to v30.2 (#3274)

This release contains improvements for v3 transactions, which we want
for #3192.

Add per-peer profit scoring (#3247)

We create a new set of actors that keep track of payment statistics
across our peers and rank them to identify the top profit earners.
Based on those statistics, the actors issue recommendations to:

- allocate more liquidity towards nodes that are generating revenue and
may run out of liquidity in the next few days
- reclaim liquidity from inactive channels
- change our relay fees to optimize increases or decreases in outgoing
flow and volume

Fix race condition in `Postman` causing flaky `OfferPayment` tests (#3270)

The integration test "send blinded multi-part payment a->b->c (single
channel a->b)" in `OfferPaymentSpec` fails intermittently. The root
cause is a race condition in `Postman` where the subscription for an
onion message reply is registered after the message is sent, allowing
the reply to arrive and be silently dropped before the subscription
exists.

In integration tests where all 3 nodes run on the same JVM, the onion
message round-trip (Alice -> Bob -> Carol -> creates invoice -> Carol
-> Bob -> Alice) can complete in just a few milliseconds - fast enough
to beat the `Subscribe` message to the `Postman`'s mailbox.

This explains why the test is flaky: it usually works (round-trip slower
than subscribe), but occasionally fails (round-trip faster than
subscribe, reply dropped).

We simply move the subscription registration to *before* the message is
sent to the network.

Add bitcoin rpc call to check if an address belongs to our wallet (#3267)

We use the "ismine" property that is returned by the "getaddressinfo" RPC call.

Improve support for plugin-defined features (#3264)

We allow injecting non-official features via plugins, by extending the
`CustomFeaturePlugin` trait. However, we previously added those features
to a set of unknown features: we can do better and add them to the known
features map, which provides pretty-printing in logs and simplifies
debugging.

We also had an issue where `hasFeature` would always return `false` for
plugin features, even when they were supported by both nodes, because we
only looked at our `activated` features map which only contains official
features.

We get rid of the `UnknownFeature` class entirely and keep the encoded
feature bits after deserializing them. We take those feature bits into
account in `hasFeature`, which fixes the issue.

This creates a slightly weird asymmetry because our local features,
which are created based on our `eclair.conf`, will not have this field
set while features that are read from network messages or the DB will
have this field set. A better architecture would be to remove the
`activated` map entirely and directly wrap the feature bits, while
providing helper functions to interact with features without having
to directly read a `BitVector`. This is a larger refactoring though,
because we'd need to add support for filtering `init` / `node` features
using bitmasks, which isn't entirely trivial to handle and requires
careful management of plugin features.

Fix flaky onion message test (#3266)

The "automatically connect to known nodes" test was flaky, because there
is a race with our code that lazily removes node announcements from our
DB after channels close (in `handleChannelSpent` in `Validation.scala`).

We now explicitly wait for gossip to ensure that we cannot run into this
race condition, which should fix the flakyness during CI.

Reject offers with amount set to `0` (#3265)

This doesn't make any sense, the field should be omitted if any amount
is acceptable: setting it to `0` is confusing.

See https://github.com/lightning/bolts/pull/1316

Remove support for zlib encoding for channel queries (#3263)

We removed support for sending compressed data when it was removed from
the spec (see https://github.com/ACINQ/eclair/pull/2244). We meant to
remove support for receiving compressed data as well as a follow-up, but
it slipped for a long time! There's no reason to keep supporting it now.

Plugin validation of interactive transactions (#3258)

We add a new `ValidateInteractiveTxPlugin` trait that can be extended
by plugins that want to perform custom validation of remote inputs and
outputs added to interactive transactions. This can be used for example
to reject transactions that send to specific addresses or use specific
UTXOs.

Auto-refresh relay fees from conf can be disabled (#3260)

There is a performance hit on restart if there is a large number (>100s) of channels, so it can be disabled with a new `eclair.relay.fees.reset-existing-channels` setting.

The default behavior is unchanged.

Add `ChannelFundingCreated` event (#3256)

We add a `ChannelFundingCreated` event that is emitted when the funding
transaction or a splice transaction has been signed and can now be
published, either by us or by our peer (depending on who funds).

This can be handy to detect peers that are using black-listed inputs and
immediately close the channel before it confirms and can be used.

Lazily load peer storage at node restart (#3257)

This avoids a herd effect when restarting the node.

Don't automatically use `scid_alias` for public channels (#3255)

In #3250 we started implicitly choosing the best channel type based on
activated features when not explicitly specified by the node operator.
However, we automatically used `scid_alias` if it was supported by both
peers, but it is only allowed for unannounced channels. We now restrict
this to follow the BOLTs.

Add basic CLAUDE.md (#3254)

This was generated using Opus 4.6 with the `/init` command. It provides
overall context about the codebase to every Claude session, to avoid
repeating work to build that context (consume less token whenever using
Claude for a task).

Fix flaky wallet funding tests (#3253)

We fix a few flaky tests in our channel FSM that test scenarios where
errors are received while creating funding transactions. The issue was
that we used a dummy wallet that could complete its call before our
calls to `awaitCond(alice.stateName == WAIT_FOR_FUNDING_INTERNAL)`.

We use a new dummy on-chain wallet that never responds to transaction
funding calls to allow tests to inject failure events.

Add claude files to `.gitignore` (#3252)

Claude adds a `.claude` dir at the root of the project. In a sandboxing setup we may want to use other custom claude-related directories, hence the wildcard.

Select `channel_type` for automatic channel creation (#3250)

When creating channels automatically, we don't necessarily have access
to our peer's features. It is useful to automatically select the channel
type based on local and remote features. We now allow this behavior in
the `OpenChannelInterceptor`.

Note that when we add official support for taproot channels and 0-fee
commitments, we will hard-code in `eclair` which type of channels we
think is best. If node operators want to override this order, they will
need to explicitly provide the `channel_type` parameter (which can be
done when using the API).

Fix balance fuzz tests (#3251)

Those tests started randomly failing after moving to anchor outputs.
The reason is that anchor outputs have a bigger commit weight than non
anchor channels, and with our initial balance allocation we sometimes
reached a point where the *remote* had exhausted all of its balance to
pay commit fees (when the feerate was around 10 000 sat/kw), so the
*local* couldn't send HTLCs (`RemoteCannotAffordFeesForNewHtlc`). This
is an expected situation where we cannot do anything except wait for
pending HTLCs to resolve before sending other HTLCs.

We simply increase the initial balance to avoid running into this case.
We also remove the `skip` case (which wasn't working at all, we should
have used the `cancel` scalatest helper if we wanted to interrupt the
test without failing it) and allow failures during the initial HTLC
setup (which can happen if HTLCs are larger than the available balance).

(Minor) Upgrade postgres libs (#3238)

Upgrade the jdbc driver to the latest version.

Add type to local/remote error metrics (#3249)

We were only splitting by origin (local/remote).

This compensates for #3236.

Prioritize private channels when relaying payments (#3248)

When relaying payments, we want to select private channels first and
keep as much liquidity available as possible in public channels, to
ensure that we don't send a `channel_update` that would otherwise
disable the public channel (and thus make the private channels also
unusable since they aren't visible by path-finding algorithms) or
limit the `htlc_maximum_msat` of this public channel (which also
indirectly applies to private channels).

We also change the order in which we select channels that have the
same visibility: we prioritize channels with smaller balances, to
ensure that we keep our larger balances for larger payments. When
balances are equal, we prioritize the largest channel, which creates
a larger inbound liquidity to allow receiving larger payments.