Skip to content

Commit

Permalink
Merge pull request lightningdevkit#3536 from TheBlueMatt/2025-01-0.1-…
Browse files Browse the repository at this point in the history
…backports

Backports for 0.1.0
  • Loading branch information
TheBlueMatt authored Jan 16, 2025
2 parents c8ec8eb + a016cc9 commit 2a642f8
Show file tree
Hide file tree
Showing 32 changed files with 749 additions and 227 deletions.
32 changes: 28 additions & 4 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
fail-fast: false
matrix:
platform: [ ubuntu-latest, windows-latest, macos-latest ]
toolchain: [ stable, beta, 1.63.0 ] # 1.63.0 is the MSRV for all crates.
toolchain: [ stable, beta, 1.63.0 ] # 1.63.0 is the MSRV for all crates but `lightning-transaction-sync`.
exclude:
- platform: windows-latest
toolchain: 1.63.0
Expand All @@ -44,6 +44,27 @@ jobs:
- name: Set RUSTFLAGS to deny warnings
if: "matrix.toolchain == '1.63.0'"
run: echo "RUSTFLAGS=-D warnings" >> "$GITHUB_ENV"
- name: Run CI script
shell: bash # Default on Winblows is powershell
run: CI_ENV=1 CI_MINIMIZE_DISK_USAGE=1 ./ci/ci-tests.sh

build-tx-sync:
strategy:
fail-fast: false
matrix:
platform: [ ubuntu-latest, macos-latest ]
toolchain: [ stable, beta, 1.75.0 ] # 1.75.0 is the MSRV for `lightning-transaction-sync`.
runs-on: ${{ matrix.platform }}
steps:
- name: Checkout source code
uses: actions/checkout@v4
- name: Install Rust ${{ matrix.toolchain }} toolchain
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile=minimal --default-toolchain ${{ matrix.toolchain }}
rustup override set ${{ matrix.toolchain }}
- name: Set RUSTFLAGS to deny warnings
if: "matrix.toolchain == '1.75.0'"
run: echo "RUSTFLAGS=-D warnings" >> "$GITHUB_ENV"
- name: Enable caching for bitcoind
id: cache-bitcoind
uses: actions/cache@v4
Expand All @@ -57,7 +78,7 @@ jobs:
path: bin/electrs-${{ runner.os }}-${{ runner.arch }}
key: electrs-${{ runner.os }}-${{ runner.arch }}
- name: Download bitcoind/electrs
if: "matrix.platform != 'windows-latest' && (steps.cache-bitcoind.outputs.cache-hit != 'true' || steps.cache-electrs.outputs.cache-hit != 'true')"
if: "steps.cache-bitcoind.outputs.cache-hit != 'true' || steps.cache-electrs.outputs.cache-hit != 'true'"
run: |
source ./contrib/download_bitcoind_electrs.sh
mkdir bin
Expand All @@ -69,7 +90,7 @@ jobs:
echo "ELECTRS_EXE=$( pwd )/bin/electrs-${{ runner.os }}-${{ runner.arch }}" >> "$GITHUB_ENV"
- name: Run CI script
shell: bash # Default on Winblows is powershell
run: CI_ENV=1 CI_MINIMIZE_DISK_USAGE=1 ./ci/ci-tests.sh
run: CI_ENV=1 CI_MINIMIZE_DISK_USAGE=1 ./ci/ci-tx-sync-tests.sh

coverage:
strategy:
Expand Down Expand Up @@ -229,7 +250,10 @@ jobs:
run: |
cd fuzz && cargo update -p regex --precise "1.9.6" --verbose && cd ..
- name: Sanity check fuzz targets on Rust ${{ env.TOOLCHAIN }}
run: cd fuzz && RUSTFLAGS="--cfg=fuzzing --cfg=secp256k1_fuzz --cfg=hashes_fuzz" cargo test --verbose --color always
run: |
cd fuzz
RUSTFLAGS="--cfg=fuzzing --cfg=secp256k1_fuzz --cfg=hashes_fuzz" cargo test --verbose --color always
cargo clean
- name: Run fuzzers
run: cd fuzz && ./ci-fuzz.sh && cd ..

Expand Down
174 changes: 174 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,177 @@
# 0.1 - Jan XXX, 2025 - XXX

## API Updates
* The `lightning-liquidity` crate has been moved into the `rust-lightning`
git tree, enabling support for both sides of the LSPS channel open
negotiation protocols (#3436).
* Since its last alpha release, `lightning-liquidity` has also gained support
for acting as an LSPS1 client (#3436).
* This release includes support for BIP 353 Human Readable Names resolution.
With the `dnssec` feature enabled, simply call `ChannelManager`'s
`pay_for_offer_from_human_readable_name` with a list of lightning nodes that
have the `dns_resolver` feature flag set (e.g. those running LDK with the
new `lightning_dns_resolver::OMDomainResolver` set up to resolve DNS queries
for others) and a Human Readable Name (#3346, #3179, #3283).
* Asynchronous `ChannelMonitorUpdate` persistence (i.e. the use of
`ChannelMonitorUpdateStatus::InProgress`) is now considered beta-quality.
There are no known issues with it, though the likelihood of unknown issues
is high (#3414).
* `ChannelManager`'s `send_payment_with_route` and `send_spontaneous_payment`
were removed. Use `send_payment` and `send_spontaneous_payment_with_retry`
(now renamed `send_spontaneous_payment`) instead (#3430).
* `ChannelMonitor`s no longer need to be re-persisted after deserializing the
`ChannelManager` before beginning normal operation. As such,
`ChannelManagerReadArgs::channel_monitors` no longer requires mutable
references (#3322). See the Backwards Compatibility section for more info.
* Additional information is now stored in `ChannelMonitorUpdate`s which may
increase the average size of `ChannelMonitorUpdate`s when claiming inbound
payments substantially. The expected maximum size of `ChannelMonitorUpdate`s
shouldn't change materially (#3322).
* Redundant `Event::PaymentClaimed`s will be generated more frequently on
startup compared to previous versions.
`Event::PaymentClaim{able,ed}::payment_id` has been added to allow for more
robust handling of redundant events on payments with duplicate
`PaymentHash`es (#3303, #3322).
* `ChannelMonitorUpdate::update_id`s no longer have a magic value (of
`u64::MAX`) for updates after a channel has been closed. They are now
always monotonically increasing (#3355).
* The MSRV of `lightning-transaction-sync` has been increased to rustc 1.75 due
to its HTTP client dependencies (#3528).
* The default `ProbabilisticScoringFeeParameters` values now recommend specific
ratios between different penalties, and default penalties now allow for
higher fees in order to reduce payment latency (#3495).
* On-chain state resolution now more aggressively batches claims into single
transactions, reducing on-chain fee costs when resolving multiple HTLCs for a
single channel force-closure. This also reduces the on-chain reserve
requirements for nodes using anchor channels (#3340).
* A `MigratableKVStore` trait was added (and implemented for
`FilesystemStore`), enabling easy migration between `KVStore`s (#3481).
* `InvoiceRequest::amount_msats` now returns the `offer`-implied amount if a
Bitcoin-denominated amount was set in the `offer` and no amount was set
directly in the `invoice_request` (#3535).
* `Event::OpenChannelRequest::push_msat` has been replaced with an enum in
preparation for the dual-funding protocol coming in a future release (#3137).
* `GossipVerifier` now requires a `P2PGossipSync` which holds a reference to
the `GossipVerifier` via an `Arc` (#3432).
* The `max_level_*` features were removed as the performance gain compared to
doing the limiting at runtime was negligible (#3431).
* `ChannelManager::create_bolt11_invoice` was added, deprecating the
`lightning::ln::invoice_utils` module (#3389).
* The `bech32` dependency has been upgraded to 0.11 across crates (#3270).
* Support for creating BOLT 12 `invoice_request`s with a static signing key
rather than an ephemeral one has been removed (#3264).
* The `Router` trait no longer extends the `MessageRouter` trait, creating an
extra argument to `ChannelManager` construction (#3326).
* The deprecated `AvailableBalances::balance_msat` has been removed in favor of
`ChannelMonitor::get_claimable_balances` (#3243).
* Deprecated re-exports of `Payment{Hash,Preimage,Secret}` and `features` were
removed (#3359).
* `bolt11_payment::*_from_zero_amount_invoice` methods were renamed
`*_from_variable_amount_invoice` (#3397)
* Offer `signing_pubkey` (and related struct names) have been renamed
`issuer_signing_pubkey` (#3218).
* `Event::PaymentForwarded::{prev,next}_node_id` were added (#3458).
* `Event::ChannelClosed::last_local_balance_msat` was added (#3235).
* `RoutingMessageHandler::handle_*` now all have a `node_id` argument (#3291).
* `lightning::util::persist::MonitorName` has been exposed (#3376).
* `ProbabilisticScorer::live_estimated_payment_success_probability` was added
(#3420)
* `EcdsaChannelSigner::sign_splicing_funding_input` was added to support an
eventual splicing feature (#3316).
* `{Payment,Offer}Id` now support lowercase-hex formatting (#3377).

## Bug Fixes
* Fixed a rare case where a BOLT 12 payment may be made duplicatively if the
node crashes while processing a BOLT 12 `invoice` message (#3313).
* Fixed a bug where a malicious sender could cause a payment `Event` to be
generated with an `OfferId` using a payment with a lower amount than the
corresponding BOLT 12 offer would have required. The amount in the
`Event::Payment{Claimable,Claimed}` were still correct (#3435).
* The `ProbabilisticScorer` model and associated default scoring parameters
were tweaked to be more predictive of real-world results (#3368, #3495).
* `ProbabilisticScoringFeeParameters::base_penalty_amount_multiplier_msat` no
longer includes any pending HTLCs we already have through channels in the
graph, avoiding over-penalizing them in comparison to other channels (#3356).
* A `ChannelMonitor` will no longer be archived if a `MonitorEvent` containing
a preimage for another channel is pending. This fixes an issue where a
payment preimage needed for another channel claim is lost if events go
un-processed for 4038 blocks (#3450).
* `std` builds no longer send the full gossip state to peers that do not
request it (#3390).
* `lightning-block-sync` listeners now receive `block_connected` calls, rather
than always receiving `filtered_block_connected` calls (#3354).
* Fixed a bug where some transactions were broadcasted one block before their
locktime made them candidates for inclusion in the mempool (though they would
be automatically re-broadcasted later, #3453).
* `ChainMonitor` now persists `ChannelMonitor`s when their `Balance` set first
goes empty, making `ChannelMonitor` pruning more reliable on nodes that are
only online briefly (e.g. mobile nodes, #3442).
* BOLT 12 invoice requests now better handle intermittent internet connectivity
(e.g. on mobile devices with app interruptions, #3010).
* Broadcast-gossip `MessageSendEvent`s from the `ChannelMessageHandler` are now
delivered to peers even if the peer is behind in processing relayed gossip.
This ensures our own gossip propagates well even if we have very limited
upload bandwidth (#3142).
* Fixed a bug where calling `OutputSweeper::transactions_confirmed` with
transactions from anything but the latest block may have triggered a spurious
assertion in debug mode (#3524).

## Performance Improvements
* LDK now verifies `channel_update` gossip messages without holding a lock,
allowing additional parallelism during gossip sync (#3310).
* LDK now checks if it already has certain gossip messages before verifying the
message signatures, reducing CPU usage during gossip sync after the first
startup (#3305).

## Node Compatibility
* LDK now handles fields in the experimental range of BOLT 12 messages (#3237).

## Backwards Compatibility
* Nodes with pending forwarded HTLCs or unclaimed payments cannot be
upgraded directly from 0.0.123 or earlier to 0.1. Instead, they must
first either resolve all pending HTLCs (including those pending
resolution on-chain), or run 0.0.124 or 0.0.125 and resolve any HTLCs that
were originally forwarded or received running 0.0.123 or earlier (#3355).
* `ChannelMonitor`s not being re-persisted after deserializing the
`ChannelManager` only applies to upgraded nodes *after* a startup with the
old semantics completes at least once. In other words, you must deserialize
the `ChannelManager` with an upgraded LDK, persist the `ChannelMonitor`s as
you would on pre-0.1 versions of LDK, then continue to normal startup once,
and for startups thereafter you can take advantage of the new semantics
avoiding redundant persistence on startup (#3322).
* Pending inbound payments paying a BOLT 12 `invoice` issued prior to upgrade
to LDK 0.1 will fail. Issued BOLT 12 `offer`s remain payable (#3435).
* `UserConfig::accept_mpp_keysend` was removed, thus the presence of pending
inbound MPP keysend payments will prevent downgrade to LDK 0.0.115 and
earlier (#3439).
* Inbound payments initialized using the removed
`ChannelManager::create_inbound_payment{,_for_hash}_legacy` API will no
longer be accepted by LDK 0.1 (#3383).
* Downgrading to prior versions of LDK after using `ChannelManager`'s
`unsafe_manual_funding_transaction_generated` may cause `ChannelManager`
deserialization to fail (#3259).
* `ChannelDetails` serialized with LDK 0.1+ read with versions prior to 0.1
will have `balance_msat` equal to `next_outbound_htlc_limit_msat` (#3243).

## Security
0.1 fixes a funds-theft vulnerability when paying BOLT 12 offers as well as a
funds-lockup denial-of-service issue for anchor channels.
* When paying a BOLT 12 offer, if the recipient responds to our
`invoice_request` with an `invoice` which had an amount different from the
amount we intended to pay (either from the `offer` or the `amount_msats`
passed to `ChannelManager::pay_for_offer`), LDK would pay the amount from the
`invoice`. As a result, a malicious recipient could cause us to overpay the
amount we intended to pay (#3535).
* Fixed a bug where a counterparty can cause funds of ours to be locked up
by broadcasting a revoked commitment transaction and following HTLC
transactions in specific formats when using an anchor channel. The funds can
be recovered by upgrading to 0.1 and replaying the counterparty's broadcasted
transactions (using `Confirm::transactions_confirmed`) (#3537). Thanks to
Matt Morehouse for reporting and fixing this issue.
* Various denial-of-service issues in the formerly-alpha `lightning-liquidity`
crate have been addressed (#3436, #3493).


# 0.0.125 - Oct 14, 2024 - "Delayed Beta Testing"

## Bug Fixes
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ members = [
"lightning-background-processor",
"lightning-rapid-gossip-sync",
"lightning-custom-message",
"lightning-transaction-sync",
"lightning-macros",
"lightning-dns-resolver",
"lightning-liquidity",
"possiblyrandom",
]

exclude = [
"lightning-transaction-sync",
"no-std-check",
"msrv-no-dev-deps-check",
"bench",
Expand Down
3 changes: 2 additions & 1 deletion ci/check-lint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -93,4 +93,5 @@ RUSTFLAGS='-D warnings' cargo clippy -- \
-A clippy::unnecessary_to_owned \
-A clippy::unnecessary_unwrap \
-A clippy::unused_unit \
-A clippy::useless_conversion
-A clippy::useless_conversion \
-A clippy::unnecessary_map_or `# to be removed once we hit MSRV 1.70`
41 changes: 2 additions & 39 deletions ci/ci-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
set -eox pipefail

RUSTC_MINOR_VERSION=$(rustc --version | awk '{ split($2,a,"."); print a[2] }')
HOST_PLATFORM="$(rustc --version --verbose | grep "host:" | awk '{ print $2 }')"

# Some crates require pinning to meet our MSRV even for our downstream users,
# which we do here.
Expand All @@ -11,19 +10,6 @@ function PIN_RELEASE_DEPS {
# Starting with version 1.39.0, the `tokio` crate has an MSRV of rustc 1.70.0
[ "$RUSTC_MINOR_VERSION" -lt 70 ] && cargo update -p tokio --precise "1.38.1" --verbose

# Starting with version 0.7.12, the `tokio-util` crate has an MSRV of rustc 1.70.0
[ "$RUSTC_MINOR_VERSION" -lt 70 ] && cargo update -p tokio-util --precise "0.7.11" --verbose

# url 2.5.3 switched to idna 1.0.3 and ICU4X, which requires rustc 1.67 or newer.
# Here we opt to keep using unicode-rs by pinning idna_adapter as described here: https://docs.rs/crate/idna_adapter/1.2.0
[ "$RUSTC_MINOR_VERSION" -lt 67 ] && cargo update -p idna_adapter --precise "1.1.0" --verbose

# indexmap 2.6.0 upgraded to hashbrown 0.15, which unfortunately bumped their MSRV to rustc 1.65 with the 0.15.1 release (and 2.7.0 was released since).
[ "$RUSTC_MINOR_VERSION" -lt 65 ] && cargo update -p [email protected] --precise "2.5.0" --verbose

# Starting with version 0.23.20, the `rustls` crate has an MSRV of rustc 1.71.0
[ "$RUSTC_MINOR_VERSION" -lt 71 ] && cargo update -p [email protected] --precise "0.23.19" --verbose

return 0 # Don't fail the script if our rustc is higher than the last check
}

Expand All @@ -35,15 +21,12 @@ PIN_RELEASE_DEPS # pin the release dependencies in our main workspace
# The addr2line v0.21 crate (a dependency of `backtrace` starting with 0.3.69) relies on rustc 1.65
[ "$RUSTC_MINOR_VERSION" -lt 65 ] && cargo update -p backtrace --precise "0.3.68" --verbose

# Starting with version 0.5.9 (there is no .6-.8), the `home` crate has an MSRV of rustc 1.70.0.
[ "$RUSTC_MINOR_VERSION" -lt 70 ] && cargo update -p home --precise "0.5.5" --verbose

# proptest 1.3.0 requires rustc 1.64.0
[ "$RUSTC_MINOR_VERSION" -lt 64 ] && cargo update -p proptest --precise "1.2.0" --verbose

export RUST_BACKTRACE=1

echo -e "\n\nChecking the full workspace."
echo -e "\n\nChecking the workspace, except lightning-transaction-sync."
cargo check --verbose --color always

# When the workspace members change, make sure to update the list here as well
Expand All @@ -58,7 +41,6 @@ WORKSPACE_MEMBERS=(
lightning-background-processor
lightning-rapid-gossip-sync
lightning-custom-message
lightning-transaction-sync
lightning-macros
lightning-dns-resolver
lightning-liquidity
Expand All @@ -83,25 +65,6 @@ cargo check -p lightning-block-sync --verbose --color always --features rpc-clie
cargo test -p lightning-block-sync --verbose --color always --features rpc-client,rest-client,tokio
cargo check -p lightning-block-sync --verbose --color always --features rpc-client,rest-client,tokio

if [[ "$HOST_PLATFORM" != *windows* ]]; then
echo -e "\n\nChecking Transaction Sync Clients with features."
cargo check -p lightning-transaction-sync --verbose --color always --features esplora-blocking
cargo check -p lightning-transaction-sync --verbose --color always --features esplora-async
cargo check -p lightning-transaction-sync --verbose --color always --features esplora-async-https
cargo check -p lightning-transaction-sync --verbose --color always --features electrum

if [ -z "$CI_ENV" ] && [[ -z "$BITCOIND_EXE" || -z "$ELECTRS_EXE" ]]; then
echo -e "\n\nSkipping testing Transaction Sync Clients due to BITCOIND_EXE or ELECTRS_EXE being unset."
cargo check -p lightning-transaction-sync --tests
else
echo -e "\n\nTesting Transaction Sync Clients with features."
cargo test -p lightning-transaction-sync --verbose --color always --features esplora-blocking
cargo test -p lightning-transaction-sync --verbose --color always --features esplora-async
cargo test -p lightning-transaction-sync --verbose --color always --features esplora-async-https
cargo test -p lightning-transaction-sync --verbose --color always --features electrum
fi
fi

echo -e "\n\nTest futures builds"
cargo test -p lightning-background-processor --verbose --color always --features futures
cargo test -p lightning-background-processor --verbose --color always --features futures --no-default-features
Expand Down Expand Up @@ -145,7 +108,7 @@ cargo test -p lightning-invoice --verbose --color always --no-default-features -
echo -e "\n\nTesting no_std build on a downstream no-std crate"
# check no-std compatibility across dependencies
pushd no-std-check
cargo check --verbose --color always --features lightning-transaction-sync
cargo check --verbose --color always
[ "$CI_MINIMIZE_DISK_USAGE" != "" ] && cargo clean
popd

Expand Down
Loading

0 comments on commit 2a642f8

Please sign in to comment.