LSPS5 implementation #3662
LSPS5 implementation #3662
Conversation
|
👋 Thanks for assigning @tnull as a reviewer! |
|
This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3662 +/- ##
==========================================
- Coverage 89.16% 89.14% -0.02%
==========================================
Files 156 160 +4
Lines 123828 125718 +1890
Branches 123828 125718 +1890
==========================================
+ Hits 110408 112070 +1662
- Misses 10750 10896 +146
- Partials 2670 2752 +82 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Wow, thank you for looking into this! I did a first pass, and it looks pretty amazing already!
Before going too much into further details, here are a few general comments upfront:
-
I'm generally no fan of introducing additional dependencies here, an in particular not
reqwestandtokio. I think following the pattern so farBroadcastNotificationscould be a request that the user handles with any HTTP client they want and then could call back intoLSPS5ServiceHandler. Alternatively, we could also use a trait similar to the currentHTTPClient, but I don't think we want to keep the default implementation. Note that the blockingreqwestvariant wraps atokioruntime internally, and therefore should never (1, 2, ...) be used together. I guess technically we could consider a defaultasyncversion of the trait that usesasyncreqwest, but I would prefer to simply have well-documented trait on our end that the user can implement however they choose to. Also note that stackingtokioruntimes is heavily discouraged in general, so assuming our users would themselves use atokioruntime, we shouldn't wrap one inLSPS5ServiceHandler. -
Note that
lightning-liquidityis optionally no-stdcompliant, so please don't rely onstdwherever possible, often it's just a matter of usingcoreinstead and importing the respective types fromcrate::prelude. If you really find yourselves needing to usestd, make sure it's feature gated behindfeature = "std"and we provide an alternative for users that don't support it. -
Minor: Regarding formatting we're using tabs, not spaces. Feel free to run
./contrib/run-rustfmt.shafter each commit to run our formatting scripts. -
This PR in its current scope is great, just want to note that eventually we need to add persistence for the state. As we haven't fully fleshed out the persistence strategy for
lightning-liquidityin general yet, it's actually preferred to defer this to a follow-up, but just wanted to mention it. Also note that some changes toMessageQueue/EventQueuewill happen inlightning-liquidity: IntroduceEventQueuenotifier and wake BP for message processing #3509, but will ping you to rebase once that has been merged. (just sidenotes here).
I hope these initial points make sense, let me know if you have any questions, or once you made corresponding changes and think this is ready for the next round of review!
|
👋 The first review has been submitted! Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer. |
Thanks for asking! I'm totally fine to keep this (with its current scope) in a single PR, as long as we keep the commit history pretty clean to allow continuing review to happen commit-by-commit. To this end, please make sure add any fixup commits clearly marked (e.g. via a |
|
Btw, I'm not sure if you're familiar with the previous attempt of implementing LSPS5: #3499 Given this is a clean slate, not sure how much there is to learn, but still might be worth a look. Also not sure if @johncantrell97 would be interested in reviewing this PR, too, as he's familiar with the codebase and LSPS5. |
martinsaposnic
force-pushed
the
lsps5
branch
2 times, most recently
from
1d4b47c to
edf5346
Compare
|
@tnull, ready for the next review round!
CI is failing because of the usage of the |
martinsaposnic
force-pushed
the
lsps5
branch
7 times, most recently
from
9809682 to
af5929e
Compare
|
🔔 1st Reminder Hey @tnull @valentinewallace! This PR has been waiting for your review. |
1 similar comment
|
🔔 1st Reminder Hey @tnull @valentinewallace! This PR has been waiting for your review. |
|
🔔 1st Reminder Hey @tnull! This PR has been waiting for your review. |
lightning-liquidity/src/lsps0/ser.rs
Outdated
| @@ -23,12 +23,14 @@ use crate::lsps2::msgs::{ | |||
| }; | |||
| use crate::prelude::HashMap; | |||
|
|
|||
| use chrono::DateTime; | |||
There was a problem hiding this comment.
nit: I think importing this is fine (although not doing so was a choice before), but now we started, let's drop the chrono:: prefix below.
There was a problem hiding this comment.
I prefer the style used before. Deleting the import...
| //! URL utilities for LSPS5 webhook notifications | ||
|
|
||
| use crate::alloc::string::ToString; | ||
| use crate::prelude::Vec; |
There was a problem hiding this comment.
I think this needs a rebase on main as we dropped most of the prelude reexports in #3701. Instead, this needs to use alloc::vec::Vec directly, for example.
| url: String, | ||
| } | ||
|
|
||
| /// Implementation of methods for the Url struct |
There was a problem hiding this comment.
nit: impl blocks really don't need to be commented. Let's drop this line.
| /// | ||
| /// # Returns | ||
| /// A Result containing either the parsed URL or an error message | ||
| pub fn parse(url_str: &str) -> Result<Self, String> { |
There was a problem hiding this comment.
Mhh, let's not return an allocated type (String) in the error variant here. If we think we need to be able to communicate why the error failed, we should introduce an enum ParseError for it. If we don't think it matters all too much, we could just return Err(()). If we really think the error type needs to return free-form strings (which is usually not preferable to have a clear API of documented enum variants), they should be &'static str if possible to avoid the allocations.
| /// 1. Start with a letter (a-z, A-Z) | ||
| /// 2. Contain only letters, digits, plus (+), period (.), or hyphen (-) | ||
| fn is_valid_scheme(scheme: &str) -> bool { | ||
| if scheme.is_empty() { |
There was a problem hiding this comment.
I still think the is_empty check is redundant now that we map_or(false,.. below? Or would a non-empty scheme ever result in chars.next() being Some?
| // | ||
| // This file is licensed under the Apache License, Version 2.0 <LICENSE-APACHE | ||
| // or http://www.apache.org/licenses/LICENSE-2.0> or the MIT license | ||
| // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your option. You may not use this file except in accordance with one or both of these |
There was a problem hiding this comment.
nit: Somehow this line is sticking out.
| }; | ||
| use crate::message_queue::MessageQueue; | ||
|
|
||
| use bitcoin::secp256k1::PublicKey; |
There was a problem hiding this comment.
Please try to group the imports a bit better. They should be grouped by crate and the groups sorted from "most local" to "least local" (i.e., start with crate, end at std/core)
| /// Time provider for LSPS5 service | ||
| time_provider: Arc<dyn TimeProvider>, | ||
| /// Map of recently used signatures to prevent replay attacks | ||
| recent_signatures: Arc<Mutex<VecDeque<(String, Duration)>>>, |
There was a problem hiding this comment.
Do we need to make this Arc?
| config: LSPS5ClientConfig, | ||
| ) -> Option<Self> { | ||
| let max_signatures = config.signature_config.max_signatures.clone(); | ||
| Some(Self { |
There was a problem hiding this comment.
Same as in service: let's reuse _new_with_custom_time_provider
| } | ||
|
|
||
| /// Handle received messages from the LSP | ||
| pub fn handle_message( |
There was a problem hiding this comment.
I don't think we should expose this in the API?
- Add 'time' feature flag to allow disabling time-dependent functionality - Include 'time' in default features - Allow users to disable SystemTime::now without disabling all std features - Follows pattern established in other crates (e.g., lightning-transaction-sync) - Improves compatibility with WASM environments
- do new_from_duration_since_epoch (instead of From<Duration>) - Avoid doing ambiguous timestamp types - Add abs_diff function to use on client / service
Adds a new url_utils.rs module that provides: - A lightweight URL parser specialized for LSPS5 webhook validation - An implementation focusing on scheme and host extraction - RFC-compliant scheme validation - Tests for various URL scenarios This implementation allows validating webhook URLs without depending on the external url crate
martinsaposnic
force-pushed
the
lsps5
branch
5 times, most recently
from
58ba6b0 to
29fa657
Compare
- Replace String fields with UntrustedString wrapper - Change parse to take owned String and return Err(()) - Simplify error handling, drop free‑form error messages - Add is_https(), url_length(), is_public() helpers - Make url() public, remove scheme()/host() getters - Clean up parsing logic to avoid extra allocations - Update tests to use .to_string() and new API - Add proptest for robust property‑based testing - Remove redundant comments and dead code
- Define LSPS5Request and LSPS5Response enums for webhook registration, listing, and removal. - Implement WebhookNotification and associated helper constructors for different notification types. - Implement serialization/deserialization support with comprehensive tests. - Improve LSPS5 message types, validation, and testing - Replace generic String types with strongly-typed Lsps5AppName and Lsps5WebhookUrl with built-in length and format validation - Restructure imports to follow one-per-line convention - Add constants for notification method strings - Make WebhookNotificationMethod enum more consistent with LSPS5 prefix - Use explicit serde_json::json and serde_json::Value instead of imports - Improve code documentation with proper ticks and references - Add comprehensive test vectors from the BLIP-0055 specification
- Use UntrustedString for LSPS5AppName and LSPS5WebhookUrl - Move URL validation to url_utils for consistency - Use chars().count() for app name length check - Remove jsonrpc field from WebhookNotification struct - Always serialize jsonrpc as 2.0 and Visitor for deserialization - Avoid heap allocations in WebhookNotification deserialization - Update tests for new constructors and error types
- Introduce LSPS5ServiceEvent for LSPS-side webhook events including registration, listing, removal, and notification. - Define LSPS5ClientEvent for handling webhook outcomes on the client (Lightning node) side. - Outline WebhookNotificationParams enum to support notification-specific parameters. - Improve LSPS5 event documentation and field naming - Rename client/lsp fields to counterparty_node_id for consistent terminology - Replace generic String types with more specific Lsps5AppName and Lsps5WebhookUrl - Add comprehensive documentation for all events and fields - Include format specifications (UTF-8, ISO8601) and size constraints - Add request_id field to all relevant events for consistent request tracking - Provide detailed descriptions of error codes and their meanings - Use complete sentences in documentation comments
- Replaces raw error code/message pairs with a structured LSPS5Error type - Changes String timestamp fields to use the proper LSPSDateTime type - Improves documentation with references to MAX_APP_NAME_LENGTH constant
Implements the LSPS5 webhook registration service that allows LSPs to notify clients of important events via webhooks. This service handles webhook registration, listing, removal, and notification delivery according to the LSPS5 specification. Some details: - A generic HttpClient trait is defined so users can provide their own HTTP implementation - A generic TimeProvider trait is defined with a DefaultTimeProvider that uses std functionality - Uses URL utils to validate webhook URLs according to LSPS5 requirements - Uses secure message signing logic from the lightning::util::message_signing module - Works with the events and messages defined in earlier commits - Tests will be provided in a future commit
- Use LSPSDateTime for webhook timestamps for future LDK serialization - Make MIN_WEBHOOK_RETENTION_DAYS and PRUNE_STALE_WEBHOOKS_INTERVAL_DAYS Durations - Remove unnecessary Arc from Mutex fields (webhooks, last_pruning) - Use unwrap() for Mutex locks as per project convention - Move pruning interval to a const - Set last_pruning in prune_stale_webhooks after pruning - Remove unnecessary reassignments and error handling on locks - Use new_with_custom_time_provider for handler construction - Set PROTOCOL_NUMBER to 5 for LSPS5 - Rename _new_with_custom_time_provider to new_with_custom_time_provider - Make notification_cooldown_hours a Duration - Minor doc and formatting improvements
Implements the client-side functionality for LSPS5 webhook registration, allowing Lightning clients to register, list, and remove webhooks with LSPs. This client handler processes responses and verifies webhook notification signatures. Key features: - Full client API for webhook registration operations - Per-peer state tracking for pending requests - Automatic request timeout and cleanup - Security validation for webhook URLs - Notification signature verification - Add store_signature and check_signature to prevent replay attacks - Some tests are provided but more will come in a future commit This implementation pairs with the service-side LSPS5 webhook handler to complete the webhook registration protocol according to the LSPS5 specification.
- Use Duration type consistently for time-related configurations instead of u64 - Replace raw durations with LSPSDateTime for timestamp tracking and comparison - Do unwrap directly on mutex locks - Clean up code organization (imports, visibility modifiers) - Use the new LSPS5Error
Fully integrates the LSPS5 webhook components into the lightning-liquidity framework, enabling usage through the LiquidityManager. It includes - Registering LSPS5 events in the event system - Adding LSPS5 module to the main library exports - Updating LSPS0 serialization to handle LSPS5 messages - Adding LSPS5 configuration options to client and service config structures - Implementing message handling for LSPS5 requests and responses - Adding accessor methods for LSPS5 client and service handlers With this change, LSPS5 webhook functionality can now be accessed through the standard LiquidityManager interface, following the same pattern as other LSPS protocols.
…orrectness of the signing logic
|
@tnull thanks a lot for the review! All review comments have been addressed in fixup commits. Let me know what you think! Side note. There are a few things I’m still not fully convinced about:
|
|
🔔 1st Reminder Hey @tnull! This PR has been waiting for your review. |
A complete implementation for LSPS5 (spec defined here lightning/blips#55)
Reviewing commit by commit is recommended (~40% of the added lines are tests)
Notes:
lightning-liquidity: IntroduceEventQueuenotifier and wake BP for message processing #3509 once that PR is merged