BOLT12 Offers Tracking Issue

[TheBlueMatt]

The following tasks are required to implement BOLT 12 Offers in LDK:

• Data abstractions
  • Offer encoding and building BOLT 12 offer encoding and building #1719, BOLT 12 spec updates #1972
  • Bech32 encoding and parsing BOLT 12 offer parsing #1726
  • InvoiceRequest encoding, parsing, and building BOLT 12 invoice_request encoding and building #1738
  • Signature calculation and verification BOLT 12 invoice_request encoding and building #1738
  • Refund encoding, parsing, and building BOLT 12 refund encoding and building #1908, BOLT 12 spec updates #1972
  • Invoice encoding, parsing, and building BOLT 12 invoice encoding and building #1926, Pre-work for BOLT 12 invoices #1927, Expose signable hashes for BOLT 11 and BOLT 12 invoices #2162, Expose description from Invoice's offer/refund #2206
  • Test vectors Update BOLT 12 test vectors #2334
  • Fuzz testing BOLT 12 deserialization fuzzers #1977
• Onion messages
  • Sending, forwarding, and receiving Onion messages v1 #1503, Onion messages: support reply paths  #1652, Expose onion messages in public API  #1650
  • Onion message handling Onion messages: flip feature bit 🎉 #1688
  • Custom onion messages Support custom onion messages #1748
  • Onion message router trait BOLT 12 Offers message handling support #2294
  • Onion message pathfinding Support onion message pathfinding #1724
  • Onion message replies Onion messages: support reply paths  #1652, BOLT 12 Offers message handling support #2294
  • Fuzz testing Onion messages: add fuzz testing  #1648, Re-add one-hop onion message fuzzing test #2686
• Route blinding
  • Spec test vectors Implement route blinding test vectors  #3204
  • OM BlindedPath construction Onion messages v1 #1503
  • Payment BlindedPath construction Add API for constructing blinded payment paths #2412, Aggregate BlindedPayInfo for blinded routes #2514
    • Follow up with custom TLV support for blinded paths Support sending custom TLVs to to blinded recipients #2975, Introduce Custom Data support for ReceiveTlvs #2830
    • Follow up by setting each hop's PaymentConstraint::htlc_minimum_msat to the max min_htlc for the other nodes in its anonymity set (e.g. for a blinded route A>B>C, set B's min_htlc to the max min_htlc for A's peers)
  • BlindedRoute pathfinding Blinded pathfinding groundwork #2146, Remove redundant final_cltv_delta param from get_route #2239, Prefactor PaymentParameters for blinded recipients #2258, Respect route hint max_htlc in pathfinding  #2305, Routing to blinded payment paths #2120
    • Score blinded paths that we've received over so we improve blinded path selection
    • Support blinded pathfinding to paths where we are the intro node (to be done as a follow-up to offers support)
    • Track failed blinded paths in PaymentParameters similar to the existing previously_failed_channels field: Avoid retrying over previously failed blinded paths #2818
  • Fuzz test blinded pathfinding Blinded pathfinding fuzzing #2366
  • BlindedRoute payments
    • Split onion payload abstraction into forwarding vs receiving Route blinding groundwork  #2128
    • Address follow-up from Route blinding groundwork  #2128: #2128 follow-ups #2801
    • Sending and receiving to 1-hop blinded paths Support constructing blinded path onion keys  #2411, Route blinding MVP #2413, Fix PaymentPathFailed::payment_failed_permanently on blinded path fail #2576
    • Forwarding and receiving to multi-hop blinded paths Route blinding: support forwarding as the intro node #2540, Support receiving to multi-hop blinded paths #2688, Follow-ups to #2688 #2791, Complete route blinding support #2812, Test blinded forwarding #2823
    • Allowing paying to blinded paths where we are the intro node Blinded paths with unannounced introduction nodes #3132
    • Add a random delay before returning any errors if we are the intro node (SHOULD in the spec)
    • Address follow-up from Route blinding: support forwarding as the intro node #2540: Route blinding: support forwarding as the intro node #2540 (review), Route blinding: support forwarding as the intro node #2540 (comment)
  • Support probing blinded PaymentParameters (i.e. strip the blinded path off before sending the probe)
  • Support receiving to blinded paths with phantom Blided Paths to Phantom #4313
• Offers message flow
  • Stateless message verification Stateless BOLT 12 message verification #1989
  • InvoiceRequest and Invoice message handling interface BOLT 12 Offers message handling support #2294
  • ChannelManager implementation of message handling BOLT 12 outbound PaymentId #2468, BOLT 12 Invoice payments #2371, BOLT 12 Offers message flow #2039
  • Stateless Offer and Refund builder utilities BOLT 12 Offers utilities #2578
  • Multi-hop blinded paths in Offer and Refund messages Multi-hop blinded paths in Offers #2690
  • Multi-hop blinded paths in InvoiceRequest onion message reply paths Multi-hop blinded paths in Offers #2690
  • Multi-hop blinded payment paths in Bolt12Invoice messages Multi-hop blinded paths in Offers #2690
  • Three-hop blinded paths Three-hop blinded paths #2793
  • Blinded path choice API (i.e. number of paths, number of hops, compact or not)
  • Correlate InvoiceError with outbound payment using reply paths Update payment and generate Event from OffersMessageHandler #2837
  • InvoiceRequest retries Retry InvoiceRequest messages #2836
  • Fuzz testing
• Miscellaneous
  • Builder support in language bindings Offers builders for C-bindings #2903
  • Non-BTC currencies in Offer
  • Proof of payment
  • Authenticate messages using path_id Authenticate use of offer blinded paths #3139
  • Full phantom node support Blided Paths to Phantom #4313
  • Integration testing Functional tests for BOLT 12 Offers payment flow #2697
  • Update test vectors BOLT 12 Offer test vectors #2682
  • Compact blinded paths (i.e., using short channel ids) Compact BlindedPath representation for onion messaging #2921

The things left for us to ship BOLT12, in an issue so we don't lose track of things:

• unsolicited invoices
• struct deserialization fuzzers
• fuzzers for the builders, if its doable
• onion message handler for BOLT12 messages
• Metadata storage in Offers which can be used to check the correctness of invoice requests
• non-builder offer/invoice_request/invoice constructor for bindings - cc Ability to construct an invoice from bindings #1868
• Blinded path payment
• Blinded path receiving
• Blinded path forwarding (not strictly required but probably required to test the above and we need it eventually)

After we go public, we should also:

• Support for non-Bitcoin offers (my fault, I thought the sender had to convert, vs passing no amount in the invoice_request)

Anything I'm missing here @jkczyz?

[jkczyz]

We also need to do spontaneous/unsolicited invoices, i.e., BOLT11-style invoices not in response to an InvoiceRequest but rather scanned directly by the user. This was a late addition to the spec. lightning/bolts@7b0c0fc

I think we can punt on this for now similar to non-Bitcoin-denominated offers. Or a volunteer can work on it in parallel. It would be similar to Refund in that it will have its own struct (probably named SpontaneousInvoice) and share the underlying TLV streams. Like Refund it has it's own bech32 prefix (lni instead of lnr).

[jkczyz]

For bindings, we probably only need non-builder constructors for Offer and Refund as LDK would handle constructing the others. Though it would still be nice to expose them directly if someone wants to use them without a ChannelManager. Just may be a lower priority.

[TheBlueMatt]

Right, like BOLT11 we'll eventually have folks who want to use them, so we definitely need to support it, but it won't be the "common case". I'm not convinced we shouldn't drop the builders and replace them entirely or at least have a simple util method that calls the builder as appropriate, should be pretty trivial if we go the util method route.

[valentinewallace]

We may also eventually want utilities similar to lightning-invoice::utils::create_invoice_from_channelmanager that will automatically supply the payment hash and blinded payment paths

[jkczyz]

FYI, I repurposed this issue to track all work for BOLT 12 Offers. @valentinewallace Could you update the route blinding portion as needed (i.e., split into more tasks and add PRs)?