Merge pull request #276 from benthecarman/macaroons

Replace HMAC Auth with scoped macaroons

Author: tnull

CONTRIBUTING.md

@@ -19,8 +19,14 @@ cargo run --bin ldk-server ./contrib/ldk-server-config.toml
 ## Testing
 
 ```bash
-cargo test                     # Run all tests
-cargo test --all-features      # Run tests with all features
+cargo test                     # Run workspace tests
+cargo test --all-features      # Run workspace tests with all features
+```
+
+Run the end-to-end tests from their separate workspace:
+
+```bash
+cargo test --manifest-path e2e-tests/Cargo.toml -- --test-threads=4
 ```
 
 ## Code Quality
@@ -50,7 +56,15 @@ cargo fmt --all
 2. Regenerate protos (see above)
 3. Create handler in `ldk-server/src/api/` (follow existing patterns)
 4. Add route in `ldk-server/src/service.rs`
-5. Add CLI command in `ldk-server-cli/src/main.rs`
+5. Map the RPC to its required permission in `method_authorization` in `ldk-server/src/macaroons/authorization.rs`.
+   Unmapped methods return `UNIMPLEMENTED`, even for admin tokens.
+6. Add CLI command in `ldk-server-cli/src/main.rs`
+7. For a unary RPC, add the MCP tool in `ldk-server-mcp/src/tools/` and update the tool list test
+   in `ldk-server-mcp/tests/integration.rs`. Add a live test in `e2e-tests/tests/mcp.rs` if applicable.
+8. Test allowed and denied requests, including admin access.
+
+If the RPC needs a new permission, add it to `ldk-server-grpc/src/permissions.rs` and
+`ALL_PERMISSIONS`. Update the presets that need it and add it to `docs/api-guide.md`.
 
 ## Configuration
 

Cargo.lock

@@ -1,6 +1,6 @@
 [workspace]
 resolver = "2"
-members = ["ldk-server-cli", "ldk-server-client", "ldk-server-grpc", "ldk-server", "ldk-server-mcp"]
+members = ["ldk-server-cli", "ldk-server-client", "ldk-server-grpc", "ldk-server", "ldk-server-mcp", "ldk-server-macaroons"]
 exclude = ["e2e-tests"]
 
 [profile.release]

Cargo.toml

@@ -19,6 +19,7 @@ a Lightning node while exposing a robust, language-agnostic API via [Protocol Bu
 - `ldk-server-cli`: CLI client for the server API
 - `ldk-server-client`: Rust client library for authenticated TLS gRPC calls
 - `ldk-server-grpc`: generated protobuf and shared gRPC types
+- `ldk-server-macaroons`: shared token parsing, signing, derivation, and request binding
 - `ldk-server-mcp`: stdio MCP bridge exposing unary `ldk-server` RPCs as MCP tools
 
 ### Features

README.md

@@ -15,24 +15,107 @@ underlying LDK Node documentation.
 
 ## Authentication
 
-Every gRPC request must include an `x-auth` metadata header with an HMAC-SHA256 signature:
+Each request needs a hex-encoded v2 macaroon in the `macaroon` header:
 
+```text
+macaroon: <hex-encoded-request-macaroon>
 ```
-x-auth: HMAC <unix_timestamp>:<hmac_hex>
+
+Keep your original macaroon private. The client uses it to make a token for each request,
+as described below. All requests use TLS.
+
+You can add caveats to make a restricted copy of your macaroon without contacting the server.
+A caveat limits its permissions, allowed methods, or expiry time. Added caveats can only reduce
+access. See [Restrictions](#restrictions) for examples.
+
+### Request binding
+
+The Rust client, CLI, and MCP handle request binding automatically. They keep your macaroon
+private and send a copy tied to the request's method, body, and time.
+
+Custom clients must add one final caveat:
+
+```text
+request = <unix-seconds> <RpcMethod> <body-sha256>
 ```
 
-Where:
+Use Unix time in seconds, a method name such as `OnchainSend`, and the lowercase SHA-256 hash
+of the exact gRPC body, including its five-byte frame header. Rust clients can use
+`macaroon::bind_macaroon_to_request`.
+
+Client and server clocks must be within 60 seconds. The token cannot authorize a different
+request, but the same request can still be replayed while the token is valid.
+
+See [Request proof format](request-binding.md) for exact encoding rules and an example.
+
+### Restrictions
 
-- `unix_timestamp` is the current time in seconds since the Unix epoch
-- `hmac_hex` is the hex-encoded result of
-  `HMAC-SHA256(api_key_bytes, timestamp_be_bytes || grpc_request_body_bytes)`
-    - `api_key_bytes` is the API key string encoded as UTF-8 bytes
-    - `timestamp_be_bytes` is the timestamp as a big-endian 8-byte unsigned integer
-    - `grpc_request_body_bytes` is the raw gRPC request body sent over HTTP/2, including
-      the 5-byte gRPC message frame
+A caveat is a condition that limits what a macaroon can do. All caveats must pass:
 
-The server rejects requests where the timestamp differs from the server's clock by more than
-**60 seconds**.
+| Caveat | Meaning |
+|--------|---------|
+| `permissions = node:read,payments:read` | Allow only these permissions |
+| `method = GetNodeInfo` | Allow this RPC method and `GetPermissions` |
+| `time-before = 1800000000` | Expire at this Unix time in seconds |
+
+Added caveats can only reduce access. They cannot restore permissions or extend the expiry time.
+`GetPermissions` ignores method restrictions, but still requires a valid, unexpired macaroon.
+
+Derive a restricted copy without contacting the server:
+
+```bash
+ldk-server-cli derive-macaroon "$MACAROON" \
+  --caveat 'permissions = node:read' \
+  --caveat 'method = GetNodeInfo' \
+  --caveat "time-before = $EXPIRY_UNIX_SECONDS"
+```
+
+The command prints a hex token. Rust clients can use `macaroon::derive_macaroon`.
+Give the copy to the application and keep the original private.
+
+### Create and revoke tokens
+
+Use `CreateMacaroon` to give each client a token you can revoke separately. New tokens keep
+all the caller's restrictions. Revoking the caller's token does not revoke these new tokens.
+
+Copies made with `derive-macaroon` share the original token's ID. Revoking that ID blocks
+all those copies. The server cannot list copies made locally.
+
+Revocation and expiry block new requests. Existing event streams stay open until the client
+disconnects or the server stops. Reconnecting requires a valid token.
+
+See [Macaroon Management](#macaroon-management) for the RPCs and
+[Operations](operations.md#macaroons) for storage and recovery.
+
+### Macaroon Permissions
+
+Choose the permissions each client needs, or use `admin` by itself for full access.
+The CLI also has `readonly`, `invoice`, and `admin` presets.
+RPCs with no permission mapping return `UNIMPLEMENTED`, even for admin tokens.
+
+| Permission | Access |
+| ---------- | ------ |
+| `node:read` | Node information, balances, and pathfinding scores |
+| `onchain:receive` | Create on-chain receive addresses |
+| `onchain:send` | Send on-chain funds |
+| `invoices:create` | Create BOLT11/BOLT12 invoices and incoming refund requests |
+| `payments:read` | Read payments, forwarded payments, and forwarding statistics |
+| `payments:claim` | Claim or fail held BOLT11 payments |
+| `payments:send` | Send BOLT11, BOLT12, spontaneous, unified, and refund payments; splice out |
+| `channels:read` | List channels |
+| `channels:manage` | Open, configure, cooperatively close, or splice funds into channels |
+| `channels:force_close` | Force-close channels |
+| `peers:read` | List peers |
+| `peers:manage` | Connect or disconnect peers |
+| `messages:sign` | Sign messages and create BOLT12 payer proofs |
+| `messages:verify` | Verify message signatures |
+| `graph:read` | Read network graph data |
+| `utilities:read` | Decode invoices and offers |
+| `events:read` | Subscribe to the event stream |
+| `macaroons:manage` | Create, list, and revoke macaroons within your permissions |
+
+MCP provides token management through `create_macaroon`, `list_macaroons`, `revoke_macaroon`,
+and `get_permissions`.
 
 ## TLS
 
@@ -67,9 +150,10 @@ Errors are returned as standard gRPC status codes:
 | gRPC Code                 | Meaning                                                          |
 |---------------------------|------------------------------------------------------------------|
 | `INVALID_ARGUMENT` (3)    | Malformed request or invalid parameters                          |
+| `PERMISSION_DENIED` (7)   | Missing permission or a caveat that does not pass                    |
 | `FAILED_PRECONDITION` (9) | Lightning operation error (e.g., insufficient balance, no route) |
 | `INTERNAL` (13)           | Server-side bug                                                  |
-| `UNAUTHENTICATED` (16)    | Missing or invalid `x-auth` header                               |
+| `UNAUTHENTICATED` (16)    | Missing, invalid, or revoked macaroon                               |
 
 The `grpc-message` trailer contains a human-readable error description.
 
@@ -237,6 +321,18 @@ Use events as notifications. After reconnecting, reconcile recoverable state wit
 `GetPaymentDetails`, `ListPayments`, `ListForwardedPayments`, and `ListChannels`. Some event fields
 cannot be recovered through these APIs.
 
+### Macaroon Management
+
+| RPC | Description |
+|-----|-------------|
+| `CreateMacaroon` | Create a macaroon and return its private hex token in `token` |
+| `ListMacaroons` | List IDs, names, permissions, and caveats, without secrets |
+| `RevokeMacaroon` | Revoke a macaroon by ID |
+| `GetPermissions` | Show the caller's ID, name, usable permissions, and caveats |
+
+The first three RPCs require `macaroons:manage` or `admin`. You can only create or revoke
+tokens whose permissions you have. The last unrestricted admin token cannot be revoked.
+
 ### Metrics
 
 Metrics are served as a plain HTTP GET endpoint (not gRPC):

docs/api-guide.md

@@ -115,7 +115,7 @@ cooldown_secs = 3600
 
 Where local ldk-server data is stored. Defaults to `~/.ldk-server/` on Linux and
 `~/Library/Application Support/ldk-server/` on macOS. This directory is still used for
-the node mnemonic, API key, TLS material, and logs when LDK Node state and history
+the node mnemonic, macaroon credentials, TLS material, and logs when LDK Node state and history
 use PostgreSQL.
 
 ### `[storage.postgres]`
@@ -257,7 +257,11 @@ Two resolution methods are supported via the `mode` field:
   tls.crt                # TLS certificate (PEM)
   tls.key                # TLS private key (PEM)
   <network>/                # e.g., bitcoin/, regtest/, signet/
-    api_key                # API key
+    macaroons/
+      admin.macaroon       # Admin token (0400)
+      roots/               # Private root keys (0700)
+        admin.toml         # Admin root key and permissions (0400)
+        <id>.toml          # Root key and permissions from CreateMacaroon (0400)
     ldk-server.log         # Log file
     ldk_node_postgres.lock  # Present when LDK Node state uses PostgreSQL
     ldk_node_data.sqlite   # LDK Node state, payments, and forwarding history
@@ -271,4 +275,4 @@ guidance.
 
 When `[storage.postgres]` is configured, LDK Node wallet state, channel state, payment
 history, and forwarding history are stored in PostgreSQL instead of `ldk_node_data.sqlite`.
-The storage directory remains required for the mnemonic, API key, TLS material, and logs.
+The storage directory remains required for the mnemonic, macaroon credentials, TLS material, and logs.

docs/configuration.md

@@ -71,28 +71,32 @@ gRPC service listening on 127.0.0.1:3536
 NODE_URI: <node_id>@<address>
 ```
 
-Two files are auto-generated on first run:
+The server creates these files on first run:
 
-| File            | Location                          | Purpose                                  |
-|-----------------|-----------------------------------|------------------------------------------|
-| API key         | `<storage_dir>/<network>/api_key` | 32-byte random key (stored as raw bytes) |
-| TLS certificate | `<storage_dir>/tls.crt`           | Self-signed ECDSA P-256 certificate      |
+| File | Location | Purpose |
+|------|----------|---------|
+| Admin macaroon | `<storage_dir>/<network>/macaroons/admin.macaroon` | Full API access |
+| TLS certificate | `<storage_dir>/tls.crt` | Secure client connections |
 
 The default storage directory is `~/.ldk-server/` on Linux and
 `~/Library/Application Support/ldk-server/` on macOS.
 
-### Reading the API Key
+### Client Macaroons
 
-The API key file contains raw bytes. To get the hex string the CLI and client library expect:
+The CLI reads `admin.macaroon` automatically. This file contains a hex token.
+Keep it private, and never give clients files from `macaroons/roots/`.
+
+Create a restricted token for each application:
 
 ```bash
-xxd -p -c 64 ~/.ldk-server/bitcoin/api_key
+ldk-server-cli create-macaroon my-app --preset readonly
+ldk-server-cli create-macaroon invoice-app --preset invoice
 ```
 
 ## First Commands
 
 If the CLI and server share the same machine and use the default storage directory, the CLI
-auto-discovers the API key and TLS certificate, so no flags are needed:
+auto-discovers the macaroon and TLS certificate, so no flags are needed:
 
 ```bash
 # Check the node is running
@@ -111,7 +115,7 @@ details explicitly:
 ```bash
 ldk-server-cli \
   --base-url localhost:3536 \
-  --api-key <hex_api_key> \
+  --macaroon <hex_macaroon> \
   --tls-cert /path/to/tls.crt \
   get-node-info
 ```

docs/getting-started.md

@@ -59,7 +59,7 @@ the following config to `/etc/logrotate.d/ldk-server` (adjust the log path to ma
 
 - Network graph data (re-synced from gossip or RGS)
 - Fee rate cache (re-fetched from the chain backend)
-- The API key (can be regenerated, but clients will need the new one)
+- Macaroon credentials (can be replaced, but clients will need new tokens)
 - The TLS certificate (can be regenerated, but clients will need the new one)
 
 > **Warning:** Do not restore a backup onto two running nodes simultaneously. Running the
@@ -68,13 +68,39 @@ the following config to `/etc/logrotate.d/ldk-server` (adjust the log path to ma
 
 ## Security
 
-### API Key
+### Macaroons
 
-- Auto-generated as 32 random bytes on first startup
-- Stored at `<network_dir>/api_key` with `0400` permissions (read-only for owner)
-- The hex-encoded form of this key is used for HMAC authentication
-- Treat it as a secret: anyone with the API key and network access to the gRPC port can
-  control the node
+Keep `<network_dir>/macaroons/` private. It contains the default admin token in `admin.macaroon`
+and the server's root keys in `roots/`. Give clients tokens, never root keys.
+
+Use `create-macaroon` to give each client a token you can revoke separately. Use `derive-macaroon`
+to make a restricted copy. Give each client only the permissions it needs.
+See the [API guide](api-guide.md#authentication) for restrictions and request binding.
+
+To replace an admin token, create and save a new admin token, then revoke the old ID.
+Replace `admin.macaroon` with the new token or pass it with `--macaroon`.
+The API prevents revocation of the last unrestricted admin token.
+
+#### Recovery
+
+Back up `roots/` and `admin.macaroon` together. Old root files can restore revoked access.
+Deleting all roots invalidates every token; the server creates a new admin token on restart.
+
+At startup, the server repairs a missing or invalid `admin.macaroon` from `roots/admin.toml`
+and logs the change. It keeps valid tokens, even if restricted or expired. It warns if the
+file holds a request token; replace that file with a reusable token.
+
+If the original admin root is gone but other roots remain, the server warns instead of replacing
+it. Use another admin token to create a replacement and save it as `admin.macaroon`.
+
+Duplicate root names or IDs stop startup. Move conflicting files out of `roots/` and restart.
+Files ending in `.tmp` are ignored.
+
+Root-file caveat edits take effect after restart. `GetPermissions` shows them, and newly issued
+tokens inherit them. Tokens issued earlier have separate roots and do not change.
+
+The server logs successful token creation and revocation, including who made the change and
+which token it affects. Logs contain no tokens or root secrets.
 
 ### TLS
 
@@ -187,7 +213,7 @@ To allow clients to connect from other machines:
    (e.g., `0.0.0.0:3536`).
 3. **Distribute the TLS certificate:** Copy `<storage_dir>/tls.crt` to each client machine.
    Clients must pin this certificate since it is self-signed.
-4. **Share the API key:** Provide the hex-encoded API key to authorized clients.
+4. **Share the macaroon:** Provide the hex-encoded macaroon to authorized clients.
 
 If you regenerate the TLS certificate (by deleting `tls.crt` and `tls.key` and restarting),
 all clients will need the new certificate.