[NUT-XX] Compact Nut Filters

25.md

@@ -0,0 +1,118 @@
+# NUT-25: Compact Nut Filters
+
+`optional`
+
+`depends on: NUT-06`
+
+---
+
+This NUT describes a structure for compact filters on ecash notes and other information, for client-side use, primarily for recovery flows. The filter construction proposed uses Golomb-Coded Sets (GCS) for efficient compression.
+
+Clients can query and use these filters to test for set membership, which is particularly useful for checking the spent state of ecash notes without revealing which notes are being checked, and for checking if blind signatures have been issued without revealing the specific blind signatures.
+
+## Specification
+
+### Golomb-Coded Set (GCS) Filters
+
+A Golomb-Coded Set (GCS) is a probabilistic data structure that allows for compact representation of a set of items. It enables checking for set membership with a certain false positive rate, but no false negatives. GCS filters are constructed by hashing items to a range, sorting the hashed values, and then encoding the differences between successive values using Golomb-Rice coding. This method provides significant compression, making them suitable for efficient transmission and client-side processing.
+
+Each filter is defined by:
+
+- `N`: The cardinality of the set it encodes.
+- `P`: The bit length of the remainder code in Golomb-Rice coding.
+- `M`: The inverse of the target false positive rate.
+
+### Mint Responsibilities
+
+Mints **MUST** generate GCS filters containing sets of items associated with specific keysets. These filters are generated at self-determined intervals. The mint is responsible for ensuring the filters are available for clients to query.
+
+For each keyset, mints generate:
+
+- A filter encoding the `Y` (nullifiers) values of all spent ecash notes.
+- A filter encoding the `B_` (blinded_messages) values of all issued ecash notes.
+
+### Wallet Behavior and Recovery Flow
+
+Wallets utilize GCS filters during recovery to determine the status of ecash notes and blind signatures leaking as little sensitive information as possible.
+
+**Restore Flow for Spent Ecash Notes:**
+
+1.  The wallet identifies the `keyset_id` for which it needs to check spent notes.
+2.  The wallet queries the Mint's `GET v1/filter/spent/{keyset_id}` endpoint to retrieve the GCS filter for spent nullifiers.
+3.  Upon receiving the `GetFilterResponse`, the wallet extracts the `content` (the GCS filter bytes), `n`, `p`, and `m` parameters.
+4.  For each ecash note the wallet possesses for that `keyset_id`, it computes the nullifier `Y`.
+5.  The wallet then queries the GCS filter with its `Y` values. If a `Y` value doesn't match in the filter, the note is considered unspent, while it's "maybe" spent otherwise. Due to the probabilistic nature of GCS, false positive are possible, meaning a note might be marked as spent when it is not. For this reason, wallets **SHOULD** check the state of all "maybe" spent notes.
+
+**Restore Flow for Issued Blind Signatures:**
+
+1.  The wallet identifies the `keyset_id` for which it needs to check issued blind signatures.
+2.  The wallet queries the Mint's `GET v1/filter/issued/{keyset_id}` endpoint to retrieve the GCS filter for issued blind signatures.
+3.  Upon receiving the `GetFilterResponse`, the wallet extracts the `content` (the GCS filter bytes), `n`, `p`, and `m` parameters.
+4.  For each deterministically derived ([NUT-13](13)) `blinded_message` (`B_`) of `keyset_id`, it queries the GCS filter with its `B_` value. If a `B_` value doesn't match in the filter, the blind signature was not issued, while it's "maybe" issued otherwise. Similar to spent notes, false positives are possible. Wallets **SHOULD** handle this by attempting to retrieve the signed blind signature and gracefully handling errors.
+
+### Querying for Spent or Issued Ecash Filters
+
+Wallets **MAY** query the following endpoints:
+
+- `GET v1/filter/spent/{keyset_id}` to get the GCS filter that encodes the `Y` (nullifiers) values of all the spent ecash from `keyset_et_id`.
+- `GET v1/filter/issued/{keyset_id}` to get the GCS filter that encodes the `B_` (blinded_messages) values of all the issued ecash from `keyset_id`.
+
+The Mint **MUST** respond with a `GetFilterResponse`, which has the following structure.
+
+```json
+{
+  "n": "<int>",
+  "p": "<int | null>",
+  "m": "<int | null>",
+  "content": "<base64_str>",
+  "timestamp": "<int>"
+}
+```
+
+Where:
+
+- `n` is the number of items in the filter.
+- `p` is the bit parameter of the Golomb-Rice coding. If `null`, then the client assumes `p = 19`.
+- `m` is the inverse of the false positive rate. If `null`, then the client assumes `m = 784931`.
+- `content` is a base-64 string encoding the bytes of the filter. It is typically computed as:
+  ```python
+  content = b64encode(filter_bytes).decode()
+  # And vice-versa
+  filter_bytes = b64decode(content)
+  ```
+- `timestamp` is the Unix epoch (in seconds) when the filter was created. The Mint might choose not to re-compute the filter upon every request, and instead serve one from cache memory and computed an updated version after arbitrary amount of time.
+
+### Implementation Details
+
+See the reference implementation in [cashu-ts](https://github.com/cashubtc/cashu-ts/blob/aeb85d6b03fa30cc2a2cfa7c3c647ed17cb6501f/src/gcs.ts) for details.
+
+### False Positive Rate (FPR) For Bulk Tests
+
+Each individual look-up has $1 - \frac{1}{M}$ chance of being a _true positive_. We can consider a bulk test as $x$ independent look-ups, so the chance that at _all_ of them are true positives is $\bigl(1 - \frac{1}{M}\bigr)^x$.
+
+Therefore, the chance of any one of them being a _false positive_ (or equivalently, **not** all of them being _true positives_) is $1 - \bigl(1 - \frac{1}{M}\bigr)^x$.
+
+For $M = 784931$, this turns out to be:
+
+| $x$  | $M$    | $P_M(x)$    |
+| ---- | ------ | ----------- |
+| 1    | 784931 | 0.000001274 |
+| 10   | 784931 | 0.00001274  |
+| 300  | 784931 | 0.000382126 |
+| 5000 | 784931 | 0.006349745 |
+
+## Mint Info Settings
+
+Mints that support this NUT **MUST** announce it in their [NUT-06](06.md) `GetInfoResponse`. They can do so by updating the response to signal support for NUT-25.
+
+```json
+{
+    ...,
+    "nuts": {
+        ...,
+        "25": {"supported": true}
+    }
+}
+```
+
+[13]: 13.md

README.md

@@ -40,6 +40,7 @@ Wallets and mints `MUST` implement all mandatory specs and `CAN` implement optio
 | [22][22] | Blind authentication              | [Nutshell][py], [cdk-cli]                                                          | [Nutshell][py], [cdk-mintd], [nutmix]            |
 | [23][23] | Payment Method: BOLT11            | [Nutshell][py], [cdk-cli]                                                          | [Nutshell][py], [cdk-mintd], [nutmix]            |
 | [24][24] | HTTP 402 Payment Required         | -                                                                                  | -                                                |
+| [25][25] | Compact Nut Filters               | [Nutshell][py], [cdk-mintd], [cashu-ts][ts]                                        | [Nutshell][py], [cdk-mintd]                      |
 
 #### Wallets:
 
@@ -97,3 +98,4 @@ Wallets and mints `MUST` implement all mandatory specs and `CAN` implement optio
 [22]: 22.md
 [23]: 23.md
 [24]: 24.md
+[25]: 25.md

tests/25-test.md

@@ -0,0 +1,34 @@
+# NUT-25 Test Vectors
+
+The following list of items should encode to the target filter `7sdQJ7OweaujLCqS7KDHzu/3pySZrDsatjQA`, with parameters:
+
+- `p = 19`
+- `m = 784931`
+
+```json
+[
+  "c2735796c1d45c68e7f03d3ea3bfcf5d6f10e6eb480e57fc3dccaf8ce66990dfc5",
+  "3c7ac2a233f8d5439be8cf3109d314e7da476e1ca762dc05f64ca3d5acac2da1fa",
+  "73e199a811db202ef7fbb1699b0e4859d15735c8f7f838fd9e50b37dc47c0ff4b9",
+  "02f171db2b577f6d586580651da4951c2e1506454bb9b76077d7a9fdb8606cf2f6",
+  "106954852453d217ad91e3b14c37bcb6adf62b038cc6a6a281f63edf78de2c7819",
+  "621e006de8d41b14491933e695985a730179003846b739224316af578fc49c1ee8",
+  "59b759ecda3c4d9027b9fe549fe6ae33b1bf573b9e9c2d0cdf17d20ea38794f1b7",
+  "cfcc8745503e9efb67e48b0bee006f6433dec534130707ac23ed4eae911d60eec2",
+  "f1d57d98f80e528af885e6174f7cd0ef39c31f8436c66b8f27c848a3497c9a7dfb",
+  "5a21aa11ccd643042f3fe3f0fcc02ccfb51c72419c5eab64a3565aa8499aa64cdf"
+]
+```
+
+Matching any given item from this list should return `True`, while matching any item from the following list
+should return `False`:
+
+```json
+[
+  "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3",
+  "d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6",
+  "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff00",
+  "ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffee",
+  "1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2b3c"
+]
+```