Private Overlay Data and Recoverable Paid Lookup

Abstract and status

A private overlay is an overlay node that holds non-public information. Existing off-chain submission values and lookup context are useful primitives for that model. This proposal specifies the missing durability, authority and acquisition contract so a music seller can accept a song key privately, publish a free catalogue and sell access through an authenticated lookup service.

This is a proposed opt-in extension. It preserves existing offChainValues, BEEF and lookup context encodings and does not reinterpret BRC-81 as the complete definition of private overlays. It uses BRC-192 representations, BRC-194 selection and BRC-193 errors. It works with strict or federated overlay collections.

1. Trust boundaries and ports

The node's trusted HTTP boundary creates an immutable RequestContext with authenticated caller identity, validated certificate/policy claims, evaluated access partition, selected service/profile and any verified funding reference. An application payload, off-chain byte array or lookup question MUST NOT be able to set authenticated, paid, recipient or equivalent trusted facts. In-process callers need an explicit trusted context factory and the same checks; omitting it selects ordinary unauthenticated behavior.

Private data enters only selected authorized nodes. Generic multi-host broadcast, public GASP, proof assembly, telemetry, crash reports and public subscription logs MUST NOT copy private bytes. Node operators must document whether local topic/lookup plugins share one trust boundary. Passing secret bytes to every installed plugin is not isolation between mutually untrusted plugins; such deployments require separate protected execution or separate nodes.

An optional PrivatePublicationStore supports atomic reservation, protected payload persistence, binding to exact public evidence and idempotent completion. An optional AcquisitionStore owns challenges, payment association, frozen asset/terms, delivery material and recovery deadlines. Implementations may share a database but MUST keep these concerns separate from public output retention. Encryption at rest, access restrictions and backup/key lifecycle are deployment requirements; this BRC does not prescribe a database or a new encryption format.

2. Private publication

POST /overlay/v1/private/publish, selected with #private-publish-v1, accepts:

type Publish = {
  version: 1; requestId: string; topic: string;
  evidence: Evidence; assetId: Hex32;
  schema: string; privateValues: Bytes;
  extensions?: Record<string, unknown>; critical?: string[]
}

requestId has BRC-193's retry-ID constraints. schema names an installed application validation profile, not executable remote code. assetId is an immutable domain asset commitment; the schema defines its preimage. privateValues is at most the advertised bound and at most 1 MiB in this profile. Domain validation MUST bind the private values to the exact asset/output, verify publisher authority and validate the key/content relationship before exposing availability. Merely knowing a txid is not authority to replace a stored key.

An accepted request returns PublicationResult as defined in section 7. The publication ID is digest("private-publication", {chain, publisher, topic, requestId}), where chain is the verified evidence chain and publisher is the authenticated authorized caller. Reusing it with a different semantic request is conflict; alternate proof bytes for the same transaction may be merged after validation. POST /overlay/v1/private/status accepts {version:1, publicationId:Hex32} and returns the same record only to an authorized publisher.

ready requires durable private storage, successful public admission, installed lookup binding and recoverable post-commit work. On a storage architecture without one atomic transaction across those components, use a durable staged intent and reconciler: reserve and persist protected bytes first, admit once, then mark ready only after the binding is durable. A crash can leave pending; it cannot fabricate ready. Existing classic admission callbacks that merely log plugin failure do not satisfy this contract without that extra barrier. Dry-run never stores or releases a key.

Replayed public admission or GASP reconstruction can rebuild public indexes but cannot manufacture private bytes. A node lacking them reports unavailable; it does not advertise paid key availability. An authorized private replication channel needs independent receiver authorization and protected transport/storage. It is not implied by public federation.

3. Catalogue and acquisition request

Browsing remains an ordinary uncharged lookup, optionally live under BRC-193. The public catalogue may identify an authorized seller, immutable asset ID, available offers and current listing outpoints. It MUST NOT include the key or imply that every catalogue host can sell it. A client explicitly selects one seller and one offer before any payment. Multi-host discovery MUST NOT fan out a paid request.

POST /overlay/v1/private/acquire, selected with #paid-lookup-v1, accepts:

type Acquire = {
  version: 1; requestId: string; service: string;
  assetId: Hex32; listing: Outpoint; termsDigest: Hex32;
  recipient: Identity; request: Bytes;
  extensions?: Record<string, unknown>; critical?: string[]
}

The caller MUST equal recipient in this profile. Delegated purchase is a future explicit profile. request carries the application purchase request, including a signed LCH License Request when BRC-198 is selected. The service's immutable rules define parsing, asset/terms binding and output/context schema. acquisitionId = digest("acquisition", {chain, seller, buyer, service, requestId}). The chain comes from the verified listing; seller and buyer are authenticated identities. The entire normalized Acquire body is frozen for that ID. A changed asset, recipient, offer or body conflicts even if its price is the same.

Before returning a payment challenge, the seller freezes the relevant listing evidence, asset version, price, terms, key availability and recovery obligation. It MUST be able to honor that snapshot if the catalogue listing is later replaced or locally withdrawn. New acquisitions can stop; an already funded acquisition retains its separate rights and recovery state.

4. BRC-105 funding and recovery

The endpoint uses BRC-103/104 over HTTPS and a BRC-105 402 challenge. The challenge body is:

type Challenge = {
  version: 1; acquisitionId: Hex32; requestDigest: Hex32;
  seller: Identity; buyer: Identity; assetId: Hex32; termsDigest: Hex32;
  satoshis: U64; derivationPrefix: string;
  acceptancePolicy: ReleasePolicy; rulesDigest: Hex32;
  payableUntil: U64; recoveryUntil: U64
}

requestDigest = digest("acquire-request", Acquire). The BRC-105 amount, prefix and payee headers MUST agree with the body; [object Object] MUST be positive and fit the BRC-100 SatoshiValue range. An uncharged private-context service uses an explicitly selected uncharged lookup policy, not a zero-valued funding transaction under this paid profile. recoveryUntil MUST be at least payableUntil + 86400 seconds, with checked arithmetic. The prefix is uniquely bound to this acquisition and its request digest in durable storage before the challenge is sent. A retry before funding receives the same challenge while payment construction is permitted; after payableUntil it returns quoted status and the saved challenge through uncharged recovery, never a new 402 authorizing construction. A client MUST NOT construct a new payment at or after payableUntil.

The client persists the challenge and exact payment bytes before retrying the same Acquire body with the BRC-105 payment header. The seller validates the actual payment, caller, challenge, amount and its stated transaction-acceptance policy, not a caller's claim to have paid. The same payment can fund only this acquisition. This profile requires the exact challenged amount. Wrong or reused payment returns an error without creating another challenge under that request ID.

A Bitcoin transaction does not provide a trusted creation timestamp. Consequently, until recoveryUntil the seller MUST accept first delivery of an otherwise valid payment for the original frozen challenge, even after payableUntil. That grace is part of the obligation, not proof the client obeyed its creation deadline. The seller retains the reserved asset/key state accordingly. It MUST NOT reject a valid recovery solely because its HTTP session changed or the offer/catalogue disappeared. After recovery expiry, retention/reissue policy may allow more; it cannot be silently promised without storage.

Wallet internalization, ledger mutation and HTTP response are not one distributed transaction. A durable state machine records quoted -> funding-pending -> funded -> delivery-pending -> delivered. Persist the payment's exact identity before an external wallet call, use an idempotent wallet operation and reconcile uncertain results. Never repeat royalty accrual or generate a replacement payment merely because the reply was lost. A definitively invalid candidate leaves the acquisition quoted; once funding has been reserved, uncertainty stays funding-pending until reconciled. A terminal local rejection is retained as failed, not silently reset to a new invoice. A recipient-visible funded state is published only after evidence/acceptance and durable association succeed.

5. Results and lookup context

Acquisition status and recovery return:

type Acquired = {
  version: 1; acquisitionId: Hex32;
  status: 'quoted' | 'funding-pending' | 'funded' | 'delivery-pending'
    | 'delivered' | 'failed' | 'expired';
  recoveryUntil: U64; challenge: Challenge;
  funding?: { chain:Chain; txid:Hex32; outputIndex:U32 };
  reason?: string; acceptance?: ReleaseEvidence;
  result?: { evidence: Evidence; context: Bytes; schema: string }
}

result is present only for delivered. It contains the frozen purchased output's BEEF and private lookup context, not necessarily the currently advertised successor. context is interpreted only under the selected service schema. A content-access schema can return a decryption capability; BRC-198 is an optional binding for signed licensing evidence. Raw keys require authenticated confidential transport and protected recipient storage; BRC-78 wrapping adds recipient-bound encryption when selected. Authentication signatures alone do not encrypt the response.

POST /overlay/v1/private/recover accepts {version:1, acquisitionId:Hex32} and returns Acquired to the same buyer. This route is uncharged and may report pending before payment arrives. Repeating the original request/payment is also valid recovery and MUST NOT charge again. A wrong buyer gets the same not-found response as a nonexistent record, without revealing asset or payment details. Recovery limits cannot force a second purchase to obtain an already paid result.

The existing BRC-24 context channel remains available. A plugin MAY reuse the protected acquisition service internally to generate that context, but cannot claim this profile by adding a key to an arbitrary legacy query. The new route supplies explicit selection, identity, payment binding and recovery. A source adapter marks such context private and does not leak it into shared catalogue projections.

6. Migration and conformance

An application can adopt this profile without changing its existing public token format. A migration from a separate entitlement/key service MUST preserve its funded acquisitions and recovery obligations, and MUST NOT claim that public token admission implies protected-data readiness. Version private payload/encryption schemas explicitly. Application guides specify their own publication, accounting, payout and historic-data migration procedures; those product schemas are not part of this contract.

Conformance requires trusted-context rejection of payload impersonation; selective private publication; crash recovery on both atomic and staged admission paths; unavailable-key behavior; frozen listing turnover; wrong buyer; tampered amount/terms; duplicate payment; lost payment reply; pending internalization; no duplicate royalty; late first delivery; offer expiry; no paid-request fan-out; and public log/feed/GASP isolation. These tests use local fixtures and authorized principals. The contract does not promise atomic fair exchange: a seller can fail to deliver an off-chain key, and the selected acceptance policy can carry unconfirmed-payment risk.

7. Complete publication and acquisition contract

type PublicationResult = {
  version:1; publicationId:Hex32; txid:Hex32;
  status:'pending'|'ready'|'unavailable'|'rejected'|'expired';
  reason?:string; updatedAt:U64
}
type WalletFundingOperation = {
  id:Hex32; acquisitionId:Hex32;
  funding:{chain:Chain,txid:Hex32,outputIndex:U32};
  buyer:Identity; seller:Identity; satoshis:U64;
  derivationPrefix:string; derivationSuffix:string; beef:Bytes
}

Publication semantic digest is digest("publication-request", Publish with evidence.beef removed). It includes every other field, including txid, outputIndex, privateValues, schema and extensions. A proof variant may change only evidence.beef, must verify the same raw target/output, and does not replace that digest. Two request IDs may publish the same binding if their asset/schema/privateValues match exactly; they reference one protected blob with separate operation fences. A different protected value for the same (chain,topic,txid,outputIndex,assetId,schema) conflicts. A replacement requires a new public output or a distinct expressly versioned schema, not last-writer-wins.

Publication transitions are absent → pending (authorized bytes plus intent durable), pending → ready (admission and lookup binding durable), pending → rejected (definitive validation/admission failure), pending → expired (staged-work deadline reached before an external effect), and ready → unavailable (protected bytes/key/binding cannot currently be recovered). Uncertain external admission stays pending and is reconciled before expiry/cleanup; expiration cannot assert rollback. Unavailable → ready requires verified restoration of the original bytes. Rejected/expired records retain request fences and reasons; duplicate status cannot restart the operation. Request/status authorization is current publisher authority; inaccessible records return not-found. Protected temporary bytes with no admission/effect/obligation may be deleted after a declared finite staging retention, retaining their semantic digest and terminal fence. Admitted bytes underpinning outstanding acquisitions cannot be deleted on that schedule.

Before issuing a challenge, retain the signed capability, installed rules and complete acceptancePolicy, verified output/asset/terms, protected result material, response-size allowance and storage capacity through the promised recovery interval. Readiness loss prevents new quotes. It never cancels an existing quote's obligation; restoration may require protected backups. Recovery replicas must possess that material and authority or report unavailable. A key custodian is responsible for preserving encryption keys/backups through all obligations; publication eviction, catalogue withdrawal and public evidence GC do not authorize their deletion.

Derive the one expected standard BRC-29 P2PKH locking script from authenticated buyer/seller, protocol [2,"3241645161d8"], and key ID derivationPrefix + " " + derivationSuffix. Parse the complete Atomic BEEF and verify its target transaction. Scan all outputs for that exact script: exactly one must exist and its value must equal challenged satoshis; zero matches, duplicates (even with differing amounts), underpayment and overpayment are invalid. Funding identity is {chain,txid,outputIndex} of that output, never the BEEF hash or prefix alone. Alternate valid BEEF proofs for the same raw transaction/output are the same funding. A funding outpoint is globally unique in this seller's acquisition ledger; another acquisition cannot reserve it. Prefix is permanently tied to one acquisition; changing suffix/output/txid after reservation conflicts.

This explicitly specializes BRC-105: a used prefix/payment may be replayed only for its original acquisition and buyer, with no further charge/effect. Invalid payment returns invalid, not a replacement challenge; a request without payment cannot generate a fresh invoice under that ID. Exact-amount matching is stricter than BRC-105's generic at-least rule. Generic AuthFetch payment retries are not suitable unless wrapped by this persisted, selected-host acquisition procedure.

The validation/commit sequence is: validate request/challenge/authority and candidate bytes; establish exact output and acceptance-policy eligibility; atomically reserve acquisition+funding+WalletFundingOperation; call the wallet operation; reconcile its result; atomically record wallet receipt, funded ledger and a delivery intent; then issue protected material and record delivered. An unresolved evidence/policy check is pending work, not verified funding. The operation ID is digest("wallet-funding", {seller,acquisitionId,funding}). A compatible wallet adapter exposes internalizeOnce(operation) and getInternalization(id) returning accepted with exact funding/receipt, rejected with reason and no effect, absent, or unknown. Its journal and wallet effect must be atomic or the wallet must offer an equivalent idempotent identifier with durable lookup. A server-side mutex alone is insufficient. Lost reply calls getInternalization; accepted advances once, absent retries the same operation, unknown remains pending. A plain BRC-100 internalizeAction without a provable replay/reconciliation contract cannot claim this adapter capability. No inferred duplicate creates a new payment.

Acquired's required status rules are: quoted has no funding/acceptance/result/reason; funding-pending has funding and no result; funded and delivery-pending have funding plus acceptance and no result; delivered has funding, acceptance and result; failed has reason and any retained funding/acceptance but no result; expired has reason, no funding and no result. Challenge is always the saved quote. If an initial invalid candidate was never reserved it is a request error, not a failed acquisition. Funded is an observable durable state even if a fast implementation immediately progresses to delivery-pending/delivered. Failed identifies a local terminal decision with retained evidence and does not prove a signed payment can never settle elsewhere. Missing material after acceptance is a reported failure/unavailability, not a claim of absent payment or authority to repay. Clients keep a separate received/valid/usable/unusable state as in BRC-196.

At quote time require now < payableUntil, and payableUntil no later than the selected Offer's creation cutoff. RecoveryUntil is at least payableUntil + max(86400, advertised recoverySeconds, applicable Offer recoveryPeriodSeconds). All additions are checked. First complete authenticated payment receipt strictly before recoveryUntil durably pins a validation/delivery obligation; processing may finish later. At the exact deadline a previously unreceived payment is expired; a previously pinned candidate continues validation. Once funded, the undelivered obligation remains recoverable until delivered or a retained definitive failure is reported, even beyond recoveryUntil; delivery cannot be discarded while the issuer is unavailable. Recovery of delivered material is guaranteed until max(original recoveryUntil, deliveredAt+86400). These rules take precedence over subsequent Offer expiry, withdrawal, HTTP session expiry and manifest expiry. They do not extend authorization to construct a new payment after payableUntil. An unfunded, unpinned quote can expire at recoveryUntil and release capacity; retain its fence so it cannot become a new invoice.

The payment header is at most 96 KiB UTF-8 JSON; total decoded HTTP header fields at most 128 KiB, and Atomic BEEF at most 64 KiB decoded. Prefix/suffix are nonempty at most 128 ASCII characters and validated for BRC-29. Proxies must support these bounds or the service cannot advertise this profile. Larger payments require a separately negotiated BRC-118 transport profile; it is not implicitly enabled here. Requests/responses are bounded by the smaller selected profile allowance and 4 MiB. Preflight reserves the complete result envelope, not only secret length. Never accept payment for a result known not to fit. All protected responses/errors follow BRC-193 cache/CORS/logging requirements, and payment headers/secrets are excluded from telemetry. These declared finite limits are part of feasibility checking before any wallet construction.

BRC-192 section 10 supplies a provisional compatible spend graph and source membership; neither is proof that a paid lookup satisfied its release policy. Keep the exact funding evidence and frozen release/recovery basis. A membership withdrawal, replacement or reorganization cannot automatically charge again, erase an acquisition or convert non-final intent into mined payment.