TalkCo's architecture after the alternatives are forced apart

The problem is that the desirable properties do not arrive in one package. Local-first databases make a device productive while disconnected. Private queues reduce correlation. Mesh protocols survive network loss. Group protocols coordinate membership. Storage systems provide durability. A single overlay usually makes one of those concerns authoritative, and that authority becomes the new platform centre.

The research therefore refines the second-round recommendation rather than replacing it: TalkCo should own the encrypted event and authority model, then use several narrow fabrics behind adapters. Distribution is a property of the boundaries and failure domains, not a requirement to select one fashionable network.

#1. The refined decision

Adopt this shape as the implementation target:

┌─────────────────────────────────────────────────────────────────────┐
│ TalkCo client                                                       │
│                                                                     │
│ UI → local policy → SQLite/FTS5 → encrypted event log               │
│                  ↘ platform keystore                               │
│                                                                     │
│ TalkCo authority layer: contexts, signatures, capabilities, epochs  │
│                                                                     │
│ Sync scheduler and metadata accounting                              │
│       ┌──────────────┬──────────────┬──────────────┬──────────────┐ │
│       │ private queue │ direct/QUIC  │ local mesh   │ group relay  │ │
│       │ SimpleX-like  │ iroh-like    │ Briar/SSB-   │ p2panda/     │ │
│       │ delivery      │ delivery     │ like         │ Marmot-like  │ │
│       └──────────────┴──────────────┴──────────────┴──────────────┘ │
│                              │ opaque envelopes                     │
│                 signed manifest and repair protocol                 │
└──────────────────────────────┬──────────────────────────────────────┘
                               │
       S3 / Tahoe-style / Storj / self-hosted object adapters
       ciphertext chunks, erasure-coded shards, leases, repair proofs

TalkCo owns:

• the event identity, causal ancestry, signatures, and validation rules;
• the mapping from a local authorized view to encrypted protocol objects;
• context membership, revocation, recovery, and key-epoch transitions;
• the metadata accounting and routing choice for each object class;
• immutable ciphertext manifests, tombstones, leases, and repair policy;
• local search and all plaintext projections.

Adapters may deliver, relay, discover, copy, cache, or repair opaque objects. They may not define membership, ordering, authorization, deletion precedence, or the canonical plaintext view.

Confidence is high for this separation and medium for each concrete adapter. The claim-evidence record is architecture-exhaustion.case.json.

#2. The architecture families that survived comparison

The important result is not that every family should be implemented. It is that their boundaries are now legible: SimpleX-like queues solve delivery privacy, Briar-like links solve physical/network partitioning, MLS/Marmot-like layers solve group epochs, and Tahoe/Storj-like systems solve ciphertext availability. None owns the complete TalkCo object model.

#3. The object-to-fabric map

Choosing a fabric per object avoids paying the metadata and coordination cost of the most public transport for every operation.

MLS authenticates and advances group epoch/transcript state; TalkCo policy authorizes credentials, membership transitions, grants, and revocation. Neither MLS arrival order nor a relay may resolve a disagreement between those authorities silently.

#4. Security and access control after the new evidence

#4.1 Keep protocol semantics separate from implementation maturity

The direct-message seam remains PQXDH/ratchet semantics, with an implementation selected only after license, platform, and test-vector review. The one-shot wrapping seam remains HPKE with a hybrid ML-KEM/X25519 profile. The current IETF HPKE post-quantum draft defines hybrid ML-KEM/X25519 and ML-KEM/P-256/P-384 options, but it is still a draft, so TalkCo must persist a suite identifier and support migration rather than freeze provisional identifiers. HPKE-PQ draft

MLS remains the group prototype. The active MLS post-quantum ciphersuite draft is still work in progress and distinguishes PQ confidentiality from PQ authentication. TalkCo should therefore implement a versioned GroupCryptoProvider, test a hybrid suite, and keep application signatures and steward approvals independent of MLS ciphersuite selection. MLS PQ ciphersuites draft

#4.2 Treat key transparency as evidence, not authority

The current Key Transparency architecture and protocol drafts describe fork detection, multiple independent logs, monitors, migration, stale-response bounds, and the consequences of lost client state. Those mechanisms can strengthen a context-scoped directory, but they do not remove the need for trusted monitors, a connected gossip path, or explicit deployment assumptions. KT architecture draft, KT protocol draft

TalkCo should use transparency only for:

• a sphere or community namespace rather than a global identity graph;
• a small set of independently operated logs or witnesses;
• offline-retained tree heads and bounded stale-response policy;
• explicit log migration and tombstone handling;
• a monitor that produces a portable, non-repudiable fork proof.

The client still validates an invitation, grant, key epoch, and signature locally. Transparency detects an equivocation pattern; it does not make an unauthorized key valid.

#4.3 Marmot is a benchmark, not a blind dependency

Marmot is the strongest current evidence that Nostr-style transport and MLS-style groups can be composed. It should be tested against an independent TalkCo event envelope and metadata accounting. The audit evidence makes the gate concrete:

• document the event/package relationship and threat model before adoption;
• separate exporter secrets and media keys with explicit domain separation;
• guarantee unique media nonces across repeated sends and epochs;
• bound SQLite history and pending-welcome reads;
• isolate storage state, key-package state, and group-state lifecycles;
• verify that the pinned revision contains the audited fixes, inspect the post-audit delta and LastResort KeyPackage defaults, and review the license of any Rust MDK dependency.

The benchmark may result in a Marmot adapter, a plain MLS adapter, or a TalkCo-specific group envelope. The group authority remains TalkCo's in all three cases.

#5. The distributed storage protocol

The second round's ciphertext manifest idea survives intact, but the alternatives clarify which parts should be borrowed.

plaintext
  → chunking and authenticated encryption
  → opaque ciphertext chunks
  → optional erasure coding across declared failure domains
  → signed ShardSetManifest
  → provider leases and placement receipts
  → independent availability challenges and ciphertext-only repair

The minimum objects are:

• EncryptedBlob: immutable ciphertext bytes, size, ciphertext digest/commitment, algorithm suite, and chunking parameters;
• ShardSetManifest: shard hashes, coding parameters, failure-domain labels, and repair epoch;
• LogicalAssetManifest: signed succession from a logical asset to one or more blob generations;
• Grant: context-scoped capability and recipient/key reference, never a provider credential;
• ProviderLease: expiry and retention request, not proof of application ownership;
• PossessionProof: evidence that a provider can answer a challenge over a ciphertext shard;
• Tombstone: a signed logical removal or access-revocation event, distinct from physical garbage collection.

The storage adapter must offer only narrow operations such as put_shard, get_shard, stat_shard, renew_lease, challenge_shard, and delete_shard. A provider may expose a bucket listing to its operator, but TalkCo must not use that listing as a canonical person, relationship, membership, or application index.

#Storage choices

Syncthing is a useful block-transfer benchmark between trusted devices, not the untrusted archive. Its block exchange, discovery, and relay separation can inform the ShardTransport interface.

Repair must reconstruct ciphertext only. It must not require a plaintext key, infer new recipients, mutate an event log, or silently replace a manifest. The simulation must use correlated failure domains—households, operators, jurisdictions, and network partitions—not only random node loss. The storage adapter must declare its control-plane dependency: Tahoe immutable reads may proceed from a capability and known placement; Ceph needs a monitor map before direct OSD reads; S3, Storj, and Walrus normally involve service metadata; Hypercore can read directly once peers are known but usually discovers them through a routing layer.

Deletion remains a vocabulary, not a promise: logical removal, future-access revocation, tombstone propagation, lease expiry, physical garbage collection, and crypto-erasure are different operations. No architecture can erase plaintext already viewed or unknown copies in external backups.

#6. Combinations explicitly rejected

These combinations are attractive because they reduce integration work. They are rejected because they collapse too many authorities into one dependency.

#7. Implementation sequence and falsifying gates

The sequence is deliberately staged so that an attractive network cannot hide a broken authority model.

#Gate 0: freeze the contracts

Write the Rust traits and test vectors before selecting a network:

• EventCodec, EventValidator, ContextAuthority, GrantValidator;
• GroupCryptoProvider, DirectSessionProvider, KemSuite;
• SyncTransport, QueueTransport, MeshTransport, BlobStore, RepairWorker;
• stable event IDs, causal ancestry, suite IDs, context IDs, and domain-separation labels.

Fail if a transport can create an event that local validation cannot explain, or if a storage backend can mutate application state without a signed manifest.

#Gate 1: two-device local-first core

Build encrypted SQLite state, local FTS5 projections, signed events, duplicate suppression, offline queues, key rotation, and recovery from an encrypted bundle. Exercise direct messages, membership changes, tombstones, and a deliberately stale replica.

Fail if local search requires network access, if stale state can grant access, if a replay changes a projection, or if recovery silently creates a new identity.

#Gate 2: transport matrix

Run the same event harness over:

1. a dumb relay;
2. an iroh-style direct/QUIC path;
3. a SimpleX-inspired pairwise queue;
4. a local Bluetooth/Wi-Fi/removable-media path;
5. a p2panda-style signed-log sync path.

Capture endpoint identifiers, relay identifiers, topics, timing, packet sizes, retries, and cross-context correlation under explicit metadata accounting. Measure partition recovery and relay compromise separately from cryptographic compromise.

Fail if a transport is required to establish membership, if private contexts are trivially linkable through stable routing identifiers, or if the client cannot switch transports without changing event identity.

#Gate 3: group and key-access matrix

Compare plain MLS/OpenMLS framing with a Marmot-shaped relay envelope. Add concurrent joins, removals, stale key packages, forked commits, lost devices, exporter misuse, media resend, and transparency-log equivocation. Use the current draft suite identifiers only as versioned experiments; never bake provisional identifiers into an irreversible store format.

Fail if a removed member decrypts a future epoch, a simultaneous commit is silently selected by arrival order, a transparency fork lacks portable proof, media key/nonce derivation permits reuse, or TalkCo policy, p2panda authorization state, and MLS membership are allowed to resolve a disagreement by arrival order.

#Gate 4: ciphertext storage matrix

Run identical manifests against S3, a controlled Tahoe-style grid, and one self-hosted multi-node substrate. Add host loss, correlated operator loss, stale manifests, partial uploads, lease expiry, duplicate shards, malicious shard responses, and repair during a partition.

Fail if a provider can forge a valid manifest, if repair needs plaintext, if logical revocation depends on provider deletion, or if the redundancy profile fails the declared failure domains.

#Gate 5: community distribution and recovery

Only after the preceding gates pass, test independent operators, scoped transparency logs, steward rotation, recovery bundles, old-device pairing, and exit/import. A user must be able to export verified encrypted state and rejoin through a different relay/storage set without re-registering with a global account service.

Threshold recovery is now a first-class experimental seam, not a promise for v0. A RecoveryStore may distribute recovery-bundle assistance across independent trustees or devices, but recovery nodes must never become account owners, sphere owners, or a permanent identity directory. A threshold OPRF/proactive-secret-sharing design such as Kintsugi is a research reference only; its global username-to-node locator model would violate TalkCo's private-namespace boundary. The v0 path remains an encrypted user-held bundle plus old-device pairing.

#Gate 6: recovery, admission, and evidence semantics

Test threshold recovery, committee rotation, old-share rejection, password-guessing limits, and recovery-metadata correlation separately from signing and governance. Anonymous invite or rate-limit credentials may be tested as a bounded abuse-control mechanism using Privacy Pass or AnonCreds, but they do not provide Sybil resistance without an issuer or attester authority.

Use distinct evidence records rather than an overloaded availability label:

• PossessionProof: a host answered a challenge over ciphertext;
• IntegrityProof: the response matched a signed shard commitment;
• FreshnessReceipt: a lease or repair epoch was current;
• AvailabilityObservation: an independent observer retrieved the object;
• RepairReceipt: a worker replaced a missing or corrupt shard.

None of these proves future availability, failure-domain independence, legal compliance, or that an unknown replica was erased. A Merkle challenge-response is a useful storage experiment; Filecoin-style proofs remain disproportionate for v0. A crypto-erasure or revocation claim fails while a recovery bundle, trustee share, retained epoch secret, or old device can still recover the relevant key.

The distributed-systems lane also narrows the architecture claim. Replaceability reduces lock-in but does not prove coordinator independence. A two-of-three steward rule is ThresholdAuthorizedRoot, not Byzantine consensus. A quorum or BFT protocol would require explicit membership, quorum intersection, reconfiguration, fork, and liveness assumptions. Use consensus only for exceptional, online community-root transitions if the threat model eventually justifies it.

#8. Cryptographic corrections from the verification lanes

• Describe the direct-message target as PQXDH plus Triple Ratchet semantics with an ML-KEM Braid-style sparse post-quantum ratchet, not as an unspecified “PQ ratchet.” The prekey service remains untrusted and needs identity pinning or scoped transparency.
• Keep RFC 9180 HPKE as the abstraction, with hybrid ML-KEM/X25519 behind a versioned KemSuite. The current HPKE-PQ and PQ-MLS drafts have provisional identifiers and must not define irreversible TalkCo wire IDs.
• Use OpenMLS classical suites for the first executable group prototype, with a GroupCryptoProvider seam for PQ-MLS experiments. The active PQ-MLS draft includes ML-DSA-authenticated suites as well as classical-authenticated hybrid suites; this is a long-term target, not a launch dependency. Pin exact providers and test secret-tree persistence, crash consistency, and advisory status.
• Keep independent signatures for v0. FROST is now RFC 9591, but it remains classical, non-robust, and not a consensus, custody, recovery, or storage-repair protocol.
• Keep minimal signed COSE/CBOR capabilities. Borrow attenuation and delegation invariants from UCAN/Biscuit without importing DIDs, Datalog, or an unrestricted policy language. Capability possession never implies decryption, and revocation is an authorization-state change rather than erasure of plaintext already viewed.
• Treat key transparency as scoped equivocation evidence with monitors, gossip, stale-response bounds, and migration—not as identity creation or decentralization.

#9. Current decisions and remaining questions

#Distribution corrections from the adversarial lane

• p2panda's current workspace has generic authentication, encryption, discovery, and authorization components. The correct boundary is not “p2panda has no authorization”; it is “p2panda does not define TalkCo's application authority, recovery, deletion, or governance semantics.”
• p2panda and iroh are not independent black-box alternatives if p2panda uses iroh underneath. Compare p2panda protocol semantics over iroh, raw iroh with TalkCo events, and at least one non-iroh path such as a dumb relay or local mesh.
• iroh's v1.0 maturity makes it a credible transport candidate, but its endpoint IDs, relay URLs, addresses, connected-peer metadata, and discovery path remain visible to infrastructure. Maturity does not reduce the metadata accounting.
• The first implementation ranking is now: Rust event core → SQLite/rusqlite local store → iroh 1.0 transport → dumb-relay fallback, with p2panda 0.7 as a pinned semantic-sync adapter. p2panda's evolving Postcard wire format must remain separate from TalkCo's canonical COSE/deterministic-CBOR event encoding.
• SQLite session/changesets can optimize local movement and conflict application, but they do not replace signed event identity, capability validation, key epochs, membership semantics, or deletion policy. FTS5 remains a rebuildable local projection.
• Loro 1.0 is a credible Rust/WASM/Swift bounded-draft candidate. Automerge and Yrs remain alternatives where their interoperability or ecosystem is needed. None may authorize security state.
• Native SQLite, browser/WASM SQLite, IndexedDB, and mobile storage have different worker, quota, crash, background, and durability behavior. The first platform must be selected by a cross-shell golden-vector and suspend/resume experiment, not hidden behind a generic storage abstraction.
• PowerSync, Electric, RxDB, Turso/libSQL, PGlite, and Dexie are implementation references or opaque ciphertext/UI adapters. Their hosted or source-database authority must not be mistaken for TalkCo's event authority.
• An explicitly modeled authorization CRDT may be admitted for a narrow membership state if stale-state, concurrent-removal, and failure semantics are proven. Arbitrary CRDT merges remain prohibited for key epochs, revocation, recovery, deletion, or governance.
• Global DHTs remain unsuitable for private namespaces because signatures prevent forgery but not Sybil routing, eclipse, withholding, censorship, or lookup correlation. Optional Tor/mixnet routing can protect selected discovery or recovery traffic, but it is a privacy plane, not a database or storage layer.
• Replaceability is a design property, not evidence of coordinator independence. Each operation must state which relay, discovery service, transparency log, quorum/threshold signer, repair worker, or endpoint availability assumption it makes.

#Settled for the next prototype

• Rust-owned protocol core with SQLite, FTS5, and platform-backed key protection.
• Signed, encrypted, versioned TalkCo events above the local database.
• Context-scoped identities and capabilities; no global private directory.
• Separate direct, group, delivery, discovery, storage, and repair seams.
• Plaintext search remains local and rebuildable.
• S3-compatible ciphertext storage is the first adapter; Tahoe is the semantic reference; Ceph and SeaweedFS are the serious self-hosted repair/placement benchmarks.
• CRDTs are limited to encrypted drafts and user-owned mergeable artifacts; Loro is the first new bounded-draft candidate to benchmark.
• The first transport experiment is iroh 1.0 with a non-iroh dumb-relay control adapter; p2panda 0.7 is the first semantic-sync adapter, pinned and isolated. The first client platform remains an explicit gate.
• Recovery is an explicit seam: encrypted bundle and old-device pairing first, threshold custody only as an adversarial experiment.
• Storage evidence is split into possession, integrity, freshness, observed retrieval, and repair; no proof is called “availability” without stating the exact observation.
• “More distributed” means responsibility-partitioned and coordinator-independence-tested; it does not follow from node count or adapter replaceability.

#Remaining research: freeze contracts before comparing packages

The broad architecture survey is complete. The remaining research is narrower: define the semantics that every adapter must obey, then measure the implementations against them. The following items are ordered by whether they can invalidate the first harness.

1. Identity and authority taxonomy. Distinguish person or account roots, device credentials, context identities, group credentials, recovery credentials, and steward authority. SQLite remains derived state; MLS validates group cryptographic state; p2panda contributes only adapter assertions; recovery credentials remain scoped. A disagreement must produce accept, reject, or quarantine, never an arrival-order winner.

Exit criterion: define a deterministic transition function and golden vectors for replay, stale replicas, concurrent add/remove, rollback, expired grants, tombstones, recovery imports, and MLS/application disagreement. Every event-arrival permutation must produce the same authorization result.

1. Atomic transition and crash semantics. Decide how event append, projection update, outbox publication, ratchet advancement, MLS persistence, grant transitions, and recovery-state changes become durable together. A component-local storage API is not automatically a cross-component transaction. OpenMLS persistence and its secret-tree persistence advisory make this a security boundary, not only a reliability concern.

Exit criterion: specify one SQLite transaction or durable intent/commit journal, inject process death after every write, and prove restart yields either the complete transition or the previous state. No envelope may be published from uncommitted cryptographic state.

1. First platform and at-rest threat model. Choose one initial pass/fail shell rather than pretending native, browser, and mobile storage are equivalent. Define whether protection covers encrypted event payloads, database pages, FTS projections, WAL/journals, temporary files, backups, exports, logs, crash dumps, thumbnails, and retained ratchet/MLS/recovery secrets. SQLite WAL and SQLCipher’s design notes should inform the experiment; they do not replace it.

Exit criterion: record the filesystem, SQLite mode, keystore boundary, backup policy, suspend/crash model, and attacker matrix. Disk-image inspection after representative operations and forced crashes must find no undeclared plaintext class. Browser and mobile become separate probes with their own lifecycle and background-delivery tests.

1. Migration, time, and resource bounds. Version event schemas, canonical encodings, capabilities, crypto suites, database schemas, and group-state persistence independently. Define downgrade behavior, offline expiry semantics, lease freshness, clock rollback handling, causal-depth limits, envelope limits, skipped-key limits, pending-proposal limits, recovery-attempt limits, and projection rebuild limits.

Exit criterion: golden vectors cover unknown versions, downgrade attempts, clock rollback, no-clock operation, oversized inputs, cyclic or deep ancestry, and migration restart. Every untrusted collection has a documented maximum and deterministic failure behavior.

1. Direct-session semantics. If direct messages are in the first vertical slice, define per-device sessions, offline initiation, duplicates, out-of-order delivery, skipped-key bounds, backup rollback, and deletion semantics now. Compare concrete ratchet libraries later; a mock DirectSessionProvider should pass deterministic vectors first. The PQ ratchet choice remains deferred until the real message schedules are measured.
2. Metadata observer model. Define a normalized trace schema before comparing iroh, a dumb relay, a queue-shaped adapter, or local mesh. It must model local process, OS/network, relay, discovery, storage, group-member, colluding-provider, and global-passive observers, and distinguish observed, inferred, unavailable, redacted, and padded fields.

Exit criterion: every adapter emits endpoint and relay identifiers, identifier lifetimes, topics, byte counts, timing buckets, retries, batching, padding, and correlation hypotheses in the same trace format. Linkability and padding effectiveness are measured by the harness rather than asserted from protocol descriptions.

1. Group implementation pin and comparison. Pin an immutable OpenMLS revision, crypto provider, storage provider, feature set, license, advisory state, migration path, and persistence format before the group gate. The release page currently lists OpenMLS v0.8.1 as latest, so a major-version label is insufficient. Marmot’s protocol is adopted, while marmot-ts remains alpha; that makes Marmot a later wire-envelope benchmark, not a first dependency.

Exit criterion: run identical concurrent join, remove, stale-key-package, fork, exporter, media-resend, crash, and multi-device scenarios through plain OpenMLS and Marmot framing. Fail on silent policy/MLS disagreement, future-epoch decryption by removed members, or non-portable fork evidence.

1. Canonical storage schema before storage backends. If attachments are part of the first event format, freeze deterministic fixtures for ciphertext commitments, EncryptedBlob, ShardSetManifest, logical generations, grants, leases, tombstones, repair epochs, and the five evidence records. Start with a fake blob store and S3 compatibility; only then compare Tahoe semantics, Ceph, SeaweedFS, Garage, Walrus, or IPFS Cluster under correlated failure.

Exit criterion: stale, corrupt, partial, duplicate, expired, and repaired shards produce deterministic results; repair changes ciphertext custody but cannot change logical event identity or require plaintext keys.

1. Dependency and license readiness. Treat exact pins, lockfiles, transitive licenses, MSRV, target builds, advisories, reproducibility, and upgrade ownership as an adoption gate for each selected dependency. Do not spend another research round auditing every deferred candidate. The current iroh release list already has patch-level movement beyond the report’s “iroh 1.0” shorthand; pin the exact release or commit. Keep p2panda 0.7 isolated because its group-encryption and multi-device work is still evolving in the 0.7 release.

#What should be measured, not researched

Once the contracts exist, the main unknowns are empirical:

• crash recovery, WAL behavior, duplicate suppression, replay handling, and projection rebuilds;
• suspend/resume, keystore failures, quota, process locking, background delivery, build size, and battery cost;
• transport linkability, timing and packet-size leakage, relay compromise, partition recovery, and transport switching without changing event identity;
• concurrent group commits, removals, stale key packages, forks, exporter misuse, media resend, and crash during persistence;
• recovery latency, stale-bundle rejection, duplicate identity prevention, password-guess limits, and recovery metadata;
• storage repair under operator, household, jurisdiction, and network-partition failures.

#What should remain deferred

Production PQ-MLS or HPKE-PQ identifiers, a custom PQ ratchet, Marmot as a dependency, actual SimpleX integration, threshold recovery, deployed Key Transparency, global DHTs, mixnets, economic storage networks, and ordinary consensus should not block the first harness. The PQ-MLS and HPKE-PQ drafts remain versioned experiments. Scoped transparency, admission credentials, and trustee committees advance only after the identity/device and v0 recovery models establish a concrete requirement.

The implementation order is therefore:

identity/device taxonomy → authority oracle → atomic commit protocol → platform/at-rest contract → migration/time/resource bounds → mock direct/group/recovery/storage seams → two-device harness → metadata traces → pinned group harness → adapter and storage comparisons

The next implementation decision should be made by the falsifying harness, not by package popularity: choose the smallest stack that passes the authority, metadata, partition, and repair gates, then keep the other candidates as replaceable adapters.