TalkCo's distributed boundary: local authority, replaceable custody

The problem is not a shortage of distributed components. TalkCo can become less dependent on a central service while still leaving one gateway, indexer, key directory, repair scheduler, or identity provider with effective control. It can also become more distributed while exposing every private context through public topics, stable endpoint identifiers, or searchable provider metadata.

The design therefore distributes responsibility per object and per failure domain. It keeps private authority at endpoints, makes coordination bounded and replaceable, and treats every network or storage substrate as an adapter with a declared metadata budget.

#1. Architecture decision

Build TalkCo as a Rust-owned, local-first encrypted event system with:

• SQLite-backed device state and local-only search;
• context-scoped cryptographic identities and signed protocol events;
• causal/eventual replication for authored state and bounded CRDT use for drafts;
• quorum-certified community root operations;
• replaceable transports, relays, discovery paths, and repair workers;
• a TalkCo-specific ciphertext storage protocol over replaceable storage substrates.

Confidence is high for the layered shape and medium for exact PQ suites, overlay choice, redundancy parameters, and the first platform. The full decision record and claim-evidence case are in distributed-architecture.case.json.

#2. Definitions and non-goals

The design does not promise perfect unlinkability, metadata-free operation, immediate writes through every partition, trustless governance, universal deletion after decryption, protection against a modified client copying plaintext, or a CRDT representation for every object.

#3. The boundary map

┌──────────────────────────────────────────────────────────────────────┐
│                         TalkCo client                                │
│                                                                      │
│  UI/editor ── local policy ── KeyStore / platform keystore           │
│      │                         │                                     │
│      ▼                         ▼                                     │
│  Local projections ◄──── SQLite + FTS5 + SQLCipher                   │
│      ▲                         │                                     │
│      └──── signed, encrypted, versioned event log                    │
│                              │                                       │
│       COSE + deterministic CBOR; Protobuf/Buf transport schemas      │
│                              │                                       │
│       ┌──────────────────────┼──────────────────────┐                │
│       ▼                      ▼                      ▼                │
│  Pairwise/group crypto   Capabilities/quorums   Key transparency     │
│  PQXDH/ratchets/MLS      grants/2-of-3 roots     scoped logs/monitors │
│       │                      │                      │                │
│       └──────────────────────┼──────────────────────┘                │
│                              ▼                                       │
│                    Sync and transport adapter                        │
│                 p2panda / iroh / relay / future overlay              │
└──────────────────────────────┬───────────────────────────────────────┘
                               │ opaque encrypted envelopes
                ┌──────────────┼──────────────┐
                ▼              ▼              ▼
          Relays/store     Discovery        Optional KT replicas
          and-forward      and rendezvous   no global directory
                               │
                               ▼
                    Signed manifest chain
                               │
                               ▼
                 Encrypted fixed-size blob chunks
                               │
                 erasure-coded ciphertext shards
                               │
             S3 / Tahoe-style / Storj / self-hosted stores
                               │
                        repair workers
                    reconstruct ciphertext only

#4. Authority and consistency are per object

There is no single correct consistency model for TalkCo. Authorship, membership, key epochs, manifests, and search projections have different conflict semantics.

CRDT convergence is useful for drafts and user-owned documents. It is not an authorization decision and must not decide membership, revocation, key epochs, recovery, governance, or deletion precedence.

#5. Ranked implementation choices

#Local state and synchronization

1. SQLite + FTS5 + SQLCipher/platform keystore — adopt for v0. SQLite supplies mature local ACID storage and FTS5 supplies a rebuildable local index. SQLCipher or platform encryption protects local files, but TalkCo still owns key lifecycle and event authorization. File size, WAL, timestamps, and FTS structure are local metadata. See SQLite FTS5, SQLite WAL, and SQLCipher.
2. TalkCo signed event layer — mandatory. Use versioned encrypted envelopes, explicit predecessors and causal parents, COSE signatures, deterministic CBOR for signed objects, and Protobuf/Buf where a versioned transport schema is useful. See COSE, CBOR, and Buf compatibility checks.
3. p2panda — first distributed-sync prototype behind SyncTransport. Its signed logs, causal ordering, partial replication, and Rust fit are strong matches. It is pre-1.0; application authorization remains TalkCo’s responsibility, and peers, topics, timing, and discovery can still leak context. See p2panda and p2panda sync.
4. iroh — transport comparison, not an architecture commitment. Rust/QUIC, NAT traversal, and relays are useful for the same encrypted envelope harness. Endpoint IDs, relay use, timing, and direct network paths can correlate contexts. See iroh documentation.
5. Automerge or Yjs — bounded drafts only. Nest the document inside a TalkCo encrypted artifact. Actor IDs, history, tombstones, room names, and awareness metadata remain leakage risks. See Automerge and Yjs.
6. libp2p, Waku, and Hypercore/Autobase — later comparisons. libp2p provides broad DHT, pubsub, relay, and rendezvous primitives but exposes lookup/topic metadata and does not provide consensus. Waku offers relay/store/filter patterns but Store is not canonical durability and Filter subscriptions reveal interest. Hypercore/Autobase offers signed append-only logs but does not solve authorization, repair, or durable storage and adds a JavaScript/Rust boundary. See libp2p, Waku, Hypercore, and Autobase.

Hosted sync databases such as RxDB, PowerSync, Electric, Turso Sync, PouchDB/CouchDB, and Zero remain useful for UI or ciphertext-only experiments. They retain a backend authority model and must not become TalkCo’s private-content authority.

#Key exchange and access control

#Storage substrates

TalkCo should own six storage-level objects: EncryptedBlob, ShardSetManifest, LogicalAssetManifest, Tombstone, ProviderLease, and AvailabilityProof. The mutable application state is the signed manifest chain; the blob substrate stores opaque immutable bytes.

The storage boundary should expose narrow operations such as put_shard, get_shard, stat_shard, renew_lease, delete_shard, challenge_shard, record_receipt, claim_repair, and append_signed_manifest. It should not expose global listings, people or relationship lookup, plaintext search, provider-side application authorization, arbitrary mutable overwrite, or host decryption keys.

Erasure coding occurs after encryption. Small hot objects may use replication; large or cold objects may use erasure coding after failure-domain simulation. Repair workers reconstruct ciphertext only, use bounded leases and repair epochs, and must be tested against correlated household, network, power, geographic, and operator failures rather than node count alone.

Deletion means several different things: logical deletion, future access revocation, tombstoning, lease expiry, physical garbage collection, and crypto-erasure. TalkCo must not promise erasure of plaintext already decrypted, unknown ciphertext copies, external backups, public replicas, or recovery bundles that still contain a key.

#6. Key hierarchy and mechanism separation

The five unresolved architectural contradictions have these working resolutions:

• Signal fork versus Rust core: retain Signal protocol semantics as a reference and implement a narrow Rust-owned adapter. Direct libsignal reuse remains gated by API stability, license, platform, and security review.
• TACo commitment versus conditional access: commit to a ConditionalAccess interface; treat TACo as one candidate implementation for exceptional content, not ordinary messages.
• PQ confidentiality versus PQ authenticity: hybrid ML-KEM/X25519 can provide post-quantum confidentiality while Ed25519 remains classical authentication. Long-lived governance, migration, and recovery records should support dual Ed25519/ML-DSA signatures. Do not claim full PQ security until authentication, recovery, and governance are included. See FIPS 203 and FIPS 204.
• Key transparency versus no global directory: use blinded, sphere/community-scoped logs with local pinning, independent monitors, and optional peer gossip. Do not create a global lookup or equivalence graph.
• Threshold signatures versus threshold custody, erasure coding, and Byzantine consensus: signatures authorize records; custody releases key material; erasure coding reconstructs encrypted bytes; Byzantine consensus orders committee state. Separate APIs, threat models, and tests are mandatory.

#7. Minimal v0 and staged adoption

#Minimal v0 composition

• Rust protocol core.
• SQLite + FTS5 + SQLCipher/platform keystore.
• Signed, encrypted, versioned append-only events.
• COSE + deterministic CBOR for signed records; Protobuf/Buf where useful for transport.
• HPKE hybrid experiment and OpenMLS group prototype behind crypto adapters.
• Independent two-of-three community signatures.
• Minimal scoped capability grants.
• p2panda sync adapter plus a dumb relay fallback.
• S3 ciphertext blob adapter.
• Fixed-size encrypted chunks, signed manifests, tombstones, leases, and ciphertext-only repair.
• Local-only FTS and no global private index.

#Roadmap

1. Freeze the object authority/consistency matrix, key hierarchy, metadata budgets, and adapter traits.
2. Build the two-device encrypted-event slice with offline writes, duplicate delivery, local search, pairing, and membership removal.
3. Run identical envelopes over p2panda and iroh; measure convergence, mobile behavior, metadata, battery, and coordinator dependence.
4. Build a five-host ciphertext harness with S3 and one self-hosted substrate; test manifests, repair, migration, expiry, tombstones, and failure domains.
5. Add two-of-three roots, fork handling, scoped key transparency, and recovery bundles.
6. Trial Storj, Tahoe-style storage, libp2p/Waku, Garage/SeaweedFS, and conditional access only after the preceding evidence gates pass.

Every dependency needs a stable adapter boundary, reproducible tests, compatible licensing, documented metadata leakage, explicit failure behavior, no authority over TalkCo identity or security state, migration evidence, and an acceptable maintenance history.

Hard rejects are global plaintext state, public people/content search, reusable cross-sphere identifiers, central permanent recovery, online centralized coordination for ordinary private reads, public enumeration of private DHT namespaces, hosts with default decryption rights, unversioned remote policies, provider listings as canonical state, universal deletion claims, and generic LWW/CRDT merges for membership, revocation, or key epochs.

#8. Smallest falsifying experiment

Build a disposable harness containing:

• two devices and two unrevealed spheres owned by one device-local root;
• two direct-message peers and one three-steward community;
• two key-transparency replicas and three relays/rendezvous nodes;
• five storage hosts across at least three failure domains and one repair worker;
• S3 plus one self-hosted storage adapter;
• p2panda and iroh transport adapters.

Exercise offline creation, duplicate delivery, reordering, replay, simultaneous MLS Commits, steward equivocation, hybrid key establishment, group removal, capability attenuation/expiry/replay, key-directory forks, corruption, withholding, sleeping hosts, correlated failures, repair races, provider migration, tombstones, leases, legal hold, crypto-erasure, recovery, and removal of relay/discovery/repair/steward roles.

The design is falsified if a host decrypts content; a coordinator is required for ordinary reads; duplicate delivery changes meaning; a removed member decrypts a future epoch; membership conflicts silently resolve; fewer than the quorum authorizes a root transition; stale manifests resurrect state; two unrevealed spheres share a recoverable protocol identifier; repair requires plaintext; migration changes object meaning; key-transparency forks remain undetectable under the stated assumptions; or traces reveal unacceptable cross-sphere correlation.

Acceptance evidence should include Wycheproof and official ML-KEM/ML-DSA vectors, property/state-machine tests, TLA+/TLC models for epochs/recovery/quorum/tombstones, Testcontainers multi-node runs, Toxiproxy fault injection, packet-capture metadata summaries, cross-implementation COSE/CBOR and HPKE tests, explicit t-1 quorum failures, and migration/revocation/deletion-boundary tests. See Wycheproof, TLA+, Testcontainers, and Toxiproxy.

#9. Open questions ranked by impact

#10. Do next

1. Freeze the object authority/consistency matrix as the protocol contract.
2. Define LocalStore, EventLog, SyncTransport, KeyStore, BlobStore, ManifestStore, and RecoveryStore traits.
3. Build the two-device encrypted event and local-search slice.
4. Prototype OpenMLS membership and epoch removal behind a crypto adapter.
5. Compare p2panda and iroh with identical envelopes and failure schedules.
6. Build the five-host ciphertext storage harness with S3 and one self-hosted substrate.
7. Run cross-sphere packet/storage correlation tests before adding public discovery.
8. Freeze dependency-license and metadata-leakage gates before selecting production packages.