FEN Backend Architecture — Garage S3

Status: normative design · companion to fen.md and canonical-serialization.md. Decision context: the "run nothing" constraint is dropped. FEN operates one small, dumb, ciphertext-only component (Garage + a tiny provisioner), starting on a Synology NAS and migratable to a VPS/cloud as usage grows. WebDAV and other targets are completely removed.

1. The Trust Model

FEN is designed so the backend cannot read any data.

Can see: encrypted event/attachment bytes (random-looking), the per-group bucket id, per-member logs/<pubkey>.jsonl file names (so: group structure + pseudonymous member pubkeys), sizes, timestamps, and client IPs.
Cannot see: group names, member real identities, expense amounts, notes, balances — anything content. The XChaCha20-Poly1305 + Ed25519 layer (unchanged) guarantees this.

2. Architecture Components

flowchart LR
  subgraph client[Member device]
    APP[FEN app + S3Backend<br/>SigV4 on-device]
  end
  subgraph edge[Cloudflare Tunnel / Tailscale Funnel]
    CF[stable domains<br/>s3.fenapp.net · api.fenapp.net]
  end
  subgraph host[Your host: Synology NAS today, VPS later]
    GARAGE[(Garage<br/>S3 data plane<br/>1 bucket / group)]
    PROV[Provisioner<br/>control plane<br/>~200 LOC]
  end
  APP -- "list/get/put/delete (SigV4, per-group key)" --> CF --> GARAGE
  APP -- "create/rotate/close group (Ed25519-signed)" --> CF --> PROV
  PROV -- "admin API v2: CreateBucket/CreateKey/AllowBucketKey/UpdateBucket" --> GARAGE

Two planes, deliberately separated:

Data plane = Garage. Members talk to it directly with S3 SigV4 using a per-group key. The provisioner is never in the data path, so data throughput scales with Garage alone.
Control plane = Provisioner. A tiny authenticated service that touches only group lifecycle (create, rotate key, close/delete). It holds the Garage admin token; clients never do.

3. Storage model — one bucket per group, one key per group

At group creation the provisioner calls the Garage admin API v2:

CreateBucket with globalAlias = grp_<groupId>.
UpdateBucket to set a quota (maxSize, maxObjects) — e.g. 100 MB / 5 000 objects per group.
CreateKey → returns { accessKeyId, secretAccessKey }.
AllowBucketKey granting that key read + write on that bucket only.

Garage's permission model is per-access-key-per-bucket, so one bucket + one key gives clean, hard isolation: that key can touch only that group's bucket.

4. `StorageDescriptor`

The shared descriptor is created by the provisioner and returned to the group owner over the signed HTTPS POST /v1/groups control-plane call. After that initial owner binding, the secret reaches other members only through encrypted member-facing channels: the encrypted invite package (&p=) at join and the epoch-encrypted StorageKeyRotated event when the credential is rotated (§6.1):

final class S3Descriptor {
  const S3Descriptor({
    required this.endpoint,       // https://s3.fenapp.net   (stable domain)
    required this.region,         // "garage"
    required this.bucket,         // grp_<groupId>
    required this.accessKeyId,    // per-group
    required this.secretAccessKey,// per-group BEARER SECRET — invite-encrypted only
  });
}

5. On-bucket layout

(bucket grp_<groupId>)/
  roster/roster.jsonl                 # encrypted roster events
  logs/<member_pubkey>.jsonl          # one append-only file per author
  epochs/<epoch>.json                 # KeyEpochUpdate payloads
  attachments/<sha256(ciphertext)>    # encrypted receipt blobs

Single-writer-per-object. Each member uses one active device (v1), so every object has exactly one writer and concurrent writes never collide. In v2 (multi-device), this will shard to logs/<pubkey>/<deviceId>.jsonl to preserve the single-writer property.

5.1 Friend direct-add relay

The known-friend direct-add path deliberately does not add a server mailbox, pubkey-indexed lookup service, or any backend-visible friend graph. Delivery reuses a group log that the organiser and invitee already share: the organiser wraps the target group's invite package to the friend's known public key using the existing X25519/HKDF primitives, then writes a GroupInviteRelayed event to the selected relay group.

A relay group is eligible only if it is active and shared by both parties: open, not closed, not a cold/local-only archive, not scheduled for backend deletion, not already purged, backend-writable/reachable by the organiser, and normally syncable by the invitee. If several eligible active shared groups exist, clients choose the one with the newest creation date. If none exists, the UI falls back to the normal invite-link flow.

6. Provisioner API (control plane)

A stateless ~200-line service (Go/Rust). All requests are signed by the owner's Ed25519 key and rate-limited.

Endpoint	Body (signed)	Effect	Returns
`POST /v1/groups`	`{ groupId, ownerPubkey, ts, sig }`	`CreateBucket` + quota + `CreateKey` + `AllowBucketKey(read,write)`	`S3Descriptor`
`POST /v1/groups/:id/rotate-key`	`{ ownerPubkey, ts, sig }`	new `CreateKey`+`AllowBucketKey`; old key stays valid, scheduled for deny at `denyOldAt` (§6.1)	new `S3Descriptor` + `denyOldAt`
`POST /v1/groups/:id/rotate-key/complete`	`{ ownerPubkey, ts, sig }`	`DenyBucketKey` on every superseded key now (early completion of §6.1 phase 2); idempotent	ok
`DELETE /v1/groups/:id`	`{ ownerPubkey, ts, sig }`	Executes `DeleteBucket` on Garage immediately.	ok
`POST /v1/groups/:id/close`	`{ ownerPubkey, ts, sig }`	Denies write access (current and superseded keys), and schedules bucket for deletion in 45 days.	ok
`POST /v1/groups/:id/invites`	`{ ownerPubkey, ts, sig }`	`CreateKey`+`AllowBucketKey(read,write)`; recorded with an `expiresAt` deadline (§6.2)	`{ inviteId, accessKeyId, secretAccessKey, expiresAt }`
`POST /v1/groups/:id/invites/:inviteId/complete`	`{ memberPubkey, ts, sig }`	Rejects if past `expiresAt`; if unused, atomically records `memberPubkey` as the winner and denies the invite credential; if already used by the same `memberPubkey`, retries idempotently; if used by a different pubkey, rejects as already used	current `S3Descriptor` (the group's shared credential)

POST /v1/groups is not optional for Managed-tier groups. The client may create the local event log while offline, but it must not mark the group syncable, mint invites, or attempt S3 sync until this call has returned and the real descriptor has been stored. Placeholder or locally-random S3 credentials are invalid and must be treated as a local-only/unbound state. This integration gap is tracked by implementation-review issue #68.

6.1 Two-phase key rotation (credential hand-off)

The per-group S3 key is shared by every member, and the bucket it protects is the only in-band channel for delivering a replacement. If rotate-key denied the old key immediately, rotation would cut off every remaining member at once with no way to receive the new credential (design-review finding #1 / issue #15). Rotation is therefore two-phase; the client-side protocol (StorageKeyRotated / StorageKeyAck events, epoch interleaving on removal) is specified in data-model.md §3.10, the security analysis of the overlap window in security.md §4.8.

Rotate. POST …/rotate-key mints and grants the new key and returns the full new S3Descriptor plus denyOldAt = now + ROTATE_GRACE_PERIOD_SECONDS (default 7 days). The old key keeps read+write; the provisioner records it as pending deny with that deadline. Rotating again while a rotation is pending is allowed — each superseded key keeps its own deadline, so a removed member's exposure is never extended by a later rotation.
Hand off (client side). The organiser publishes the new descriptor to remaining members as an epoch-encrypted StorageKeyRotated event, readable via the old credential; members switch and ack.
Complete. When all remaining members have acked, the owner calls POST …/rotate-key/complete and the provisioner denies every pending key immediately. If that call never comes, the provisioner's background reaper denies each pending key at its denyOldAt deadline — the overlap window is bounded server-side, independent of client liveness.

The provisioner never reads the event log and never learns the roster: when to complete early is entirely the organiser's judgement; the deadline is the provisioner's only own decision. close demotes superseded keys to read-only alongside the current key (members mid-hand-off keep read access through the close grace period); bucket deletion denies everything.

6.2 Invite credentials (single-use, short-TTL)

Embedding the group's shared S3 key directly in an invite link made a leaked link a permanent, irrevocable compromise: expiry was client-enforced only, and there was no way to distinguish a link that had merely been seen from one that had been used (design-review finding #2 / issue #17). Invites instead get their own, narrower credential:

Create. POST …/invites mints a fresh Garage key scoped to the group's bucket exactly like the shared key (finer per-key ACLs are out of scope), records it with expiresAt = now + INVITE_TTL_SECONDS (default 30 minutes) and usedAt = null, and returns the credential to the owner for embedding in the invite package (fen.md §2.4).
Complete. After the user explicitly accepts the invite in the app (fen.md Flow 4), the joiner calls POST …/invites/:inviteId/complete, signed by their own freshly-generated member keypair (there is no roster to check it against — what authorizes the call is knowledge of the invite itself), immediately after recording their own MemberJoined event locally (event log + outbox — the current v1 client does not perform an S3 write with the invite credential itself; see fen.md §2.4 for exactly what is and isn't implemented). The provisioner rejects the call if the invite is past expiresAt. If unused, it atomically records this memberPubkey as the winner, denies the invite credential (single-use), and returns the group's current shared S3Descriptor. If a retry arrives for an invite already used by the same memberPubkey, the call is idempotent and returns the same shared descriptor again; this handles failures after claim but before the descriptor reached the client. If the invite was used by a different pubkey, the call fails loudly as already used/compromised. This resumable-completion requirement is tracked by issue #69.
Reap. The background reaper (§6.1's same loop) denies and deletes invite credentials once used or past expiresAt, whichever comes first — the used ones are already denied by step 2, so this is final cleanup; the never-used, expired ones are denied here for the first time.

Net effect: an unredeemed leaked invite self-expires within INVITE_TTL_SECONDS; one redeemed by someone other than the intended invitee makes the real invitee's own completion call fail loudly (visible, not silent) instead of quietly handing them a dead credential. Recovery from a compromised redemption is the existing two-phase key rotation (§6.1) — key-epoch rotation and member removal remain out of v1 scope (appendices.md §9.2).

7. Deployment & Zero-Footprint Scaling (v1 Managed Tier)

The managed-backend deployment is specified as code in infra/managed-backend: Garage, the provisioner, and one cloudflared container share a Docker network. The checked-in Cloudflare Tunnel config records the intended ingress: s3.fenapp.net to Garage's S3 API at http://garage:3900 and api.fenapp.net to the provisioner at http://provisioner:8080, with Garage's admin API kept internal at http://garage:3903.

This makes the intended NAS deployment reproducible, but it does not verify that the public hostnames are live. If the cloudflared container is run with only TUNNEL_TOKEN (no --config and no credentials file), the tunnel is remotely managed and this repo's local ingress file has no effect on live routing. Going live in that mode requires the operator to add Public Hostname routes for the existing tunnel in Cloudflare Zero Trust: Networks -> Tunnels -> [tunnel] -> Public Hostname -> Add a public hostname, with api.fenapp.net served by http://provisioner:8080 alongside the existing s3.fenapp.net -> http://garage:3900 route. cloudflared tunnel route dns can create DNS CNAMEs, but it does not create the ingress service mapping for a remotely-managed tunnel. If the deployment migrates to a locally-managed tunnel created with cloudflared tunnel create and a credentials file, the checked-in cloudflared/config.yml becomes the actual ingress source of truth.

Zero-Footprint Dual-Track Deletion: FEN achieves a strict zero-footprint for closed groups.

Managed Tier: The owner calls /v1/groups/:id/close which makes the bucket read-only and triggers a 45-day server-side deletion timer in the Provisioner. As a first line of defense, the owner's app attempts to call DELETE /v1/groups/:id exactly 30 days later. If the client fails or is uninstalled, the Provisioner's daily cron job physically wipes the Garage bucket after 45 days.
BYO Tier (Self-Hosted): Users connecting their own S3 clusters have a local setting: "Purge closed groups from server after 30 days". Since there is no Provisioner, if toggled ON, the app wakes up after 30 days and directly issues S3 DeleteObject commands to wipe the bucket. If OFF, the data stays in their bucket forever.

Orphaned Group Pruning: To prevent abandoned groups from accumulating indefinitely on the Managed Tier, the Provisioner's cron job executes an automated sweep based on S3 metadata: Deep Inactivity Sweep (6 Months): The Provisioner checks the LastModified timestamp of the newest object in the logs/ directory. If no data has been written in 6 months, the bucket is permanently deleted. The client app is responsible for tracking this timeframe and warning the user to archive or send a GroupPing keep-alive event before the 6-month timer expires.

v2 (Bring Your Own S3 Backend): Power users can paste a raw connection string (endpoint, bucket, keys) for any S3-compatible storage backend (Garage, MinIO, AWS S3, etc.). The FEN app connects directly to their bucket using SigV4. They do not run the FEN Provisioner or the FEN backend stack, which is why the client app itself must handle the optional 30-day purge. WebDAV is fundamentally unsupported.

1. The Trust Model​

2. Architecture Components​

3. Storage model — one bucket per group, one key per group​

4. StorageDescriptor​

5. On-bucket layout​

5.1 Friend direct-add relay​

6. Provisioner API (control plane)​

6.1 Two-phase key rotation (credential hand-off)​

6.2 Invite credentials (single-use, short-TTL)​

7. Deployment & Zero-Footprint Scaling (v1 Managed Tier)​