Blob uploaders: a write-only credential for the blob bucket

The founder's item 1 of 2026-09-21 (platform/STATE.md, 03:40 UTC): a scoped, long-lived, revocable credential that can only drop files into a catalog's blob bucket <handle>--<catalog>--blobs (b-<uuid> on catalogs created before 2026-09-20; the API's bucket field and the recipe's blob_bucket spell it, this document writes <blobs> for it). For a relay, a camera, a field device, a partner that sends you files, a CI job that publishes artefacts: anything that must write and must never read, list what others wrote, or come near the tables and the catalog.

tablemere catalog uploader create --catalog garden --name camera-north --prefix photos/north/ --save ~/camera-north.json
tablemere catalog uploader list   --catalog garden
tablemere catalog uploader revoke --catalog garden --uploader camera-north

tablemere connect says so in its notes (uploaders_note in GET /v1/connection): the catalog's own credential is for engines and for you, never for a device.

What the credential is, exactly

One IAM identity in the store, u-<warehouse_uuid>-<8 hex of the uploader id>, with one inline policy (PutUserPolicy, name uploader) that the control plane writes once and never re-puts:

{"Version": "2012-10-17", "Statement": [
  {"Sid": "PutOnly", "Effect": "Allow", "Action": ["s3:PutObject"],
   "Resource": ["arn:aws:s3:::<blobs>/photos/north/*"]},
  {"Sid": "ListOwnPrefix", "Effect": "Allow", "Action": ["s3:ListBucket"],       <- only with list: true
   "Resource": ["arn:aws:s3:::<blobs>"],
   "Condition": {"StringLike": {"s3:prefix": ["photos/north/*"]}}}
]}

No prefix (prefix: "") means the whole blob bucket; list: true without a prefix lists the whole blob bucket (no Condition), and the answer's scope says which. A prefix without a trailing slash confines to keys that start with it: photos covers photos.jpg and photos/a.jpg both. The prefix grammar is what S3 keys allow minus what the policy engine would read as a pattern: letters, digits, . _ - / = @ :; no leading slash, no empty or dot segments, no * ? $.

expires_at is recorded and shown (status: expired after it); the key itself stays valid until revoked. Revocation is the only thing that stops a holder today.

What was read in the store's source before this was built

SeaweedFS 4.47, branch tablemere/4.47, weed/s3api (cited in app.py's uploader block; every line below is a claim the live matrix at the end checks):

• PutUserPolicy stores the inline policy AND recomputes the identity's legacy Actions (a lossy projection: Write:b-<uuid>/photos/north/*, List:b-<uuid>, Conditions dropped). The S3 server, though, hydrates inline policies into identity.PolicyNames on every configuration reload (hydrateRuntimePolicies, called from the credential-manager load path that the propagation after each IAM write triggers), and VerifyActionPermission then routes to authorizeViaAttachedPolicies: the policy engine is authoritative and the legacy Actions are not consulted. That is what makes a fine-grained, prefix-conditioned policy hold.
• The engine matches fine-grained actions: s3:PutObject does not imply s3:GetObject (write-only is real).
• policy_engine/types.go multipartActionSet: s3:PutObject implicitly covers CreateMultipartUpload, UploadPart, UploadPartCopy, CompleteMultipartUpload, AbortMultipartUpload, ListParts and ListMultipartUploads. The routes for all of them carry ACTION_WRITE (s3api_server.go; the bucket-level ListMultipartUploads is ACTION_READ). s3:ListMultipartUploadParts is still "not a valid action" for GetActions, the PutUserPolicy validator (CLAUDE.md), so it must not appear in a policy; it is not needed.
• s3:ListBucket is evaluated on the bucket ARN with s3:prefix taken from the request's ?prefix= (policy_engine.ExtractConditionValuesFromRequest); a listing without ?prefix= has no context value and StringLike returns false. The device must ask for its own prefix.

The IAM cost, stated

Every IAM write costs the store a propagation deadline (10 s at the host's identity count on 2026-09-20, about 0.1–0.3 s after the fixture purge / patch #4; STATE.md 21:10 UTC). Designed for the fewest writes:

The unit tests count them through the fake's call log (control_plane/tests/test_uploaders.py). Each uploader is one more identity in the store, so make one per device or partner, not one per file.

Consuming what was uploaded

The uploader cannot read; you do, from the catalog side, with the catalog's own credential (tablemere connect, blob_bucket and blob_example_python in GET /v1/connection):

-- DuckDB, with the S3 secret of the connect recipe (tablemere_s3)
SELECT filename, size, content
FROM read_blob('s3://<blobs>/photos/north/*.jpg');
-- Parquet dropped by a partner, straight into a table of the catalog
INSERT INTO garden.sensors SELECT * FROM read_parquet('s3://<blobs>/partner-acme/2026-09/*.parquet');

Index files in a table (the Garden Lake pattern: photos in the bucket, rows in garden.images) and hand download links out from your side with generate_presigned_url('get_object', …); the uploader has no GetObject to presign with. Deleting or moving what was uploaded is yours too.

Prefix design

• One prefix per device or partner (photos/north/, partner-acme/), so revoking one revokes one.
• Put the date in the key, not in the prefix, unless you want the credential to expire with the month (--prefix drops/2026-09/ is a cheap time box).
• A prefix that engines can glob: read_parquet('s3://<blobs>/partner-acme/*/*.parquet').
• The blob bucket counts against the catalog's storage tier like the table bucket does (/v1/usage).

Expiring what uploaders drop: lifecycle rules (added 2026-09-21)

The catalog credential at write level can put S3 lifecycle rules on the blob bucket — expire camera/ after 90 days, abort multipart uploads not completed in 7 days — and they apply to everything under the prefix, what uploaders wrote included. The uploader itself cannot set or read them (measured 403 on ?lifecycle while its PutObject was 200), nor can the read-only identity. tablemere catalog lifecycle set <catalog> --prefix photos/north/ --expire-days 90; the API, the boto3 form, when deletion happens (up to 24 h after an object is due) and why table buckets never carry rules: platform/docs/lifecycle.md.

Revocation and rotation

DELETE /v1/warehouses/{id}/uploaders/{id|name}: the key is deleted at the store before anything else, so a request signed with it is refused at once and a multipart upload in flight cannot complete. There is no rotate: a lost or leaked secret is revoke + create (the name is free again; 6 IAM writes in all). Deleting the catalog removes every uploader identity with it. uploader.create and uploader.revoke are audited (name, prefix, list, identity, key id, the writes; never the secret).

Routes

POST   /v1/warehouses/{id}/uploaders                 {name, prefix?, list?, expires_at?}   write level; 201; the secret once
GET    /v1/warehouses/{id}/uploaders?include_revoked=  read level; never a secret
DELETE /v1/warehouses/{id}/uploaders/{uploader_id|name}                                   write level; the key first

CLI: tablemere catalog uploader create|list|revoke. create refuses to run without --save FILE (0600, nothing printed) or --show-key (printed once), so no identity is ever created whose secret nobody received; with --json, --save masks the secret everywhere in the answer, recipes included.

Live matrix, measured 2026-09-21 (real output; both runs 30 probes)

Two runs of the same matrix with a stdlib SigV4 signer (the CLI's doctor signer plus a payload hash):

1. The routes, end to end, against a real SeaweedFS 4.47 (this worktree's control plane on the laptop, talking to the local stack's al-weed through the gateway): POST /v1/warehouses 41.4 s (the 4 tenant IAM writes), POST …/uploaders 201 in 20.6 s twice (iam_writes: [CreateUser, CreateAccessKey, PutUserPolicy]), GET …/uploaders 200 with no secret in the body, DELETE …/uploaders/{id} 200 in 30.7 s (storage: {access_key: ok, user_policy: ok, user: ok}), DELETE /v1/warehouses/{id} 202, tombstone with uploader_<short>_key|policy|user: ok for both uploaders.
2. The store on al-lake1 (https://s3.tablemere.com, https://catalog.tablemere.com): the exact IAM / S3Tables calls the route makes, as the admin, from the host, with a fresh t-, two u- identities and a fresh table bucket; everything removed afterwards. The control plane build with the route was not deployed for this run (another session owns deploys; the branch is not pushed). IAM timings on the host: CreateUser 0.27 s, CreateAccessKey 10.3 s, PutUserPolicy 10.4 s, every Delete* 10.3 s (the propagation deadline of STATE 21:10 UTC), and 20–31 s per write in the second run while another writer held the IAM lock.

Status codes as answered, identical in both runs (relay = prefix photos/, lister = the same with list: true, tenant = t-<uuid>, the positive control with the same key shape):

Two observations from the runs that are not about the uploader: (a) a 4-byte PAR1 stand-in for a parquet file was answered 500 by the host's table bucket (200 by the local one), so the positive control uses a real parquet written by DuckDB; (b) DeleteTableBucket answered 409 BucketNotEmpty for 20–30 s after the last object under x/y/data/ was deleted (204), then 200: the emptiness check lags object deletion, which is also why a tombstone can record table_bucket: HTTP 409 until the startup sweep retries.