Skip to content
Back to skills

Blobstore

ASecurity

File uploads and downloads (images, documents, attachments) — declare a forge.Bucket per env, hand a workload its connection with forge.bucket_env, presign an upload in a handler with forge/pkg/blobstore, PUT it straight from the browser, confirm it with Stat. Dev runs MinIO; Reliant hosting provisions the bucket (forge.OnHosted); anywhere else use any S3-compatible bucket (S3, R2, GCS).

  • 10 stars
  • 0 votes
  • 0 copies
  • 1 view
  • Added November 13, 2026
developmentrustgodockerkubernetesawsapidatabasefrontendbackend

Works with

  • cli
  • api

Security analysis

A100/100

Scanned November 13, 2026

npx -y skills add reliant-labs/forge --skill blobstore --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Blobstore?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Blobstore
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/reliant-labs-blobstore/badge)](https://www.skillsdirectory.com/skills/reliant-labs-blobstore)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: storage/blobstore
description: File uploads and downloads (images, documents, attachments) — declare a forge.Bucket per env, hand a workload its connection with forge.bucket_env, presign an upload in a handler with forge/pkg/blobstore, PUT it straight from the browser, confirm it with Stat. Dev runs MinIO; Reliant hosting provisions the bucket (forge.OnHosted); anywhere else use any S3-compatible bucket (S3, R2, GCS).
---

# File uploads: forge.Bucket + forge/pkg/blobstore

The bytes of an upload never pass through your API. The browser asks the API
for a **presigned URL**, PUTs the file straight to object storage with it, and
tells the API it is done; the API **confirms what landed** and records it.
Downloads are presigned GET URLs the same way. One protocol everywhere: the
S3 API, which MinIO speaks in dev and AWS S3, Cloudflare R2 and Google Cloud
Storage (through its S3 interoperability, with an HMAC key) speak in prod.

```
browser ──CreateUpload(name, type, size)──▶ API ──PresignPut──▶ {PUT url, headers}
browser ──PUT bytes (exact size + type)──▶ bucket
browser ──CompleteUpload(id)──▶ API ──Stat──▶ size/type match? record the row
browser ──GET presigned url──▶ bucket          (API: PresignGet)
```

## 1. Declare the bucket, per env

Buckets are a `Bundle.buckets` entry, like a `forge.ManagedDatabase`: the
**env** says where each one lives. The scaffolded `deploy/kcl/dev/main.k`
carries the dev half switched off; turn it on and name the bucket:

```kcl
_object_storage = True
_minio = forge.BucketOnMinIO {
    port = plugin.resolve_port("acme-dev-minio", 9000)
} if _object_storage else None
_buckets: [forge.Bucket] = [forge.Bucket {name = "uploads", runtime = _minio}]
```

`forge env up dev` then starts docker-compose.yml's `minio` service (the
`forge.minio_workload` the template adds), creates every declared bucket, and
reports it healthy only once they exist. The data persists in the
`minio_data` volume. A project scaffolded before this existed has no `minio`
service: copy it, and its `minio_data` volume, from a fresh
`forge project new` scaffold's docker-compose.yml.

`forge.Bucket` fields:

| field | meaning |
|---|---|
| `name` | how `forge.BucketRef` names it, and the real bucket name unless the runtime says otherwise (3..63 of `a-z0-9-`) |
| `public_read` | every object readable without a signature (avatars, a public gallery). Off for anything private: hand out `PresignGet` URLs |
| `cors_origins` | the browser origins that upload directly. The dev MinIO applies them (none declared = any origin). On `forge.OnHosted` the platform adds the env's own hosted frontends and custom domains, so list only OTHER origins. On `forge.BucketOnS3` forge does not manage the bucket: set its CORS where it lives, allowing `PUT` and `GET` and the `Content-Type` header |
| `deletion_policy` | `"retain"` (default) or `"delete"`: what removing the declaration does to a bucket forge manages (`forge.OnHosted`). Retain leaves the bucket and every object; delete is irreversible and only happens because someone typed it |
| `runtime` | where it lives — required, no default |

Runtimes:

- **`forge.BucketOnMinIO {port}`**: the env's MinIO (dev, e2e). Credentials
  are the dev MinIO's own (`minioadmin`), committed on purpose like the dev
  postgres password: it binds loopback.
- **`forge.BucketOnS3 {...}`**: a bucket you operate or buy. forge renders
  the connection and never creates, configures or deletes the bucket.

  ```kcl
  forge.Bucket {name = "uploads", runtime = forge.BucketOnS3 {
      endpoint = "https://<account>.r2.cloudflarestorage.com"   # S3: https://s3.<region>.amazonaws.com
      region = "auto"                                            # R2 "auto"; GCS "us-east-1" works
      bucket = "acme-prod-uploads"                               # the real name; default: forge.Bucket.name
      access_key_id = forge.ManagedSecret {name = "UPLOADS_ACCESS_KEY_ID"}
      secret_access_key = forge.ManagedSecret {name = "UPLOADS_SECRET_ACCESS_KEY"}
  }}
  ```

  The two credentials are references, never literals: a `forge.ManagedSecret`
  (the env's secret store, `forge secret set --env prod UPLOADS_SECRET_ACCESS_KEY`
  with the value on stdin; what a hosted workload reads) or a `forge.SecretRef`
  (a Kubernetes Secret on a cluster you operate). GCS: endpoint
  `https://storage.googleapis.com` and an HMAC key of a service account that
  holds only `roles/storage.objectAdmin` on that one bucket.
- **`forge.OnHosted {}`**: Reliant hosting provisions it. See "On Reliant
  hosting" below.

### On Reliant hosting

```kcl
# deploy/kcl/prod/main.k
_buckets: [forge.Bucket] = [forge.Bucket {name = "uploads", runtime = forge.OnHosted {}}]
```

The env publishes the bucket to the control plane as a `forge.dev/v1alpha1
ManagedBucket`, next to its Workloads, and the platform:

- creates one cloud bucket for this env (Google Cloud Storage; its real name
  is the platform's, `UPLOADS_BUCKET` carries it), with uniform access and
  public access prevented unless `public_read`;
- creates a service account that can read, write, list and delete objects in
  **that bucket only**, and an HMAC key for it;
- writes the Secret `forge-bucket-uploads` in the env's namespace, one key per
  `BucketRef.key`: `endpoint` and `public_endpoint` are
  `https://storage.googleapis.com`, `region` is `auto`, `path_style` is
  `true`, and the two credentials are the HMAC key;
- sets the bucket's CORS (`PUT`, `GET`, `HEAD`; `Content-Type`) to your
  `cors_origins` plus the env's hosted frontend origins and bound custom
  domains.

A hosted workload's `forge.bucket_env("UPLOADS", "uploads")` then lowers to
`bucketRef` env values, which become `secretKeyRef`s on that Secret. The pod
does not start until the Secret exists, so a deploy that reports the workload
ready also proved the bucket. Nothing changes in your code: the same seven
variables fill `blobstore.S3Config`. The browser talks to GCS directly, so
uploads never touch the cluster.

What is refused, at render:

- a workload on a cluster or host reading a hosted bucket (its credential
  exists only in the control plane's namespace): bind the bucket to
  `forge.BucketOnS3` in that env, or run the workload on `forge.OnHosted`;
- a hosted bucket name over 40 characters (the platform prefixes it);
- a hosted bucket in a Bundle with no `control_plane`.

Removing the declaration leaves the bucket and its objects in place unless
`deletion_policy = "delete"`.

#### Verify a hosted bucket

`forge env deploy` publishes the bucket but cannot observe it: the control
plane reports no status for a ManagedBucket, so the deploy lists it as
`not_waited` (in `--json` too), never as ready. A workload that reads it
turning ready proves only that the credential Secret exists. A bucket no
workload reads (a bucket-only env) is proved by nothing. After the first
deploy of a hosted bucket, and after any change to its `public_read` or
`cors_origins`, check it from the deployed app, through the same handlers your
users hit:

1. **Upload**: from a page served on one of the bucket's origins, presign a
   PUT (section 4) and send it from the browser (section 5). A CORS error in
   the browser console means the origin is missing from `cors_origins` or the
   platform has not applied it yet.
2. **Confirm and download**: `Stat` the key (section 6), then fetch a
   presigned GET and compare the bytes.
3. **Private stays private**: fetch the object's plain URL,
   `https://storage.googleapis.com/<UPLOADS_BUCKET>/<key>`, with no signature.
   It must be refused (403) unless the bucket is `public_read`, and must
   succeed when it is.

For a bucket-only env, run the check once a workload that reads the bucket is
deployed; until then nothing has used the credential.

**Recovering** from a bad bucket change rolls forward, as every `forge env deploy`
does (`deploy/release`, "Verify + recover"): restore the previous declaration (or
drop the bucket's referrers) in the tree and `forge env deploy <env>` again;
binding the env to an older release is the same promote. Either keeps the
bucket, its objects and its credential: a bucket is never release-bound, and
the default `deletion_policy = "retain"` leaves it in place even when the
declaration is removed. Never set `deletion_policy = "delete"` to
undo a change; it destroys every object. Do not delete the `forge-bucket-<name>`
Secret to "reset" a credential: the platform owns it, and the workloads that
read it stop starting until it is rewritten.

Every env a workload runs in must declare each bucket the workload references;
the render names a missing one.

## 2. Hand the workload its connection

In `deploy/kcl/workloads.k`, on the workload that serves the upload RPCs:

```kcl
env = forge.bucket_env("UPLOADS", "uploads")
```

That is seven `forge.BucketRef`s: `UPLOADS_ENDPOINT`, `UPLOADS_PUBLIC_ENDPOINT`,
`UPLOADS_REGION`, `UPLOADS_BUCKET`, `UPLOADS_ACCESS_KEY_ID`,
`UPLOADS_SECRET_ACCESS_KEY`, `UPLOADS_PATH_STYLE`, which are the fields of
`blobstore.S3Config`. Each lowers per env and per runtime. For a dev MinIO
bucket they are literals, and the endpoint is `localhost` for a host process,
the cluster's host gateway for a k3d pod, and `http://minio:9000` (the compose
service, on the compose network) for a container in the MinIO's own compose
file, whose compose process env carries them for the file to pass through
(`UPLOADS_ENDPOINT: ${UPLOADS_ENDPOINT}`). `PUBLIC_ENDPOINT` stays
`localhost`, because the browser is on this machine and the presigned URL has
to be signed for the host the browser actually dials. For a `BucketOnS3`
bucket you get the declared values, and the credentials go through their own
secret channels. One fact on its own is `forge.BucketRef {name = "uploads",
key = "bucket"}`.

Read them through the typed config, so nothing calls `os.Getenv`: a block in
`proto/<project>/config/v1/config.proto`, composed on `AppConfig`, then
`forge generate`:

```protobuf
message UploadsConfig {
  string endpoint = 1 [(forge.v1.config) = {env_var: "UPLOADS_ENDPOINT"}];
  string public_endpoint = 2 [(forge.v1.config) = {env_var: "UPLOADS_PUBLIC_ENDPOINT"}];
  string region = 3 [(forge.v1.config) = {env_var: "UPLOADS_REGION"}];
  string bucket = 4 [(forge.v1.config) = {env_var: "UPLOADS_BUCKET"}];
  string access_key_id = 5 [(forge.v1.config) = {env_var: "UPLOADS_ACCESS_KEY_ID"}];
  string secret_access_key = 6 [(forge.v1.config) = {env_var: "UPLOADS_SECRET_ACCESS_KEY", sensitive: true}];
  bool path_style = 7 [(forge.v1.config) = {env_var: "UPLOADS_PATH_STYLE"}];
}
```

## 3. Build the store once, in the composition root

`internal/app/compose.go` builds the store from that config block and passes
it to the service as an interface:

```go
uploads, err := blobstore.NewS3(blobstore.S3Config{
    Endpoint:        cfg.GetUploads().GetEndpoint(),
    PublicEndpoint:  cfg.GetUploads().GetPublicEndpoint(),
    Region:          cfg.GetUploads().GetRegion(),
    Bucket:          cfg.GetUploads().GetBucket(),
    AccessKeyID:     cfg.GetUploads().GetAccessKeyId(),
    SecretAccessKey: cfg.GetUploads().GetSecretAccessKey(),
    PathStyle:       cfg.GetUploads().GetPathStyle(),
})
```

`NewS3` makes no network call. A wrong key or a missing bucket shows up on the
first `Stat`, `List` or `Delete`, or as a refused upload. Every call gets a
span, the `blobstore.{calls,errors,duration}` metrics and a log record, and
every request to the storage service is a traced client span. A missing object
(`ErrNotFound`) is logged as an expected outcome, not as an error.

The consumer depends on the 5-method interface `blobstore.Store`, or on a
narrower one it declares itself, and never on `*blobstore.S3`.

## 4. Presign the upload in the handler

**Keys are yours, and they are tenant-scoped.** Generate the key on the
server, never from the user's file name: `<tenant>/<kind>/<uuid>`. Wrap the
store in `blobstore.Scoped(store, tenantID)` per request, so the handler
cannot name another tenant's object whatever key it is handed: keys with a
leading `/`, empty segments, `.` or `..` are refused (`ErrInvalidKey`).

```go
func (s *service) CreateUpload(ctx context.Context, in CreateUploadInput) (CreateUploadResult, error) {
    if !allowedTypes[in.ContentType] {
        return CreateUploadResult{}, svcerr.InvalidArgument("unsupported file type")
    }
    store, err := blobstore.Scoped(s.deps.Uploads, in.TenantID)
    if err != nil {
        return CreateUploadResult{}, err
    }
    key := "attachments/" + uuid.NewString()
    req, err := store.PresignPut(ctx, key, blobstore.PutOptions{
        ContentType:   in.ContentType,
        ContentLength: in.Size,     // exact size; signed
        MaxBytes:      25 << 20,    // your limit; ErrTooLarge above it
        TTL:           10 * time.Minute,
    })
    if err != nil {
        return CreateUploadResult{}, err // svcerr-classified: InvalidArgument for a bad request
    }
    // Record a pending row (tenant, key, declared type and size) before you return.
    return CreateUploadResult{Key: key, URL: req.URL, Method: req.Method, Headers: req.Headers}, nil
}
```

**Why the client states the size up front.** The presigned PUT signs the
exact `Content-Length` and the `Content-Type`, so the storage service rejects
any body of a different size or type with a 403. That bounds an upload on
every S3-compatible service. A presigned POST with `content-length-range`
could bound a maximum instead, but Cloudflare R2 doesn't implement it.

## 5. Upload from the browser

```ts
const { key, url, method, headers } = await client.createUpload({
  name: file.name,
  contentType: file.type,
  size: BigInt(file.size),
});
const res = await fetch(url, { method, headers, body: file }); // the browser sets Content-Length from the body
if (!res.ok) throw new Error(`upload failed: ${res.status}`);
await client.completeUpload({ key });
```

Send the `headers` exactly as you got them, and the same `File` whose size you
declared. A 403 means the size, the type or the signature did not match: an
edited file, a wrong `Content-Type`, or an expired URL.

## 6. Confirm, then serve

```go
obj, err := store.Stat(ctx, key)
switch {
case errors.Is(err, blobstore.ErrNotFound):
    return svcerr.FailedPrecondition("the upload has not arrived")
case err != nil:
    return err
case obj.Size != pending.Size || obj.ContentType != pending.ContentType:
    _ = store.Delete(ctx, key)
    return svcerr.InvalidArgument("the upload does not match what was declared")
}
// mark the row complete
```

Downloads: `store.PresignGet(ctx, key, blobstore.GetOptions{TTL: 5 * time.Minute, DownloadName: row.FileName})`.
`DownloadName` makes it an attachment saved under that name. `List(ctx,
prefix, ListOptions{After, Limit})` pages a prefix in key order (1000 max per
page), and `Delete` is idempotent. Clean up pending rows whose upload never
completed with a worker: list or delete their keys.

## 7. Test without a bucket

`blobstore.NewMemory()` is a `Store` with the S3 backend's key and option
validation. Its presigned URLs are `memory://` URLs nothing fetches. Play the
browser with `mem.Put(key, data, contentType)` and assert with `Stat`, `List`
or `mem.Get`. forge's own S3 backend is tested against a real MinIO; your
service tests need only the fake.

## Rules

- The API never proxies upload bytes. Presign and confirm.
- Keys are server-generated and tenant-scoped (`Scoped`). A user's file name
  goes in your row, and in `DownloadName`, never in the key.
- Always `Stat` before you trust an upload, and compare it with what you
  presigned.
- Private by default: `public_read` only for content that is public on
  purpose.
- Credentials are references (`ManagedSecret` / `SecretRef`), never literals,
  outside the dev MinIO.

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in development →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.