SeaweedFS

SeaweedFS is an open-source storage system that serves an S3-compatible object store, a POSIX file system and Iceberg table buckets from the same data. It is built for many small files: each blob is one disk read away, and capacity grows by starting another volume server. The project wiki owns the full reference; this page covers the architecture, the S3 setup and where it fits.

What it is

One Go binary, weed, that can run as a master, a volume server, a filer, an S3 gateway, a WebDAV server, an Admin UI or all of them at once. The licence is Apache 2.0. The project is led by Chris Lu, who also sells a SeaweedFS Enterprise Edition through seaweedfs.com with data recovery, self-healing storage and configurable erasure coding.

The current line is 4.x, released roughly every one to two weeks; 4.48 shipped on 2026-09-28. Two changes in the 4.x line matter for anyone upgrading from 3.x:

• 4.00 (November 2025) made the official Docker image run as a non-root user (UID/GID 1000). A data directory owned by root stops being writable after the upgrade.
• 4.01 changed the S3 data path. The S3 gateway now reads and writes volume servers directly, and the filer only holds metadata. Before, every byte went through the filer. The release notes cite about 30% lower write latency.

Recent releases also add a Rust volume server, which the README describes as a drop-in for the Go one on the same on-disk format, and S3 Tables with a built-in Iceberg REST catalog.

The design comes from Facebook's Haystack paper (blob store), with erasure coding from f4. It belongs to the same family as Ceph RGW, MinIO and Garage: self-hosted storage that speaks the S3 API.

How it works

Three roles carry the system:

• Master. Tracks which volume lives on which volume server and hands out file ids. It tracks volumes, not files, so a cluster with billions of files has only a few thousand volumes to track. One master is enough for most clusters; three form a Raft group for failover. The master is not in the read path.
• Volume server. Stores blobs ("needles") in append-only volume files and keeps a 16-byte in-memory index entry per blob, so a read is one seek. Large files are split into chunks of a few MB, each its own blob. Replication and erasure coding work at the volume level.
• Filer. Adds directories and file names on top of blobs. Its metadata lives in a pluggable store: LevelDB by default, or PostgreSQL, MySQL, Redis, etcd, Cassandra and others.

The S3 gateway is stateless. It maps buckets to filer directories (each bucket is its own collection, so a bucket delete is instant), checks the caller's identity and actions, and since 4.01 moves the object bytes straight to and from volume servers.

Space is reclaimed later. A delete marks the needle as deleted; vacuum (compaction) rewrites a volume without its deleted needles once garbage passes a threshold. Balancing, erasure coding of warm volumes and repair also run as separate jobs from weed shell or the maintenance worker. Nothing rebalances by itself when a server is added.

When to use it

Reach for it when:

• A cluster or homelab needs a self-hosted S3 endpoint for backups: Velero, CloudNativePG/Barman, K3s etcd snapshots, restic or rclone.
• The workload is many small files, where one-seek reads and low per-file overhead matter.
• One process on one node is enough today, but the option to add volume servers later should stay open.
• Per-bucket credentials are needed without running a separate IAM service.
• The same data should also be reachable as a FUSE mount, WebDAV or Iceberg tables.

Skip it when:

• A managed object store (S3, R2, B2) is acceptable; there is nothing to operate.
• The need is block storage for databases or VM disks. Ceph RBD or a CSI block driver fits that, not an object store.
• The team cannot schedule upgrades every few weeks. The release cadence is fast and fixes land in new versions, not old branches.
• Strict, audited parity with every AWS S3 edge case is a hard requirement. Test the exact client first.

At a glance

Core concepts

• Volume — a large append-only file (30 GB by default) that holds many needles. The unit of replication, erasure coding and vacuum.
• Needle — one blob inside a volume, addressed by a file id like 3,01637037d6.
• Collection — a group of volumes. Each S3 bucket is a collection.
• Replication placement — three digits xyz: extra copies in other data centers, other racks and other servers. 000 is one copy; 001 adds one copy on another server.
• Filer store — the database that holds directory metadata.
• Identity — an S3 user with one or more key pairs and a list of actions, global (Read) or bucket-scoped (Read:velero).

Key facts

Item	Value
Licence	Apache 2.0 (Enterprise Edition sold separately)
Current release	4.48 (2026-09-28)
Default ports	Master 9333, volume 8080, filer 8888, S3 8333; gRPC is HTTP port + 10000
Docker user	UID/GID 1000 since 4.00
Install paths	Single binary, Docker image chrislusf/seaweedfs, Helm chart, operator, CSI driver
S3 auth sources	Config file (-s3.config), filer-stored config, environment variables, in that priority

SeaweedFS vs MinIO vs Ceph RGW

Aspect	SeaweedFS	MinIO	Ceph RGW
Licence	Apache 2.0	AGPLv3	LGPL
Project state	Active, releases every few weeks	GitHub repository archived	Active
Minimum footprint	One process, one directory	One process, one directory	Monitors, OSDs and a gateway
Scale-out unit	Add one volume server	Add a whole server pool	Add OSDs
Small-file reads	One disk seek per blob	Per-object metadata file on each drive of the erasure set	Through RADOS
Data protection	Replication on hot data, erasure coding on warm data	Erasure coding on all data	Replication or erasure coding per pool

How to integrate

Single node, all roles in one process (master, volume server, filer and S3 gateway):

weed server -dir=/data -s3 -s3.config=/etc/seaweedfs/s3.json

Per-bucket identities in s3.json. Each identity sees only its own buckets:

{
  "identities": [
    {
      "name": "admin",
      "credentials": [{"accessKey": "ADMIN_KEY", "secretKey": "ADMIN_SECRET"}],
      "actions": ["Admin", "Read", "Write", "List", "Tagging"]
    },
    {
      "name": "cluster-a",
      "credentials": [{"accessKey": "CLUSTER_A_KEY", "secretKey": "CLUSTER_A_SECRET"}],
      "actions": ["Read:velero", "Write:velero", "List:velero"]
    }
  ]
}

After an edit, send SIGHUP to the process (or restart it); the file is not reloaded automatically.

Kubernetes with the official Helm chart:

helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm
helm install seaweedfs seaweedfs/seaweedfs -n seaweedfs --create-namespace -f values.yaml

master:
  replicas: 1
volume:
  replicas: 1
  dataDirs:
    - name: data
      type: persistentVolumeClaim
      size: 200Gi
      maxVolumes: 0        # size the volume count from the disk
filer:
  replicas: 1
s3:
  enabled: true
  enableAuth: true
  credentials:
    admin:
      accessKey: admin
      secretKey: change-me
  createBuckets:
    - name: velero

Point a client at it. Most S3 clients need a custom endpoint and path-style addressing:

aws --endpoint-url http://seaweedfs-s3.seaweedfs:8333 s3 ls s3://velero/

In the GL stack

SeaweedFS 4.48 is the S3 backup store for every cluster behind builddaily.io, paiddaily.io and the other products.

• Where it runs. It runs on the management cluster as a single weed server -s3 process, so the master, volume server, filer and S3 gateway share one process.
• Where the data lives. Its data directory is a folder on the Mac, mounted into the Lima VM. If the VM is lost or rebuilt, the backups are still on the host disk.
• Buckets. There is one bucket per backup source: postgres, neo4j, velero and etcd.
• Access. Each cluster has its own S3 identity, limited to the buckets that cluster writes. A compromised cluster cannot read or delete another cluster's backups.
• Why SeaweedFS. It replaced MinIO as the backup target because MinIO's community edition is archived and gets no more fixes.
• Retention checks. A nightly backup-retention job fails if the newest backup in a set is older than 36 hours. For the small sets, it also keeps one copy per month for six months.

Gotchas

• Non-root image since 4.00. The container runs as UID/GID 1000. A host-mounted or pre-existing data directory owned by root gives permission errors after the upgrade. Change the owner or set the pod's fsGroup.
• The config file is not live-reloaded. Changes to -s3.config take effect only after SIGHUP or a restart.
• Config sources replace each other. A config file, filer-stored config and environment variables are not merged. A higher-priority source replaces the lower one completely, so adding a config file drops identities created through the Admin UI.
• Environment-variable credentials are global admin. AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY create one unscoped admin identity and show in process lists. Use a config file for anything shared.
• Volume count caps capacity. weed server defaults to 8 volumes. Set -volume.max (or maxVolumes: 0 in the chart) so the volume count matches the disk, or writes fail with no writable volumes while the disk still has space.
• Deletes do not free disk at once. Space returns only after vacuum compacts a volume. Retention jobs that delete a lot still need vacuum to run before the disk shrinks.
• One node means one copy. Replication 000 on a single volume server has no redundancy. Protect the data directory itself, or copy critical buckets off the machine.
• The Admin UI binds to loopback without auth. weed mini keeps the Admin UI on loopback unless WEED_ADMIN_PASSWORD or mTLS is set. -admin.allowInsecureBind removes that guard and belongs only on an isolated network.

Risks

• Maintainer concentration. A large share of core commits come from the project lead, and an Enterprise Edition funds the work. The code is Apache 2.0 and forkable, but the roadmap depends on a small group.
• Fast release cadence. New versions ship every week or two, and fixes go into the newest version. Pin a version, read the release notes, and upgrade often.
• Low lock-in. Clients speak plain S3, so moving to another S3 store is a bucket copy with rclone or aws s3 sync. The on-disk volume format is SeaweedFS-specific, so recovery without SeaweedFS is not practical.
• Security surface. Recent releases include many S3 authorization fixes (object-lock permissions, policy evaluation, STS session handling). Keep the S3 port inside the cluster network and keep up with patches.

Alternatives

• MinIO — the common S3 choice for years; the community repository is now archived, so it fits only existing installs or a paid commercial edition.
• Garage — AGPLv3, built for small geo-distributed clusters on modest hardware; pick it when nodes sit in different places and a simple replication model is the goal.
• Ceph RGW (with Rook) — pick it when object, block and file storage are needed from one cluster and the team can run Ceph.
• RustFS — an Apache 2.0 reimplementation of MinIO's storage model; pick it for MinIO-compatible behaviour from an active project.

Related

• Velero, CloudNativePG, Neo4j and K3s — the clients that write their backups and etcd snapshots to SeaweedFS.
• Lima — the VM layer that mounts the host folder holding the data directory.
• MinIO — the S3 store SeaweedFS replaced.
• Backup layer and Data layer — how storage and backups fit together across the clusters.
• Official: SeaweedFS wiki, releases, S3 credentials, Amazon S3 API.