Photos, videos, documents and backups are blobs (binary large objects): big, immutable once written, and read by key. They don't belong in a relational database. Object storage (Amazon S3, Google Cloud Storage, Azure Blob) stores them cheaply and durably, a CDN serves them, and a processing pipeline turns uploads into the sizes and formats clients need.
In simple words
The database is the library catalogue card; object storage is the warehouse shelf with the actual book. The card says where the book is. A processing pipeline is the bindery that makes the pocket and hardcover editions after a new book arrives.
Big bytes skip your servers: clients upload straight to storage, and processing happens asynchronously.
Deep Dive: Object Storage in One Picture
bucket: user-media
key: photos/2026/09/29/u123/ab12cd.jpg -> bytes + metadata (type, size, etag)
- flat namespace of keys (the "/" is just part of the name)
- PUT / GET / DELETE whole objects (or ranges); no in-place edits
- very high durability via replication/erasure coding across devices and zones
- storage classes: hot (standard) -> infrequent access -> archive (cheap, slow)
- lifecycle rules move/delete objects by age
The database keeps only metadata (owner, object key, size, dimensions, status); the object store keeps the bytes.
Deep Dive: Upload Flow with Pre-signed URLs
1. client -> API: POST /uploads {filename, size, type}
2. API: validate, create media row (status=PENDING),
return a pre-signed PUT URL (valid ~15 min, for one exact key)
3. client -> object store: PUT bytes directly (multipart for large files,
resumable: retry only failed parts)
4. object store -> event (e.g. "object created") -> queue
5. workers: virus scan, thumbnails / transcoding, extract metadata
6. media row -> READY with variant keys; client polls or gets notified
Pre-signed URLs keep gigabytes of upload traffic off your API servers, which then only handle small JSON requests.
Deep Dive: The Processing Pipeline
Images: generate fixed sizes (thumbnail, medium, large) and modern formats (WebP/AVIF); strip EXIF location data.
Video: split into chunks, transcode each chunk in parallel into several bitrates/resolutions, package for adaptive streaming (HLS/DASH). See video streaming.
Documents: extract text for search, generate previews.
Workers are stateless and idempotent (the same object key processed twice produces the same outputs), scaled by queue depth.
Key Pointers
Blobs in object storage, metadata in a database, delivery through a CDN.
Clients upload directly with pre-signed URLs; multipart for large files.
Processing is asynchronous and event-driven; track status in the metadata row.
Use content-addressed keys (hash of content) to deduplicate identical files.
Lifecycle policies cut cost: old or rarely read media move to cheaper tiers.
Follow-up Questions
Why not store images in the database as BLOB columns?
It bloats the database (backups, replication, buffer cache all carry the bytes), costs far more per GB, and serving them makes the database do a file server's job. Object storage plus CDN is cheaper and scales independently.
How do you keep private files private while still using a CDN?
Keep the bucket private and serve through signed URLs or signed cookies with short expiry, generated by your API after checking permissions.
The upload finished but processing failed. What does the user see?
The media row stays PROCESSING/FAILED; the job retries with backoff and lands in a dead-letter queue after N failures, and the UI shows a retry option rather than a broken image.
Trade-offs
Object storage trades rich queries and in-place updates for cost, durability and scale. Pre-generating many variants costs storage but makes reads fast; generating on demand (and caching at the CDN) saves storage but adds first-request latency.