· Part 2/6 · Choosing AWS services under constraint
Storage is chosen by access pattern, not by durability
Every managed storage service quotes durability that is effectively identical. The decision is made somewhere else entirely — by how the data is reached, and by who can prove who reached it.
AWS’s storage decision guide covers thirteen services. Read the durability column and you learn nothing: the numbers are all extravagant and all effectively the same. Nobody has ever chosen between S3 and EFS on durability, and no comparison table admits it.
The decision is made on two things instead. How does code reach the bytes, and can you say afterwards who reached them.
The first one is an API question wearing a costume
There is a real, permanent divide running through that list of thirteen: object storage and filesystems are different things, and the adapters that make one look like the other are the source of a specific kind of production incident.
Object storage gives you a key and a blob. Writes are whole-object. There is no partial update, no rename that is atomic in the way a filesystem’s is, and no lock. A filesystem gives you all of those, and code written against one assumes them without ever saying so.
The failure mode is not “it does not work”. It is that it works for months. A document pipeline mounts a bucket through a filesystem adapter, writes a temporary file, renames it into place, and reads it back — and this is fine until two workers do it at the same moment on the day volume doubles, and one of them reads a half-written object that a real filesystem would never have shown them.
If the constraint is existing code opens files by path, that is not a preference to be negotiated. It eliminates half the list.
The second one is the question nobody asks until it is too late
Can you say, months later, who read a particular file?
Object storage can answer this, because every read is an API call and API calls can be recorded, per object, per principal. A mounted filesystem generally cannot — the cloud API recorded that a volume was attached to an instance, and everything after that happened inside an operating system that was not keeping that record for you.
For most data nobody cares. For personal data under GDPR, for anything covered by an access policy someone signed, or for the document store behind a retrieval system, the ability to reconstruct who saw what is the whole compliance story — and it is decided by which storage service you picked, on day one, invisibly.
You cannot add it later. That is the point. An access log that starts in month nine answers no question about month three.
Run the sieve
Which of these are true for you?
Still standing7 of 7
Amazon S3
Objects addressed by key, with lifecycle rules, versioning and per-request access records.
Amazon EBS
A block device attached to one instance. What a database expects to be given.
Amazon EFS
A shared POSIX filesystem several machines can mount at once.
Amazon FSx for Lustre
A parallel filesystem for work that reads enormous amounts fast — training sets, simulation, genomics.
Amazon FSx for Windows File Server
SMB shares with the directory semantics a Windows estate already depends on.
AWS Storage Gateway
A local cache that presents cloud storage as something on-premises equipment recognises.
AWS Backup
Policy-driven copies with a retention schedule somebody can be shown.
Our reading, not a specification. Service behaviour changes — check any elimination that decides something against the service page before you design around it.
Two constraints worth turning on together: existing code opens files by path and every read has to be attributable. That pair is common in regulated document work, and watching it empty the board is the most useful thing this widget does. When it does, the answer is not a storage service. It is an application-level access layer in front of one — which is a design decision worth arriving at deliberately rather than discovering.
A worked example
A document extraction pipeline for identity documents. Files arrive from a customer, are classified, have fields extracted, and are retained for a statutory period. Three constraints, none of them technical in origin:
- Retention is a legal period, not a caching policy. That is lifecycle rules, and it wants a service where retention is declarative and evidenced rather than a cron job somebody wrote.
- Most of it is never read again. The expensive class is the one you keep hot for the 2% of documents that get reopened.
- Every read is attributable, because these are identity documents and somebody will eventually ask.
That points at object storage with per-request access records, and it points away from a mounted filesystem — which means the extraction code that wanted a filesystem needs to be the part that changes. Finding that out in week one costs a design conversation. Finding it out in month six costs the pipeline.
The cost trade that is actually a latency trade
Storage classes are usually presented as a cost decision: colder is cheaper. The part that catches people is that retrieval from a cold class is not just billed differently, it is shaped differently — there is a delay, and it is long enough to be an architectural fact rather than a performance detail.
Which means the tiering decision is really a question about your own business process: when someone asks for a five-year-old document, how long may they wait? If the honest answer is “while they are on the phone”, the cheap class is not cheap, it is wrong. If the answer is “we respond to these requests in five working days”, then the cheap class is correct and the saving is real.
That question has nothing to do with storage. It is the same shape as every decision in this series: the constraint that decides lives outside the comparison table, in how the organisation actually works.
Next: databases, where the equivalent hidden question is which query you cannot avoid.