The Anatomy of a CSV File
A close-up follow-up to Beyond “Just a File”. That piece mapped the whole landscape of file formats; this one zooms all the way in on the one format everybody still exchanges data with — the humble CSV — and asks four questions: how its data is structured, what the RFC 4180 standard actually says, how it’s stored, and where it shows up in the real world.
A · How is the data structured?
First, pin down what a CSV actually is. It’s all of these at once:
And a few facts fall straight out of that:
- Every data point is just text. A value that is practically a number, date, time or boolean is still stored as a string.
- It’s table-like. Data sits in rows and columns — ideally every row has the same number of fields.
- Rows are separated by
\n(line feed / newline); fields are separated by,(comma). - CSV enforces nothing. No schema, no validator. Which is exactly why you can meet a CSV whose rows don’t all have the same field count — semi-structured, or even unstructured, data wearing a
.csvextension.
The simple case
When delimiters only ever separate fields, parsing is trivial — split on \n, then split on ,:
1001,File
1002,CSV
1003,Fixed-Length file
1004,JSON
1005,XML
1006,COBOL Data File
1007,Length-Prefixed File
1008,Apache Parquet
1009,Apache ORC
1010,Apache Avro
Every row has two clean fields. A basic string split is all you need.
The complex case
It gets interesting the moment a delimiter shows up inside the data. Look at rows 2 and 4:
1001,File
1002,"CSV,PSV,TSV"
1003,Fixed-Length file
1004,"JSON,XML"
1005,COBOL Data File
Those inner commas are data, not field separators — so the field is wrapped in double quotes. A naive split on , now falls apart:
One line, two ways to read it
split(",")
1002"CSVPSVTSV"
4 fields ✗
This is the whole reason a standard had to exist. Because CSV enforces nothing, someone could just as easily wrap with single quotes (') or back-quotes (`) — and every tool would be left guessing. So the industry agreed on one: RFC 4180.
B · RFC 4180 — the standard, and the real world
RFC 4180 is the closest thing CSV has to a rulebook. But it is just a standard, not followed religiously — real files deviate constantly. The left column is the theory; the right is what actually lands in your inbox:
| Aspect | RFC 4180 | Real world |
|---|---|---|
| Line breaks | Records separated by \r\n (CRLF) | \r\n\n |
| Delimiter | Fields separated by , | ,|\t\x01\x00 |
| Quoting | Fields with commas, line breaks or quotes are wrapped in " | "'quote-allquote-minimal |
| Escaping quotes | A " inside a field becomes "" | ""\" |
| Field count | Every row has the same number of fields | generally followed |
| Header | An optional header row may lead, in the same format | generally followed |
| Encoding | — | UTF-8ASCII |
That gap between the two columns is exactly why almost every language and tool in the data space — traditional or modern — ships a real CSV reader/writer instead of leaving you to split(',').
C · How is the data stored?
Structure is about what the bytes mean; storage is about how they sit on disk. A handful of properties define CSV as a storage format:
Row-oriented & sequential
Native to record-by-record processing — you read it top to bottom, one row at a time.
Mutable
Rows can be added, updated or deleted easily — as long as the file lives on one machine, isn't too big, and the filesystem allows it.
Schema-less
Every data point is text, so there are no real types — no int, float, double or bool.
Encoding matters
The one encoding that really matters is the character set — utf-8 / ascii / ebcdic. Know your machine's native set and how your tool handles it.
Can you read it in parallel?
A CSV is row-oriented and sequential — yet Spark (and friends) can still split a large CSV by byte ranges and run a task per split in parallel:
one big CSV → byte-range splits → parallel tasks
…but only as long as all four of these hold:
\n or \r\n.multiline: false).bz2).And the headline cost: reading a CSV is expensive, and it gets more expensive as the CSV gets more complex. Every ambiguity the reader has to resolve is work.
D · CSV-like files in the real world
For a format with “no strict rules,” CSV is astonishingly load-bearing. You meet it as an input or an output constantly:
Data exchange
Vendors, partners and government orgs still swap data as CSV-like files — so CSV is a frequent input and final output in pipelines.
Operational reporting
In traditional settings, reports are delivered as CSV or Excel — which is why CSV so often sits at the end of a reporting pipeline.
APIs
Plenty of APIs speak CSV — the Salesforce API, for instance, can return requested data in CSV-like formats.
Logs
Log data is commonly written as CSV-like or JSON, so CSV frequently shows up as the input to log-processing pipelines.
Input in, native out — then back again
When CSV is an input, the first move is usually to convert it into a tool-native format (Parquet, if you’re on Spark). When CSV is the required output, the last move converts your native format back to CSV:
And the format never quite stops surprising you — delimiters aren’t always commas, quoting isn’t always double quotes. I’ve received CSVs that use || (double pipe) as the delimiter. No strict rules, remember.
A follow-up to Beyond “Just a File” — originally shared as a LinkedIn carousel. Feel free to read and react to it there too.