Streaming Large CSV Files in the Browser Without Freezing

Nymphsoft Guide · 2026-08-19

A writeup of a real problem hit while adding "large-file mode" to the CSV → JSON converter, and a bug a randomized test caught along the way.

Why opening a large CSV freezes the tab

The simplest CSV → JSON converter works like this: read the whole file as one string with FileReader.readAsText(), put that string into a <textarea>, parse it, and put the stringified result into another <textarea>. For a CSV that's a few KB, that's completely fine. But once the file is tens or hundreds of MB, three separate spots make the browser freeze.

The moment a multi-hundred-MB string is assigned to textarea.value — browsers' text rendering/layout engines simply aren't built to handle an editable text field of that size.
Looping over millions of rows synchronously — this alone is usually hundreds of ms to a few seconds, but if the main thread is fully blocked the whole time, a "page unresponsive" warning is easy to trigger.
Calling JSON.stringify on the entire result array at once and assigning it back to a textarea.value — the first problem repeats on the output side.

All three share the same root cause: treating the whole file as one indivisible blob.

The approach: a stateful incremental parser

The fix is to read the file in chunks (typically tens to hundreds of KB) via File.stream() and process each chunk as it arrives. The catch is that CSV parsing is stateful — whether you're currently inside a quoted field, or whether the previous character was the first half of an escaped double-quote, has to carry over across chunk boundaries without getting lost.

So the character-by-character state machine from the batch parser stays exactly the same, except instead of local variables that disappear when the function returns, they live in a closure.

function createStreamingCsvParser(delimiter, onRow) {
  let field = "";
  let row = [];
  let inQuotes = false; // false | true | "maybe-close"
  let pendingCR = false;

  function push(chunk) {
    for (let i = 0; i < chunk.length; i++) {
      // ... one character at a time, resuming from whatever
      // the previous push() call left in scope
    }
  }

  function finish() { /* flush the last pending field/row */ }
  return { push, finish };
}

The hard part is a chunk ending at exactly the wrong spot. Say a field contains an escaped double-quote (""), and the first of those two characters happens to be the very last character of the current chunk. Is it a closing quote, or the start of an escape? You can't know yet — you have to look at the first character of the next chunk. To handle this, inQuotes got a third state beyond true/false: "maybe-close", which defers the decision to the next push() call.

The bug: one truthy check broke everything

The first version looked like this:

if (inQuotes) {
  // handle characters inside a quoted field
}
if (inQuotes === "maybe-close") {
  // resolve the deferred chunk-boundary decision
}

Looks reasonable, but "maybe-close" is truthy in JavaScript. So whenever inQuotes === "maybe-close", the first if (inQuotes) caught it before the second block ever ran — the boundary-resolution branch was dead code, silently. No error, no crash — it just quietly produced the wrong result. That's the most dangerous kind of bug.

What caught it wasn't a handful of hand-written test cases — it was a fuzz test: take the same text, split it at random positions, feed it through the streaming parser, and compare the result against the batch parser (the original, known-correct, parse-it-all-at-once version) every time.

for (let trial = 0; trial < 20; trial++) {
  const boundaries = randomChunkBoundaries(text.length, avgChunkSize);
  const streamed = streamingParseAll(text, ",", boundaries);
  assert.deepEqual(streamed, parseCSV(text, ","));  // cross-check
}

Shrinking the chunk size down to a single character turned nearly every quote character into "the last character of this chunk," which forced the "maybe-close" branch to fire constantly — and most test cases failed immediately. The fix was a single line: if (inQuotes) became if (inQuotes === true).

Takeaways

When you add a "deferred decision" state to a state machine, double-check it can't accidentally satisfy some other check. Mixing a real value (true/false) with a "pending" marker of the same type is exactly how a truthy/falsy check silently takes the wrong branch.
Code that handles chunk boundaries has to be tested with random boundaries. Hand-picked test cases only split at the boundaries you thought to try. Real network or file streams don't extend you that courtesy.
If a known-correct implementation already exists (the batch parser), reuse it as the answer key for the new one (the streaming parser). Not having to redefine "what's correct" from scratch is what made writing the fuzz test a five-minute job instead of a multi-hour one.

This streaming parser is exactly what powers the CSV ↔ JSON converter's large-file mode, which kicks in automatically for files over 2MB.

Frequently Asked Questions

Why not use a Web Worker?

Moving parsing to a Web Worker avoids blocking the main thread, but transferring the result back via postMessage becomes its own copy cost for large data. Processing in chunks and periodically yielding to the event loop kept the tab responsive enough without the added complexity of a Worker.

Do all CSV parsing problems need a streaming parser like this?

No. For files under a few MB, reading the whole thing into a string and parsing it in one pass feels instant. Streaming is an optimization that only matters once holding the entire file as one string becomes the bottleneck.

Why doesn't the JSON → CSV direction stream too?

CSV → JSON can finalize each row as soon as it's read. JSON → CSV needs the header — the union of keys across every object — before it can emit any row, which requires at least two passes. That direction doesn't stream yet.