Lesson 06

Streams for large payloads

Stream large files or responses instead of loading everything into memory.

streams
memory
http

Good Code

src/routes/download.ts

import { createReadStream } from "node:fs";
import { pipeline } from "node:stream/promises";
import type { ServerResponse } from "node:http";

export async function sendExport(filePath: string, response: ServerResponse) {
  response.setHeader("content-type", "text/csv");

  try {
    await pipeline(createReadStream(filePath), response);
  } catch (error) {
    console.error({ error, filePath }, "Export stream failed");
    if (!response.headersSent) {
      response.statusCode = 500;
      response.end("Could not stream export.");
    }
  }
}

Bad Code

src/routes/download.ts

import { readFile } from "node:fs/promises";
import type { ServerResponse } from "node:http";

export async function sendExport(filePath: string, response: ServerResponse) {
  const file = await readFile(filePath);

  response.setHeader("content-type", "text/csv");
  response.end(file);
}

Review Notes

What to review

Good Code

The good version streams the file through pipeline, which manages backpressure and surfaces stream failures.

Bad Code

The bad version reads the entire file into memory before sending it, which can pressure memory or crash the process for large exports.

Takeaways

For large payloads, prefer pipeline-based streaming with error handling.