Production & Deployment · Node.js Notes

#package.json for production services

package.json is the manifest of a Node project: what it's called, how Node should interpret its files, which commands it supports, what it depends on and which runtime it expects. It isn't code, but it decides what code gets installed and how it runs — so it's part of your production surface (and your supply chain).

The mental model is two files with two jobs. package.json states intent: "I need express, any 5.x from 5.2.1 up". package-lock.json records fact: "this install used express 5.2.1 from this URL with this hash, and here are the exact versions of every other package it pulled in". You edit the first; npm writes the second; you commit both.

package.json

{
  "name": "orders-service",
  "version": "1.4.2",
  "private": true,
  "type": "commonjs",
  "main": "dist/server.js",
  "scripts": {
    "build": "tsc",
    "build:watch": "tsc --watch",
    "dev": "node --watch --env-file-if-exists=.env dist/server.js",
    "start": "node dist/server.js",
    "test": "jest",
    "test:coverage": "jest --coverage"
  },
  "dependencies": {
    "express": "^5.2.1",
    "pino": "^10.4.0"
  },
  "devDependencies": {
    "@types/node": "^24.0.0",
    "jest": "^30.5.2",
    "supertest": "^7.3.1",
    "typescript": "^7.0.2"
  },
  "engines": {
    "node": ">=24 <25"
  },
  "packageManager": "npm@11.19.0"
}

What’s happening

name, version, private: version follows semver (MAJOR.MINOR.PATCH). "private": true makes npm publish refuse — cheap insurance for an internal service whose name might be free on the public registry.
type: "commonjs" (the default) or "module" decides how .js files in this package are loaded; .cjs and .mjs always mean CommonJS and ESM regardless (see CommonJS vs ES modules). It changes interpretation only — it doesn't convert require code to import.
main is the classic entry point. Libraries use exports instead, which both maps entry points (".": "./dist/index.js", "./client": "./dist/client.js") and blocks deep imports of anything not listed (see Module resolution). A private service rarely needs either; start runs the file directly.
scripts: npm start and npm test are built-in names; anything else is npm run build. start runs the compiled output, not a watcher. dev uses Node's own --watch (restart on change) and --env-file-if-exists (load .env if present) — both built into Node 22+, no nodemon or dotenv needed.
dependencies vs devDependencies: dependencies are needed at runtime (express, pino); devDependencies are needed to build or test (TypeScript, Jest, type definitions). Production installs skip the latter. A package your running code requires must be in dependencies, or the production image crashes with Cannot find module — tests pass locally because dev installs have everything.
engines documents the Node range. With npm it's advisory: I set "node": "<21" on a dependency and npm install on Node 26 printed npm warn EBADENGINE Unsupported engine and installed anyway. With --engine-strict (or engine-strict=true in .npmrc) the same install failed with npm error notsup Required: {"node":">=18 <21"}. packageManager names the exact tool version; it's only enforced if you use Corepack.

AdvancedVersion ranges and the lockfile

Ranges say what a fresh resolution may pick. I checked them with the semver package npm itself uses:

semver-demo.cjs

// semver-demo.cjs — what each range accepts
const semver = require('semver');

const versions = ['1.2.3', '1.2.9', '1.3.0', '1.9.9', '2.0.0'];
for (const range of ['1.2.3', '~1.2.3', '^1.2.3']) {
  console.log(range.padEnd(7), versions.filter((v) => semver.satisfies(v, range)).join(' '));
}

const zero = ['0.2.3', '0.2.9', '0.3.0'];
console.log('^0.2.3 ', zero.filter((v) => semver.satisfies(v, '^0.2.3')).join(' '));
console.log('^0.0.3 ', ['0.0.3', '0.0.4'].filter((v) => semver.satisfies(v, '^0.0.3')).join(' '));
console.log('prerelease in ^1.2.3?', semver.satisfies('1.3.0-beta.1', '^1.2.3'));

// 1.2.3   1.2.3
// ~1.2.3  1.2.3 1.2.9
// ^1.2.3  1.2.3 1.2.9 1.3.0 1.9.9
// ^0.2.3  0.2.3 0.2.9
// ^0.0.3  0.0.3
// prerelease in ^1.2.3? false

What’s happening

An exact version 1.2.3 accepts only itself. ~1.2.3 allows patch updates: >=1.2.3 <1.3.0, so 1.2.9 yes, 1.3.0 no.
^1.2.3 allows minor and patch: >=1.2.3 <2.0.0. That's npm's default when you npm install express, on the promise that minor versions don't break you.
The caret is stricter below 1.0: ^0.2.3 means <0.3.0, and ^0.0.3 means exactly 0.0.3. Semver treats 0.x as unstable, so the leftmost non-zero part is the one that's locked. People are often surprised that ^0.2.3 won't pick up 0.3.0.
Prereleases (1.3.0-beta.1) are excluded from normal ranges unless the range itself names a prerelease.
Ranges explain why the lockfile matters: without it, two installs a week apart can resolve different versions of any of hundreds of transitive packages. The sandbox for this site lists 12 direct dependencies; its package-lock.json has 466 package entries, each with an exact version, a resolved tarball URL and an integrity hash (sha512-…) that npm checks after download.

Advancednpm ci vs npm install

npm install resolves: it reads package.json, uses the lockfile as a guide, updates node_modules incrementally, and rewrites the lockfile if they disagree. npm ci reproduces: it installs exactly what the lockfile says, or fails. I ran both in a tiny project with local packages (--offline, so nothing was downloaded):

Output

# 1. No lockfile yet
$ npm ci
npm error code EUSAGE
npm error The `npm ci` command can only install with an existing package-lock.json or
npm error npm-shrinkwrap.json with lockfileVersion >= 1. …

# 2. Someone adds a dependency to package.json but doesn't commit the updated lockfile
$ npm ci
npm error `npm ci` can only install packages when your package.json and package-lock.json
npm error or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install`
npm error before continuing.
npm error Missing: shout@1.0.0 from lock file
$ echo $?
1

# 3. A stray file in node_modules
$ touch node_modules/.marker
$ npm install        →  up to date      (.marker still there)
$ npm ci             →  added 2 packages (.marker gone: node_modules was deleted first)

# 4. Production installs
$ npm ci --omit=dev                 →  added 1 package   (only the dependency)
$ NODE_ENV=production npm ci        →  added 1 package   (same: NODE_ENV=production implies --omit=dev)
$ npm ci --production               →  npm warn config production Use `--omit=dev` instead.

What’s happening

npm ci requires a lockfile. That's deliberate: CI should never be the place where versions get chosen.
When package.json and the lockfile disagree, npm install would quietly fix the lockfile; npm ci fails the build and names what's missing. That catches the "works on my machine because I didn't commit the lockfile" mistake before it ships.
npm ci deletes node_modules and installs from scratch, so leftovers from earlier installs (or a developer's manual hacking) can't leak into the build. npm install leaves what's there.
--omit=dev skips devDependencies. Setting NODE_ENV=production has the same effect on npm, which is a trap in Docker: set it in the build stage before npm ci and your build tools disappear (more in A production Dockerfile). --production still works but is deprecated.
Rule: developers run npm install (and commit the lockfile changes); CI and Dockerfiles run npm ci.

#Peer dependencies

A normal dependency says "I need this package, give me my own copy if necessary". A peer dependency says "I work with this package, and the app must provide it — one shared copy". It exists for plugins: an Express middleware, a React component library, an ESLint plugin. If each plugin bundled its own Express or React, the app would end up with several copies of the framework, and plugins would register themselves on a copy the app isn't using.

You have a real one in the sandbox: express-rate-limit declares

express-rate-limit/package.json (excerpt)

"peerDependencies": {
  "express": ">= 4.11"
}

Advancednpm ls: the app's copy satisfies the peer

and npm ls express shows npm satisfied it with the app's own copy rather than a nested one:

Output

node@1.0.0 /…/.sandbox/node
+-- express-rate-limit@8.7.1
| `-- express@5.2.1 deduped
`-- express@5.2.1

Advancednpm 7+: auto-installed peers and conflicts

The behaviour you need to know: since npm 7, peers are installed automatically if the app doesn't list them, and a peer conflict is an error. I reproduced a conflict with two local packages: host-plugin@1.0.0 has peerDependencies: { "host-framework": "^1.0.0" }, and the app depends on host-framework@2.0.0:

Output

$ npm install
npm error code ERESOLVE
npm error ERESOLVE unable to resolve dependency tree
npm error
npm error While resolving: app@1.0.0
npm error Found: host-framework@2.0.0
npm error node_modules/host-framework
npm error   host-framework@"file:../host-framework-2.0.0.tgz" from the root project
npm error
npm error Could not resolve dependency:
npm error peer host-framework@"^1.0.0" from host-plugin@1.0.0
npm error node_modules/host-plugin
npm error   host-plugin@"file:../host-plugin-1.0.0.tgz" from the root project
npm error
npm error Fix the upstream dependency conflict, or retry this command with --force or
npm error --legacy-peer-deps to accept an incorrect (and potentially broken) dependency resolution.

$ npm install --legacy-peer-deps
added 2 packages
$ node -p 'require("host-plugin").version'
2

What’s happening

npm builds the tree: the root needs host-framework@2.0.0 and host-plugin@1.0.0. The plugin's peer range ^1.0.0 means >=1.0.0 <2.0.0, and there can be only one host-framework the plugin shares with the app.
2.0.0 is outside ^1.0.0, so no single version satisfies both, and npm 7+ stops with ERESOLVE, printing both sides: what it found (Found: host-framework@2.0.0) and who wanted something else (peer … from host-plugin@1.0.0).
--legacy-peer-deps restores npm 6 behaviour: ignore peers entirely. The install succeeds — and the plugin now runs against version 2, which its author never declared support for. The last line proves it: require("host-plugin") re-exports the framework, and its version is 2.
The right fixes, in order: upgrade the plugin to a version whose peer range includes 2.x; or stay on framework 1.x; or, if you've verified it works, pin the resolution with npm overrides in package.json. --force and --legacy-peer-deps hide the problem rather than solving it — and if you use them, every developer and CI job must use them too (usually via .npmrc).

AdvancedGuidance for library authors

#A production Dockerfile

A good image is small, reproducible, unprivileged, and stops cleanly. It contains your compiled code, production dependencies and a Node runtime — not your TypeScript compiler, test framework, .git folder or .env file. The tool for that is a multi-stage build: earlier stages have everything needed to build; the final stage copies in only the results.

Which base image: node:24-slim. As of October 2026, Node 24 is the Active LTS line; Node 26 (which this site's examples run on) is still "Current" and only becomes LTS later in October 2026 — production images should track LTS. slim is Debian with the extras removed: much smaller than the full image, and unlike Alpine it uses glibc, so prebuilt native modules work without surprises. In a real project pin an exact version (node:24.x.y-slim) or a digest, so a rebuild next month doesn't silently change the runtime.

Dockerfile

# syntax=docker/dockerfile:1

ARG NODE_IMAGE=node:24-slim

# ---------- 1. deps: production dependencies only ----------
FROM ${NODE_IMAGE} AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --omit=dev

# ---------- 2. build: all dependencies, then compile ----------
FROM ${NODE_IMAGE} AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .
RUN npm run build

# ---------- 3. runtime: what actually ships ----------
FROM ${NODE_IMAGE} AS runtime
ENV NODE_ENV=production
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY package.json ./
USER node
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=15s --retries=3 \
  CMD ["node", "-e", "fetch('http://127.0.0.1:3000/livez').then(r => process.exit(r.ok ? 0 : 1), () => process.exit(1))"]
CMD ["node", "dist/server.js"]

What’s happening

# syntax=docker/dockerfile:1 opts into the current BuildKit Dockerfile syntax, which is what makes RUN --mount=type=cache available. ARG NODE_IMAGE before the first FROM lets every stage share one base-image setting (and lets CI override it with --build-arg).
Stage deps: copy only package.json and package-lock.json, then npm ci --omit=dev. Copying the manifests before the source is the key to layer caching: Docker reuses a layer if its inputs didn't change, so editing src/ doesn't re-run the install. The cache mount keeps npm's download cache between builds without baking it into the image.
Stage build: a fresh npm ci with devDependencies (TypeScript, types), then COPY . . and npm run build produces dist/. Notice NODE_ENV=production is not set here — it would make npm ci skip devDependencies and tsc wouldn't exist. If you have native modules that compile from source, this is also where you'd apt-get install python3 make g++; slim doesn't include them, and they never reach the runtime stage.
Stage runtime starts again from the clean base and copies in just three things: production node_modules from deps, dist/ from build, and package.json. No compiler, no tests, no source, no npm cache. ENV NODE_ENV=production here is for your app and libraries (Express, for example, uses it to cache views and shorten error output).
USER node: the official images include an unprivileged node user (uid 1000). The files were copied as root and are only readable by node, which is what you want — a compromised process can't rewrite its own code. If the app must write somewhere (uploads, temp files), create that one directory and chown it.
HEALTHCHECK lets plain Docker and Compose mark the container unhealthy. slim has no curl, so it uses Node's built-in fetch; I checked the one-liner exits 0 for a 200, and 1 for a 500 or a refused connection. Kubernetes ignores HEALTHCHECK and uses its own probes (see Readiness and liveness probes).
CMD ["node", "dist/server.js"] is the exec form (JSON array): Node itself becomes PID 1 and receives SIGTERM directly. The shell form (CMD node dist/server.js) or CMD ["npm", "start"] puts a shell or npm in front of Node, and signals may not reach your app — so the shutdown handler never runs and the platform kills it hard after the grace period.

Advanced.dockerignore: a small build context

.dockerignore

node_modules
dist
coverage
.git
.env*
npm-debug.log*
Dockerfile
.dockerignore

What’s happening

.dockerignore controls what COPY . . can see (the "build context"). Without it, your local node_modules — possibly with macOS-compiled native binaries — would be copied over the clean one, and a .git folder or .env with real secrets would end up inside an image layer.
dist and coverage are excluded because the build stage produces its own; a stale local dist/ must not leak in.
A smaller context also makes builds faster, because the whole context is sent to the builder before anything runs.

AdvancedEdge case: signals in containers

#Logging without blocking

Logs are a second output stream of your service, and they cost real work on the main thread: building the object, serialising it to JSON, and handing bytes to a destination. Do it on every request and it matters; do it to a slow destination synchronously and it can block the event loop.

The mental model: logging is a data pipeline with a producer you can't slow down (your request handlers) and a sink that may be slow (a disk, a pipe, a network). When the sink can't keep up, only three things can happen: the producer waits (blocking), a buffer grows (memory), or records are dropped. A good logging setup makes that choice deliberately — and in containers the usual choice is: write structured JSON lines to stdout, and let the platform's log agent ship them.

First, what stdout actually does. I wrote 8 MB to process.stdout in a loop, with stdout pointing at a file and then at a slow pipe:

stdout-kind.cjs

// stdout-kind.cjs
const big = 'x'.repeat(1024 * 1024) + '\n';
const t = Date.now();
let ok = true;
for (let i = 0; i < 8; i++) ok = process.stdout.write(big) && ok;
process.stderr.write(`write() returned ${ok}, writableLength=${process.stdout.writableLength}, loop took ${Date.now() - t} ms\n`);

// $ node stdout-kind.cjs > out.txt                       (macOS, Node 26)
// write() returned true, writableLength=0, loop took 3 ms
// $ node stdout-kind.cjs | (sleep 1; cat > /dev/null)
// write() returned false, writableLength=8388616, loop took 2 ms

What’s happening

To a file, write() is synchronous: each call finishes the disk write before returning, writableLength (bytes waiting in Node's buffer) is 0, and write() returns true. Fast here because the disk is fast; on a slow or full disk, every logger.info would stall the event loop for that long.
To a pipe on macOS, write() is asynchronous: the loop finished in 2 ms, write() returned false (that's backpressure — "stop, I'm full"), and 8,388,616 bytes sat in memory waiting for the reader. Ignore that false in a busy service with a slow log consumer and memory grows until something gives.
Which one you get depends on the platform. Node's documentation describes stdout to files as synchronous everywhere, and pipes as synchronous on Linux — which is what a container's stdout usually is. So in production, slow log consumption tends to show up as event-loop blocking rather than memory growth.
Either way: log less on hot paths, keep lines small, and use a logger that buffers sensibly (below) rather than many tiny synchronous writes.

AdvancedStructured logging with winston

The winston example from my notes, made structured and run:

winston-demo.cjs

// winston-demo.cjs
const winston = require('winston');

const logger = winston.createLogger({
  level: process.env.LOG_LEVEL || 'info',
  format: winston.format.combine(
    winston.format.timestamp(),
    winston.format.errors({ stack: true }),
    winston.format.json()
  ),
  defaultMeta: { service: 'orders' },
  transports: [
    new winston.transports.Console(),
    new winston.transports.File({ filename: 'error.log', level: 'error' }),
  ],
});

logger.debug('cache miss', { key: 'user:42' }); // below "info": dropped
logger.info('order created', { orderId: 'o_123', durationMs: 42 });
logger.error('payment failed', { orderId: 'o_123', err: new Error('card declined') });

AdvancedOutput: winston's JSON lines

Output

{"durationMs":42,"level":"info","message":"order created","orderId":"o_123","service":"orders","timestamp":"2026-10-07T18:55:13.752Z"}
{"err":{},"level":"error","message":"payment failed","orderId":"o_123","service":"orders","timestamp":"2026-10-07T18:55:13.752Z"}
--- error.log:
{"err":{},"level":"error","message":"payment failed","orderId":"o_123","service":"orders","timestamp":"2026-10-07T18:55:13.752Z"}

What’s happening

createLogger builds a pipeline: each call becomes an info object, passes through the format chain (add timestamp, expand errors, serialise to JSON), then goes to every transport whose level allows it.
level: 'info' means debug is dropped before any formatting work — that's why there's no "cache miss" line. Setting LOG_LEVEL=debug turns it on without a code change.
defaultMeta adds service: 'orders' to every line, so a log platform can filter by service. The fields are real JSON keys (orderId, durationMs), which you can search and aggregate — the point of structured logging.
The File transport has level: 'error', so only the error line lands in error.log. Each transport is its own stream with its own buffering.
Look at "err":{} — the error vanished. Error's message and stack are non-enumerable own properties, so JSON.stringify sees an empty object, and format.errors() only handles an error passed as the message or as the whole meta object. I checked the variants: logger.error(err) and logger.error('payment failed', err) both include the stack; { err } nested in meta loses it. A failed payment logged without its reason is a very bad day.

Advancedpino: JSON lines off the main thread

pino takes a different approach: JSON lines only, minimal work on the main thread, numbers for levels, and anything fancy (pretty printing, shipping) done elsewhere.

pino-demo.cjs

// pino-demo.cjs
const pino = require('pino');

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  base: { service: 'orders' }, // replaces the default { pid, hostname }
  redact: ['req.headers.authorization', 'user.password'],
});

logger.debug({ key: 'user:42' }, 'cache miss'); // dropped
logger.info({ orderId: 'o_123', durationMs: 42 }, 'order created');
logger.info({ req: { headers: { authorization: 'Bearer abc.def' } }, user: { id: 7, password: 'hunter2' } }, 'login');

const reqLog = logger.child({ requestId: 'r-1' });
reqLog.warn('slow downstream');

const err = new Error('card declined');
logger.error({ err, orderId: 'o_123' }, 'payment failed');

AdvancedOutput: pino lines with redaction

Output

{"level":30,"time":1791399330177,"service":"orders","orderId":"o_123","durationMs":42,"msg":"order created"}
{"level":30,"time":1791399330177,"service":"orders","req":{"headers":{"authorization":"[Redacted]"}},"user":{"id":7,"password":"[Redacted]"},"msg":"login"}
{"level":40,"time":1791399330177,"service":"orders","requestId":"r-1","msg":"slow downstream"}
{"level":50,"time":1791399330177,"service":"orders","err":{"type":"Error","message":"card declined","stack":"Error: card declined\n    at Object.<anonymous> (…/pino-demo.cjs:17:13)…"},"orderId":"o_123","msg":"payment failed"}

What’s happening

The argument order is object first, message second: logger.info({ orderId }, 'order created'). Levels are numbers (30 info, 40 warn, 50 error) and time is epoch milliseconds — cheaper to produce than ISO strings, and every log platform understands them.
redact replaces the listed paths with "[Redacted]" before serialisation. The bearer token and the password never reach stdout. Redaction by path is much more reliable than hoping nobody logs req.headers.
logger.child({ requestId: 'r-1' }) returns a logger whose lines all carry requestId. pino serialises the child's bindings once, so per-line cost stays low — this is how you attach request context (see Logging vs tracing for doing it automatically).
The err key is special: pino's standard error serialiser turns an Error under err into { type, message, stack }. The same mistake as winston's { err } works correctly here.
By default pino writes to stdout through its own buffered writer. For heavy volume, pino.transport({ target: … }) moves formatting and shipping into a worker thread, so the main thread only hands over bytes.

AdvancedSecurity: secrets in logs

#Logging vs tracing

Three signals, three questions:

Signal	Answers	Shape
Logs	"What happened, in detail?"	discrete events with fields
Metrics	"How much, how often, how fast — overall?"	numbers over time (rate, p99, error %)
Traces	"Where did this request spend its time, across services?"	a tree of timed spans

A trace follows one request through every service it touches. Each unit of work — an HTTP handler, a DB query, an outgoing call — is a span with a start, a duration, attributes and a parent. All spans of one request share a trace ID, which travels between services in the W3C traceparent header, e.g. 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01 (version, trace ID, parent span ID, flags). The tracing backend reassembles them into a waterfall that shows "this checkout took 1.8 s, of which 1.4 s was the payment service waiting on its database".

Logs and traces meet at the correlation ID: put the same request (or trace) ID on every log line, and you can jump from a slow span to exactly the logs it produced. The hard part in Node is getting that ID into a log call ten functions deep without passing it through every function signature. That's what AsyncLocalStorage is for: a value that follows the async flow of one request — through awaits, timers and callbacks — the way a thread-local follows a thread in Java.

correlation.cjs

// correlation.cjs
const { AsyncLocalStorage } = require('node:async_hooks');
const { randomUUID } = require('node:crypto');
const { setTimeout: sleep } = require('node:timers/promises');
const express = require('express');
const pino = require('pino');

const requestContext = new AsyncLocalStorage();

const logger = pino({
  base: undefined,
  timestamp: false,
  mixin: () => ({ requestId: requestContext.getStore()?.requestId }),
});

// Deep in the call stack: no requestId parameter anywhere
async function loadOrders(userId, delayMs) {
  logger.info({ userId }, 'querying orders');
  await sleep(delayMs); // the request "pauses" here; others run meanwhile
  logger.info({ userId }, 'orders loaded');
  return [{ id: 'o_1' }];
}

const app = express();

app.use((req, res, next) => {
  const requestId = req.get('x-request-id') ?? randomUUID();
  res.set('x-request-id', requestId);
  requestContext.run({ requestId }, next);
});

app.get('/orders', async (req, res) => {
  const orders = await loadOrders(req.query.user, Number(req.query.delay));
  res.json(orders);
});

// Two overlapping requests: A is slow, B is fast
const request = require('supertest');
Promise.all([
  request(app).get('/orders?user=rohit&delay=50').set('x-request-id', 'req-A'),
  request(app).get('/orders?user=neha&delay=10').set('x-request-id', 'req-B'),
  request(app).get('/orders?user=romeo&delay=20'),
]).then((responses) => {
  console.log('response headers:', responses.map((r) => r.headers['x-request-id']));
  console.log('outside any request:', requestContext.getStore());
});

AdvancedOutput: request IDs per request

Output

{"level":30,"requestId":"req-A","userId":"rohit","msg":"querying orders"}
{"level":30,"requestId":"req-B","userId":"neha","msg":"querying orders"}
{"level":30,"requestId":"e55684f0-55e0-426f-b948-8a36d7392d8c","userId":"romeo","msg":"querying orders"}
{"level":30,"requestId":"req-B","userId":"neha","msg":"orders loaded"}
{"level":30,"requestId":"e55684f0-55e0-426f-b948-8a36d7392d8c","userId":"romeo","msg":"orders loaded"}
{"level":30,"requestId":"req-A","userId":"rohit","msg":"orders loaded"}
response headers: [ 'req-A', 'req-B', 'e55684f0-55e0-426f-b948-8a36d7392d8c' ]
outside any request: undefined

What’s happening

The middleware reads an incoming x-request-id (so an ID set by the gateway or an upstream service is kept) or makes one with randomUUID(), echoes it in the response header, and calls requestContext.run({ requestId }, next). Everything that runs inside next — and everything it schedules asynchronously — sees that object as the store.
pino's mixin runs for every log line and merges its result into the line. It calls requestContext.getStore(), so loadOrders logs the right ID without ever receiving it as a parameter.
The three requests interleave: all three log "querying orders", then wait 50, 10 and 20 ms. B finishes first, then the third request, then A. Despite that, every "orders loaded" line carries its own request's ID — Node propagates the store through the await sleep(...) continuation of each request separately.
The third request had no header, so it got a UUID, and the same UUID came back in its response header — which is what lets a user's support ticket ("request e556… failed") be matched to logs.
Outside any run(), getStore() is undefined, so code at startup simply logs without an ID.
If you used a module-level let currentRequestId instead (I tried it): the "querying" lines look right, but all three "orders loaded" lines got the third request's UUID, because by the time A and B woke up, the variable had been overwritten. One process serves many requests at once (How one process serves many requests), so request state can't live in a global.

AdvancedProduction tracing with OpenTelemetry

In production you'd usually let OpenTelemetry do this: it uses the same AsyncLocalStorage mechanism to track the active span, automatically creates spans for incoming HTTP requests, Express routes, outgoing fetch/http calls and database drivers, propagates traceparent to downstream services, and can inject trace_id/span_id into pino or winston lines. A sketch — it assumes the @opentelemetry/* packages are installed and an OTLP collector is reachable:

instrumentation.cjs

// instrumentation.cjs — load before the app: node --require ./instrumentation.cjs dist/server.js
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');

const sdk = new NodeSDK({
  serviceName: 'orders',
  traceExporter: new OTLPTraceExporter(), // endpoint from OTEL_EXPORTER_OTLP_ENDPOINT
  instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();

// Elsewhere — a manual span around one piece of work:
const { trace, SpanStatusCode } = require('@opentelemetry/api');
const tracer = trace.getTracer('orders');

async function chargeCard(order) {
  return tracer.startActiveSpan('chargeCard', async (span) => {
    span.setAttribute('order.id', order.id);
    try {
      return await payments.charge(order);
    } catch (err) {
      span.recordException(err);
      span.setStatus({ code: SpanStatusCode.ERROR });
      throw err;
    } finally {
      span.end();
    }
  });
}

What’s happening

Instrumentation must load before the libraries it patches, which is why it's a separate file preloaded with --require: it hooks http, express, database drivers and so on as they're first required.
NodeSDK wires together the instrumentations, the exporter (spans are batched and sent over OTLP to a collector) and the service name that labels every span.
startActiveSpan('chargeCard', fn) creates a span as a child of whatever span is active (here, the HTTP request span) and makes it active inside fn, so a DB query inside payments.charge becomes its child.
recordException and setStatus(ERROR) mark the span failed in the waterfall; finally { span.end() } records the duration even when it throws. A span you never end() is never exported.

Where does the ID go?

In correlation.cjs, loadOrders calls setTimeout(() => logger.info('late'), 100) instead of awaiting. Which requestId does the "late" line have?

Show answer

The ID of the request that called loadOrders.

What’s happening

AsyncLocalStorage captures the current store when an async resource — a timer, a promise, a socket callback — is created, and restores it when that resource's callback runs.
The setTimeout is created inside requestContext.run(...) for that request, so 100 ms later, even if other requests ran in between and even after the response was sent, getStore() returns that request's { requestId }.
The exception is code that escapes the async chain — for example a callback registered on a long-lived emitter that was created outside the request, or some connection-pool libraries that reuse callbacks. Those can lose or mix context, which is why tracing libraries patch them specifically.

#Graceful shutdown

Every deploy, scale-down and node drain ends the same way: the platform sends your process SIGTERM, waits a grace period (Docker: 10 s by default; Kubernetes: terminationGracePeriodSeconds, 30 s by default), then sends SIGKILL, which can't be caught. Whatever your process was doing when SIGKILL lands — a checkout half-written, a response half-sent — is cut off.

Graceful shutdown is using that grace period well: stop taking new work, finish the work you have, close resources in order, exit with the right code — and give up at a deadline rather than letting the platform kill you mid-cleanup. The mental model is a restaurant at closing time: lock the front door (stop listening), let seated customers finish (drain in-flight requests), then cash up and turn off the lights (close the DB pool, flush telemetry), and if the last table is still there at midnight, ask them to leave (deadline).

server.cjs

// server.cjs
const express = require('express');
const { setTimeout: sleep } = require('node:timers/promises');

const t0 = Number(process.env.T0 ?? Date.now()); // share the driver's clock
const log = (msg) => console.log(`[server +${String(Date.now() - t0).padStart(4)}ms] ${msg}`);

let shuttingDown = false;
const inFlight = new Set(); // responses not yet finished

const app = express();

app.use((req, res, next) => {
  inFlight.add(res);
  res.on('close', () => inFlight.delete(res));
  next();
});

app.get('/livez', (req, res) => res.send('ok'));
app.get('/readyz', (req, res) => res.status(shuttingDown ? 503 : 200).send(shuttingDown ? 'draining' : 'ready'));
app.get('/slow', async (req, res) => {
  log('GET /slow started');
  await sleep(2000); // e.g. a slow checkout call
  res.json({ done: true });
  log('GET /slow finished');
});

const server = app.listen(Number(process.env.PORT ?? 3000), () => {
  log(`listening on ${server.address().port}`);
});

async function closeResources() {
  await sleep(100); // stand-in for db.end(), queue.close(), tracer.shutdown()
  log('db pool closed');
}

function shutdown(signal) {
  if (shuttingDown) return; // a second signal must not start a second shutdown
  shuttingDown = true;
  log(`${signal} received: draining ${inFlight.size} in-flight request(s)`);

  // Tell keep-alive clients not to reuse the socket once their response is done.
  for (const res of inFlight) if (!res.headersSent) res.setHeader('Connection', 'close');

  const deadline = setTimeout(() => {
    log('deadline hit: forcing remaining connections closed');
    server.closeAllConnections();
    process.exit(1);
  }, 10_000);
  deadline.unref(); // this timer alone must not keep the process alive

  server.close(async (err) => {
    log('server closed: every connection has ended');
    try {
      await closeResources();
      process.exit(err ? 1 : 0);
    } catch (cleanupErr) {
      log(`cleanup failed: ${cleanupErr.message}`);
      process.exit(1);
    }
  });
}

if (!process.env.NO_HANDLER) {
  process.on('SIGTERM', () => shutdown('SIGTERM'));
  process.on('SIGINT', () => shutdown('SIGINT'));
}

AdvancedDriving a SIGTERM shutdown test

A driver plays the orchestrator: it starts the server on a free port, starts a slow request, sends SIGTERM 300 ms later, tries a new request, and reports how everything ended.

shutdown-demo.cjs

// shutdown-demo.cjs — plays the orchestrator
const { spawn } = require('node:child_process');

const t0 = Date.now();
const log = (msg) => console.log(`[driver +${String(Date.now() - t0).padStart(4)}ms] ${msg}`);

const child = spawn(process.execPath, ['server.cjs'], {
  env: { ...process.env, PORT: '0', T0: String(t0) }, // port 0 = any free port
  stdio: ['ignore', 'pipe', 'inherit'],
});
child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => process.stdout.write(chunk));

child.stdout.once('data', async (line) => {
  const port = Number(line.match(/listening on (\d+)/)[1]);
  const base = `http://127.0.0.1:${port}`;

  const slow = fetch(`${base}/slow`).then(
    async (res) => log(`/slow → ${res.status} ${await res.text()} (connection: ${res.headers.get('connection')})`),
    (err) => log(`/slow failed: ${err.cause?.code ?? err.message}`)
  );

  await new Promise((r) => setTimeout(r, 300));
  log('sending SIGTERM');
  child.kill('SIGTERM');

  await new Promise((r) => setTimeout(r, 200));
  await fetch(`${base}/livez`).then(
    (res) => log(`new request → ${res.status}`),
    (err) => log(`new request refused: ${err.cause?.code ?? err.message}`)
  );
  await slow;
});

child.on('exit', (code, signal) => log(`server exited: code=${code} signal=${signal}`));

AdvancedOutput: the in-flight request finishes

Output

$ node shutdown-demo.cjs
[server +  52ms] listening on 52537
[server +  67ms] GET /slow started
[driver + 366ms] sending SIGTERM
[server + 368ms] SIGTERM received: draining 1 in-flight request(s)
[driver + 572ms] new request refused: ECONNREFUSED
[driver +2073ms] /slow → 200 {"done":true} (connection: close)
[server +2069ms] GET /slow finished
[server +2069ms] server closed: every connection has ended
[server +2170ms] db pool closed
[driver +2172ms] server exited: code=0 signal=null

$ NO_HANDLER=1 node shutdown-demo.cjs
[server +  59ms] listening on 53925
[server +  77ms] GET /slow started
[driver + 378ms] sending SIGTERM
[driver + 381ms] /slow failed: UND_ERR_SOCKET
[driver + 381ms] server exited: code=null signal=SIGTERM
[driver + 579ms] new request refused: ECONNREFUSED

What’s happening

Start: the server listens on a random free port (PORT=0), and the driver parses the port from the first log line. Both processes print times from the same clock (T0), so the timestamps line up; two lines can appear slightly out of order because two processes are writing to the terminal.
+67 ms: GET /slow starts and the handler awaits a 2-second sleep. The middleware has put its res in inFlight.
+368 ms, SIGTERM: because we registered process.on('SIGTERM'), Node does not exit — our shutdown runs. It sets shuttingDown (now /readyz would say 503), marks the one in-flight response Connection: close, arms a 10 s deadline with unref() (so the timer itself never keeps the process alive), and calls server.close().
server.close() stops listening immediately — the driver's new request at +572 ms gets ECONNREFUSED — and closes connections that are already idle. Its callback waits until every remaining connection has ended. The /slow handler is still running and isn't interrupted.
+2069 ms: /slow responds 200 {"done":true} with connection: close, so the socket closes right after the response; that was the last connection, so the close callback fires in the same millisecond. Then closeResources() (DB pool, queues, telemetry flush) and process.exit(0) at +2170 ms. The user's request finished; nothing was cut off.
Without a handler: SIGTERM kills Node immediately (code=null signal=SIGTERM, which a shell reports as exit status 143 = 128 + 15). The in-flight request dies with UND_ERR_SOCKET — undici's error for a socket closed mid-response. That's what every deploy looks like to your users without graceful shutdown.
If you drop the Connection: close loop (I ran that version first): the response goes out with connection: keep-alive, the client keeps the idle socket open for reuse, and server.close() waits for it — the close callback came about 3 seconds after the response, only when that idle socket was finally closed by a keep-alive timeout. server.close() only closes connections that are idle at the moment you call it; ones that become idle later stay open until a keep-alive timeout. With a long-lived client that can eat the whole grace period. And if a request truly hangs, the deadline branch runs: I shortened it to 1 s and saw deadline hit…, the client's request fail, and code=1.

AdvancedEdge case: Kubernetes endpoint removal lag

AdvancedLong-lived connections and other shutdown work

Other things a real shutdown handles: long-lived connections (WebSockets, SSE) need an explicit "please reconnect" and a deadline, because they never end on their own; queue consumers should stop fetching new messages and finish or release the current one; and process.on('uncaughtException') handlers should log and exit rather than try to keep serving (see Floating promises and unhandled rejections). If you run under the cluster module, the primary process forwards the shutdown to its workers.

Why unref()?

In server.cjs, what would change if you removed deadline.unref()?

Show answer

In this code, nothing visible — but it would matter if the code relied on the event loop emptying instead of calling process.exit.

What’s happening

An active timer keeps Node's event loop alive. unref() tells Node "don't stay alive just for this timer".
In server.cjs, the success path calls process.exit(0) explicitly, so the pending 10 s timer is irrelevant — exit happens anyway.
Many shutdown handlers instead let the process end naturally once the server and pools are closed (no process.exit), which is a nice proof that nothing was leaked. In that style, a ref'd deadline timer would keep the process alive for the full 10 s after a clean drain, and then exit with code 1 from the deadline branch — a successful shutdown reported as a failure.

#Readiness and liveness probes

An orchestrator asks your service two different questions, and mixing them up causes outages:

Liveness — "Is this process healthy enough to keep running, or should I restart it?" A failing liveness probe kills and restarts the container.
Readiness — "Should I send it traffic right now?" A failing readiness probe removes the instance from the load balancer, but leaves it running.

A third, startup, answers "has it finished booting?" and holds off the other two until it passes, so a slow start isn't mistaken for a dead process.

The mental model: liveness is "is the shop owner conscious?"; readiness is "is the shop open?". The owner can be fine while the shop is closed — during opening, while the delivery is late (a dependency is down), or while closing up at the end of the day (draining).

health.cjs

// health.cjs
const express = require('express');

const withTimeout = (promise, ms) =>
  Promise.race([
    promise,
    new Promise((_, reject) => setTimeout(() => reject(new Error(`timed out after ${ms} ms`)), ms).unref()),
  ]);

function createHealth({ checkDb }) {
  const state = { started: false, draining: false };
  const router = express.Router();

  // Liveness: "is this process able to answer at all?" No dependency checks.
  router.get('/livez', (req, res) => res.send('ok'));

  // Readiness: "should the load balancer send me traffic right now?"
  router.get('/readyz', async (req, res) => {
    if (!state.started) return res.status(503).json({ ready: false, reason: 'starting' });
    if (state.draining) return res.status(503).json({ ready: false, reason: 'draining' });
    try {
      await withTimeout(checkDb(), 500);
      res.json({ ready: true });
    } catch (err) {
      res.status(503).json({ ready: false, reason: `db: ${err.message}` });
    }
  });

  return { router, state };
}

module.exports = { createHealth };

Advancedhealth-demo.cjs: liveness and readiness

health-demo.cjs

// health-demo.cjs
const express = require('express');
const request = require('supertest');
const { createHealth } = require('./health.cjs');

let db = 'up';
const checkDb = () =>
  db === 'up' ? Promise.resolve()
  : db === 'down' ? Promise.reject(new Error('ECONNREFUSED'))
  : new Promise(() => {}); // 'hung': never settles

const health = createHealth({ checkDb });
const app = express().use(health.router);

const probe = async (label) => {
  const [live, ready] = await Promise.all([request(app).get('/livez'), request(app).get('/readyz')]);
  console.log(label.padEnd(18), 'livez', live.status, '| readyz', ready.status, ready.text);
};

(async () => {
  await probe('booting');
  health.state.started = true;            // after config, pools and caches are ready
  await probe('started');
  db = 'down';
  await probe('db down');
  db = 'hung';
  await probe('db hung');
  db = 'up';
  health.state.draining = true;           // first thing the SIGTERM handler does
  await probe('draining');
})();

AdvancedOutput: probes through each state

Output

booting            livez 200 | readyz 503 {"ready":false,"reason":"starting"}
started            livez 200 | readyz 200 {"ready":true}
db down            livez 200 | readyz 503 {"ready":false,"reason":"db: ECONNREFUSED"}
db hung            livez 200 | readyz 503 {"ready":false,"reason":"db: timed out after 500 ms"}
draining           livez 200 | readyz 503 {"ready":false,"reason":"draining"}

What’s happening

Booting: the process is up (livez 200) but hasn't finished connecting pools or warming caches, so readyz is 503 starting. The platform doesn't route to it yet — that's what prevents the burst of errors when a new version starts listening before it can actually serve.
Started: the app sets state.started = true once initialisation is done, and readiness checks the database. checkDb resolves → 200.
DB down: readiness is 503 with a reason, so the load balancer sends traffic to other instances — but liveness stays 200. Restarting this process wouldn't fix the database; it would just add a cold start.
DB hung: the check never settles, and without withTimeout the probe request would hang until the orchestrator's own probe timeout. With it, readiness answers 503 in 500 ms with a useful reason. The unref() stops that timeout timer from keeping the process alive.
Draining: the first thing the SIGTERM handler from Graceful shutdown does is set this flag, so the instance leaves the pool while it finishes its in-flight requests. Liveness is still 200 — the process is fine, just closing.
Liveness is deliberately trivial: if the event loop can run this handler, the process is alive. If the loop is blocked for longer than the probe timeout, liveness fails and a restart is the right answer.

AdvancedKubernetes probe configuration

The matching Kubernetes configuration (illustrative):

deployment.yaml (excerpt)

spec:
  terminationGracePeriodSeconds: 40
  containers:
    - name: orders
      image: registry.example.com/orders:1.4.2
      ports:
        - containerPort: 3000
      startupProbe:
        httpGet: { path: /livez, port: 3000 }
        periodSeconds: 2
        failureThreshold: 30        # up to 60 s to boot
      readinessProbe:
        httpGet: { path: /readyz, port: 3000 }
        periodSeconds: 5
        timeoutSeconds: 2
        failureThreshold: 2
      livenessProbe:
        httpGet: { path: /livez, port: 3000 }
        periodSeconds: 10
        timeoutSeconds: 2
        failureThreshold: 3         # ~30 s of failures before a restart
      lifecycle:
        preStop:
          exec: { command: ["sleep", "5"] }   # let endpoint removal catch up

What’s happening

startupProbe checks every 2 s, allowing 30 failures (60 s) before giving up. Liveness and readiness don't run until it passes, so a slow boot isn't killed by liveness.
readinessProbe reacts fast (two failures, 10 s) because taking an instance out of rotation is cheap and reversible.
livenessProbe is slower to trigger (three failures, 30 s) because a restart is expensive and, if the cause is shared (a dependency, a traffic spike), restarting every pod at once makes things worse.
preStop sleeps 5 s before Kubernetes sends SIGTERM, covering the endpoint-removal race; terminationGracePeriodSeconds: 40 must fit the 5 s sleep plus the app's own drain deadline (10 s in server.cjs) with room to spare. The grace period counts from the start of preStop.

AdvancedPitfall: dependency checks in liveness

#Zero-downtime deployment

From my notes: PM2, load balancers, blue-green and canary. They're all variations on one idea: zero downtime needs overlapping healthy capacity. If one process serves all traffic and you stop it to start the new version, there's a gap, however short. With several instances behind a load balancer you can always replace some while the others serve.

Output

single instance:   users → v1        deploy: stop v1 ……… start v2      ← downtime gap
behind an LB:      users → LB → A, B, C   replace A, then B, then C    ← always ≥ 2 serving

AdvancedPrerequisites: readiness and draining

But capacity alone isn't enough — every strategy below also depends on the two previous topics: new instances must only get traffic when ready, and old instances must drain gracefully. A load balancer that stops routing to an instance doesn't help the request that instance was in the middle of.

AdvancedRolling deployments

Rolling

Replace instances a few at a time: start a v2, wait until it's ready, drain and stop a v1, repeat. Kubernetes Deployments do this by default.

deployment.yaml (excerpt)

spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1          # at most 1 extra pod above 3 during the rollout
      maxUnavailable: 0    # never fewer than 3 ready pods

What’s happening

maxSurge: 1 lets Kubernetes create a 4th pod (v2) before removing anything.
maxUnavailable: 0 means it may only terminate a v1 pod once the v2 pod passes readiness — so capacity never drops below 3.
The cycle repeats three times: A(v2) B(v1) C(v1) → A(v2) B(v2) C(v1) → all v2. Each terminated v1 gets SIGTERM and drains.
During the rollout v1 and v2 serve traffic at the same time, so they must be compatible with each other's data, sessions and API — the reason Database migrations without downtime exists. Rollback is another rolling update, which takes as long as the rollout.

AdvancedBlue-green deployments

Blue-green

Run two full environments. Blue is live (v1); deploy v2 to green, test it with no user traffic, then switch the router/DNS/load balancer from blue to green in one step. Keep blue around for a while: rollback is switching back.

Pros: a clean cutover, the new version is tested in a production-like environment before users see it, instant rollback.
Cons: you pay for double capacity during the switch; the database is usually shared, so schema changes must still work for both; in-flight requests and sessions on blue still need draining.

AdvancedCanary releases

Canary

Send a small slice of real traffic to v2 — 1–5% — while the rest stays on v1, and compare: error rate, p95/p99 latency, CPU and memory, and business metrics (checkouts per minute). If healthy, increase in steps (5% → 25% → 50% → 100%); if not, route everything back. From my notes: deploy to a small set of servers, route a portion of traffic via the load balancer, monitor, then gradually increase.

The catch is that canaries are only as good as your automated judgement: "the process didn't crash" isn't enough — a release can return HTTP 200 with wrong data. Define the metrics and thresholds that roll back automatically before you start. Tools like Argo Rollouts or a service mesh implement weighted routing on Kubernetes.

AdvancedPM2 on a single host

Without an orchestrator, PM2 runs N copies of your app in cluster mode and can reload them one at a time.

ecosystem.config.js

// ecosystem.config.js — illustrative; assumes pm2 is installed globally
module.exports = {
  apps: [
    {
      name: 'orders',
      script: 'dist/server.js',
      instances: 4,          // or 'max' for one per CPU core
      exec_mode: 'cluster',  // workers share the port via the cluster module
      wait_ready: true,      // a new worker counts as up only after process.send('ready')
      listen_timeout: 10000, // …or after this many ms
      kill_timeout: 15000,   // ms between SIGINT and SIGKILL for an old worker
    },
  ],
};
// in server.js, once listening and connected:  process.send?.('ready');
// deploy:  pm2 reload orders      (restart = all at once; reload = one by one)

What’s happening

exec_mode: 'cluster' uses Node's cluster module: a primary process owns the port and hands connections to the workers.
pm2 reload orders starts a new worker, waits until it's up, then stops one old worker, and repeats — a rolling update inside one machine. pm2 restart stops everything first, which is a gap.
wait_ready plus process.send('ready') is PM2's readiness signal: without it, PM2 considers a worker up as soon as it starts, before it has connected to the database.
kill_timeout is the grace period PM2 gives an old worker after asking it to stop (PM2 sends SIGINT by default — handle it like SIGTERM, as server.cjs does). The default is short, so raise it to cover your drain.
On Kubernetes, skip PM2 for this: the platform already does rolling updates, restarts and readiness, and one process per container keeps signals and resource limits simple (see Do you need cluster on Kubernetes?).

#Database migrations without downtime

Your code deploy can be perfect and the database migration can still take you down. During any rolling, blue-green or canary deploy, old and new versions run at the same time against the same database. So a migration that drops or renames a column the old version still reads breaks every request still served by the old version — for the whole rollout.

The fix is expand and contract (also called parallel change): split one breaking change into several backward-compatible steps, each deployed separately.

Output

expand   → add the new shape alongside the old (old code doesn't notice)
migrate  → new code writes both, reads new-with-fallback; backfill old rows
contract → once nothing reads the old shape, stop writing it, then remove it

AdvancedExpand and contract: renaming a column

Renaming users.name to users.full_name, run for real on node:sqlite:

migration-demo.cjs

// migration-demo.cjs — renaming users.name to users.full_name with no downtime
const { DatabaseSync } = require('node:sqlite');
const db = new DatabaseSync(':memory:');

db.exec(`CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)`);
db.exec(`INSERT INTO users (name) VALUES ('Rohit'), ('Neha')`);

// The two app versions that will run side by side during the rollout
const v1 = {
  create: (n) => db.prepare('INSERT INTO users (name) VALUES (?)').run(n),
  read: (id) => db.prepare('SELECT name FROM users WHERE id = ?').get(id)?.name,
};
const v2 = {
  create: (n) => db.prepare('INSERT INTO users (name, full_name) VALUES (?, ?)').run(n, n), // dual write
  read: (id) => {
    const row = db.prepare('SELECT full_name, name FROM users WHERE id = ?').get(id);
    return row?.full_name ?? row?.name; // fall back for rows v1 wrote
  },
};

// 1. EXPAND: additive, backward compatible. v1 never notices.
db.exec('ALTER TABLE users ADD COLUMN full_name TEXT');

// 2. ROLLOUT: v1 and v2 serve traffic at the same time
v1.create('Romeo');   // id 3, full_name NULL
v2.create('Priya');   // id 4, both columns
console.log('v1 reads 4:', v1.read(4), '| v2 reads 3:', v2.read(3));

// 3. BACKFILL (in batches on a real table)
db.exec('UPDATE users SET full_name = name WHERE full_name IS NULL');
console.log('rows missing full_name:', db.prepare('SELECT COUNT(*) AS n FROM users WHERE full_name IS NULL').get().n);

// 4. v3 uses only full_name. It can't stop writing name while name is NOT NULL:
const v3 = {
  create: (n) => db.prepare('INSERT INTO users (full_name) VALUES (?)').run(n),
  read: (id) => db.prepare('SELECT full_name FROM users WHERE id = ?').get(id)?.full_name,
};
try {
  v3.create('Arjun');
} catch (err) {
  console.log('v3 insert before contract:', err.message);
}

// 5. CONTRACT: only once no v1 or v2 instance is left
db.exec('ALTER TABLE users DROP COLUMN name');
v3.create('Arjun');
console.log('v3 reads 1 and 5:', v3.read(1), v3.read(5));
try {
  v1.read(1);
} catch (err) {
  console.log('a straggling v1 after contract:', err.message);
}

// v1 reads 4: Priya | v2 reads 3: Romeo
// rows missing full_name: 0
// v3 insert before contract: NOT NULL constraint failed: users.name
// v3 reads 1 and 5: Rohit Arjun
// a straggling v1 after contract: no such column: name

What’s happening

Expand: ADD COLUMN full_name TEXT is nullable and additive. v1's queries only mention name, so they keep working — this migration can run before any new code is deployed.
Rollout: v1 and v2 serve at once. v1 creates Romeo (id 3) with full_name NULL; v2 creates Priya (id 4) writing both columns. Each can read the other's rows: v1 reads Priya via name, and v2 reads Romeo via its fallback full_name ?? name. Output: v1 reads 4: Priya | v2 reads 3: Romeo.
Backfill copies name into full_name for old rows — rows missing full_name: 0. On a big table you do this in batches (by id range) so you don't lock the table or flood replication.
v3 reads and writes only full_name. Deploying it while name is still NOT NULL fails: NOT NULL constraint failed: users.name. So the contract phase has its own order: first relax the old column's constraint (or keep writing it), then deploy code that ignores it.
Contract: once no v1 or v2 instance is running, DROP COLUMN name. v3 now works (Rohit Arjun). Any v1 still alive — a forgotten cron job, a slow canary rollback — fails with no such column: name, which is exactly why dropping comes last, in a separate release after the old code is gone.
That's at least three deploys for one rename. It feels slow, but each step can be rolled back on its own, and at no point does a running version see a schema it doesn't understand.

AdvancedRisk: locking DDL and long migrations

#Gitflow vs trunk-based development

My notes title this "Gitflow vs feature branches". A small correction first: both styles use feature branches. The real difference is how long branches live and how many permanent branches you keep.

Gitflow has two permanent branches — main (what's in production, tagged by release) and develop (the integration branch) — plus three kinds of temporary ones: feature/* off develop, release/* off develop for stabilising a version, and hotfix/* off main for urgent fixes, merged back into both. It was designed for software shipped as versioned releases: mobile apps, desktop software, libraries, anything where several versions are supported at once.

Trunk-based development has one long-lived branch, main. Everyone integrates into it at least daily through short-lived branches (hours to a day or two) and small pull requests; main is always releasable; unfinished work is merged hidden behind feature flags rather than kept on a branch. Releases come from main (or from short release branches cut from it). It's the default for continuously deployed services — which is what most Node backends are.

Shell

# Gitflow: a feature, then a release
git switch develop && git switch -c feature/refund-api
# …days or weeks of commits…
git switch develop && git merge --no-ff feature/refund-api
git switch -c release/1.5.0 develop      # stabilise: only fixes from here
git switch main && git merge --no-ff release/1.5.0 && git tag v1.5.0
git switch develop && git merge --no-ff release/1.5.0   # bring the fixes back

# Trunk-based: a small change, merged the same day
git switch main && git pull
git switch -c refund-api-validation
# …a few small commits, behind a flag: if (flags.refundApi) { … }…
git push -u origin refund-api-validation   # PR → CI → review → merge to main → deploy

What’s happening

In Gitflow the feature lives on its own branch for as long as it takes, so it drifts from develop; the merge at the end is where conflicts and integration bugs appear ("merge hell").
Cutting release/1.5.0 freezes features while QA stabilises; fixes go onto the release branch and must be merged into both main and develop, or the fix is lost in the next release.
In trunk-based development the branch exists for hours. The change is small enough to review properly, CI runs on every merge to main, and conflicts stay tiny because nobody is far from main.
The feature flag is what makes this possible: half-built code reaches production switched off, can be turned on for internal users or a canary percentage, and turned off again without a deploy — a release decision separated from a deploy.

AdvancedReference: Gitflow vs trunk-based

	Gitflow	Trunk-based
Long-lived branches	`main` + `develop`	`main` only
Branch lifetime	days to weeks	hours to a day or two
Integration	at merge time, often late	continuous
Unfinished work	on a branch	merged, behind a flag
Fits	versioned releases, several supported versions	continuous delivery of services
Main risk	painful merges, slow feedback	needs strong CI, tests and flag hygiene

AdvancedPrerequisites for trunk-based development