To: the internet

TO: THE INTERNET  ·  CC: EVERYONE WHO RUNS INSTALL

More package versions are published to npm each day than any team could read. That is not a staffing problem. There is no reviewer pool large enough, and there never was one.

So the industry settled on lists. Known bad names, known bad hashes, advisories filed after the incident. A list is a record of what already happened to somebody else. It holds right up until the name is new.

An advisory is a receipt. We would rather have the recording.

What a recording contains

One package at one version, installed and imported in a container that holds nothing real. Every file it opened. Every process it started. Every address it reached for. Each action tied back to the process that produced it, because an install is noisy and only some of the noise is the package.

Read one line of that record and nothing is unusual. Packages read files. Build tools compress things. Installers write to temporary directories. What makes it a finding is the order.

 0.31s  FILE  read     ~/.npmrc
 0.52s  FILE  read     ~/.aws/credentials
 2.04s  PROC  spawn    gzip
 4.20s   ..   idle
 6.72s  NET   connect  185.62.188.14:443
 9.98s  NET   connect  185.62.188.14:443

verdict: review    window: 10.5s    attributed: postinstall

Where the recording stops

Behaviour outside the observation window is out of scope, and we say so on the verdict rather than in a footnote. A package that waits a week, or reads the clock, or looks for a build server and stays quiet when it finds one, comes back clean. Clean here means nothing notable observed. It does not mean nothing is there.

A sandbox is table stakes. Deciding what counts as evidence is the open problem, and it is the part nobody wants to publish numbers on.

So here are ours

Detection is a trade, and we would rather show the arithmetic than the marketing. Widening a rule catches more packages. It also flags projects whose only offence was an unusual repository layout. Every rule we ship has a figure attached to both sides of that trade.

• Precision is published as measured, not rounded toward the number we would prefer.
• Every verdict ships with the trace it came from, so you can disagree with us on the evidence rather than on trust.
• A run that was cut short is filed under review, because no coverage is not the same as nothing found.

That is the whole of it. Watch the code run, write down what happened, attribute it to the process that did it, and state plainly what was never covered. The rest is a list, and lists arrive late.