Your SBOM Depends On Which Tool Generated It

Most teams treat an SBOM like a hash: run the tool, get the answer, attach it to the release. The research published this autumn says that is the wrong mental model. Run two reputable tools over one repository and you get two different bills of materials, and the difference is not noise you can patch away.

Measured twice, independently

The first study is "Mind the Gap: How SBOM Specification Ambiguities Lead to Divergent Software Bills of Materials" by Alan Prado, Olivier Zendra, Philippe Boinot and Olivier Barais, submitted to arXiv on 17 September 2026. The authors evaluated three widely used SBOM generators against more than 3,000 JavaScript and Rust projects, using baselines derived from dependency lockfiles. The tools differed in both dependency coverage and SBOM completeness.

The second is "When SBOMs Differ", presented by Lukas Gehrke of the Technical University of Munich at SCORED 2026, part of OpenSSF Community Day Europe in Prague, on 6 October 2026. That work assembled 468,550 unique SBOMs from 93,445 open-source repositories — taken from the GitHub dependency graph or generated with Syft and Trivy — compared the dependency sets the two generators produce, examined how each encodes dependency relationships, and scored the results against current regulatory guidelines using the sbomqs quality tool. Adherence varied depending on which tool produced the SBOM. The dataset is published; the session abstract does not give per-tool disagreement rates, so treat the direction as established and the magnitude as something you should measure on your own repositories.

Two different research groups, two different corpora, two different ecosystems, same conclusion. That is the part that should change your process.

The disagreement is in the specification, not the code

This is the finding worth sitting with. Earlier work had already observed that generators differ; what was unclear was whether the cause was implementation error or deliberate design choice. Mind the Gap’s answer is that most discrepancies are systematic, stemming from different assumptions about four things: dependency scope, naming, provenance and representation. A smaller set come from inconsistent support for fields the SBOM specifications do define.

Unpack those four and you can predict most of the divergence yourself:

Ambiguity	The question the spec leaves open
Scope	Do dev dependencies, optional dependencies, build-only tooling and transitive packages belong in the bill?
Naming	Which identifier names a component, and how is it normalised across ecosystems?
Provenance	What counts as evidence that a component is present — a manifest entry, a lockfile entry, a file on disk?
Representation	How are relationships between components encoded, and can a consumer reconstruct the graph?

The authors are explicit that many of these discrepancies cannot be fixed with simple patches and instead require clearer standardization, and they recommend that future SBOM standards define canonical rules for dependency scope, provenance and representation. That is a request for the specifications to say what the tools are currently guessing.

If this pattern feels familiar, it is the same shape we found when asking where version ordering is actually written down: every ecosystem behaves as if there is a shared standard, and the normative text turns out to be a library, a test file or nothing at all. Supply chain metadata keeps running into the same wall: the behaviour is universal, the specification is not.

The lockfile is the only honest baseline

Notice the methodological choice in Mind the Gap: the baselines were derived from dependency lockfiles, not from another SBOM. That is the right call, and it is a useful instruction for your own pipeline. Your lockfile is the artefact that decided what was installed; an SBOM is one tool’s interpretation of it, layered with assumptions about which of those entries deserve to be reported.

Which means the practical check is cheap. Export the dependency list your package manager already knows about, export the component list from your SBOM, and diff them. Most teams have never done this once. A JSON formatter and a text diff checker are enough to do it by hand the first time, before you decide whether it is worth automating.

Expect the gaps to run in both directions. A scanner reading files on disk sees things the lockfile never named; a scanner reading the manifest misses what was vendored or patched in. We have written before about how much of what ships in a published package is decided by packaging rules nobody reads, and about how little of a registry’s own metadata contract is formally specified. An SBOM inherits every one of those ambiguities.

One generator’s documented blind spots

You do not need a paper to see this effect; one of the three sources in the TU Munich dataset publishes its own limits. GitHub’s dependency graph documentation lists, per ecosystem, which files it reads — and the recommended ones are nearly all lockfiles, because they lock "the package versions to those included in the build" and so let Dependabot "find vulnerable versions in both direct and indirect dependencies."

Then read the caveats. For Python projects declaring dependencies in setup.py, GitHub says "we may not be able to parse and list every dependency in your project." GitHub Actions workflows count as manifests only when they "must be located in the .github/workflows/ directory of a repository to be recognized as manifests." And for actions, "alerts are only generated for actions that use semantic versioning, not SHA versioning" — so the pinning practice security teams spent two years promoting is the one that drops out of alerting.

None of that is a flaw. It is a documented set of design decisions, which is exactly the point: another tool made different ones, and the two outputs are both defensible. If you are evaluating scanners, the security and privacy category on TechLogHub lists the commercial options and our open-source listings cover the generators these studies actually used.

Why December 2027 turns this into a compliance problem

The Cyber Resilience Act is the reason this research exists. The European Commission’s own CRA page gives three dates: "The CRA entered into force on 10 December 2024", "with reporting obligations to apply as of 11 September 2026", and "The main obligations introduced by the Act will apply from 11 December 2027."

Worth noting how the Commission presents it: that overview page does not mention software bills of materials at all. The obligation lives in the legal text, where recital 22 describes market surveillance authorities being able to request from manufacturers "the software bills of materials (SBOMs) that they have generated pursuant to this Regulation." Mind the Gap frames the deadline as SBOMs becoming mandatory from December 2027, and spells out the consequence: because generation is becoming a legal compliance requirement, the choice of tool can shape the output and potentially cause undetected non-compliance.

"Undetected" is the operative word. An SBOM that is systematically incomplete still validates, still scores, still attaches to a release. Nothing fails. The reporting side of the same regulation already has clocks attached; the documentation side has the harder failure mode, because there is no alert for a bill of materials that is quietly wrong. If you are picking tooling now, the method in how to compare developer tools applies: compare outputs on your own repository, not feature lists.

What to do before the deadline

Treat the SBOM as a build output with a toolchain, not as a fact about your software. That means pinning the generator and its version in CI and recording both in the document, so a bill of materials from last March can be explained. It means running a second generator once, diffing the component sets, and writing down what the difference is — if you cannot explain it, you do not yet know what your SBOM claims.

Then decide your scope policy explicitly: dev dependencies in or out, build tooling in or out, and the same answer in every repository. The research says the tools disagree because nobody told them; your organization can at least tell them. Score the output with a quality tool like sbomqs so you learn which fields your generator omits before an auditor does, and keep the lockfile alongside the SBOM in the release artefacts, since it is the only thing that records what was actually installed.

FAQ

Which SBOM generator is the most accurate?

Neither study crowns one, and that is deliberate. The divergence traced back to different, defensible assumptions about scope, naming, provenance and representation rather than to correctness. The useful question is which tool’s assumptions match the claim you need to make, and whether you can state those assumptions when asked.

Does the CRA say I must produce an SBOM?

The regulation treats SBOMs as something manufacturers generate and may be asked to hand over: recital 22 refers to market surveillance authorities requesting the SBOMs manufacturers "have generated pursuant to this Regulation." The main obligations apply from 11 December 2027 per the Commission’s own page. For what your specific product class owes, read the legal text or take advice — this is not legal guidance.

Can I just use the GitHub dependency graph?

It is one of the three sources in the TU Munich dataset, so it is a legitimate input. Read its documented coverage first: lockfiles give the best results, setup.py parsing is explicitly best-effort, and Actions alerts only cover semantically versioned references rather than SHA-pinned ones.

How do I know whether my SBOMs disagree?

Generate one with two tools on the same commit, flatten both component lists, sort them and diff. Then diff each against your lockfile, which is the baseline the researchers used. Count what only one tool found, and categorise those by scope, naming, provenance or representation — that bucket tells you which assumption differs.

Will the specifications fix this?

Eventually, perhaps. Mind the Gap recommends that future standards define canonical rules for dependency scope, provenance and representation, and notes that many current discrepancies cannot be resolved with simple patches. Standards work on that scale takes years, and the compliance date is December 2027, so plan to document your own choices rather than wait.

---

An SBOM nobody can reproduce is a document, not evidence. Pin the tool, diff the output, write down the assumptions.