Reachability is not confidentiality: the 2.3.1 exposure change

Summary

Earlier releases treated reaching a file as proof that its contents were sensitive. If a scan found a document through a browsable directory listing, the report carried a verified confidential-exposure finding. 2.3.1 changes that: reaching a file is now a listed_file observation, and only content-specific proof contracts promote it. Server-state exposures that are deterministic, such as a browsable directory index or a served /metrics endpoint, are still proven. The change lowered the Juice Shop benchmark's enforced recall from 0.44 to 0.33, and the maintainers chose to record that as a visible declared gap rather than hide it.

Problem

The 2.3.1 release notes describe the correction: "Reaching a file — including through a browsable directory listing or a well-known location such as RFC 9116 /.well-known/security.txt — is recorded as a listed_file observation, not a verified confidential finding. A filename, a path, or a directory entry is not proof the content is sensitive."

That matters for a product whose findings carry a proof state. A verified finding is supposed to mean deterministic evidence was collected, so an over-promoted exposure is not a cosmetic severity problem. It is a proof-contract violation that a downstream consumer, a CI gate, or a customer review would have relied on.

How it works

Observation by default. Any reachable file, whether from a directory listing, a well-known location, or content discovery, becomes a listed_file observation.

Promotion only through content contracts. The release notes list what promotes a file to a proven exposure: "the secret-material classes (private key, cloud credential, environment secret), a recovered version-control store, or a backup/source artifact recognized by shape."

Server state is still proven. "Exposures that are themselves deterministic server state are still proven: a browsable directory index and a served /metrics endpoint report at high, verbose error disclosure at medium." Framework-fingerprint and API-specification observations report at informational severity, and actuator detection "now keys on the Spring vendor media type instead of generic _links/status shapes." The notes close the section with: "Location alone never establishes sensitivity, on any target."

The narrow secret set is a contract. AGENTS.md describes the unauthenticated data_exposure proof as recognising "only a narrow, entropy-screened set of self-evident secret formats. JWTs, bearer tokens, SSNs, card numbers, and Google API keys are excluded because public endpoints may legitimately issue them or documentation samples match. A public value outside that set is an observation, not a verified exposure; widening the set is a proof-contract change, never a per-target tuning."

Benchmark consequence

The Juice Shop answer key includes exposed-confidential, a specific document reachable under /ftp. Under the corrected policy its confidentiality is not deterministically provable, so the fixture at v2.3.1 moves it to a declared gap:

• min_expected_recall drops from 0.44 (4 of 9) to 0.33 (3 of 9): sqli-search plus the two deterministic exposures, the /metrics endpoint and the /ftp listing.
• min_verified_high_critical is 3, with the comment "per-file confidentiality is no longer force-verified".
• max_known_expectation_gaps grows from 5 to 6.

The fixture's own words: "This is a characterized change to what the benchmark measures, not a blanket threshold cut: the answer key is unchanged, the new gap names its reason, and a NEW miss, a recall drop below the reconciled level, or a seventh gap still fails."

Boundaries

• The release notes state "No change to the deterministic proof boundary." The correction narrows what the exposure engine claims; it does not add a proof path.
• The other changes in 2.3.1 are unrelated to detection: non-destructive target archiving, evidence-deletion consistency with ownership receipts, atomic request-collection uploads, and release-hygiene checks that reject incomplete E2E scorecards.
• Release status at the tag: the release-notes index records the 2.3.1 candidate as "prepared 2026-09-09; not yet published", and the release-mapping table row for 2.3.1 was still pending for commit and image digests. The annotated tag v2.3.1 is dated 2026-09-11. The stable installer channel advances only after publication and a public smoke test.

Limitations

• A genuinely confidential document that is merely reachable now surfaces as an observation. Reviewers must open listed files rather than trust a severity label. This trades benchmark recall for precision on purpose.
• The proof contracts for secret material, version-control stores, and backup artifacts are shape-based. A sensitive file of an unrecognised shape is not promoted.
• This article quotes the release notes and fixture; it did not re-run the benchmark before and after the change.

Product implications

• Treat listed_file observations as a review queue. The findings model keeps severity, proof state, and lifecycle as separate dimensions for exactly this case.
• CI consumers that gate on verified high findings get fewer false promotions from directory listings. The trade is visible in the benchmark fixture rather than absorbed silently.
• The same rule applies to AI-assisted work: a Hunt candidate that says "this file looks sensitive" is an observation until a proof contract confirms it.

References

• Release notes 2.3.1
• Release notes index
• Juice Shop release fixture at v2.3.1
• Agent operating policy: findings, evidence, and proof
• Benchmark methodology article
• Scan product page