Contact

Method

Publication level beats audit count

·The Openscore VPN team

transparency · audits · methodology

The number of audits a provider claims is the easiest figure on its homepage to inflate and the hardest to check. So we do not score it. We score what a reader can actually open.

In our schema an audit is never a tally mark. Each one carries a scope, what was examined: a no-logs configuration, the infrastructure, the client applications, a browser extension, or the protocol, and, decisively, a publication level. There are four, and the distance between them is the whole story:

  • full-report-public, the complete report is published and anyone can read the findings, the method, and the caveats.
  • full-report-gated, the full report exists but sits behind a login or a request form.
  • summary-only, a page describes an audit; the report itself is not available.
  • claimed-unpublished, an audit is asserted to have happened, with nothing to read at all.

A marketing line that reads “independently audited five times” collapses all four into a single boast. Five summary-only or claimed-unpublished entries are five statements about audits, not five audits you can inspect. One full-report-public report you can open and disagree with is worth more than all of them, because it is the only one of the set that is actually evidence.

Why this falls straight out of the evidence rule

The site’s first rule is that a fact reaches a scoring rule only if it cites a primary source. A published audit report is a primary document: we read the finding out of it and cite the page it appears on. A provider’s summary of an audit is a different kind of object, it is the provider characterising a document we cannot see. Scoring it as though we had read the audit would be scoring the marketing, and it would launder the provider’s own account of the evidence into a number that looks independent.

This is the same instinct that makes us refuse to cite competing review sites: a conclusion we cannot trace to a source we read ourselves is someone else’s claim wearing our typeface. A summary of an audit is a claim; the report is the source.

What the score does with it

Publication level therefore drives the transparency contribution, not the count. A full, public report earns real credit. A gated one earns less, and says so, because a reader cannot verify what a login stands between them and. A summary alone and an unpublished claim earn little to nothing: there is no primary document behind them for us to cite, so under the evidence rule there is barely a fact there to score.

The effect is that a provider cannot buy a transparency score by accumulating audits it will not show you. The one lever that moves the number is the one that also lets you check our work: publish the report. That is the behaviour the score is built to reward, and the only one it can honestly measure.

Every claim in this note rests on the site's published method. Read it in full on the methodology page, or return to the notebook.