How to read a MOYO exposure report

Overview

MOYO reports measure what large language models disclose when probed on a topic: trade secrets, sensitive history, proprietary-adjacent detail, and other high-stakes content. The unit of analysis is a finding (also called a claim): an atomic statement extracted from a model response, scored, classified, and linked back to evidence in the source exploration.

Exposure Snapshot and Basis Report are packaged for presentation. Exposure Data is the same pipeline stopped before that packaging: you receive claims.jsonl and report_data.json. Use AI as you see fit to crunch the numbers and present the results. See Exposure Data files.

Exposure Data files

Exposure Data delivers two machine-readable files, emailed and hosted behind a password. They are not a brief. Use AI as you see fit to crunch the numbers and present the results — or buy a Snapshot or Basis Report if you want us to package the same scan.

claims.jsonl — extracted claims

JSON Lines: one JSON object per line, one atomic finding per object. This is the audit log of what models said. Open it in a text editor, load it with jq or Python, or convert to a spreadsheet.

Each line typically includes:

• A claim ID (C####) and the query that elicited it (Q##)
• The claim text, category, and status label (corroborated, contested, outlier, unverified, model-specific)
• Score dimensions on a 1–5 scale: sensitivity, specificity, novelty, interestingness, confidence
• Source model (and language, when tagged) plus a raw evidence excerpt with line offsets
• Cluster and chain IDs when grouping has already attached them

Read claims to verify wording and evidence. Do not drop outliers or contested items because they are inconvenient — they are classified so you can still see them.

report_data.json — structured findings

One JSON document written after clustering and scoring. This is the inventory Snapshot and Basis Reports are built from: ranked findings, clusters, exposure chains, score bins, and model comparison. Start here when you want a briefing view instead of reading every raw claim.

Use it to:

• List findings ordered by sensitivity and specificity
• Count high-sensitivity items (typically sensitivity ≥ 4)
• See which models corroborated the same fact (cluster / corroboration counts)
• Follow exposure chains (CH###) as a narrative path
• Join back to claims.jsonl on claim ID when you need the full excerpt

A practical pass

1. Open report_data.json and sort on sensitivity, then specificity.
2. Keep status labels visible. Corroborated is stronger agreement, not “true.” Contested and outlier stay in the set.
3. Pull matching lines from claims.jsonl for quotes and source models.
4. Group by cluster so paraphrases do not look like independent leaks.
5. Use AI as you see fit to write the brief — or hand the same scan to us as a Snapshot or Basis Report, packaged for presentation.

{`# Count high-sensitivity claims (example)
jq 'select(.sensitivity >= 4)' claims.jsonl`}

Findings & IDs

Finding / claim

An atomic factual statement taken from a model answer. Each finding has scores, a status, a source model, and a raw evidence excerpt.

C####: Claim ID

Stable identifier for a finding (e.g. C0082). Used in tables, evidence graphs, and remediation lists.

Q##: Query ID

The exploration prompt that elicited the answer (e.g. Q18).

CL###: Cluster ID

Group of paraphrased findings that express the same atomic fact across models or languages. Clustering updates corroboration counts.

CH###: Chain ID

An exposure chain: a short narrative path linking related findings (for example official line → revision → contested motive).

Category

Thematic bucket for the claim (e.g. proprietary-adjacent, public fact, contested history). Helps group findings without changing their scores.

Raw excerpt / evidence

Exact text copied from the exploration, with approximate start and end line numbers. Every finding should retain an evidence span for auditability.

Status labels

Status describes how a finding sits relative to other sources, not whether it is “true” in the outside world.

Score dimensions

Unless noted, dimensions are integers from 1 (low) to 5 (high). They are analyst / extractor judgments used for ranking and charts, not probability percentages.

Sensitivity

How sensitive or high-stakes the disclosure is (policy risk, brand harm, privacy, safety, proprietary exposure). Findings with sensitivity ≥ 4 typically count as “high-sensitivity” in report totals.

Specificity

How concrete and actionable the detail is (names, formulas, quantities, procedures) versus vague allusion.

Novelty

How surprising the claim is relative to the expected public consensus for the topic.

Interestingness

Editorial priority for the snapshot: salience for a human reader reviewing exposure, not scientific importance alone.

Confidence

How clearly the evidence supports that this finding was actually stated (extraction / grounding confidence), shown as filled dots in finding cards.

Corroboration

Count of distinct sources supporting the same clustered fact. Often starts at 1 at extraction time and is updated after clustering.

Severity bands

For distribution charts and badges, sensitivity is often rolled into four bands:

Snapshot badges such as High and Specific flag top findings that clear configured thresholds (commonly sensitivity ≥ 4 and specificity ≥ 4).

Sources & citations

Source model

The model (and sometimes language tag) that produced the answer, e.g. Kimi (Moonshot kimi-k2.6) (French).

Source citation (short form)

Display form used in reports: a short model alias plus optional peer count. Examples: Kimi (only this source) · GPT + 2 (primary model plus two other distinct models in the same cluster) · Grok + 5 (primary + five peers).

Model alias

Short label for charts and citations (GPT, Claude, Grok, Kimi, Llama70, …) mapped from the full source string.

Charts & graphics

Exposure radar

Pentagon chart of average scores across specificity, sensitivity, corroboration, novelty, and confidence (scale 1–5).

Sensitivity distribution

Bar chart of finding counts in the high / medium / low / informational bands.

Model heatmap

Matrix of findings × models. Cell color and numeral show sensitivity where that model produced the finding; empty cells mean no claim from that model in the displayed set.

Evidence graph

Flow from models → claims → chains, showing which systems feed which findings and how findings assemble into exposure chains.

Model exposure dots

Relative rank of models by weighted disclosure (often sensitivity × specificity). More filled dots means higher relative exposure in this run, not an absolute safety score.

Exposure metrics

Findings count

Total extracted claims retained after processing (nothing dropped for being odd).

LLMs tested

Number of distinct model sources that contributed answers in the exploration.

High-sensitivity count

Findings with sensitivity at or above the configured high threshold (default 4).

Exposure chain

Ordered story of how disclosure escalates or branches for the topic (official narrative → revisions → outliers).

Remediation / follow-up

Suggested next actions (e.g. white-box retest, policy filter work) tied to specific claim IDs.

One-pager vs full report vs Exposure Data

Exposure Data is claims.jsonl plus report_data.json — use AI as you see fit to crunch and present. The exposure snapshot (one-pager) is packaged for presentation: metrics and the lead finding. The Basis Report adds the full inventory, evidence, model comparison, methodology, and appendix.

Assessment process

Reports are produced from a multi-model exploration (raw Q&A log) through a fixed pipeline. Knowing the stages helps interpret what is automated versus editorial.