Can Radiology Reports Teach an MRI Model?

The competition has thousands of knee-MRI studies, but only a small number have complete expert labels for all twelve findings. It also has radiology reports.

That looks like an easy shortcut: read the reports, turn them into labels, train a model.

It is not that simple.

Radiology reports are rich, but they are not a tidy spreadsheet. They can be multilingual. A finding can be uncertain, historical, negated, implied, described in a different anatomical vocabulary, or simply omitted.

A report is evidence. It is not automatically ground truth.

This post is about turning that evidence into a training signal without erasing the uncertainty that makes it risky.

Weak supervision, in normal language

Weak supervision means using a label source that is helpful but imperfect.

For this project:

Expert label:   a carefully reviewed answer for a target finding
Weak label:     a useful hint inferred from a report
Missing label:  no safe training signal

The distinction matters because the model will learn whatever we present as truth. If every missing mention becomes “absent,” the model learns a systematic lie: that clinicians always write down every normal structure.

They do not.

1. Preserve what the report actually said

The first layer is a conservative rule system. It parses reports, recognizes sections, finds candidate evidence, and preserves the original source span internally for audit.

For each target, it tries to distinguish states such as:

This is why “not mentioned” deserves its own state. It is not the same as “normal.”

Interview concept — label masking: tell the loss function to ignore a target when the label is unknown. The model is then not punished for disagreeing with a blank.

2. Why multilingual reports make shortcuts dangerous

The corpus contains more than one language and report style. A simple keyword list breaks easily:

• a negation can reverse a finding;
• one term can refer to a history rather than the current scan;
• anatomy can be implied rather than named directly;
• sentence boundaries and headings may vary by language;
• generic bone-marrow language does not always mean traumatic contusion.

The engineering response was not “translate everything and trust the translation.” Translation could add another model layer and another opportunity to lose clinical nuance. Instead, the parser keeps the original text, uses multilingual section clues, and routes uncertainty to review rather than pretending language detection is perfect.

Learning: when the label source is already noisy, every extra transformation needs to earn its risk.

3. Use disagreement as information

Rules are precise but limited. A model can catch paraphrases that rules miss, but models can also sound confident while being wrong.

The compromise was a bounded consensus process:

Rule evidence + primary extractor
          ↓
Ambiguous cases only → second opinion
          ↓
Still disputed? → one bounded tie-breaker
          ↓
Agreement: retain cautiously
Disagreement: low weight or mask

The key word is bounded. A stronger model was not allowed to rewrite the entire dataset. It was used only where the earlier sources disagreed, and even then a majority decision stayed low-weight.

That is different from declaring one model an oracle. It treats model disagreement as a measurement of uncertainty.

4. What the final label set looks like

The resulting label set contains 34,577 trainable report-target pairs. Every one carries provenance and a reliability weight.

The model will not receive these tiers as a claim of clinical probability. They are training controls: stronger evidence influences the loss more; weak or missing evidence influences it less or not at all.

Interview concept — soft target: instead of a rigid 0 or 1, a training target can encode cautious confidence. That is useful for weak labels, but it must be calibrated later against held-out expert data.

5. The important thing we did not do

We did not say, “The language model labelled it, so it is correct.”

We did not train on every finding mentioned by any method. We did not treat report silence as normality. And we did not let a late tie-breaker upgrade a disputed label into high confidence.

That restraint is not a loss of ambition. It is protection against a failure mode that looks productive early and damages the image model later.

What needs to happen next

The report-derived labels are a hypothesis about useful supervision. The first image baseline must test that hypothesis:

1. train under the frozen split;
2. evaluate only against held-out expert labels;
3. compare label v1 and v2 under the same conditions;
4. inspect per-target failures, not only the headline average.

If the additional low-weight labels help, great. If they hurt a target, the system is designed so we can trace the provenance, revise the policy, and rerun a bounded experiment.

That is the goal of a label factory: not to create a permanent answer, but to make uncertainty auditable and improvements testable.

Next: The easiest way to “win” is to accidentally cheat →

This post shares aggregate methodology only. It contains no report excerpts, images, patient/study identifiers, private prompts, code, weights, or case-level labels. It is not medical advice or evidence of clinical validity.