Trust & Reliability Center
Measured on a held-out half of a labelled corpus; thresholds are chosen on the other half. Jev reads text about a photo, not the pixels. These results may not cover every live decision or deployment.
228 authored scenarios · 0 labelled ledger photos
Every case was written from the rubric text by the project's developers with an AI assistant, not by independent labellers. None is a real field photo, so these figures show how Jev reads clear and borderline descriptions, not its accuracy on real submissions.
Each held-out set has fewer than 100 cases, so every rate carries a wide 95% interval (shown on each card). ECE depends on binning and sample size; the Brier score complements it, and neither is ground truth.
Evidence Relevance
Distinguishes physical fieldwork evidence from screenshots, people-only photos, and unusable images, from text about the photo.
On 59 held-out cases, Jev’s top answer was right 100.0% of the time (95% interval 94%–100%).
Threshold 0.73 was chosen on 61 separate tuning cases. Below it, decisions go to the human review queue.
Held-out cases Jev got wrong (0)
None in this held-out set.
Site Identification
Matches a photo to a registered site from its text and GPS distance, or answers none.
On 31 held-out cases, Jev’s top answer was right 83.9% of the time (95% interval 67%–93%).
Threshold 0.96 was chosen on 29 separate tuning cases. Below it, decisions go to the human review queue.
Held-out cases Jev got wrong (5)
- plot-b → noneat 81%No GPS, but the text names a feature that only Plot B has.
- plot-b → noneat 79%No GPS, but the text names a feature that only Plot B has.
- plot-b → noneat 77%No GPS, but the text names a feature that only Plot B has.
- plot-b → noneat 75%No GPS, but the text names a feature that only Plot B has.
- plot-c → plot-bat 62%GPS is ambiguous between two plots; only Plot C's description matches the text.
SRC Site Response Grade
Jev scores the site response using available captions and CV measurements against the project rubric.
On 17 held-out cases, Jev’s top answer was right 88.2% of the time (95% interval 66%–97%).
Threshold 0.67 was chosen on 31 separate tuning cases. Below it, decisions go to the human review queue.
Held-out cases Jev got wrong (2)
- 1 → 0at 92%Only lighting changed.
- 1 → 2at 55%Only lighting changed; brightness alone is not progress.