RxScribe-Bench
A multi-axis benchmark for evaluating vision-language models on Indian outpatient prescriptions, with four axes tied to clinical severity instead of one blended accuracy number.
Correctness
in %Hallucination
total · lower betterEngagement
in %Robustness
in %Introducing RxScribe-Bench
Outpatient prescriptions can be notoriously hard to interpret, especially in the Indian context. The challenge goes beyond illegible handwriting: dense shorthand, unstructured layouts, mixed languages, and terms that at times only the prescribing doctor and a neighborhood pharmacist may fully understand can make misinterpretation common and consequential. Errors in reading medicine names, dosages, and instructions are common for both humans and AI models. Besides misreading, AI models may generate information not present in the original prescription.
We assembled and de-identified a dataset of 200 real prescriptions from Indian outpatient clinics to evaluate vision-language models. These prescriptions often combine multilingual notes, abbreviations, unstructured layouts, and local terms that make interpretation tough.
Read ‘500 mg’ as ‘50 mg’ and the patient takes a tenth of the intended dose. Read ‘Dr Sharma’ as ‘Dr Sharme’ and nothing happens. The standard approach to evaluating prescription OCR uses a single error rate, so it scores those two mistakes the same. RxScribe-Bench addresses this by measuring not just the presence but also the nature of errors.
Most benchmarks focus only on extraction accuracy and do not distinguish between models that fabricate information and those that admit inability to interpret a field.
We evaluated four advanced vision-and-language models from OpenAI, Google, Anthropic, and Meta. Each model received a handwritten prescription and a JSON schema to return a structured record. No OCR pre-processing, retrieval tools, or multiple attempts were provided. We ran three attempts per model per image, totalling over 2,400 scored runs.
Instead of a single accuracy metric, we evaluate each model along four axes: Correctness, Hallucination, Engagement, and Robustness. Each axis represents a distinct failure mode.
Anatomy of a prescription
| Tab Azithromycin | 500mg | 1-0-0 | 3 days | Before food |
| Syr Ambrodil-S | 2 tsp | TID | 5 days | Blank |
| Tab Montelukast | 10mg | 0-0-1 | 7 days | After dinner |
Note: All details shown in this image, including names, registration numbers, dates, and content, are for illustrative purposes only and do not represent any real person, patient, or medical facility.
How each model reads
Fewer hallucinations makes a longer bar, so every axis points the same way. The unscaled numbers and ranks are in the panel below.
Claude Opus 5 has the highest attempt rate on clear fields (89.5%) and sits within half a point of the top on hard fields (80.1%), making it uniformly engaged. However, it trails correctness and attempted-only robustness versus the others. On absent fields, where silence is correct, it fills in the most, making it the least conservative. Claude Opus 5 combines the highest clear-field engagement with the lowest accuracy.
Throughout the evaluation, none of the models performed consistently. GPT-5.6 achieved the highest reading accuracy but exhibited the slowest processing speed. Claude Opus 5 processed hard fields more quickly but also had the lowest correctness scores. Muse Spark 1.2 prioritised safety by skipping hard fields, resulting in the lowest engagement score.
Task curation
The prescription images used in this benchmark were collected after appropriate approvals from participating clinics and prescribing clinicians. All personally identifiable information (PII) was manually identified and redacted using a Safe Harbor de-identification process before inclusion in the benchmark.
Each image was independently annotated by two annotators, whose extractions agreed exactly on 96% of fields across the complete dataset. Every field also carries one of four legibility labels: Clear, Partial, Blur, or Not Visible, which later sections use to separate hard fields from clear and absent ones.
The four axes
Once a model prediction and ground truth are aligned, evaluation becomes less about whether a model was wrong and more about how it was wrong. We found that prescription-reading errors fall into four distinct categories: misreading, fabrication, disengagement, and sensitivity to difficult handwriting. Each category becomes one axis of evaluation.
Correctness
Correctness checks if the model read the field and filled it in correctly. We grade medicine-table fields as pass or fail. A model that reads ‘Paracetamol 500 mg’ as ‘Paracetamol 50 mg’ is simply wrong, as changing one digit turns it into a different dose. Clinical and non-clinical fields use ANLS, so a close reading can still earn partial credit. If a model produces ‘Dr Sharma’ as ‘Dr Sharme’, for example, it can receive partial credit because the error is unlikely to affect patient care. For medication, a near-miss on a dose isn't close enough.
These score labels are illustrative abstractions of the grading rules, not scores from a specific benchmark sample.
{ "medicine_duration": "7 days" }{ "medicine_freq": "BID" }{ "prescriber_reg_no": "DMC/R/48271" }Pooled F score from Table 3. Medicine-table fields use pass/fail grading. The Clinical + non-clinical series pools the clinical tier (diagnoses, symptoms, vitals, history, allergies, and recommended tests) with the non-clinical tier (prescriber, facility, and document fields). It is not non-clinical-only, and the clinical tier is not reported on its own. Overall is the full-benchmark score, so the three series are not additive. GPT-5.6 leads overall and on Medicine-only; Gemini 3.1 Pro leads the pooled Clinical + non-clinical split.
Hallucination
Hallucination counts anything a model asserts that isn't supported in the source document. However, this has one exception: active ingredients. It is something the model infers using its pharmacological understanding, and that is not present in the source document itself. A model has to infer them from the brand name. The hallucination axis does not count that inference as a fabrication. It only counts it when the inference is wrong. We split it into three severity tiers because an invented medication is a very different problem from an invented clinic address.
These score labels are illustrative abstractions of the grading rules, not scores from a specific benchmark sample.
{ "salt": "Fluticasone Propionate / Oxymetazoline" }{ "hospital": "ABC clinic" }{ "diagnosis": "Acute URTI" }Hallucination events across 600 runs per model from Table 4. Lower is better. Muse Spark 1.2 records the fewest events in every severity tier, at 2.3 per run. Claude Opus 5 records the most, at 3.7 per run.
Engagement
When a model declines to fill in a field when there is nothing to read, it abstains from engaging with the task on valid grounds. Engagement score tracks how often a model does that, and on what grounds. There are two ways to disengage: first, when there is nothing to read; second, when a field is blurred or partially legible.
We penalise a model if a human annotator can recover a field to some extent, but the model cannot produce the same output. Leaving such a field blank is treated as a miss, not a careful response. Fields fall into three groups:
n = 1,740The rate at which a model attempts a field with difficult handwriting rather than skipping it.Should attempt
n = 19,665The same rate on legible content, which acts as a baseline.Should attempt
n ≈ 11,215Where there's genuinely nothing on the sheet and silence is the correct answer.Should stay quiet
These score labels are illustrative abstractions of the grading rules, not scores from a specific benchmark sample.
{ "gender": "[blank]" }{ "facility_name": "A-3, Gandhi Apartment" }{ "heart_rate": "90" }Attempt and appropriate-silence rates from Table 5. Hard and clear fields should be attempted. Absent fields should remain blank, so higher is better for all three series. The absent-field series restores the restraint measure that an attempt-only view misses.
Robustness
Robustness shows how a model scores when the handwriting gets illegible or unreadable. This score was added to check a model's performance in hard fields versus clear fields. So, during annotation, we labelled every blurred or partially legible field as “hard”. Keeping hard fields separate helped us identify models that are actually robust. It is published twice: a delivered form that charges every skipped hard field as a zero, and an attempted-only form computed over just the fields a model chose to answer.
Hard-field scores under the two accounting rules in Table 6. Delivered score counts every skipped hard field as zero. Attempted-only score excludes skipped fields, which rewards selective answering. Gemini 3.1 Pro leads the delivered score.
Stability and efficiency
Two modifiers sit alongside the axes, and neither lines up with the quality ordering. GPT-5.6 is the most accurate model in this pool and also the steadiest from run to run, while Gemini 3.1 Pro, second on average correctness, is the least steady of the four. Claude Opus 5 and Muse Spark 1.2 each hit a complete zero on at least one run, which fixes their spread at its maximum.
| Model | Correctness meanMean | Std | Min scoreMin | Max scoreMax |
|---|---|---|---|---|
| Claude Opus 5 | 61.7% | 14.6% | 0.0% | 94.5% |
| Gemini 3.1 Pro | 67.0% | 15.5% | 13.0% | 97.0% |
| GPT-5.6 | 68.1% | 14.5% | 23.2% | 98.3% |
| Muse Spark 1.2 | 64.5% | 14.8% | 0.0% | 93.9% |
Muse Spark 1.2 is the cheapest model per run and Claude Opus 5 the fastest, while GPT-5.6 is the slowest and by far the most expensive, at roughly six times Muse Spark 1.2's cost per run, driven by per-token price rather than output length. Claude Opus 5 and Muse Spark 1.2 share the only parse failures in the pool: for each, all three cold runs on the largest image in the set failed to parse and were scored zero rather than excluded.
| Model | Time/runTime | Cost/runCost | Total costTotal | Tokens in/outTokens in/out | Parse errorsParse errors |
|---|---|---|---|---|---|
| Claude Opus 5 | 40.9 s | $0.1233 | $73.97 | 9,235 / 3,085 | 0.50% |
| Gemini 3.1 Pro | 52.9 s | $0.0696 | $41.76 | 6,249 / 4,888 | 0% |
| GPT-5.6 | 153.4 s | $0.2416 | $144.97 | 9,257 / 7,090 | 0% |
| Muse Spark 1.2 | 109.3 s | $0.0401 | $24.07 | 6,787 / 7,588 | 0.50% |
Conclusion
RxScribe-Bench's four-axis framework divides outputs into different failure modes. A high hallucination count indicates insufficient pharmacological knowledge, while low engagement suggests avoidance of hard-to-read data. A decline in robustness reflects selective processing of easier fields. Our dataset is stable enough to compare runs across model versions, detailed enough to isolate which change helped, and hard enough that no current frontier model saturates it.
One qualification matters when reading the pooled clinical and non-clinical score. Many non-clinical fields, such as prescriber and clinic details, are printed on the prescription and are easier to extract than handwritten medicine-table fields. A high pooled score should not be treated as a direct measure of medicine-table reading. The medicine-only split is the better signal for medication transcription, while the overall score covers the complete benchmark schema.
This fragmentation, rather than any leaderboard position, is the finding: reading, avoiding invention, engaging with hard content, and degrading gracefully are separable skills that no model masters uniformly. This evaluation should not be read as a settled ranking, or as more certain than 200 prescriptions can support.