One clinician, 200 lesions
Magnus Lynch is a consultant dermatologist and Mohs surgeon at Guy's Hospital in London and an Honorary Reader at King's College London. On October 25 he published a test he designed himself on the Lynch Lab blog: 200 skin lesions, given to OpenAI's GPT-6 Astra, Anthropic's Claude Fable 5.1, and an Inception v3 fine-tuned on public skin images. Inception v3 is a convolutional neural network (CNN) Google published in 2015.
The cases come from MILK10k, a public dataset released through the International Skin Imaging Collaboration (ISIC) in 2025, in which every lesion has a clinical close-up and a dermoscopic image and was confirmed by biopsy. The 100 lesions that should not be discharged were 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas and 20 actinic keratoses; the 100 safe to discharge were 50 benign moles and 50 seborrhoeic keratoses. The author notes that even the benign lesions were biopsied, meaning they once worried a clinician, so the set is harder than an ordinary clinic.
Both LLMs got the same two images and the same prompt at default settings, and had to give a first-choice diagnosis out of six, a confidence, a second choice, and whether the lesion could be discharged without treatment; neither refused a case. The CNN was fine-tuned for about twenty minutes on a single H100 using 68,011 images from the ISIC Archive, with the test lesions and every other photo of the same lesion or patient removed from training and validation data beforehand.
Where the gap is
First-choice accuracy: the CNN 125 of 200 (62%), GPT-6 Astra 84 and Claude Fable 5.1 85 (42% each). Allowing a second choice, the CNN reached 173 against 134 and 138. Of the 30 melanomas, the CNN named 13 as first choice, GPT-6 Astra 8 and Claude Fable 5.1 11.
The biggest gap was on the harmless lesions: of 50 moles, the CNN recognised 37, GPT-6 Astra 16 and Claude Fable 5.1 33; of 50 seborrhoeic keratoses, 26, 13 and 6. The CNN did not win everywhere. GPT-6 Astra got all 25 basal cell carcinomas, one more than the CNN, and both LLMs beat the CNN on actinic keratosis.
The author also points out that each LLM had a favourite answer: GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the melanomas, and Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were. Asked in a patient's voice instead, with only the clinical photo and the question "do I need to see a doctor?", both advised seeing a doctor in almost every case, harmless lesions included; the figure given is 99 and 98 of a further 100 biopsy-proven melanomas. A referral for nearly everything carries little triage value.
The limits the author lists
The post includes its own caveats: 200 lesions are enough to show the pattern but not to rank close results, and 30 melanomas leave wide uncertainty; each LLM case was run once, and answers vary between runs; MILK10k has been public since 2025, so the LLMs may have seen it in training (which, the author argues, would only have helped them); and Inception v3 is a baseline, while state-of-the-art skin cancer systems and regulated products perform considerably better. The test has not been peer reviewed.
The author discloses that the analysis and first drafts were produced with Claude Code, and that one of the two tested models is also Anthropic's. The cases, prompts, every LLM reply and the scoring code can be downloaded, and the CNN runs locally in a phone browser through WebGPU. For teams considering general multimodal models for first-pass medical image screening, the numbers suggest that a small model trained for a narrow task still holds a clear edge, and that a cautious "see a doctor" answer is not the same as a useful judgment.
via: Lynch Lab: A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis