PO-40 | Evaluating agreement between physician and ChatGPT in migraine prophylaxis selection: a pilot comparative study

Simone Vigneri1, MatteoZanella2, Gianfranco Sindaco2, Alberto Merlin2, Maria Mazzullo2, Marco La Grua2, Valentina Paci2, Carmen Bertone2, Gilberto Pari2 1Santa Maria Maddalena Hospital, Neurology, Occhiobello, Italy; 2Santa Maria Maddalena Hospital, Pain Medicine, Occhiobello, Italy

Background: To evaluate the level of agreement between human clinicians and an artificial intelligence (AI) model (ChatGPT) in recommending first-line prophylactic therapies for patients with migraine, and to assess the potential clinical value of AI-generated support.

Methods: We analyzed 21 consecutive patients with a confirmed diagnosis of episodic migraine. For each patient, the treating clinician selected a first-line preventive treatment among four options: amitriptyline, propranolol, topiramate or flunarizine. Independently, ChatGPT was prompted with identical anonymous clinical data to generate therapeutic recommendation. Primary outcome was the agreement rate between physician and AI choices (molecule and dose). Statistical agreement was assessed via Cohen"s Kappa. Secondary endpoints included changes in pain intensity (NRS) at 3-month follow-up, adverse events, and treatment modifications.

Results: The observed agreement between the physician and AI treatment selection was 71.4%. Cohen"s Kappa was 0.564 (z = 3.75; p < 0.001), indicating a statistically significant moderate agreement beyond chance. The average reduction in NRS at 3 months was −3.38 points.

Conclusion: This pilot study demonstrates that ChatGPT can produce clinically coherent prophylactic treatment proposals in the management of migraine, with a statistically significant moderate agreement with experienced pain physicians. Further validation on larger cohorts is warranted to explore its potential as a clinical decision support tool.