XGBoost, ALBERT, BioBERT, Med‑LLaMA evaluated for...

Pharmacovigilance is not a game of scale. It is a game of precision. When millions of adverse event reports flood databases, the task of linking a drug to a harm demands more than raw parameter counts, it demands models that understand clinical language.

The InferBERT framework offered a controlled laboratory for this question: which architecture best uncovers causal signals in drug safety data? The answer, as a rigorous 5‑fold cross‑validation repeated over 20 runs reveals, is neither the simplicity of XGBoost nor the brute force of Med‑LLaMA, but the domain‑tuned middle ground. Four models faced off.

XGBoost, a sturdy gradient‑boosted baseline. ALBERT, the original InferBERT backbone. BioBERT, a transformer steeped in biomedical literature.

And Med‑LLaMA, a medical large language model fine‑tuned with parameter‑efficient methods. Accuracy, calibration error before and after isotonic regression, and Jaccard concordance with three traditional pharmacovigilance signals, PRR, ROR, EBGM, were measured. Statistical significance was not assumed; paired t‑tests verified every comparison.

BioBERT emerged as the clear winner. It achieved the highest accuracy on both datasets, and its causal term concordance with the old‑guard signals outperformed all rivals. Med‑LLaMA, despite its enormous size, underperformed.

Calibration adjustments improved Expected Calibration Error, but the cure was not clean, accuracy and causal discovery sometimes suffered. The lesson is blunt: bigger is not better, and generic pre‑training wastes potential. Domain‑specific pre‑training, even on a model much smaller than a general‑purpose LLM, delivers superior signal extraction for pharmacovigilance.

The path forward is not scaling up indiscriminately, but investing in manageable, aware architectures that speak the language of medicine.