Preprint, not yet peer reviewed
PhysDox: Benchmarking LLMs on Physical Feasibility Auditing of Physiological Sensing Protocols
Language models are increasingly asked to help design experiments, and they write plans that read well, yet a fluent plan can still be physically impossible to carry out.
Why it matters Language models that help plan lab work often miss the physical reasons a written experiment cannot succeed, especially limits nobody states outright.
The researchers built a test of 683 expert-checked plans for measuring signals from the human body, each graded as sound, slightly flawed or fatally flawed, and asked six models to judge them. The best reached a score of only 53 out of 100 at spotting the problems. The models tended to mistake a plan that lists every step for one that would work, and they missed unstated physical limits about twice as often as plain hardware errors. The authors conclude that checking feasibility takes real physical reasoning rather than recall.
Liu et al., arXiv cs.AI, 16 November 2026.

















