Auditing Reproducibility in Non-Targeted Analysis: 103 LC/GC--HRMS Tools Reveal Temporal Divergence Between Openness and Operability
Abstract
In 2008, melamine in infant formula forced laboratories across three continents to verify a compound they had never monitored. Non-targeted analysis using LC/GC-HRMS handles these cases. But when findings trigger regulatory action, reproducibility becomes operational: can an independent laboratory repeat the analysis and reach the same conclusion? We assessed 103 tools (2004-2025) against six pillars drawn from FAIR and BP4NTA principles: laboratory validation (C1), data availability (C2), code availability (C3), standardised formats (C4), knowledge integration (C5), and portable implementation (C6). Health contributed 51 tools, Pharma 31, and Chemistry 21. Nine in ten tools shared data (C2, 90/103, 87%). Fewer than four in ten supported portable implementations (C6, 40/103, 39%). Validation and portability rarely appeared together (C1+C6, 18/103, 17%). Over twenty-one years, openness climbed from 56% to 86% while operability dropped from 55% to 43%. No tool addressed food safety. Journal data-sharing policies increased what authors share but not what reviewers can run. Tools became easier to find but harder to execute. Strengthening C1, C4, and C6 would turn documented artifacts into workflows that external laboratories can replay.
Summary
This paper addresses the critical issue of reproducibility in non-targeted analysis (NTA) using LC/GC-HRMS, which is crucial for identifying unexpected contaminants in areas like food safety and pharmaceuticals. The authors argue that while analytical performance is important, the ability for an independent laboratory to repeat an analysis and reach the same conclusion is paramount, especially when findings lead to regulatory action. To assess this, they conducted an architectural audit of 103 LC/GC-HRMS tools published between 2004 and 2025, evaluating them against six pillars derived from FAIR and BP4NTA principles: laboratory validation (C1), data availability (C2), code availability (C3), standardized formats (C4), knowledge integration (C5), and portable implementation (C6). The study's key findings reveal a temporal divergence between "openness" (data and code availability) and "operability" (validation and portable implementation). While openness has significantly increased due to journal data-sharing mandates, operability has stagnated or even declined. Specifically, data availability reached 87.38%, and code availability reached 75.73%. However, portable implementation remained the least common pillar at 38.83%. No tool addressed food safety. This creates a significant gap, as regulatory-grade reproducibility requires tools that are not only findable and accessible but also easily executable by independent labs. The authors conclude that strengthening validation, standardization, and portability is essential to transform documented artifacts into workflows that can be reliably replayed across different laboratories and jurisdictions.
Key Insights
- •Over the 21-year study period (2004-2025), openness (C2, C3, C5) increased from 56% to 86%, while operability (C1, C6) decreased from 55% to 43%, resulting in a widening gap of 43.7 percentage points.
- •Code availability (C3) showed the largest increase, rising from 42.1% in 2004-2015 to 85.2% in 2020-2025, indicating the impact of journal data-sharing policies.
- •Validation (C1) declined from 68.4% to 50.8% over the same period, suggesting that newer tools are less likely to report validation despite increasing analytical complexity.
- •The co-occurrence of validation and portable implementation (C1 x C6) was the lowest at 17.48%, highlighting a disconnect between internal assurance and external transferability.
- •Tools supporting standardized formats (C4) were nearly four times more likely to integrate external databases (C5) (OR = 3.98, 95% CI: 1.59–9.92, p = 0.0023), demonstrating shared design priorities in data interoperability.
- •Containerization (Docker, Singularity) was a strong correlate of compliance, with containerized tools achieving a mean score of 4.87 pillars versus 3.64 for non-containerized tools (p = 0.0003).
- •No tool among the 103 assessed specifically addressed food safety applications, highlighting a significant gap in this critical area.
Practical Implications
- •The six-pillar framework (C1-C6) provides a concrete checklist for developers to improve the reproducibility and usability of NTA tools.
- •Regulators can use the framework as procurement criteria to select tools that meet regulatory-grade reproducibility standards.
- •The study highlights the need for journal policies to move beyond data and code availability to also encourage or require portable implementation and validation.
- •Future research should focus on developing food-safety-specific NTA platforms that prioritize laboratory validation (C1), standardized I/O (C4), and portable implementation (C6).
- •A multi-site ring trial is suggested to determine if higher pillar scores predict better cross-laboratory agreement.