The reproducibility crisis is not unique to machine learning, but ML introduces specific new failure modes that the physical sciences must confront: data leakage between training and test sets, benchmark overfitting through repeated evaluation on the same held-out data, undisclosed preprocessing and hyperparameter tuning choices, and the conflation of held-out test performance with real-world generalisation capability. These problems are amplified in the physical sciences, where datasets are often small, measurements are expensive, and the gap between published benchmarks and laboratory deployment can be substantial.

This workshop develops good practice for the full scientific ML lifecycle, covering data curation, splitting strategies appropriate for chemical and materials data (where naive random splits can introduce systematic leakage), hyperparameter tuning protocols, and reporting standards. It examines benchmark design and its pathologies, using the QM9 molecular dataset and the Materials Project as case studies that illustrate both the value and the limitations of community benchmarks.

The session also addresses practical infrastructure for reproducible research: version control (Git), data ver- sioning (DVC), experiment tracking (MLflow, Weights & Biases), and community platforms for sharing models, data, and code (Hugging Face, Zenodo, Materials Cloud). Participants will leave with a concrete workflow for conducting and reporting ML research to the standards expected by the physical science community.

Extended Key Takeaways & Outputs

• Identification of ML-specific reproducibility failure modes in the physical sciences

• Practical data splitting strategies that prevent leakage in chemical and materials datasets

• Understanding of benchmark design, pathologies, and the gap between benchmarks and deployment

• Working knowledge of version control, experiment tracking, and model-sharing infrastructure

• Concrete workflow for reproducible ML research in the physical sciences