October 1, 2025

October 1, 2025

Present

• Armstrong Foundjem, Ben Hawks, Geoffrey Fox, Gregg Barrett, Gregor von Laszewski, Juri Papay, Marco Colombo, Matt Sinclair, Piotr Luszczek, Satoshi Iwata

Tentative Agenda

• Any New Members Introduction
• Continuing discussion of New Benchmarks and the catalog of Science benchmarks
• Summary Fox_Benchmarking for HPC AI in a rapidly changing world Aug 26, 2025
• All AI for Science https://mlcommons-science.github.io/benchmark/
  https://github.com/mlcommons-science/benchmark
• AI Benchmark Assistant Sept30Version9-21-22-23-24DeepCollector
• Time Series
  Time Series Resources Summary Table
  COMBINED2
• White Papers
• The Benchmark carpentry white paper https://www.overleaf.com/9828764221czxzxxcxmcrr#1f1c84
• New white paper on Science Benchmarks
• Any Other Business

Google meeting notes

• MLC Science WG - 2025/10/01 07:58 PDT - Notes by Gemini
• Ben Hawks, Matt Sinclair, and gyuri papay discussed the presence of AI bots in the meeting, with gyuri papay expressing concern about AI's impact on employment. Matt Sinclair, Piotr Luszczek, and Gregor von Laszewski discussed the use of AI in code generation, while Ben Hawks updated on the paper's status, indicating an extended deadline due to inconsistencies and awaiting feedback from Nhan. Gregor von Laszewski reported on the carpentry paper, requesting Armstrong Foundjem to classify entries in a table and suggested shortening the energy section, while also discussing challenges in collecting data for the related research section.

Discussion

• Gregg Barrett noted that he was reviewing the benchmark spreadsheet and thought that members of this group would find ToolUniverse of interest:
• ToolUniverse website: ToolUniverse - 600+ Scientific Tools to Democratize AI scientists
• ToolUniverse paper: [2509.23426] Democratizing AI scientists using ToolUniverse
• ToolUniverse GitHub: Democratizing AI scientists with ToolUniverse