March 5, 2025
March 5, 2025
Present
Andy Cheng, Armstrong Foundjem, Azza Ahmad, Bryan, Claus Weiland, David DeBomis, Dmitry Kondratyev, Gary Mazzaferro , Geoffrey Fox, Gregor von Laszewski, Gyuri Papay, Jeyan Thiyagalingam, Lee Sharma, Marco Colombo, Matt Sinclair, Nhan Tran, Piotr Luszczek, Pranav Gupta, Satoshi Iwata, Shirley Moore, Steven Farrell, Victor Lu, Wes Brewer
Apologies
Christine Kirkpatrick
Tentative Agenda
- Any New Members Introduction
- Dmitry Kondratyev (Purdue) from FastML presentation on "SONIC: A Portable Framework for as-a-Service ML Serving"
- Continuing discussion of the catalog of Science benchmarks based on https://docs.google.com/spreadsheets/d/1Ysk32dqkgdGfDW0rFaCpc8o1Cp6uhtJqbDFAIlhfb9o/edit?usp=sharing
- White Papers
- Please find the status and locations in minutes of the February 19 meeting.
- Any Other Business
Gemini Transcript and Summary and Meet Recording
- Gemini Transcript and Summary MLC Science WG - 2025/03/05 07:50 PST - Notes by Gemini
- Recording MLC Science WG - 2025/03/05 07:50 PST - Recording
New Members
- Bryan no information
- Dmitry Kondratyev https://www.linkedin.com/in/kondratyevd/ Research Software Engineer with Ph.D. in experimental high energy physics, Experience at CERN (CMS) and Fermilab.
- Pranav Gupta Pranav Gupta - Lowe's Companies, Inc. | LinkedIn PhD data scientist passionate about cutting edge tech, e.g., LLMs and quantum computing. I am flexible and versatile, mitigate and manage risks well, and have sound technical literacy and communication skills. prannerta100 (Pranav Gupta) · GitHub
Presentation
- Dmitry Kondratyev (Purdue) from FastML presentation on "SONIC: A Portable Framework for as-a-Service ML Serving"
- Presentation SONIC ML Commons.pdf and see video recording above
- SONIC provides Inference as a service on the server side supporting multiple clients across currently several physics experiments. The client side would be specific to each experiment.
- Industry has related applications
- They have started SuperSONIC which includes
- Load balancing across gpus
- Energy savings
- Good user dashboard
- Uses Kubernetes while SONIC not implemented with Kubernetes
- Hosted on 4 Kubernetes clustersand currently tested on 3 experiments
- SuperSONIC suitable for hosting benchmarking and has itself been benchmarked
- Can use SuperSONIC for training (Geoffrey asked) but not working on this
- Juri asked about predicting performance
- Dmitry noted that GPU is not only bottleneck and they measure I/O and communication
- Victor Lu contributed some articles on estimating complexity
- https://arxiv.org/pdf/2103.05127
- https://arxiv.org/pdf/2304.08319
- (PDF) A Review of Algorithms’s Complexities on Different Valued Sorted and Unsorted Data
- A Survey on Large-scale Machine Learning
- Computational Complexity: A Modern Approach
- https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9274431
- https://citeseerx.ist.psu.edu/document?repid=rep1\&type=pdf\&doi=30600b3fa27903201742b7fd76603760e6351a9d
Catalog of Science Benchmarks
- Based on MLCommons Science/HPC Benchmarks Overview
- Shirley Moore added PDEBench
- Victor Lu suggested Julia benchmarks https://www.geeksforgeeks.org/benchmarking-in-julia/
- It was agreed that to be useful this caalog had to add value such as the taxonomy and quality comments, and not just be a list
Merger of HPC and Science Working Groups
- Doodle polls have been started
- MLCommons Science Working Group USA-Europe Meeting
- https://doodle.com/meeting/participate/id/eVoL8k5a
- This is every other week and is at 11.05 or 12.05. Current meeting is 11.05 Eastern on Wednesday. The survey is for one week, but answer as best time if held every two weeks. Note meeting is listed starting on the hour; it will start 5 minutes after the hour.
- MLCommons Science Working Group USA-Asia Meeting
- https://doodle.com/meeting/participate/id/eg8lB9Za
- This will be held every other week and is at 7.05 pm or 8.05 pm or 9.05 pm. Currently we don't hold this meeting; just the 11.05 every other Wednesday one. The survey is for one week, but answer as best time if held every two weeks. Note meeting is listed starting on the hour; it will start 5 minutes after the hour.