Skip to content
Science

January 26, 2022

January 26, 2022

Present

Tony Hey, Geoffrey Fox, Jeyan Thiyagalingam, Christine Kirkpatrick, Gregor von Laszewski, Hai ah Nam, Juri Papay, Arjun Shankar, Aristeidis Tsaris, Gregg Barrett, Farzana Yasmin Ahmad, Junqi Yin, Bala Desinghu

Tentative Agenda

New member introductions

  • none

Christine Kirkpatrick Talk

  • Christine Kirkpatrick gave an excellent presentation MLCommons Data Assessmentwith three main topics
  • Data analysis of MLCommons benchmarks
    • The data analysis of MLCommons shows numerous inconsistencies, especially in system definition files. Some of the details were entered by hand and there are also missing data. Christine made several suggestions on how to improve data quality (see presentation slides). She discussed use of containers. The WG suggested communicating the findings to MLCommons.
  • Power consumption of computers
    • The slides about power consumption highlighted the importance of measuring energy usage not only FLOP rate and storage requirements.
    • Oakridge has a paper in SC21 to address power issues (Arjun reported)
    • Power-related dissection of OLCF's Summit: https://dl.acm.org/doi/abs/10.1145/3458817.3476188
    • MLTiny group has power consumption in its benchmark (Juri reported)
    • MLC Power WG and EE HPC WG are engaging with one another on the new Power efforts (Gregg Barrett)
  • The talk ended with a report on the formation of RDA Interest Group: FAIR for Machine Learning FAIR4MLwhich is an initiative aiming to bring together the FAIR and ML communities and is based on two BOF’s at plenary meetings where co-chairs are being sought FAIR4ML-RDA-IG-Charter

Adhering to ML Common Rules – Progress

  • Juri presented the findings Reference_Implementations_v1.pptx in respect to MLCommons compliance of Science WG benchmarks.
  • In the discussion after the presentations, we agreed to use Gregor’s GitHub for pulling together all information required for making the Science WG benchmarks MLCommons compliant and agreed on the action plan. The data will be hosted at STFC and San Diego.
  • Gregor and Juri had an earlier meeting resulting in confirmation that we can take a similar approach to the HPC WG policies. The draft was posted (see https://github.com/laszewsk/mlcommons/blob/main/science_training_policy.adoc)
  • Action needed: Application owners need to identify parameters for the benchmark and place them in the table. The best is to use an ASCII document as that is the format of other MLcommons documents
  • Aris reminded us of science working group web site Science Working Group | MLCommons with staging web site https://stagingscience--mlcommons.netlify.app/en/groups/research-science/
  • And GitHub update mechanism https://github.com/mlcommons/training_policies/pull/478
  • Action needed: there may be some applications that do not need parallel computing? this may require verbiage to for example relax the requirement to have parallel file systems.
  • Action needed: revisit the google docs document with the applications and get simple changes integrated while keeping the more complex ones open but start resolving those

Any Other Business