Index of Benchmarks

medium-range forecasting ERA5 data-driven

ClimateLearn - Weather Forcasting

Climate & Earth Science • RMSE, Anomaly correlation

Avg rating: 5.00/5

medium-range forecasting ERA5 data-driven

ClimateLearn - Downscaling

Climate & Earth Science • RMSE, Anomaly correlation

Avg rating: 5.00/5

medium-range forecasting ERA5 data-driven

ClimateLearn - Climate Projection

Climate & Earth Science • RMSE, Anomaly correlation

Avg rating: 5.00/5

neural operators CFD FNO DeepONet

CFDBench (Fluid Dynamics)

Mathematics • L2 error, MAE

Avg rating: 3.33/5

land-use zero-shot multi-task

SatImgNet

Climate & Earth Science • Accuracy

Avg rating: 3.83/5

quantum circuits state preparation error correction

Quantum Computing Benchmarks (QML)

Computational Science & AI • Fidelity, Success probability

Avg rating: 3.17/5

DFT relaxations adsorption energy graph neural networks

OCP (Open Catalyst Project)

Chemistry, Materials Science • MAE (energy), MAE (force)

Avg rating: 4.17/5

USMLE diagnostic QA medical knowledge multilingual

MedQA

Biology & Medicine • Accuracy

Avg rating: 3.50/5

DFT materials genome high-throughput

Materials Project

Materials Science • MAE, R^2

Avg rating: 1.92/5

Google-proof graduate-level science QA chemistry physics

GPQA Diamond

Biology & Medicine, Chemistry, High Energy Physics • Accuracy

Avg rating: 3.83/5

grade-school science QA challenge set reasoning

ARC-Challenge (Advanced Reasoning Challenge)

Computational Science & AI • Accuracy

Avg rating: 4.83/5

symbolic reasoning number theory algebraic geometry category theory

FrontierMath

Mathematics • Accuracy

Avg rating: 2.50/5

code synthesis scientific computing programming benchmark

SciCode

Computational Science & AI • Solve rate (%)

Avg rating: 4.50/5

algebra combinatorics number theory geometry

AIME (American Invitational Mathematics Examination)

Mathematics • Accuracy

Avg rating: 2.33/5

calculus algebra number theory geometry

PRM800K

Mathematics • Accuracy

Avg rating: 3.83/5

long-context information extraction multimodal

CURIE (Scientific Long-Context Understanding, Reasoning and Information Extraction)

Materials Science, High Energy Physics, Biology & Medicine, Chemistry, Climate & Earth Science • Accuracy

Avg rating: 3.33/5

finite element simulation PDE

FEABench (Finite Element Analysis Benchmark): Evaluating Language Models on Multiphysics Reasoning Ability

Mathematics • Solve time, Error norm

Avg rating: 3.83/5

multimodal QA figure understanding table comprehension chain-of-thought

SPIQA (Scientific Paper Image Question Answering)

Computational Science & AI • Accuracy, F1 score

Avg rating: 3.67/5

single-cell annotation biological QA autonomous discovery

BaisBench (Biological AI Scientist Benchmark) - Question Answering

Biology & Medicine • Annotation accuracy, QA accuracy

Avg rating: 4.00/5

single-cell annotation biological QA autonomous discovery

BaisBench (Biological AI Scientist Benchmark) - Cell Type Annotation

Biology & Medicine • Annotation accuracy, QA accuracy

Avg rating: 4.00/5

SELFIES GAN property optimization

MOLGEN

Chemistry • Validity%, Novelty%, QED, Docking score, penalized logP

Avg rating: 4.83/5

node prediction link prediction graph classification

Open Graph Benchmark (OGB) - Biology

Biology & Medicine • Accuracy, ROC-AUC

Avg rating: 4.83/5

classification real-time ML jet tagging QKeras

Jet Classification

High Energy Physics • Accuracy, AUC

Avg rating: 4.17/5

compression autoencoder sparse data irregular sampling

Irregular Sensor Data Compression

High Energy Physics • MSE, Compression ratio

Avg rating: 4.17/5

RL beam stabilization control systems simulation

Beam Control

High Energy Physics • Stability, Control loss

Avg rating: 3.00/5

jet classification FPGA quantization-aware training Deep Sets Interaction Networks

Ultrafast jet classification at the HL-LHC

High Energy Physics • Accuracy, Latency, Resource utilization

Avg rating: 3.17/5

quench detection autoencoder anomaly detection real-time

Quench detection

High Energy Physics • ROC-AUC, Detection latency

Avg rating: 2.17/5

FPGA Graph Neural Network hls4ml real-time inference detector control

Intelligent experiments through real-time AI

High Energy Physics • Accuracy (charm and beauty detection), Latency (micros), Resource utilization (LUT/FF/BRAM/DSP)

Avg rating: 3.00/5

neural architecture search FPGA deployment quantization pruning hls4ml

Neural Architecture Codesign for Fast Physics Applications

High Energy Physics • Accuracy, Latency, Resource utilization

Avg rating: 3.83/5

smart pixel on-sensor inference data reduction trigger

Smart Pixels for LHC

High Energy Physics • Data rejection rate, Power per pixel

Avg rating: 3.33/5

BraggNN diffraction peak finding HEDM

HEDM (BraggNN)

Materials Science • Localization accuracy, Inference time

Avg rating: 3.17/5

4D-STEM electron microscopy real-time image processing

4D-STEM

Materials Science • Classification accuracy, Throughput

Avg rating: 3.17/5

plasma in-situ vision real-time ML

In-Situ High-Speed Computer Vision

High Energy Physics • Accuracy, FPS

Avg rating: 1.50/5

MLPerf HPC - Cosmoflow

High Energy Physics • Training time, Accuracy, GPU utilization

Avg rating: 4.17/5

MLPerf HPC - DeepCAM

Climate & Earth Science • Training time, Accuracy, GPU utilization

Avg rating: 4.17/5

MLPerf HPC - Open Catalyst Project DimeNet++

Chemistry • Training time, Accuracy, GPU utilization

Avg rating: 4.17/5

MLPerf HPC - OpenFold

Biology & Medicine • Training time, Accuracy, GPU utilization

Avg rating: 4.17/5

MLCommons Science - CloudMask

Climate & Earth Science • MAE, Accuracy, Speedup vs simulation

Avg rating: 5.00/5

MLCommons Science - Earthquake

Climate & Earth Science • MAE, Accuracy, Speedup vs simulation

Avg rating: 5.00/5

MLCommons Science - Candle UNO

Biology & Medicine • MAE, Accuracy, Speedup vs simulation

Avg rating: 5.00/5

MLCommons Science - STEMDL

Materials Science • MAE, Accuracy, Speedup vs simulation

Avg rating: 5.00/5

anomaly detection proton collision real-time inference event filtering unsupervised ML

LHC New Physics Dataset

High Energy Physics • ROC-AUC, Detection efficiency

Avg rating: 3.33/5

medical AI federated evaluation privacy-preserving fairness healthcare benchmarks

MLCommons Medical AI - Pancreas Segmentation (DFCI)

Biology & Medicine • ROC AUC, Accuracy, Fairness metrics

Avg rating: 4.33/5

medical AI federated evaluation privacy-preserving fairness healthcare benchmarks

MLCommons Medical AI - Brain Tumor Segmentation (BraTS)

Biology & Medicine • ROC AUC, Accuracy, Fairness metrics

Avg rating: 4.33/5

medical AI federated evaluation privacy-preserving fairness healthcare benchmarks

MLCommons Medical AI - Surgical Workflow Phase Recognition (SurgMLCube)

Biology & Medicine • ROC AUC, Accuracy, Fairness metrics

Avg rating: 4.33/5

calorimeter simulation generative models surrogate modeling LHC fast simulation

CaloChallenge 2022

High Energy Physics • Histogram similarity, Classifier AUC, Generation latency

Avg rating: 4.50/5

PDEs CFD scientific ML surrogate modeling NeurIPS

PDEBench

Computational Science & AI, Climate & Earth Science, Mathematics • RMSE, boundary RMSE, Fourier RMSE

Avg rating: 4.50/5

density functional theory Delta Squared-ML correction reaction energetics quantum chemistry

Delta Squared-DFT

Chemistry, Materials Science • Mean Absolute Error (eV), Energy ranking accuracy

Avg rating: 3.83/5

Urban Data Layer (UDL) - PM2.5 Concentration Prediction

Climate & Earth Science • Task-specific accuracy or RMSE

Avg rating: 4.50/5

Urban Data Layer (UDL) - Built-up Area Classification

Climate & Earth Science • Task-specific accuracy or RMSE

Avg rating: 4.50/5

Urban Data Layer (UDL) - Administrative Boundaries Identification

Climate & Earth Science • Task-specific accuracy or RMSE

Avg rating: 4.50/5

Urban Data Layer (UDL) - El Nino Anomaly Detection

Climate & Earth Science • Task-specific accuracy or RMSE

Avg rating: 4.50/5

crop science prompt engineering domain adaptation question answering

LLMs for Crop Science

Climate & Earth Science • Accuracy, F1 score

Avg rating: 4.67/5

DUNE time-series real-time trigger

DUNE

High Energy Physics • Detection efficiency, Latency

Avg rating: 2.83/5

mass spectrometry molecular structure de novo generation retrieval dataset

MassSpecGym - De novo molecule generation

Chemistry • Structure accuracy, Retrieval precision, Simulation MSE

Avg rating: 3.75/5

mass spectrometry molecular structure de novo generation retrieval dataset

MassSpecGym - Molecule Retrieval

Chemistry • Structure accuracy, Retrieval precision, Simulation MSE

Avg rating: 3.75/5

mass spectrometry molecular structure de novo generation retrieval dataset

MassSpecGym - Spectrum Simulation

Chemistry • Structure accuracy, Retrieval precision, Simulation MSE

Avg rating: 3.75/5

anomaly detection gravitational waves astrophysics time-series

HDR ML Anomaly Challenge - Gravitational Waves

High Energy Physics • ROC-AUC, Precision/Recall

Avg rating: 4.17/5

anomaly detection computer vision genomics butterfly hybrids

HDR ML Anomaly Challenge - Butterfly

Biology & Medicine • Classification accuracy, F1 score

Avg rating: 3.00/5

anomaly detection climate science sea-level rise time-series remote sensing

HDR ML Anomaly Challenge - Sea Level Rise

Climate & Earth Science • ROC-AUC, Precision/Recall

Avg rating: 3.83/5

qubit readout hls4ml FPGA QICK

Single Qubit Readout on QICK System

Computational Science & AI • Accuracy, Latency

Avg rating: 3.50/5

Google-proof multiple-choice expert reasoning science QA

GPQA: A Graduate-Level Google-Proof Question and Answer Benchmark

Biology & Medicine, High Energy Physics, Chemistry • Accuracy

Avg rating: 3.67/5

sonar imagery vision-language seafloor mapping segmentation QA

SeafloorAI

Climate & Earth Science • Segmentation pixel accuracy, QA accuracy

Avg rating: 4.33/5

sonar imagery vision-language seafloor mapping segmentation QA

SeafloorGenAI

Climate & Earth Science • Segmentation pixel accuracy, QA accuracy

Avg rating: 4.33/5

superconductivity crystal structures equivariant GNN generative models

SuperCon3D - Property Prediction

Materials Science • MAE (Tc), Validity of generated structures

Avg rating: 4.17/5

superconductivity crystal structures equivariant GNN generative models

SuperCon3D - Inverse Crystal Structure Generation

Materials Science • MAE (Tc), Validity of generated structures

Avg rating: 4.17/5

geometric deep learning distribution shift OOD robustness scientific applications

GeSS - Track Pileup

High Energy Physics • Accuracy, RMSE, OOD robustness delta

Avg rating: 4.33/5

geometric deep learning distribution shift OOD robustness scientific applications

GeSS - Track Signal

High Energy Physics • Accuracy, RMSE, OOD robustness delta

Avg rating: 4.33/5

geometric deep learning distribution shift OOD robustness scientific applications

GeSS - DrugOOD

Biology & Medicine • Accuracy, RMSE, OOD robustness delta

Avg rating: 4.33/5

geometric deep learning distribution shift OOD robustness scientific applications

GeSS - QMOF

Materials Science • Accuracy, RMSE, OOD robustness delta

Avg rating: 4.33/5

source localization bioacoustics time-series SSL

Vocal Call Locator (VCL)

Biology & Medicine • Localization error (cm), Recall/Precision

Avg rating: 3.83/5

multimodal QA scientific figures image+text chain-of-thought prompting

SPIQA (LLM)

Computational Science & AI • Accuracy, F1 score

Avg rating: 4.42/5

surrogate modeling foundation model physics simulations spatiotemporal dynamics

The Well

Biology & Medicine, Computational Science & AI, High Energy Physics • Dataset size, Domain breadth

Avg rating: 4.00/5