Skip to content

Commit 172d666

Browse files
authored
docs: update readme content (#362)
1 parent b0b60ec commit 172d666

File tree

2 files changed

+14
-10
lines changed

2 files changed

+14
-10
lines changed

lambench/metrics/results/README.md

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Overview
22

3-
Large atomic models (LAM), also known as machine learning interatomic potentials (MLIPs), are considered foundation models that predict atomic interactions across diverse systems using data-driven approaches. LAMBench is a benchmark designed to evaluate the performance of such models. It provides a comprehensive suite of tests and metrics to help developers and researchers understand the accuracy and generalizability of their machine learning models.
3+
Large atomistic models (LAM), also known as machine learning interatomic potentials (MLIPs), are considered foundation models that predict atomic interactions across diverse systems using data-driven approaches. **LAMBench** is a benchmark designed to evaluate the performance of such models. It provides a comprehensive suite of tests and metrics to help developers and researchers understand the accuracy and generalizability of their machine learning models.
44

55
## Our mission includes
66

@@ -26,7 +26,7 @@ Figure 1: Generalizability on force field prediction tasks, 1 - $\bar{M}^m_{FF}$
2626
<!-- scatter plot -->
2727
Figure 2: Accuracy-Efficiency Trade-off, $\bar{M}^m_{FF}$ vs $M_E^m$.
2828

29-
# LAMBench Metrics Calculations
29+
# LAMBench Metrics Calculation
3030

3131
## Generalizability
3232

@@ -81,12 +81,16 @@ In contrast, an ideal model that perfectly matches Density Functional Theory (DF
8181

8282
### Domain Specific Property Calculation
8383

84-
For the domain-specific property tasks, we adopt the MAE as the error metric.
85-
In the Inorganic Materials domain, the MDR phonon benchmark predicts maximum phonon
86-
frequency, entropy, free energy, and heat capacity at constant volume, with each prediction type assigned a weight of 0.25.
87-
In the Molecules domain, the TorsionNet500 benchmark predicts the torsion profile energy, torsion barrier height, and the number of molecules for which the model's prediction of the torsional barrier height has an error exceeding 1 kcal/mol.
88-
Each prediction type in this domain is assigned a weight of $\frac{1}{3}$.
89-
The resulting score is denoted as $\bar M^{m}_{PC}$.
84+
For the domain-specific property calculation tasks, we adopt the MAE as the primary error metric.
85+
86+
In the Inorganic Materials domain, the MDR phonon benchmark predicts the maximum phonon frequency, entropy, free energy, and heat capacity at constant volume, while the elasticity benchmark evaluates the shear and bulk moduli. Each prediction type
87+
is assigned an equal weight of $\frac{1}{6}$.
88+
89+
In the Molecules domain, the TorsionNet500 benchmark evaluates the torsion profile energy, torsional barrier height, and the number of molecules for which the predicted torsional barrier height error exceeds 1 kcal/mol. The Wiggle150 benchmark assesses the relative conformer energy profile. Each prediction type in this domain is assigned a weight of 0.25.
90+
91+
In the Catalysis domain, the OC20NEB-OOD benchmark evaluates the energy barrier, reaction energy change (delta energy), and the percentage of reactions with predicted energy barrier errors exceeding 0.1 eV for three reaction types: transfer, dissociation, and desorption. Each prediction type in this domain is assigned a weight of 0.2.
92+
93+
The resulting error metric after averaging over all domains is denoted as $\bar M^{m}_{PC}$.
9094

9195
## Applicability
9296

@@ -122,4 +126,4 @@ The final instability metric is computed as the average over all nine structures
122126

123127
$$M^m_{\mathrm{IS}} = \frac{1}{9}\sum_{i=1}^{9} M^m_{\mathrm{IS},i}$$
124128

125-
This result is bounded within the range $[0, +\infty]$, where a lower value signifies greater stability.
129+
This result is bounded within the range $[0, +\infty)$, where a lower value signifies greater stability.

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ classifiers = [
3333
]
3434

3535
[project.optional-dependencies]
36-
deepmd = ["deepmd-kit @ git+https://github.com/iProzd/deepmd-kit.git@P0131_dpa3_optim_angle#egg=deepmd-kit", "torch>=2.1"]
36+
deepmd = ["deepmd-kit==3.1.0", "torch>=2.1"]
3737
fairchem = ["fairchem-core", "torch_geometric", "pyg_lib", "torch_scatter", "torch_sparse", "torch_cluster", "torch_spline_conv"] # See README
3838
mace = ["mace-torch"]
3939
mattersim = ["mattersim >= 1.1.1"]

0 commit comments

Comments
 (0)