Skip to content

Commit 040985f

Browse files
authored
Merge pull request #304 from deepmodeling/doc/update-efficiency-test-doc
Doc: update description for efficiency test
2 parents 0dd6915 + 8d2464a commit 040985f

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

lambench/metrics/results/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -94,11 +94,11 @@ The resulting score is denoted as $\bar M^{m}_{PC}$.
9494

9595
### Efficiency
9696

97-
To assess the efficiency of the model, we randomly selected 2000 frames from the domain of Inorganic Materials and Catalysis using the aforementioned out-of-distribution datasets. Each frame was expanded to include 800 to 1000 atoms through the replication of the unit cell, ensuring that measurements of inference efficiency occurred within the regime of convergence. The initial 20% of the test samples were considered a warm-up phase and thus were excluded from the efficiency timing. We have reported the average efficiency across the remaining 1600 frames.
97+
To assess the efficiency of the model, we randomly selected 1000 frames from the domain of Inorganic Materials and Catalysis using the aforementioned out-of-distribution datasets. Each frame was expanded to contain between 800 and 1000 atoms — dynamically determined using a binary search algorithm to fully utilize GPU capacity — by replicating the unit cell. This ensured that measurements of inference efficiency were conducted within the regime of convergence. The initial 10% of the test samples were considered a warm-up phase and thus were excluded from the efficiency timing. We have reported the average efficiency across the remaining 900 frames.
9898

99-
We define an efficiency score, $M_E^m$, by normalizing the average inference time (with unit $\mathrm{\mu s/atom}$), $\bar \eta^m$, of a given LAM measured over 1600 configurations with respect to an artificial reference value, thereby rescaling it to a range between zero and positive infinity. A larger value indicates higher efficiency.
99+
We define an efficiency score, $M_E^m$, by normalizing the average inference time (with unit $\mathrm{\mu s/atom}$), $\bar \eta^m$, of a given LAM measured over 900 configurations with respect to an artificial reference value, thereby rescaling it to a range between zero and positive infinity. A larger value indicates higher efficiency.
100100

101-
$$M_E^m = \frac{\eta^0 }{\bar \eta^m },\quad \eta^0= 100\ \mathrm{\mu s/atom}, \quad \bar \eta^m = \frac{1}{1600}\sum_{i}^{1600} \eta_{i}^{m}$$
101+
$$M_E^m = \frac{\eta^0 }{\bar \eta^m },\quad \eta^0= 100\ \mathrm{\mu s/atom}, \quad \bar \eta^m = \frac{1}{900}\sum_{i}^{900} \eta_{i}^{m}$$
102102

103103
where $\eta_{i}^{m}$ is the inference time of configuration $i$ for model $m$.
104104

0 commit comments

Comments
 (0)