Skip to content

Commit 6c94d9a

Browse files
committed
fix: cap generalizability PC M_hat_mkpi to 1
1 parent b64c5e5 commit 6c94d9a

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

lambench/metrics/vishelper/metrics_calculations.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,9 @@ def calculate_generalizability_downstream_score(self) -> dict[str, float]:
135135
else:
136136
# normalize the metric by dummy value, TODO: check if dummy is 0
137137
raw_results[column] = raw_results[column] / dummy
138+
raw_results = raw_results.map(
139+
lambda x: np.clip(x, 0, 1), na_action="ignore"
140+
) # for models that are worse than the dummy baseline, set the value to 1.0
138141

139142
# Apply penalty for specified metrics directly to $\bar{M}_i$ before domain level aggregation.
140143
# $\bar{M}_i$ is an error metric, the lower the better, so we want to penalize it by dividing

0 commit comments

Comments
 (0)