How to calculate Combined Performance Score?

[yzchen08]

Hello, I want to calculate the CPS (Combined Performance Score) based on my own model, what is the equation to calculate the CPS? It seems that the F1 part of CPS is F1 value itself, but I cannot figure out how to calculate the RMSD and Ksrme parts.

[janosh]

CPS is a dynamic metric computed based on user-defined weights so it has to be computed on-device of visitors to the website. for that reason, it's currently only implemented in TypeScript:

matbench-discovery/site/src/lib/combined_perf_score.ts

Lines 73 to 122 in 32b227a

	// Normalization reference points:
	// - F1 score for discovery already in [0,1] range, higher is better
	// - RMSD Root mean square displacement in range 0 (perfect) to RMSD_BASELINE, lower is better
	// - κ_SRME symmetric relative mean error for lattice thermal conductivity,
	// range [0,2] linearly mapped to [1,0], lower is better
	export function calculate_cps(
	f1: number | undefined,
	rmsd: number | undefined,
	kappa: number | undefined,
	config: CombinedMetricConfig, // weights for each metric
	): number | null {
	// Find weights from config by metric names
	const { F1, RMSD, kappa_SRME } = config.parts
	// Check if any metrics with non-zero weights are missing
	if (
	(F1.weight > 0 && (f1 === undefined || isNaN(f1))) ||
	(RMSD.weight > 0 && (rmsd === undefined || isNaN(rmsd))) ||
	(kappa_SRME.weight > 0 && (kappa === undefined || isNaN(kappa)))
	) {
	return null
	}
	// Skip the calculation if all weights are zero
	const total_weight = F1.weight + RMSD.weight + kappa_SRME.weight
	if (total_weight === 0) {
	return 0
	}
	// Calculate weighted sum
	let weighted_sum = 0
	// Add F1 contribution if available and weighted
	if (f1 !== undefined && !isNaN(f1) && F1.weight > 0) {
	weighted_sum += normalize_f1(f1) * F1.weight
	}
	// Add RMSD contribution if available and weighted
	if (rmsd !== undefined && !isNaN(rmsd) && RMSD.weight > 0) {
	weighted_sum += normalize_rmsd(rmsd) * RMSD.weight
	}
	// Add kappa contribution if available and weighted
	if (kappa !== undefined && !isNaN(kappa) && kappa_SRME.weight > 0) {
	weighted_sum += normalize_kappa_srme(kappa) * kappa_SRME.weight
	}
	// Return weighted average
	return weighted_sum / total_weight
	}

but it's a simple formula and any LLM should be able to convert it to Python for you