Skip to content

Commit abd9ae8

Browse files
committed
update mteb eval
1 parent 61ab7e0 commit abd9ae8

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

FlagEmbedding/evaluation/mteb/runner.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,13 @@ def read_results(self, output_folder, tasks):
6565
print('ERROR')
6666
break
6767

68-
temp_data = data['scores'][split][0]
68+
temp_datas = data['scores'][split][0]
69+
temp_data = None
70+
for td in temp_datas:
71+
if td['hf_subset'] == 'default':
72+
temp_data = td
73+
if temp_data is None:
74+
temp_data = temp_datas[0]
6975
tasks_results[t_type][task_name] = round(temp_data['main_score'] * 100, 2)
7076

7177
print(f"tasks_results: {tasks_results}")

0 commit comments

Comments
 (0)