papers/kai/data/diagnosis/verify.out

parquet rows 82326
distinct (query,passage) 674848 ambiguous 307
sample200 155761 checked 200 {'ok': 198, ('y', 0): 103, ('y', 1): 95, 'ambiguous': 2}
all 155761 checked 155761 {'ok': 155480, ('y', 1): 77531, ('y', 0): 77949, 'ambiguous': 281}
per-query shapes (n, ys, distinct passages): [((2, (0, 1), 2), 76109), ((1, (0,), 1), 1846), ((1, (1,), 1), 1697)]
class counts Counter({0: 77955, 1: 77806})
adjacent pairs (same query, y_a, y_b): [((True, 1, 0), 38115), ((False, 0, 1), 37985), ((False, 0, 0), 918), ((False, 1, 1), 844), ((False, 1, 0), 18)]
robust.microsoft-ms_marco.alias.red.0.jsonl.gz 3990 checked 3990 {'ok': 3985, ('y', 1): 2021, ('y', 0): 1964, 'ambiguous': 5}
robust.microsoft-ms_marco.swap.red.0.jsonl.gz 3995 checked 3995 {'wrong': 3988, ('y', 1): 2010, ('y', 0): 1978, 'ambiguous': 7}
   wrong {"query": "what does environmental hygiene mean", "passage": "In the Food Service context, environmental hygiene is the correct sanitizing of food and food preparation areas such as work benches and c {'relevant': [0.0, 1.0]}
   wrong {"query": "what is zulu traditional food", "passage": "Did you. Zulus traditionally sprinkle dried pelargonium leaves on dishes as an indigenous spice. The foundations of Zulu cuisine are sorghum and  {'relevant': [1.0, 0.0]}
   wrong {"query": "where is glottis located", "passage": "Glottis. The glottis is the part of the frog that inflates the lungs of the frog. It makes the frog breath. The glottis is the opening to the lungs an {'relevant': [0.0, 1.0]}
validation/calibration/rag_relevance.microsoft-ms_marco.format.red.0.jsonl.gz 813 checked 813 {'ok': 813, ('y', 0): 390, ('y', 1): 423}
validation/in_domain/rag_relevance.microsoft-ms_marco.format.red.0.jsonl.gz 2345 checked 2345 {'ok': 2340, ('y', 0): 1138, ('y', 1): 1202, 'ambiguous': 5}