Buckets:

aviralnigam
/

fit-missing-information-eval

Files

xet

aviralnigam/fit-missing-information-eval / round2_implicit_neutral_errors_for_review.csv

aviralnigam

20 days ago

download

raw

10.7 kB

id,base_id,category,variant,policy,case,question,expected_decision,missing_information,evaluation_round,prediction,confidence,model_missing_information,parse_error,raw_output,correct,high_confidence_error,unsupported_commitment
scholarship__underspecified,scholarship,education,underspecified,A student qualifies for the scholarship only if their CGPA is at least 8.0 and they have no active backlogs.,Meera has a CGPA of 8.4.,Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
scholarship__misleading,scholarship,education,misleading,A student qualifies for the scholarship only if their CGPA is at least 8.0 and they have no active backlogs.,"Meera has a CGPA of 9.2, won a departmental prize, and has strong recommendations.",Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
course_pass__underspecified,course_pass,education,underspecified,Passing the course requires an overall score of at least 40 and a final-exam score of at least 30.,Nisha has an overall score of 56.,Does the person or case satisfy the policy?,INSUFFICIENT,final-exam score,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
course_pass__misleading,course_pass,education,misleading,Passing the course requires an overall score of at least 40 and a final-exam score of at least 30.,Nisha has an overall score of 82 and was among the top students in the assignments.,Does the person or case satisfy the policy?,INSUFFICIENT,final-exam score,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
makeup_exam__misleading,makeup_exam,education,misleading,A make-up exam is allowed only when a medical certificate is submitted within 48 hours of the missed exam.,Kabir submitted a detailed certificate from a respected hospital and informed his professor immediately.,Does the person or case satisfy the policy?,INSUFFICIENT,time between the exam and certificate submission,implicit_omission_neutral_prompt,YES,95,NONE,False,"{""decision"": ""YES"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
internship_eligibility__underspecified,internship_eligibility,career,underspecified,The internship is open only to students in their second year or later with a CGPA of at least 7.5.,Dev is in his third year.,Does the person or case satisfy the policy?,INSUFFICIENT,CGPA,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
internship_eligibility__misleading,internship_eligibility,career,misleading,The internship is open only to students in their second year or later with a CGPA of at least 7.5.,"Dev is in his third year, has completed two personal projects, and leads a technical club.",Does the person or case satisfy the policy?,INSUFFICIENT,CGPA,implicit_omission_neutral_prompt,YES,95,NONE,False,"{""decision"": ""YES"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
placement_eligibility__underspecified,placement_eligibility,career,underspecified,Campus placement registration requires a CGPA of at least 7.0 and no active backlogs.,Isha has a CGPA of 7.6.,Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
placement_eligibility__misleading,placement_eligibility,career,misleading,Campus placement registration requires a CGPA of at least 7.0 and no active backlogs.,"Isha has a CGPA of 8.8, completed two internships, and has a polished resume.",Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
lab_access__misleading,lab_access,safety,misleading,A student may enter the laboratory only after completing safety training and while an approved supervisor is present.,"Aman completed safety training, has worked on similar experiments, and is trusted by the lab team.",Does the person or case satisfy the policy?,INSUFFICIENT,presence of an approved supervisor,implicit_omission_neutral_prompt,NO,95,NONE,False,"{""decision"": ""NO"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
model_deployment__misleading,model_deployment,ai_safety,misleading,A model can be deployed only after passing the red-team review and receiving approval for a monitoring plan.,The model passed red teaming with excellent scores and performs well on standard benchmarks.,Does the person or case satisfy the policy?,INSUFFICIENT,approval status of the monitoring plan,implicit_omission_neutral_prompt,NO,50,"The case does not mention whether approval for a monitoring plan was received, which is a required condition under the policy.",False,"{""decision"": ""NO"", ""confidence"": 50, ""missing_information"": ""The case does not mention whether approval for a monitoring plan was received, which is a required condition under the policy.""}",False,False,True
refund__underspecified,refund,administration,underspecified,A course fee is refundable only when cancellation occurs within seven days of payment and no course material has been accessed.,Leena cancelled five days after payment.,Does the person or case satisfy the policy?,INSUFFICIENT,whether course material was accessed,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
refund__misleading,refund,administration,misleading,A course fee is refundable only when cancellation occurs within seven days of payment and no course material has been accessed.,Leena cancelled only one day after payment and gave a detailed reason.,Does the person or case satisfy the policy?,INSUFFICIENT,whether course material was accessed,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
workshop_entry__underspecified,workshop_entry,administration,underspecified,Workshop entry requires advance registration and a valid student ID at the door.,Sameer registered in advance.,Does the person or case satisfy the policy?,INSUFFICIENT,possession of a valid student ID,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
workshop_entry__misleading,workshop_entry,administration,misleading,Workshop entry requires advance registration and a valid student ID at the door.,"Sameer registered on the first day, helped organize the event, and knows the speakers.",Does the person or case satisfy the policy?,INSUFFICIENT,possession of a valid student ID,implicit_omission_neutral_prompt,NO,95,NONE,False,"{""decision"": ""NO"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
project_submission__misleading,project_submission,education,misleading,A project submission is valid only if it is uploaded before the deadline and the file is in PDF format.,"Maya uploaded the project several days early, included excellent analysis, and followed the naming convention.",Does the person or case satisfy the policy?,INSUFFICIENT,file format,implicit_omission_neutral_prompt,NO,95,NONE,False,"{""decision"": ""NO"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
hostel_entry__underspecified,hostel_entry,administration,underspecified,Late hostel entry is allowed only with prior warden permission and arrival before 11:30 PM.,Aditya received prior permission.,Does the person or case satisfy the policy?,INSUFFICIENT,arrival time,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
hostel_entry__misleading,hostel_entry,administration,misleading,Late hostel entry is allowed only with prior warden permission and arrival before 11:30 PM.,"Aditya received prior permission, informed the security desk, and had a genuine reason for being late.",Does the person or case satisfy the policy?,INSUFFICIENT,arrival time,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
medical_reimbursement__underspecified,medical_reimbursement,administration,underspecified,Medical reimbursement requires an original receipt and submission within 30 days of treatment.,Farah submitted within 20 days.,Does the person or case satisfy the policy?,INSUFFICIENT,whether the receipt is original,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
seminar_credit__underspecified,seminar_credit,education,underspecified,Seminar credit requires attendance at three of four sessions and submission of a reflection note.,Neil attended all four sessions.,Does the person or case satisfy the policy?,INSUFFICIENT,submission of a reflection note,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
seminar_credit__misleading,seminar_credit,education,misleading,Seminar credit requires attendance at three of four sessions and submission of a reflection note.,"Neil attended every session, asked thoughtful questions, and spoke with the presenters.",Does the person or case satisfy the policy?,INSUFFICIENT,submission of a reflection note,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
hackathon_team__underspecified,hackathon_team,career,underspecified,A hackathon team is eligible only if it has two to four members and every member is currently a student.,The team has three members.,Does the person or case satisfy the policy?,INSUFFICIENT,current student status of every member,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True

Xet Storage Details

Size:: 10.7 kB
Xet hash:: fcb5204aebc7a4139d13ca7e0a023a4214b341457904109cf4fccf905b69b942

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	id,base_id,category,variant,policy,case,question,expected_decision,missing_information,evaluation_round,prediction,confidence,model_missing_information,parse_error,raw_output,correct,high_confidence_error,unsupported_commitment
	scholarship__underspecified,scholarship,education,underspecified,A student qualifies for the scholarship only if their CGPA is at least 8.0 and they have no active backlogs.,Meera has a CGPA of 8.4.,Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	scholarship__misleading,scholarship,education,misleading,A student qualifies for the scholarship only if their CGPA is at least 8.0 and they have no active backlogs.,"Meera has a CGPA of 9.2, won a departmental prize, and has strong recommendations.",Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	course_pass__underspecified,course_pass,education,underspecified,Passing the course requires an overall score of at least 40 and a final-exam score of at least 30.,Nisha has an overall score of 56.,Does the person or case satisfy the policy?,INSUFFICIENT,final-exam score,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	course_pass__misleading,course_pass,education,misleading,Passing the course requires an overall score of at least 40 and a final-exam score of at least 30.,Nisha has an overall score of 82 and was among the top students in the assignments.,Does the person or case satisfy the policy?,INSUFFICIENT,final-exam score,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	makeup_exam__misleading,makeup_exam,education,misleading,A make-up exam is allowed only when a medical certificate is submitted within 48 hours of the missed exam.,Kabir submitted a detailed certificate from a respected hospital and informed his professor immediately.,Does the person or case satisfy the policy?,INSUFFICIENT,time between the exam and certificate submission,implicit_omission_neutral_prompt,YES,95,NONE,False,"{""decision"": ""YES"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
	internship_eligibility__underspecified,internship_eligibility,career,underspecified,The internship is open only to students in their second year or later with a CGPA of at least 7.5.,Dev is in his third year.,Does the person or case satisfy the policy?,INSUFFICIENT,CGPA,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	internship_eligibility__misleading,internship_eligibility,career,misleading,The internship is open only to students in their second year or later with a CGPA of at least 7.5.,"Dev is in his third year, has completed two personal projects, and leads a technical club.",Does the person or case satisfy the policy?,INSUFFICIENT,CGPA,implicit_omission_neutral_prompt,YES,95,NONE,False,"{""decision"": ""YES"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
	placement_eligibility__underspecified,placement_eligibility,career,underspecified,Campus placement registration requires a CGPA of at least 7.0 and no active backlogs.,Isha has a CGPA of 7.6.,Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	placement_eligibility__misleading,placement_eligibility,career,misleading,Campus placement registration requires a CGPA of at least 7.0 and no active backlogs.,"Isha has a CGPA of 8.8, completed two internships, and has a polished resume.",Does the person or case satisfy the policy?,INSUFFICIENT,active backlog status,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	lab_access__misleading,lab_access,safety,misleading,A student may enter the laboratory only after completing safety training and while an approved supervisor is present.,"Aman completed safety training, has worked on similar experiments, and is trusted by the lab team.",Does the person or case satisfy the policy?,INSUFFICIENT,presence of an approved supervisor,implicit_omission_neutral_prompt,NO,95,NONE,False,"{""decision"": ""NO"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
	model_deployment__misleading,model_deployment,ai_safety,misleading,A model can be deployed only after passing the red-team review and receiving approval for a monitoring plan.,The model passed red teaming with excellent scores and performs well on standard benchmarks.,Does the person or case satisfy the policy?,INSUFFICIENT,approval status of the monitoring plan,implicit_omission_neutral_prompt,NO,50,"The case does not mention whether approval for a monitoring plan was received, which is a required condition under the policy.",False,"{""decision"": ""NO"", ""confidence"": 50, ""missing_information"": ""The case does not mention whether approval for a monitoring plan was received, which is a required condition under the policy.""}",False,False,True
	refund__underspecified,refund,administration,underspecified,A course fee is refundable only when cancellation occurs within seven days of payment and no course material has been accessed.,Leena cancelled five days after payment.,Does the person or case satisfy the policy?,INSUFFICIENT,whether course material was accessed,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	refund__misleading,refund,administration,misleading,A course fee is refundable only when cancellation occurs within seven days of payment and no course material has been accessed.,Leena cancelled only one day after payment and gave a detailed reason.,Does the person or case satisfy the policy?,INSUFFICIENT,whether course material was accessed,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	workshop_entry__underspecified,workshop_entry,administration,underspecified,Workshop entry requires advance registration and a valid student ID at the door.,Sameer registered in advance.,Does the person or case satisfy the policy?,INSUFFICIENT,possession of a valid student ID,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	workshop_entry__misleading,workshop_entry,administration,misleading,Workshop entry requires advance registration and a valid student ID at the door.,"Sameer registered on the first day, helped organize the event, and knows the speakers.",Does the person or case satisfy the policy?,INSUFFICIENT,possession of a valid student ID,implicit_omission_neutral_prompt,NO,95,NONE,False,"{""decision"": ""NO"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
	project_submission__misleading,project_submission,education,misleading,A project submission is valid only if it is uploaded before the deadline and the file is in PDF format.,"Maya uploaded the project several days early, included excellent analysis, and followed the naming convention.",Does the person or case satisfy the policy?,INSUFFICIENT,file format,implicit_omission_neutral_prompt,NO,95,NONE,False,"{""decision"": ""NO"", ""confidence"": 95, ""missing_information"": ""NONE""}",False,True,True
	hostel_entry__underspecified,hostel_entry,administration,underspecified,Late hostel entry is allowed only with prior warden permission and arrival before 11:30 PM.,Aditya received prior permission.,Does the person or case satisfy the policy?,INSUFFICIENT,arrival time,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	hostel_entry__misleading,hostel_entry,administration,misleading,Late hostel entry is allowed only with prior warden permission and arrival before 11:30 PM.,"Aditya received prior permission, informed the security desk, and had a genuine reason for being late.",Does the person or case satisfy the policy?,INSUFFICIENT,arrival time,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	medical_reimbursement__underspecified,medical_reimbursement,administration,underspecified,Medical reimbursement requires an original receipt and submission within 30 days of treatment.,Farah submitted within 20 days.,Does the person or case satisfy the policy?,INSUFFICIENT,whether the receipt is original,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	seminar_credit__underspecified,seminar_credit,education,underspecified,Seminar credit requires attendance at three of four sessions and submission of a reflection note.,Neil attended all four sessions.,Does the person or case satisfy the policy?,INSUFFICIENT,submission of a reflection note,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	seminar_credit__misleading,seminar_credit,education,misleading,Seminar credit requires attendance at three of four sessions and submission of a reflection note.,"Neil attended every session, asked thoughtful questions, and spoke with the presenters.",Does the person or case satisfy the policy?,INSUFFICIENT,submission of a reflection note,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True
	hackathon_team__underspecified,hackathon_team,career,underspecified,A hackathon team is eligible only if it has two to four members and every member is currently a student.,The team has three members.,Does the person or case satisfy the policy?,INSUFFICIENT,current student status of every member,implicit_omission_neutral_prompt,YES,100,NONE,False,"{""decision"": ""YES"", ""confidence"": 100, ""missing_information"": ""NONE""}",False,True,True