{"request": {"context": "document: The euro sign (€) is the currency sign used for the euro, the official currency of the Eurozone in the European Union (EU). The design was presented to the public by the European Commission on 12 December 1996. The international three-letter code (according to ISO standard ISO 4217) for the euro is EUR. In Unicode it is encoded at U+20AC € euro sign (HTML € € ). In English, the sign precedes the value (for instance, €10, not 10 €, unlike most other European languages). In some style guides, but not others, the euro sign is unspaced.", "question": "does the euro sign go before the number?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "This movie has made me want to become a director, and Michelle Rodriguez is brilliant. How the hell wasn't she on mtv's top 25 under 25, she beats them all. This film definitely deserved the grand jury prize at sundance, best film i have ever seen.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((request value is greater than 43) holds and the exception (account value is at most 52) does not hold); (at least one of these conditions hold: [(shipment value is at most 57); (member value is between 39 and 49, including both endpoints)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 53. The shipment value is 58. The routing reference is 251. The member value is 39. The request value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(or(0,1),unless(2,3))"}
{"request": {"context": "The piece includes a handy quiz for couples who want to diagnose their viability.", "question": "Hypothesis: \"There is a handy quiz for couples to take.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "ticket:\n  channel: email\n  body: What famous husband-and-wife team did radio ads for Blue Nun wine ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: A case is approved when member value is equal to 8, and also shipment value is less than 13, and also the elapsed days from entry start date to entry end date are at most 44, and also case verified is yes; any other case is denied. Routing references play no part in the decision.\ncase: The routing reference is 382. The entry start date is 2027-02-04. The entry end date is 2027-03-20. The shipment value is 12. The member value is 18. The case verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),and(2,3))"}
{"request": {"context": "yep that's all good-bye okay bye", "question": "Hypothesis: \"that's all, goodbye\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: what: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [[NOT: entry value is between 12 and 22, including both endpoints] UNLESS [request value is at most 42 UNLESS review value is between 55 and 65, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 263. The review value is 56. The request value is 41. The entry value is 13.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(not(0),unless(1,2))"}
{"request": {"context": "policy: Approve exactly when ((member value is between 21 and 31, including both endpoints) AND (case value is greater than 52)) AND ((shipment value is at most 60) OR (account value is greater than 45)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 21. The account value is 55. The case value is 62. The routing reference is 398. The shipment value is 61.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(and(0,1),or(2,3))"}
{"request": {"context": "- role: customer\n  content: A comedy of epically funny proportions from the guys that brought you South Park, and most of the guys from Orgazmo. This vulgur, obscence movie has utterly disgusting, eggotistical, and satirical content. It portrays incredibly cruel treatment of humans and animals. I LOVE IT!!!!! This is some funny stuff. Really funny. Two loser friends create a game in thier driveway, which explodes into a national sensation. Corruption and greed and blackmail turn the sport sour, and its up ta Coop ta fix it. And along the way, you will laugh. Alot. That's all there is. Enjoy!!!!", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "Iran has said it has frozen uranium enrichment programs WASHINGTON (CBS.MW) -- President Bush said Monday he wants to verify if Iran has indeed suspended its nuclear fuel program as it has said it has done.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "Iran has said it has frozen uranium enrichment programs WASHINGTON (CBS.MW) -- President Bush said Monday he wants to verify if Iran has indeed suspended its nuclear fuel program as it has said it has done.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Iran has said it has frozen uranium enrichment programs WASHINGTON (CBS.MW) -- President Bush said Monday he wants to verify if Iran has indeed suspended its nuclear fuel program as it has said it has done.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(entry value is at most 6); (it is not the case that (the elapsed days from case start date to case end date are at most 21))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 432. The case start date is 2027-05-03. The case end date is 2027-06-03. The entry value is 16.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,not(1))"}
{"request": {"context": "document: Have you ever heard the saying, \\\"if it's too loud, you're too old?\\\" Yes, it is the title of a truly terrible Kiss album, (aren't they all), but it is also the way I felt when walking into this place - it was so damn loud and apparently I'm too damn old! Maybe if the music would have been better it would have been more tolerable, but blasting Godsmack at 3:00 in the afternoon makes you wonder if you've been telaported into a scene from a really bad B-movie. Dark, stale and drizzled with just the right amount of tacky.\\n\\nThe trip here was short but it wasn't really intended to be - total time spent: 5 minutes. The thing that really got to all of us was the bartenders manipulating and guilting you into buying them shots, which all happened via hand gestures and lip reading due to the aforementioned mood music. I know that seems to be the thing here, but really, \\\"one for me too?\\\" It's not like you're sitting down, introducing yourself, enjoying our company or even small talking us. It's was as if we were expected to participate in this alcohol based act of philanthropy. Can you imagine walking up to the counter at In-N-Out Burger, ordering a number one combo and having the person helping", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "document: Have you ever heard the saying, \\\"if it's too loud, you're too old?\\\" Yes, it is the title of a truly terrible Kiss album, (aren't they all), but it is also the way I felt when walking into this place - it was so damn loud and apparently I'm too damn old! Maybe if the music would have been better it would have been more tolerable, but blasting Godsmack at 3:00 in the afternoon makes you wonder if you've been telaported into a scene from a really bad B-movie. Dark, stale and drizzled with just the right amount of tacky.\\n\\nThe trip here was short but it wasn't really intended to be - total time spent: 5 minutes. The thing that really got to all of us was the bartenders manipulating and guilting you into buying them shots, which all happened via hand gestures and lip reading due to the aforementioned mood music. I know that seems to be the thing here, but really, \\\"one for me too?\\\" It's not like you're sitting down, introducing yourself, enjoying our company or even small talking us. It's was as if we were expected to participate in this alcohol based act of philanthropy. Can you imagine walking up to the counter at In-N-Out Burger, ordering a number one combo and having the person helping", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "`` austin powers in goldmember '' has the right stuff for silly summer entertainment and has enough laughs to sustain interest to the end .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Expense claims of $500 or less are approved automatically. Claims above $500 require director sign-off.\ncase: The claim total is $825. Aiko, the account owner, submitted an expense claim. Receipts were attached for every line item.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "policy: A case is approved when either case value is greater than 25, except when entry value is equal to 16, or else the condition \"account value is at least 11\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The account value is 22. The entry value is 26. The case value is 36. The routing reference is 139.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(not(0),unless(1,2))"}
{"request": {"context": "policy: Approve exactly when (((case value is between 20 and 30, including both endpoints) OR (member value is less than 55)) AND (entry value is at least 45)) AND (shipment value is at least 7). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 56. The shipment value is 7. The case value is 19. The routing reference is 810. The member value is 65.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,and(1,or(2,3)))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: No Svet: Tomescu-Dita coasts to marathon win But even she didn #39;t believe it as she looked over her shoulder five times in the last 600 meters at Sunday #39;s LaSalle Bank Chicago Marathon.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: chat\n  body: No Svet: Tomescu-Dita coasts to marathon win But even she didn #39;t believe it as she looked over her shoulder five times in the last 600 meters at Sunday #39;s LaSalle Bank Chicago Marathon.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: chat\n  body: No Svet: Tomescu-Dita coasts to marathon win But even she didn #39;t believe it as she looked over her shoulder five times in the last 600 meters at Sunday #39;s LaSalle Bank Chicago Marathon.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "I told him Wednesday was the one day she wasn't there!", "question": "Hypothesis: \"I said she wasn't there on Wednesday.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when (at least one of these conditions hold: [((case value is greater than 27) holds and the exception (the elapsed days from member start date to member end date are at most 8) does not hold); (it is not the case that (shipment value is at most 41))]) holds and the exception (entry value is greater than 40) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member start date is 2027-05-23. The case value is 37. The shipment value is 51. The member end date is 2027-05-30. The entry value is 39. The routing reference is 242.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(or(not(0),unless(1,2)),3)"}
{"request": {"context": "policy: A case is approved when when the elapsed days from entry start date to entry end date are at most 44 the requirement is that the elapsed days from case start date to case end date are at most 41, and also shipment value is equal to 59, and when it is not the requirement is that the condition \"request value is between 57 and 67, including both endpoints\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry end date is 2027-05-02. The entry start date is 2027-03-19. The case start date is 2027-05-26. The request value is 67. The shipment value is 69. The routing reference is 309. The case end date is 2027-07-05.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,and(1,2),not(3))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: There are perhaps three hundred sign languages in use around the world today. The number is not known with any confidence; new sign languages emerge frequently through creolization and de novo (and occasionally through language planning). In some countries, such as Sri Lanka and Tanzania, each school for the deaf may have a separate language, known only to its students and sometimes denied by the school; on the other hand, countries may share sign languages, though sometimes under different names (Croatian and Serbian, Indian and Pakistani). Deaf sign languages also arise outside educational institutions, especially in village communities with high levels of congenital deafness, but there are significant sign languages developed for the hearing as well, such as the speech-taboo languages used in aboriginal Australia. Scholars are doing field surveys to identify the world's sign languages.", "question": "does sign language vary from country to country?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "What is a rhomboideus minor ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "What caused Harry Houdini 's death ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "Army pulls back from Gaza leaving 100 Palestinians dead The Israeli army last night began withdrawing from Palestinian areas of the northern Gaza strip, winding down the largest assault on the territory in recent years and leaving more than 100 Palestinians dead and widespread destruction of homes.", "question": "question: What is the topic of this article?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: what: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "Army pulls back from Gaza leaving 100 Palestinians dead The Israeli army last night began withdrawing from Palestinian areas of the northern Gaza strip, winding down the largest assault on the territory in recent years and leaving more than 100 Palestinians dead and widespread destruction of homes.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Army pulls back from Gaza leaving 100 Palestinians dead The Israeli army last night began withdrawing from Palestinian areas of the northern Gaza strip, winding down the largest assault on the territory in recent years and leaving more than 100 Palestinians dead and widespread destruction of homes.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: What is the most common name ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "document: Though derivative, \"Labyrinth\" still stands as the highlight of the mid-half of the six-year-old show. Finally a story allows Welling to show how he has grown as an actor. It's not easy playing a character that is the embodiment of \"truth, justice, and the American way\" on a weekly basis with very little variation. His performance, permitting him to show how one might react if he/she discovers that all that he knew may be a lie, was quite believable. Welling rose to the occasion marvelously. As always, Michael Rosenbaum, as the \"handicapped\" Lex, delivered, as did Kristen Kreuk as a too-sweet-to-be-believed Lana. Allison Mack, the ever-present Chloe, also scored as a slightly \"off-her-rocker\" version. The use of an annoying hum in the background added to the tone of the installment and made for an engaging drama.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "They'd be done in no time.", "question": "Hypothesis: \"They'd be done eating supper soon. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "the material is extremely sensitive. some of the canvas material starting sagging within the first week. beautiful piece though. i am sure one can tighten the material", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "Most bounty hunters are employed by bail bondsmen: the bounty hunter is usually paid about 10% of the total bail amount, but this commission can vary on an individual, case-by-case basis; usually depending upon the difficulty level of the assignment and the approach used to exonerate the bail bond. If the fugitive eludes bail, the bondsman, not the bounty hunter, is responsible for 100% of the total bail amount. This is a way of ensuring clients arrive at trial. As of 2003, bounty hunters claimed to catch 31,500 bail jumpers per year, about 90% of people who jump bail.", "question": "are there still bounty hunters in the us?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "document: What was Thatcher 's approval rating after 10 years in power ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "I was disappointed it didn’t maintain a seal with sparking wine. Tried it on a couple occasions only to have it leak.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: A case is approved when either either package value is at most 40, or else account value is at least 44, or else request value is at most 23, except when shipment value is at most 46; any other case is denied. Routing references play no part in the decision.\ncase: The request value is 24. The routing reference is 995. The shipment value is 45. The package value is 39. The account value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(or(0,1),unless(2,3))"}
{"request": {"context": "The color was great, but it was a cheap material and also smelled like fish.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when (review value is at most 37) holds and the exception (at least one of these conditions hold: [(case value is less than 42); (account value is less than 40)]) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 706. The review value is 37. The case value is 43. The account value is 41.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,or(1,2))"}
{"request": {"context": "ticket:\n  channel: email\n  body: Address is listed wrong- It's 4001 E Bell Rd, Phoenix, AZ 85032\\n\\nThe pizza was really good NY style. But the staff totally outshine the food. I came in with a friend asked if they could do a pizza half/half. They said they could even do quarters. We didn't need that but would be great for families:)\\n\\nMy friend is in a wheelchair and he is total care. they offered to cut the pizza in small bites. I didn't even ask. That was sweet of them to pay attention- And very helpful. We will be back:)", "question": "question: How many stars did this reviewer give?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "ticket:\n  channel: email\n  body: Address is listed wrong- It's 4001 E Bell Rd, Phoenix, AZ 85032\\n\\nThe pizza was really good NY style. But the staff totally outshine the food. I came in with a friend asked if they could do a pizza half/half. They said they could even do quarters. We didn't need that but would be great for families:)\\n\\nMy friend is in a wheelchair and he is total care. they offered to cut the pizza in small bites. I didn't even ask. That was sweet of them to pay attention- And very helpful. We will be back:)", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Sort of like a very primitive episode of \"General Hospital\" set in a natal ward (and one for tough cases at that), this fast-moving programmer has a satisfying emotional impact -- mainly because Eric Linden, as the distraught young husband in the main plot, is so palpably a wreck, and with such good reason. His expectant wife, Loretta Young, is brought to the ward at the beginning of a 20-year prison sentence for offing a lecher who probably had it coming to him; Ms. Young, as always, doesn't do anything to disinvite audience sympathy, and she's a little too good to be true, though sympathetic and lovely to look at, of course. Her difficult pregnancy and relationships with the other girls of the ward form the heart of the movie, and the outcome -- not an entirely happy one -- feels right. Aline MacMahon, \"one of the cinema's few perfect actresses,\" in the apt words of film historian David Thomson, exudes warmth and authority as the head nurse, and Glenda Farrell, as a none-too-willing new mom of twins, gets to croon \"Frankie and Johnny\" as a drunken lullaby. Frank McHugh figures in another subplot, and he gets to show more range than Warners usually permitted him. It's scaled and paced modestly, and Linden's expectant-dad panic stays with you for days", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: Robert Paul Weston (born October 21 1975) is a British-born Canadian children's author. His debut was the award-winning novel-in-verse Zorgamazoo. His short fiction has appeared in literary journals in Canada the UK and the United States.His second novel a dark fantasy for young adults entitled Dust City was published in October 2010. The story is narrated by the son of the wolf who killed Little Red Riding Hood.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "after seeing the film , i can tell you that there 's no other reason why anyone should bother remembering it .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Warranty claims made within 90 days of purchase are covered in full. Claims made after that but within 365 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on November 8, 2026. Jonas purchased a desk lamp on January 26, 2026. The claim describes a defect in the zipper.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: DECISION RULE [NOT: [IF member value is at least 14 THEN [NOT: shipment verified is yes] ELSE [ANY of: review value is equal to 22 | entry verified is yes]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 100. The member value is 15. The entry verified is yes. The review value is 33. The shipment verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(if(0,not(1),or(2,3)))"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: The claim total is $348. Lena, the warehouse lead, submitted an expense claim. Receipts were attached for every line item.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "policy: Approve exactly when (if (request value is between 55 and 65, including both endpoints), use (package value is greater than 46); otherwise use (case value is at most 23)) OR (NOT (the elapsed days from review start date to review end date are at most 51)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 65. The routing reference is 349. The package value is 45. The case value is 34. The review start date is 2027-02-15. The review end date is 2027-04-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(if(0,1,2),not(3))"}
{"request": {"context": "not all part were there so there's no way to turn it off. you have to take batteries out.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 30 days of the purchase date.\ncase: The order was paid by card and shipped to Jonas's home address. Jonas bought a desk lamp on January 10, 2026. The return request was submitted on January 14, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "Holy cow...\\n\\nI think I have a new must visit venue whenever I go to Las Vegas. Light Nightclub is put on by the Light Group of Las Vegas..and it doesn't disappoint one bit. The club consists of three levels and it's HUGE. Light shows, lasers, special effects..you name it. Hard to describe in words, but you will be in awe when you get here.\\n\\nIt opened over Memorial Day weekend and I visited this past weekend. Lines for general admission were extremely long so you best book table service or get in contact with a host!\\n\\nAs expected, table minimums is higher then other venues due to it's hype but that was expected. On a non-event weekend, expect minimums to start at 1k for the upper level tables. We ended up tipping the host and getting moved downstairs halfway through the night, and it was well worth it. Grey Goose starts off at around $575 a bottle and Ciroc goes for $670. \\n\\nIf you elect to go through general admission, be prepared to wait in line..HOWEVER..this place is awesome for people who need room to move around. The 2nd and 3rd floors have alot of room for people to hang out at. Places like Marquee and Tao crowd bodies into the main room to the point where you can't move around. At", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "Holy cow...\\n\\nI think I have a new must visit venue whenever I go to Las Vegas. Light Nightclub is put on by the Light Group of Las Vegas..and it doesn't disappoint one bit. The club consists of three levels and it's HUGE. Light shows, lasers, special effects..you name it. Hard to describe in words, but you will be in awe when you get here.\\n\\nIt opened over Memorial Day weekend and I visited this past weekend. Lines for general admission were extremely long so you best book table service or get in contact with a host!\\n\\nAs expected, table minimums is higher then other venues due to it's hype but that was expected. On a non-event weekend, expect minimums to start at 1k for the upper level tables. We ended up tipping the host and getting moved downstairs halfway through the night, and it was well worth it. Grey Goose starts off at around $575 a bottle and Ciroc goes for $670. \\n\\nIf you elect to go through general admission, be prepared to wait in line..HOWEVER..this place is awesome for people who need room to move around. The 2nd and 3rd floors have alot of room for people to hang out at. Places like Marquee and Tao crowd bodies into the main room to the point where you can't move around. At", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ANY of: account verified is yes | entry verified is yes] | [ANY of: case value is at least 38 | shipment verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment verified is no. The account verified is yes. The routing reference is 207. The case value is 37. The entry verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(or(0,1),or(2,3))"}
{"request": {"context": "document: Alcohol laws in Oregon permit the sale of beer, wine, and liquor, for on- or off-premises consumption, between 7 a.m. and 2:30 a.m. In 2004, Oregonians consumed 5,103,000 US gallons (19,320,000 L) of distilled spirits, 11,132,000 US gallons (42,140,000 L) of wine, and 80,415,000 US gallons (304,400,000 L) of beer, ranking it 27th, 19th, and 27th respectively of US states. Oregon had an estimated population of 3,594,586 in 2004, ranking it 27th among US states.", "question": "can you buy alcohol in oregon on sunday?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Girolamo da Treviso (1508 – September 10 1544) also known as Girolamo di Tommaso da Treviso the Younger and Girolamo Trevigi was an Italian Renaissance painter.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "This place totally hit the spot on a hot day by the pool. Who doesn't like booze and ice cream fused together. My wife got a coconut and pineapple rum popsicle and I went went with the Grasshopper Sunday with vodka caramel. I was a hot day by the pool and these desserts were just what the doctor ordered. I only gave 4 stars due to the fact that we went there a couple of times and it was \\\"closed due to weather\\\". If the pool is open, I expected the enclosed restaurant at the pool to be open. I guess you have to save money where you can...but still.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "This place totally hit the spot on a hot day by the pool. Who doesn't like booze and ice cream fused together. My wife got a coconut and pineapple rum popsicle and I went went with the Grasshopper Sunday with vodka caramel. I was a hot day by the pool and these desserts were just what the doctor ordered. I only gave 4 stars due to the fact that we went there a couple of times and it was \\\"closed due to weather\\\". If the pool is open, I expected the enclosed restaurant at the pool to be open. I guess you have to save money where you can...but still.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "St Andrew's Church Burnley is located in Colne Road Burnley Lancashire England. It is an active Anglican parish church in the deanery of Burnley the archdeaconry of Blackburn and the diocese of Blackburn. Its benefice is united with those of St Margaret Burnley and St James Burnley. The church has been designated by English Heritage as a Grade II listed building.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "building", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Approve exactly when (if (request value is between 55 and 65, including both endpoints), use (package value is greater than 46); otherwise use (case value is at most 23)) OR (NOT (the elapsed days from review start date to review end date are at most 51)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 65. The routing reference is 877. The package value is 45. The case value is 34. The review start date is 2027-02-15. The review end date is 2027-04-08.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(if(0,1,2),not(3))"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(the elapsed days from account start date to account end date are at most 7); (at least one of these conditions hold: [(at least one of these conditions hold: [(case value is equal to 24); (shipment value is between 22 and 32, including both endpoints)]); (entry value is greater than 35)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account start date is 2027-04-19. The routing reference is 105. The case value is 25. The entry value is 34. The shipment value is 22. The account end date is 2027-05-06.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,or(2,3)))"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 14 days of the purchase date.\ncase: The return request was submitted on June 19, 2026. The order was paid by card and shipped to Tomas's home address. Tomas bought a backpack on June 8, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "- role: customer\n  content: Good mexican food in a hotel?\\n\\nYes.\\n\\nIt's a limited menu, no combo platters for $9.95 here. Mostly tortas, street tacos, and side items. But they are delicious, none-the-less. Really fresh ingredients. And their sangria really hit the spot, ifyouknowwhatImean (no skimping on the alcohol). \\n\\nBut the main reason I eat here: chicharron aka DEEP FRIED CHEESE WALL. It's like a grilled cheese without that pesky bread.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "- role: customer\n  content: Good mexican food in a hotel?\\n\\nYes.\\n\\nIt's a limited menu, no combo platters for $9.95 here. Mostly tortas, street tacos, and side items. But they are delicious, none-the-less. Really fresh ingredients. And their sangria really hit the spot, ifyouknowwhatImean (no skimping on the alcohol). \\n\\nBut the main reason I eat here: chicharron aka DEEP FRIED CHEESE WALL. It's like a grilled cheese without that pesky bread.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "The Flash premiered in North America on October 7, 2014, where the pilot became the second-most watched premiere in the history of The CW, after The Vampire Diaries in 2009. It has been well received by critics and audiences, and won the People's Choice Award for ``Favorite New TV Drama'' in 2014. The series, together with Arrow, has spun characters out to their own show, Legends of Tomorrow, which premiered on January 21, 2016. On January 8, 2017, The CW renewed the show for a fourth season, which is scheduled to debut on October 10, 2017.", "question": "is there going ro be another season of the flash?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "I bought a jet ski here 30 years ago very happy. Just moved back to mesa and have\\na kawasaki teryk and buy alot of parts. I race locally and damage alot of parts. The parts people are very friendly and get my parts same week usually from kawasaki alway remember me that is a lost service feature by people who work in most places these days! Highly recommend these guys.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "I bought a jet ski here 30 years ago very happy. Just moved back to mesa and have\\na kawasaki teryk and buy alot of parts. I race locally and damage alot of parts. The parts people are very friendly and get my parts same week usually from kawasaki alway remember me that is a lost service feature by people who work in most places these days! Highly recommend these guys.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Approve exactly when if (account value is greater than 16), use (NOT (shipment value is between 11 and 21, including both endpoints)); otherwise use ((the elapsed days from review start date to review end date are at most 40) AND (package value is at most 48)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-02-27. The routing reference is 192. The shipment value is 12. The review end date is 2027-04-07. The account value is 27. The package value is 58.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,not(1),and(2,3))"}
{"request": {"context": "Murdoch's News Corp. to Join S P 500 (Reuters) Reuters - News Corp.  , the media\\company built by Rupert Murdoch, will join the Standard  \\Poor's 500 stock index after the close of trading on Dec. 17,\\S P said on Tuesday.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: what: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "Murdoch's News Corp. to Join S P 500 (Reuters) Reuters - News Corp.  , the media\\company built by Rupert Murdoch, will join the Standard  \\Poor's 500 stock index after the close of trading on Dec. 17,\\S P said on Tuesday.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Murdoch's News Corp. to Join S P 500 (Reuters) Reuters - News Corp.  , the media\\company built by Rupert Murdoch, will join the Standard  \\Poor's 500 stock index after the close of trading on Dec. 17,\\S P said on Tuesday.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Went twice these past two weeks. Got the \\\"calcyum\\\" juice, with kale, parsley, carrot, and apple. It was pretty good and a healthy alternative to the fast food options nearby. Also, they are really fast compared to other juice bars I've visited. My only gripe is with the service. Never really that friendly or engaging. On my last visit they wouldn't let me pay with debit, I guess the machine was busted. So, I paid with cash, received no receipt, and realised after getting in my car and heading to work that they had charged me $11 for one juice, twice the actual price. Kicking myself for not speaking up when I thought it sounded high. I will probably still pop in for a juice once in a while, because at the end of the day you go for the product. It's just a shame they don't put any focus into quality customer service.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "Went twice these past two weeks. Got the \\\"calcyum\\\" juice, with kale, parsley, carrot, and apple. It was pretty good and a healthy alternative to the fast food options nearby. Also, they are really fast compared to other juice bars I've visited. My only gripe is with the service. Never really that friendly or engaging. On my last visit they wouldn't let me pay with debit, I guess the machine was busted. So, I paid with cash, received no receipt, and realised after getting in my car and heading to work that they had charged me $11 for one juice, twice the actual price. Kicking myself for not speaking up when I thought it sounded high. I will probably still pop in for a juice once in a while, because at the end of the day you go for the product. It's just a shame they don't put any focus into quality customer service.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Customers may order at most 10 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: The order quantity is 12. Delivery was requested to a residential address. Elin placed an order for a pair of running shoes.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "slightly_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "ticket:\n  channel: chat\n  body: novak contemplates a heartland so overwhelmed by its lack of purpose that it seeks excitement in manufactured high drama .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "ticket:\n  channel: email\n  body: Wells (Priory Road) was a railway station on the Somerset and Dorset Joint Railway at Wells in the county of Somerset in England. Opening on 15 March 1859 as Wells, on the Somerset Central Railway, at that time a broad-gauge line operated by the Bristol and Exeter Railway, prior to that Company's amalgamation with the Dorset Central Railway to form the Somerset & Dorset, it was the terminus of the branch from Glastonbury.", "question": "is there a train station in wells somerset?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((request value is greater than 43) holds and the exception (account value is at most 52) does not hold); (at least one of these conditions hold: [(shipment value is at most 57); (member value is between 39 and 49, including both endpoints)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 53. The shipment value is 58. The routing reference is 724. The member value is 39. The request value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(or(0,1),unless(2,3))"}
{"request": {"context": "Update 1: German Business Sentiment Drops in Nov. Worries about the strengthening euro and concerns over persistently high oil prices helped push German business confidence down in November, a closely watched survey showed Thursday.", "question": "question: What is the topic of this article?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "Update 1: German Business Sentiment Drops in Nov. Worries about the strengthening euro and concerns over persistently high oil prices helped push German business confidence down in November, a closely watched survey showed Thursday.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Update 1: German Business Sentiment Drops in Nov. Worries about the strengthening euro and concerns over persistently high oil prices helped push German business confidence down in November, a closely watched survey showed Thursday.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: As of 2017, 24 Hour Fitness has over 4 million members and more than 420 clubs in 18 states, with more than 22,000 employees. Its major competitors in the US are Anytime Fitness, Gold's Gym, Aspen Fitness, and LA Fitness.", "question": "is la fitness the same as 24 hr fitness?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ANY of: account verified is yes | entry verified is yes] | [ANY of: case value is at least 38 | shipment verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment verified is no. The account verified is no. The routing reference is 879. The case value is 37. The entry verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(or(0,1),or(2,3))"}
{"request": {"context": "policy: Expense claims of $1,000 or less are approved automatically. Claims above $1,000 require director sign-off.\ncase: The claim total is $842. Receipts were attached for every line item. Tomas, the account owner, submitted an expense claim.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "- role: customer\n  content: In 1974, under Kreeger's leadership, GEICO began to insure the general public, after real-time access to computerized driving records became available throughout the United States, and it was briefly the fifth-largest U.S. auto insurer. By 1975, it was clear that GEICO had expanded far too rapidly (during the 1973--75 recession) when it reported a $126.5 million loss. To prevent GEICO from collapsing, a consortium of 45 insurance companies agreed to take over a quarter of its policies, and it was forced to issue a stock offering (thus diluting existing stockholders) to raise money to pay claims. It took five years (during which the company shrank significantly) and a massive reorganization to set GEICO on the path to recovery.", "question": "question: do you have to be a government employee to use geico?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: DECISION RULE [[ANY of: package signer is the same person as package designated approver | case verified is yes] UNLESS [ALL of: the elapsed days from review start date to review end date are at most 7 | account value is between 11 and 21, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review end date is 2027-05-20. The review start date is 2027-05-13. The package designated approver is Elin. The account value is 21. The routing reference is 849. The case verified is no. The package signer is Elin.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),and(2,3))"}
{"request": {"context": "document: It vacuums great, but the battery dies quick.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "the advantage of a postapocalyptic setting is that it can be made on the cheap .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: A case is approved when member value is equal to 8, and also shipment value is less than 13, and also the elapsed days from entry start date to entry end date are at most 44, and also case verified is yes; any other case is denied. Routing references play no part in the decision.\ncase: The routing reference is 382. The entry start date is 2027-02-04. The entry end date is 2027-03-20. The shipment value is 12. The member value is 18. The case verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),and(2,3))"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 45 days of the purchase date.\ncase: Sana bought a pair of running shoes on February 16, 2026. The return request was submitted on April 3, 2026. The order was paid by card and shipped to Sana's home address.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "policy: A case is approved when member value is between 30 and 40, including both endpoints, except when account verified is yes, and also package signer is the same person as package designated approver, and also review value is less than 14; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 31. The review value is 15. The account verified is yes. The routing reference is 421. The package signer is Aiko. The package designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(1,and(2,3)))"}
{"request": {"context": "This review is for the seller not the product. Terrible! This is how the box came - definitely repackaged and there’s a huge gash in the sports bra!!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "Jon turned and looked to each of them.", "question": "Hypothesis: \"He spoke to them in a calm, quiet voice.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "US tops league of e-mail spammers The US is responsible for most of the spam e-mails that fill up computer inboxes, says a report.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "US tops league of e-mail spammers The US is responsible for most of the spam e-mails that fill up computer inboxes, says a report.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "US tops league of e-mail spammers The US is responsible for most of the spam e-mails that fill up computer inboxes, says a report.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "What country is the world 's largest importer of cognac ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Expense claims of $500 or less are approved automatically. Claims above $500 require director sign-off.\ncase: The claim total is $469. Aiko, the account owner, submitted an expense claim. Receipts were attached for every line item.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "it 's uninteresting .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "One of his friends, the poet Robert Graves, wrote a poem about his antics.", "question": "question: Hypothesis: \"Another one of his friends criticized the poem Robert Graves wrote.\" How does it relate to the premise?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "- role: customer\n  content: those who are only mildly curious , i fear , will be put to sleep or bewildered by the artsy and often pointless visuals .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "document: What is deadrise , nautical , ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "Coventry High School is a public high school located in Coventry Township near Akron Ohio United States. It is the only high school in the Coventry Local School District. It has been rated Excellent with distinction for three years running.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Mrs. Inglethorp does not take her medicine that night.", "question": "question: Hypothesis: \"Mrs. Inglethorp took her medicine that night.\" How does it relate to the premise?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: what: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "Onkyo #39;s DVD Changer Spins a New Tune Choosing a DVD player used to be simple: progressive scan or no progressive scan? These days, even the DVD players on sale for \\$29 at Wal-Mart feature progressive scanning.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: what: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: what: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Onkyo #39;s DVD Changer Spins a New Tune Choosing a DVD player used to be simple: progressive scan or no progressive scan? These days, even the DVD players on sale for \\$29 at Wal-Mart feature progressive scanning.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Onkyo #39;s DVD Changer Spins a New Tune Choosing a DVD player used to be simple: progressive scan or no progressive scan? These days, even the DVD players on sale for \\$29 at Wal-Mart feature progressive scanning.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: A case is approved when when the condition \"the elapsed days from package start date to package end date are at most 58\" fails the requirement is that shipment value is less than 46, and when it is not the requirement is that request value is greater than 32; any other case is denied. Routing references play no part in the decision.\ncase: The shipment value is 46. The package end date is 2027-03-17. The package start date is 2027-01-17. The routing reference is 858. The request value is 31.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(not(0),1,2)"}
{"request": {"context": "Macropiper hooglandii commonly known as Kava is a flowering plant in the pepper family. The specific epithet honours Dutch botanist Ruurd Dirk Hoogland.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "plant", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "The Doctor of Business Administration (commonly abbreviated as DBA, DrBA, or BusD and most recently ``PH.DBA''. ) is a research doctorate awarded on the basis of advanced study and research in the field of business administration. The D.B.A. is a terminal degree in business administration. Along with the Ph.D., it represents the highest academic qualification in business administration. Successful completion of a D.B.A. or Ph.D. in business administration is required to gain employment as a full-time, tenure-track university professor or postdoctoral researcher in the field. As with other earned research doctorates, individuals with the degree are awarded the academic title doctor, which is often represented via the English honorific ``Dr.'' or the post-nominal letters ``DBA'',or ``Ph.D.''.", "question": "can you get a doctorate in business administration?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "document: The Newfoundland--Labrador fixed link refers to various proposals for constructing a fixed link consisting of bridges, tunnels, and/or causeways across the Strait of Belle Isle, connecting the province of Newfoundland and Labrador's mainland Labrador region with the island of Newfoundland. This strait has a minimum width of 17.4 km (10.8 mi).", "question": "is there a bridge between newfoundland and labrador?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: A case is approved when the condition \"either either case signer is the same person as case designated approver, or else package value is equal to 43, or else the elapsed days from entry start date to entry end date are at most 56\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The case signer is Aiko. The case designated approver is Noah. The entry start date is 2027-07-13. The package value is 44. The entry end date is 2027-09-07. The routing reference is 619.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(or(0,or(1,2)))"}
{"request": {"context": "What is the country of origin for the name Thomas ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: A case is approved when case verified is yes, except when request value is greater than 47, and also account signer is the same person as account designated approver; any other case is denied. Routing references play no part in the decision.\ncase: The case verified is yes. The account signer is Kofi. The request value is 46. The account designated approver is Kofi. The routing reference is 865.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(0,unless(1,2))"}
{"request": {"context": "- role: customer\n  content: Despite the location in south Scottsdale, Avalon is just a really classy place that has a lot to offer. The dining experience is fabulous. The food is delicious. The atmosphere is top-notch. The music is transcendent - very hypnotic. The vibe is cool. The decor is first-class, upscale NY/LAish. The service is more than decent. They have brunch on weekends, live music and cooking classes. I highly recommend Avalon. It is worth checking out. Say hello to Dominick. Thinking about a company party or group celebration. This is the spot.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "- role: customer\n  content: Despite the location in south Scottsdale, Avalon is just a really classy place that has a lot to offer. The dining experience is fabulous. The food is delicious. The atmosphere is top-notch. The music is transcendent - very hypnotic. The vibe is cool. The decor is first-class, upscale NY/LAish. The service is more than decent. They have brunch on weekends, live music and cooking classes. I highly recommend Avalon. It is worth checking out. Say hello to Dominick. Thinking about a company party or group celebration. This is the spot.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(entry value is at most 6); (it is not the case that (the elapsed days from case start date to case end date are at most 21))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 432. The case start date is 2027-05-03. The case end date is 2027-06-03. The entry value is 5.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(0,not(1))"}
{"request": {"context": "policy: Approve exactly when ((account signer is the same person as account designated approver) holds and the exception (review value is equal to 56) does not hold) holds and the exception (the elapsed days from entry start date to entry end date are at most 39) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account designated approver is Kofi. The entry start date is 2027-06-16. The account signer is Aiko. The review value is 57. The entry end date is 2027-07-26. The routing reference is 817.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(unless(0,1),2)"}
{"request": {"context": "document: Just got back from a free screening and I'm very glad I didn't pay to see this very sub-par film. The theater was full and the crowd was a mix of kids and adults. It seemed like it was just the kids who were laughing at all the slap-stick and fart jokes though (good god they loved to hit these poor mice in the crotch a lot!). The movie is pretty juvenile, unintelligent, predictable, and mostly annoying. The characters just seem to be thrown together to fill in empty space and the relationships between them all seemed very forced with no charm at all. Visually, the film is about average with nothing that really stands out. They did a decent job of mimicking the clay look from Wallace and Gromit, but other than that it's very forgettable imagery. Although I was really bored throughout the whole film, I chuckled a couple times. It's not an absolute failure, but I most definitely would not want to watch it again. If you're a parent with kids (and you don't care that your kids see mindless cheap-jokes) then feel free to take them to see it, but everyone else shouldn't waste their money.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "Ann Inc. is an American group of specialty apparel retail chain stores for women. The company headquartered in New York City and currently operates as a subsidiary of Ascena Retail Group. The stores offer classic styled suits, separates, dresses, shoes and accessories. The brand is marketed under five divisions: Ann Taylor, Loft, Lou & Grey, Ann Taylor Factory and Loft Outlet.", "question": "is ann taylor and the loft the same store?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "``Mr. Monk and the End'' is the two-part series finale of the USA Network original criminal mystery dramedy television series, Monk. It is the fifteenth and sixteenth episodes of the eighth and final season, and is the 124th and 125th episodes in the series overall. Adrian Monk (Tony Shalhoub) finally discovers his wife Trudy's (Melora Hardin) murderer after twelve years of searching, concluding a seven-year, eight-season long arc. When ``Part 2'' aired, it set a series high and a new viewership record for the most watched episode of a regular drama series ever in basic cable with 9.4 million viewers. Both parts were written by series creator Andy Breckman and directed by Randall Zisk.", "question": "did monk ever find out who killed his wife?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Applicants must be at least 21 years old to be eligible for the volunteer driver program.\ncase: Sana is 16 years old. Sana applied to join the volunteer driver program. The application form was complete and signed.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "The colors are pretty but based on the picture I certainly expected them to be fuller...they look very blah because they are so thin.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "- role: customer\n  content: Unlike some comments, mine is positive. This movie wraps around the dinner table with a group of friends, some you like, some you don't. A few are related--mother, daughter, son. Their stories are not one smooth, happy with everyone and everything, type of life--much like real life. Some story lines do not evolve, they just happen. But like true families and good friends, they stick together. The wanna-be parents who are buying a baby are such a--holes! You are happy for the ending. Poor Delmar is stuck between a rock and a boulder taking care of herself, her mom, her son, and trying to keep all their lives together. This does not end with a sunset walk or house in the 'burbs and all are living in a dream world, but is a very real life portrayal of people living day to day, month to month. Overall, this is a good story and a great movie!!", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "Shintarô Katsu, best known for the Zatôichi films, again stars in this third and final movie in the Kenji Misumi (mostly known for \"Lone Wolf and Cub), directed saga of Hanzo 'The Razor' Itami feature the big dicked one battling ninjas, rapeing 'ghosts', and uncovering shady goings on at the Shogunate treasury. The Hanzo 'plot' was kinda getting stale and repetitive. What was once novel in the first film, was not any longer. Fortunately, this one was better then the second thanks to having more humor. I'm just glad that they choose to stop at the one trilogy (I'm looking at YOU Lucas) My Grade: B DVD Extras: Merely Trailers for all 3 Hanzo the Razor films Eye Candy: Aoi Nakajima unleashes both tits, Mako Midori just her left one", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "MCI Reports \\$3.4 Billion Loss MCI Inc. (MCIP.O: Quote, Profile, Research) on Thursday reported a \\$3.4 billion third-quarter loss as it wrote down the value of its assets due to a withering price war for long-distance telephone service.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "MCI Reports \\$3.4 Billion Loss MCI Inc. (MCIP.O: Quote, Profile, Research) on Thursday reported a \\$3.4 billion third-quarter loss as it wrote down the value of its assets due to a withering price war for long-distance telephone service.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "MCI Reports \\$3.4 Billion Loss MCI Inc. (MCIP.O: Quote, Profile, Research) on Thursday reported a \\$3.4 billion third-quarter loss as it wrote down the value of its assets due to a withering price war for long-distance telephone service.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Rainfall and the tropical climate dominate the tropical rain belt, which oscillates from the northern to the southern tropics over the course of the year, roughly following the solar equator. The tropical rain belt is an area of active rain that is positioned mostly around the tropics. According to the website Journey North, the reason the rain belt is situated near the tropics can be attributed to the fact that most of the sun's radiation is directed toward the equator, which is located in the middle of the tropics. This solar radiation generates large amounts of heat near the equator providing tropical regions with higher temperatures than most other regions on Earth.", "question": "does it rain a lot in the tropics?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Back here in Tubacca he was conscious how much he had allowed his personal affairs to drift from day to day.", "question": "Hypothesis: \"He never came back to Tubacca, instead choosing to forge west into unknown lands. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 45 days of the purchase date.\ncase: The order was paid by card and shipped to Ravi's home address. Ravi bought a rain jacket on March 19, 2026. The return request was submitted on April 24, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "Nordstrom earnings beat estimates SEATTLE -- Nordstrom Inc. #39;s income soared 71 percent in its latest quarter as the clothing chain continued to focus on improving merchandise and managing expenses.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "Nordstrom earnings beat estimates SEATTLE -- Nordstrom Inc. #39;s income soared 71 percent in its latest quarter as the clothing chain continued to focus on improving merchandise and managing expenses.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Nordstrom earnings beat estimates SEATTLE -- Nordstrom Inc. #39;s income soared 71 percent in its latest quarter as the clothing chain continued to focus on improving merchandise and managing expenses.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when ((account signer is the same person as account designated approver) holds and the exception (review value is equal to 56) does not hold) holds and the exception (the elapsed days from entry start date to entry end date are at most 39) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account designated approver is Kofi. The entry start date is 2027-06-16. The account signer is Aiko. The review value is 57. The entry end date is 2027-07-26. The routing reference is 817.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(unless(0,1),2)"}
{"request": {"context": "The Brookeville Academy is located in the heart of the Historic District in Brookeville Maryland a town located twenty miles (32 km) north of Washington D.C. and two miles (3 km) north of Olney in northeastern Montgomery County Maryland. It was built in 1810 as one of the first private schools in Montgomery County Maryland. Now fully restored it serves as a unique community center suitable for meetings social and cultural events.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Hokies Take It One at a Time (AP) AP - No. 15 Virginia Tech is all alone in first place in the Atlantic Coast Conference with three games left, including one against Virginia and another against Miami.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Hokies Take It One at a Time (AP) AP - No. 15 Virginia Tech is all alone in first place in the Atlantic Coast Conference with three games left, including one against Virginia and another against Miami.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Hokies Take It One at a Time (AP) AP - No. 15 Virginia Tech is all alone in first place in the Atlantic Coast Conference with three games left, including one against Virginia and another against Miami.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "What effect does friction have on perpetual motion ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "This is a very cheaply made werewolf flick. The video is dark and poorly lit. The audio is uneven and poorly recorded and mixed. The script is cliche ridden junk with the usual characters like the tough detective who shoots werewolves with his silver handgun! [filled of course with silver bullets]. The acting is as wooden as the characters. The FX are non-existent,lots of extreme close-ups of werewolf jaws and biting. the only thing that is shown is lots of soft-core T&A. Instead of dropping $30 for this tripe check out a really great recent werewolf pic: \"Dog Soldiers\" with Sean Pertwee.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "Dozens of men and women, each chained by one ankle, surrounded each one of seven fires.", "question": "Hypothesis: \"Nobody surrounded the seven fires.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "The Northern Light was an American clipper ship. In 1853 she sailed from San Francisco California to Boston Massachusetts via Cape Horn with Captain Freeman Hatch at the helm in a record-setting 76 days 6 hours. The record still stands for a single hull vessel. In 1993 the record was soundly broken by a multi-hull sailing vessel Great American II with no cargo.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "meanoftransportation", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "This was decent for Chinese. The service was great and the people were friendly. However the food was run of the mill Chinese.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "This was decent for Chinese. The service was great and the people were friendly. However the food was run of the mill Chinese.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Two Sides to Every Story is a 1977 album by American singer-songwriter Gene Clark. The album was Clark's first release since his 1974 album No Other. Notable tracks include In the Pines (traditional) Kansas City Southern and Hear the Wind both written by Clark and Give My Love to Marie a song written by James Talley that tells the story of a dying coal miner.The album was produced by Thomas Jefferson Kaye.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "album", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: Kofi, the billing manager, submitted an expense claim. The claim total is $211. Receipts were attached for every line item.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "In my opinion, this is a pretty good celebrity skit show. I enjoyed seeing Greg Kinnear as the host. There are many reasons why I said that. Even though Hal Sparks was an okay host, I sometimes wish that Greg Kinnear hadn't left. If you ask me, it seems that nobody stays with a TV show throughout its entire run anymore. Still, I enjoyed seeing the various hosts and other people spoofing celebrities. If you ask me, that was pretty darn funny. Before I wrap this up, I must say that I kind of miss this show. Now, in conclusion, I highly recommend this show to all you die-hard sketch show fans. You will really enjoy it.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Customers may order at most 5 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Mira placed an order for a rain jacket. Delivery was requested to a residential address. The order quantity is 3.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "Weak start, solid middle, fantastic finish. That's my impression of this film, anyway. I liked Simon Pegg in the two films I've seen him in--- Hot Fuzz, and Shaun of the Dead. His role here, though, took a completely different turn. Shows his range as an actor, but nonetheless I really disliked th character as he was portrayed at the beginning. There's a kind of humour I call \"frustration comedy.\" Its supposed \"jokes\" and wit are really nothing more than painful and awkward moments. Much like the Bean character Rowan Atkinmson plays. There are a number of other comedic actors who portray similar characters too. I don't mean to bash them here, so will not. But do be warned that if you are like me, and you dislike smarmy and maddeningly bungling idiots, Pegg shows just such characteristics for the first third of this film. It DOES get better, however. I read somewhere that this is based on a true story. Hmmm. Maybe. The film's story stopped being annoying, and became kind of a triumph of the \"little guy\" in the final third. I don't need all films to be sugar and light--- but coincidentally, as this film got better, it also started to be more and more of a happy ending. It was also a pleasure to see an", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "I was so eager to see this one of my favorite TV shows.I saw Universal trademark followed with a newly acquainted title and theme song which still impress me.Computer animation on some scenery like a solid title name\"The Jetsons\" or a dimension view of a spaceship approaching an amusement park and more made this version splendid and fantastic.Shortly after that till the end...I couldn't believe my eyes!!!!How lucky I was that I could forget all I had seen.Just songs by Tiffany and its theme song in new arrangement were in my head.Anyway,I wish to see this space-aged family (also The Flintstones and Yogi Bear) in all graphic computer design as Toy story or Bug's life.The best style for Hanna-Barbera's in my opinion.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: House of Games is a wonderful movie at multiple levels. It is a fine mystery and a shocking thriller. It is blessed with marvelous performances by Lindsay Crouse and Joe Montegna, and a strong, strong cast of supporting players, and it introduces Ricky Jay, card sharp extraordinaire, prestidigitator and historian of magic. Its dialogue, written by David Mamet, is spoken as if in a play of manners and gives the movie (in which reality is often in question) an extra dimension of unrealness. On the face of it, House of Games is a convincing glimpse into the unknown world of cheats and con men, diametrically different from The Sting, which was played merely for glamour and yuks. At this level it does succeed admirably. However, you cannot escape the examination at a deeper level of the odyssey of a woman from complacent professional competence to incredible strength and self realization. The only movie I know of which treats the theme of emergence of personal strength in a woman in as worthy a way is the underrated Private Benjamin. That thoroughly enjoyable movie unfortunately diffuses its focus, hopping among several themes and exploiting the fine performance of Goldie Hawn to chase after some easy laughs. House of Games sticks to its business. As Poe once said of a good short story,", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when (both these conditions hold: [(account value is equal to 32); (the elapsed days from case start date to case end date are at most 7)]) holds and the exception ((request signer is the same person as request designated approver) holds and the exception (member value is at least 38) does not hold) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 794. The request designated approver is Noah. The account value is 31. The member value is 39. The request signer is Ravi. The case start date is 2027-04-07. The case end date is 2027-04-14.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),unless(2,3))"}
{"request": {"context": "How much does it cost , average or whatever is available , to produce and send junk mail catalogues in the US ? , DT CD NN NN ,", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Approve exactly when ((case value is between 9 and 19, including both endpoints) OR (request value is at most 12)) holds and the exception (review value is at most 42) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The case value is 20. The routing reference is 277. The review value is 43. The request value is 22.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),2)"}
{"request": {"context": "document: Rupankar Bagchi (Bengali: রুপঙ্কর বাগচী) is a Bengali singer from Kolkata West Bengal India.He has sung many famous songs in recent Bengali films like Baishe Srabon (2011) Aparajita Tumi (2012) Chalo Let's Go (2008) Hemlock society (2012) Dutta Vs Dutta (2012) and Jaatishwar (2014). He was awarded the National Film Award for Best Male Playback Singer for the song E Tumi Kemon Tumi from Jaatishwar.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Why don't you refresh your recollection for a while, and we'll talk when I'm back.", "question": "Hypothesis: \"You refresh your memories, and we can talk when I return.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when if (account value is greater than 16), use (NOT (shipment value is between 11 and 21, including both endpoints)); otherwise use ((the elapsed days from review start date to review end date are at most 40) AND (package value is at most 48)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-02-27. The routing reference is 615. The shipment value is 12. The review end date is 2027-04-07. The account value is 27. The package value is 58.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,not(1),and(2,3))"}
{"request": {"context": "The Vidyodaya Pirivena was one of the prominent piriven (a monastic college similar to a seminary for the education of Buddhist monks) in Sri Lanka.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "document: Why do airliners crash vs. gliding down ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "just dreadful .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "We always visit this movie theater because it is so close to our home. On Monday 6/27, my experience was very different. We spent 20$ for 2 to see a movie (Cars 2) and 25$ for popcorn/chips/drinks. The food was so stale! The popcorn tasted \\\"old\\\" and the chips were well over their expiration date. Very disappointing!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "We always visit this movie theater because it is so close to our home. On Monday 6/27, my experience was very different. We spent 20$ for 2 to see a movie (Cars 2) and 25$ for popcorn/chips/drinks. The food was so stale! The popcorn tasted \\\"old\\\" and the chips were well over their expiration date. Very disappointing!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "this is one of the rarest kinds of films : a family-oriented non-disney film that is actually funny without hitting below the belt .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: A case is approved when member value is equal to 8, and also shipment value is less than 13, and also the elapsed days from entry start date to entry end date are at most 44, and also case verified is yes; any other case is denied. Routing references play no part in the decision.\ncase: The routing reference is 949. The entry start date is 2027-02-04. The entry end date is 2027-03-20. The shipment value is 12. The member value is 18. The case verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),and(2,3))"}
{"request": {"context": "Troyes is also a center for shopping, with two outlet centers selling both French and international designer-name fashions and home accessories.", "question": "question: Hypothesis: \"Troyes is all factories.\" How does it relate to the premise?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [ANY of: [review value is at most 33 UNLESS member value is at least 10] | [ALL of: case verified is yes | package value is less than 26]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review value is 33. The case verified is no. The package value is 25. The routing reference is 212. The member value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),unless(2,3))"}
{"request": {"context": "I thought \"Intensive Care\" was quite bad and very unintentionally funny. But at least not as bad as I thought it might be. Sometimes it's somewhat suspenseful, but never a good shocker. SPOILER AHEAD The fun lies in ridiculous moments. But the all-time classic moment is this: Peter (Koen Wauters) is stabbed and beaten by the killer. He lies moaning in the corner of the hallway. Amy (Nada van Nie) kneels beside him and asks \"Poor Peter, shall I get you a band-aid?\". This movie was shot in Dutch and English. To spare costs, all license plates are USA, and the background in the news studio is a skyline of Manhattan. Very funny if you're Dutch and watching the original version in Dutch.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "ticket:\n  channel: chat\n  body: The exhaust hose on a portable air condition can get up to 115-120 degrees. This make sure that the heat goes out the duct and does not stay in the room that you are trying to cool. Plus it's a little quieter too. Worth it, especially if you have a permanent installation through the wall like I do.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: A case is approved when either case value is greater than 25, except when entry value is equal to 16, or else the condition \"account value is at least 11\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The account value is 22. The entry value is 26. The case value is 36. The routing reference is 955.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(not(0),unless(1,2))"}
{"request": {"context": "It can blow up the tires in a short time, more faster than other portable tire inflators. Otherwise, it has a bright led light, so It is convenient to blow up the tires in the dark environment. Work great!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The ticket was routed through the email queue. Ravi opened a priority ticket about a login failure. The first response arrived 28 hours after the ticket was opened.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "Once it rain, it sag and the water stays on top.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "ticket:\n  channel: email\n  body: IM Worm Crawls Through JPEG Hole Attack largely unsuccessful, but security experts warn of more to come.", "question": "question: What is the topic of this article?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: what: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: email\n  body: IM Worm Crawls Through JPEG Hole Attack largely unsuccessful, but security experts warn of more to come.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: IM Worm Crawls Through JPEG Hole Attack largely unsuccessful, but security experts warn of more to come.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: DECISION RULE [[case value is less than 29 UNLESS [ANY of: request value is greater than 49 | entry verified is yes]] UNLESS the elapsed days from account start date to account end date are at most 13] -> approve; otherwise reject. Ignore the routing reference.\ncase: The request value is 48. The routing reference is 271. The account end date is 2027-04-17. The account start date is 2027-03-25. The case value is 28. The entry verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(unless(0,or(1,2)),3)"}
{"request": {"context": "Alex has been trimming my trees for years working for the other guys. I have been telling him for years that he needs to go out on his own, now that he has I'm an All Star customer. I do and will recommend his work to anyone!\\n\\nKen Gilbert Arizona", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "Alex has been trimming my trees for years working for the other guys. I have been telling him for years that he needs to go out on his own, now that he has I'm an All Star customer. I do and will recommend his work to anyone!\\n\\nKen Gilbert Arizona", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "I always enjoy this movie when it shows up on TV. The one scene that always stands out, for me that is, is the one with the Myrna Loy and the painters foreman, where she gives him very explicit instructions on the colours and as soon as she goes away he turns the his guys and says \"Did you get that, that's yellow, blue, green and white\"", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when ((package verified is yes) AND (entry value is at least 36)) OR ((request value is at most 41) AND (shipment value is between 50 and 60, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 472. The entry value is 35. The request value is 51. The package verified is yes. The shipment value is 51.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),and(2,3))"}
{"request": {"context": "Received product on time, however the product failed to work. I hired a professional to install the this product. When installed the Bluetooth worked perfectly, however the power buttons failed to work. The professional said that these were faulty switches. I would not recommend this product based on the professional installers advice. I have a Ashley power recliner with a Apex motor.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "Deal with it.'", "question": "Hypothesis: \"You can go if you'd like.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [NOT: [IF member value is at least 14 THEN [NOT: shipment verified is yes] ELSE [ANY of: review value is equal to 22 | entry verified is yes]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 100. The member value is 15. The entry verified is yes. The review value is 33. The shipment verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(if(0,not(1),or(2,3)))"}
{"request": {"context": "policy: Approve exactly when ((case value is between 9 and 19, including both endpoints) OR (request value is at most 12)) holds and the exception (review value is at most 42) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The case value is 20. The routing reference is 917. The review value is 43. The request value is 22.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),2)"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 30 days of the purchase date.\ncase: The return request was submitted on March 3, 2026. The order was paid by card and shipped to Noah's home address. Noah bought a backpack on January 25, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "Karobar National Economic Daily (Nepali कारोबार राष्ट्रिय आर्थिक दैनिक) is a Nepali language Daily economic newspaper published in Kathmandu Nepal. The newspaper began its circulation in December 2009.Karobar Economic Daily is the first daily newspaper in Nepal that contains economy related news. The newspaper is published in peach newsprint making it the first in the country. The newspaper is currently published simultaneously from Kathmandu and Itahari.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "writtenwork", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Invoices paid within 10 days after the due date incur no fee. Payments between 11 and 60 days late incur a 2% fee. Payments later than 60 days incur a 10% fee and a hold on the account.\ncase: Mira's invoice for $120 fell due last quarter. The payment was made by card. Payment was received 101 days after the due date.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "Fez has been on my list of places to try for some time now and I guess I could've waited a little longer. I will say, the service was great and our server was very attentive. The food fell short of my expectations. I had the lamb sliders for dinner and I just found the meal to be lacking seasoning and flavor for lamb. Would have to say that Fez Fizzed!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "Fez has been on my list of places to try for some time now and I guess I could've waited a little longer. I will say, the service was great and our server was very attentive. The food fell short of my expectations. I had the lamb sliders for dinner and I just found the meal to be lacking seasoning and flavor for lamb. Would have to say that Fez Fizzed!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "H-2A aliens are required by law to leave the country within ten days of the termination of their employment, and generally remain in the control of the employer during this period.", "question": "Hypothesis: \"Law requires H-2ZZ workers to leave the country within 60 days after their employment has been terminated.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ANY of: case value is at most 31 | review verified is yes] | [ANY of: entry signer is the same person as entry designated approver | member verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review verified is no. The member verified is yes. The routing reference is 346. The entry designated approver is Elin. The case value is 42. The entry signer is Noah.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(or(0,1),or(2,3))"}
{"request": {"context": "Quezon City Science High School (or to its students Quesci or Kisay) is the Regional Science High School for the National Capital Region. It is the premier science high school of Quezon City and is regarded as one of the prestigious sciences triumvirate of the Republic of the Philippines along with the Philippine Science High School and Manila Science High School. It is located at Bago-Bantay Quezon City Philippines.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "document: Much bigger than expected but very nice and well packaged no damages or faults", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when (NOT ((shipment signer is the same person as shipment designated approver) OR (review value is less than 22))) holds and the exception (NOT ((case signer is the same person as case designated approver) AND (entry value is greater than 12))) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 22. The case designated approver is Mira. The routing reference is 548. The shipment designated approver is Ravi. The shipment signer is Tomas. The review value is 22. The case signer is Mira.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(not(or(0,1)),not(and(2,3)))"}
{"request": {"context": "- role: customer\n  content: It has also led to operating difficulties and to high processing costs.", "question": "Hypothesis: \"Although it has led to operating difficulties, the processing costs were lower usual.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "document: What is the definition of low island ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: A case is approved when either either package value is at most 40, or else account value is at least 44, or else request value is at most 23, except when shipment value is at most 46; any other case is denied. Routing references play no part in the decision.\ncase: The request value is 24. The routing reference is 357. The shipment value is 45. The package value is 39. The account value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(or(0,1),unless(2,3))"}
{"request": {"context": "policy: Invoices paid within 5 days after the due date incur no fee. Payments between 6 and 30 days late incur a 2% fee. Payments later than 30 days incur a 10% fee and a hold on the account.\ncase: Payment was received 15 days after the due date. The payment was made by cheque. Noah's invoice for $2,300 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "policy: A case is approved when when the condition \"the elapsed days from package start date to package end date are at most 58\" fails the requirement is that shipment value is less than 46, and when it is not the requirement is that request value is greater than 32; any other case is denied. Routing references play no part in the decision.\ncase: The shipment value is 46. The package end date is 2027-03-17. The package start date is 2027-01-17. The routing reference is 489. The request value is 31.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(not(0),1,2)"}
{"request": {"context": "There seems to be only two types of reviews of this film on the net. Those who hate it and curse Ralph Bakshis name and those love it and call it work of genious. I'm inclined to be in the middle. I'am forced to agree with most of the criticisms of this film (e.g.the cruel cutting of the story, badly rotoscoped charecters, over acting etc...) But dispite this I still love this film. The rotoscoping (when done properly)adds an eerie lifelike dimension to the charecters and the final battle scene at the end of the film is fantastic. The surrealistic scenes when the nineriders chase Frodo are stylish and well executed and the musical score... magic. Sadly the bad points outweight film but if you can bring yourself to ignore them it is a great film. (No doubt I'll be lynched by an angry mob of people who hate this film after writing this review, ah well, such is life)", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when NOT ((request value is at least 60) holds and the exception (review value is between 5 and 15, including both endpoints) does not hold). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 70. The review value is 4. The routing reference is 392.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(unless(0,1))"}
{"request": {"context": "This is a strange sex comedy because there`s very little comedy but a whole lot of sex , most of which takes place in the back of a car and is so graphic it makes BASIC INSTINCT look like an edition of TELETUBBIES . I kid you not , the sex in this film is so in your face it might just choke you , especially if you`re watching with your grandparents . As for the rest of RITA , SUE AND BOB TOO there`s not much else worth mentioning except the language where every single sentence seems to contain the F word . This a rather bleak hyper-realist British film made at a time when Channel 4 was the main investor in British movies which explains its made for television low budget feel", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: A case is approved when either either package value is at most 40, or else account value is at least 44, or else request value is at most 23, except when shipment value is at most 46; any other case is denied. Routing references play no part in the decision.\ncase: The request value is 24. The routing reference is 995. The shipment value is 45. The package value is 39. The account value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(or(0,1),unless(2,3))"}
{"request": {"context": "policy: Warranty claims made within 365 days of purchase are covered in full. Claims made after that but within 1095 days are covered at half cost. Later claims are not covered.\ncase: The claim describes a defect in the stitching. Jonas purchased a rain jacket on May 28, 2026. A warranty claim for it was filed on February 4, 2027.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ANY of: case value is at most 31 | review verified is yes] | [ANY of: entry signer is the same person as entry designated approver | member verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review verified is no. The member verified is yes. The routing reference is 975. The entry designated approver is Elin. The case value is 42. The entry signer is Noah.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(or(0,1),or(2,3))"}
{"request": {"context": "policy: Invoices paid within 10 days after the due date incur no fee. Payments between 11 and 60 days late incur a 2% fee. Payments later than 60 days incur a 10% fee and a hold on the account.\ncase: Payment was received 82 days after the due date. The payment was made by bank transfer. Lena's invoice for $120 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "policy: DECISION RULE [[NOT: entry value is between 12 and 22, including both endpoints] UNLESS [request value is at most 42 UNLESS review value is between 55 and 65, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 891. The review value is 56. The request value is 41. The entry value is 11.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(not(0),unless(1,2))"}
{"request": {"context": "- role: customer\n  content: Damon Salvatore is a fictional character in The Vampire Diaries novel series. He is portrayed by Ian Somerhalder in the television series. Initially, Damon is the main antagonist in the beginning of the show and later became a protagonist. After the first few episodes, Damon begins working alongside his younger brother, Stefan Salvatore, to resist greater threats and gradually Elena begins to consider him a friend. His transition was completed after his younger brother Stefan, who is also a vampire, convinces him to drink blood. Damon thus vows to make his brother's life sorrowful -- thus further causing a century-long rift between the two brothers, centering around Katherine and eventually a love triangle with Elena Gilbert. After on-again/off-again relationship with both brothers, Elena chooses to be with Damon in the series finale.", "question": "do elena and damon date in vampire diaries?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: A case is approved when shipment verified is yes, and also review verified is yes, except when entry value is at most 57, and also package value is at most 51; any other case is denied. Routing references play no part in the decision.\ncase: The package value is 61. The entry value is 58. The shipment verified is yes. The review verified is no. The routing reference is 499.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),and(2,3))"}
{"request": {"context": "policy: Customers may order at most 5 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Delivery was requested to a residential address. Elin placed an order for a coffee grinder. The order quantity is 5.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "Lotus pinnatus is a species of legume known by the common names meadow bird's-foot trefoil and bog bird's-foot trefoil. It is native to western North America from California into the Pacific Northwest its distribution extending into British Columbia in just a few rare occurrences near Nanaimo. It grows in moist to wet habitat such as bogs and spring meadows. It is a hairless perennial herb lined with leaves each made up of green oval leaflets each 1 to 2.5 centimeters in length.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "plant", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "document: On July 10, 2012, Stamos began performances on Broadway in Gore Vidal's The Best Man as Senator Joseph Cantwell. Stamos reprised his role as Jesse Katsopolis for a ``Jesse and the Rippers'' reunion on Late Night with Jimmy Fallon on July 19, 2013. Played straight by host Fallon (who introduced the band as being from San Francisco and having a number one song in Japan), Stamos performed a medley of songs featured on Full House including ``Forever'' and the series' theme song ``Everywhere You Look''. Bob Saget and Lori Loughlin also made cameos.", "question": "is jesse and the rippers a real band?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "Good but slightly slick. Kindle slides off my kindle pillow. Bought a Mojo case almost identical to this a year ago and it's great.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "I recently moved to the Surprise area and was looking for a groomer for my two Pomeranians. Decided to try Margos because of the reviews on here and the shop is close. The ladies were SO friendly and did a great job with my babies. Lots of shops you walk into are chaos with loud dryers barking dogs and a hectic vibe. This was totally different the dogs there were quietly getting groomed while a few daycare pups tottered around the shop. The two ladies there interviewed me on my dogs and how I wanted their coats to look. A few hours later they were ready and looked fabulous! Will definitely be back.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "I recently moved to the Surprise area and was looking for a groomer for my two Pomeranians. Decided to try Margos because of the reviews on here and the shop is close. The ladies were SO friendly and did a great job with my babies. Lots of shops you walk into are chaos with loud dryers barking dogs and a hectic vibe. This was totally different the dogs there were quietly getting groomed while a few daycare pups tottered around the shop. The two ladies there interviewed me on my dogs and how I wanted their coats to look. A few hours later they were ready and looked fabulous! Will definitely be back.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: What does a chairbound basophobic fear ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Applicants must be at least 21 years old to be eligible for the apprenticeship.\ncase: Omar is 41 years old. Omar applied to join the apprenticeship. The application form was complete and signed.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "policy: DECISION RULE [ANY of: package value is between 7 and 17, including both endpoints | [ALL of: entry value is greater than 50 | member value is greater than 7]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 414. The package value is 8. The entry value is 51. The member value is 7.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,and(1,2))"}
{"request": {"context": "What is the origin of the word , JJ .", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "good, came in with ice pack, very thoughtful", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Customers may order at most 3 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Priya placed an order for a backpack. Delivery was requested to a residential address. The order quantity is 10.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "far_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: A case is approved when either case signer is the same person as case designated approver, or else either package value is at most 53, and also review value is equal to 53, or else the condition \"entry value is greater than 38\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 54. The entry value is 49. The case signer is Sana. The case designated approver is Aiko. The package value is 64. The routing reference is 425.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(and(1,2),not(3)))"}
{"request": {"context": "Ex-Amvescap chiefs fined \\$3.34m for funds scandal Three former employees at fund manager Amvescap yesterday agreed to pay penalties totalling \\$3.34m (1.9m) to settle charges relating to the US mutual funds scandal.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "Ex-Amvescap chiefs fined \\$3.34m for funds scandal Three former employees at fund manager Amvescap yesterday agreed to pay penalties totalling \\$3.34m (1.9m) to settle charges relating to the US mutual funds scandal.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Ex-Amvescap chiefs fined \\$3.34m for funds scandal Three former employees at fund manager Amvescap yesterday agreed to pay penalties totalling \\$3.34m (1.9m) to settle charges relating to the US mutual funds scandal.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: chat\n  body: the fact that the movie is predictable is not a problem. this movie is like a beautiful painting to be enjoyed. the museum scene is like a nice music video. the apres sex scene is an all too familiar scene in all of our adult lives. but the movie would not hold any interest for me without keith gordon. keith gordon is maybe one of the most underrated actors of our time. almost everything i know about acting came from studying mostly his eyes. he had the most compelling face. his character possesses the qualities i look in a guy, sensitivity and dedication. keith gordon is gorgeous. BTW, i kinda wish he'd shave his beard now as his lips, jawline and adam's apple were his prettiest set next to his eyes.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: A case is approved when either case value is greater than 25, except when entry value is equal to 16, or else the condition \"account value is at least 11\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The account value is 22. The entry value is 26. The case value is 24. The routing reference is 139.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(not(0),unless(1,2))"}
{"request": {"context": "``The Road Not Taken'' is a narrative poem. It reads naturally or conversationally, and begins as a kind of photographic depiction of a quiet moment in woods. It consists of four stanzas of 5 lines each. The first line rhymes with the third and fourth, and the second line rhymes with the fifth (ABAAB). The meter is basically iambic tetrameter, with each line having four two-syllable feet. Though in almost every line, in different positions, an iamb is replaced with an anapest. The variation of the rhythm gives naturalness, a feeling of thought occurring spontaneously, and it also affects the reader's sense of expectation. In the only line that contains strictly iambs, the more regular rhythm supports the idea of a turning towards an acceptance of a kind of reality: ``Though as for that the passing there ... '' In the final line, the way the rhyme and rhythm work together is significantly different, and catches the reader off guard.", "question": "question: is the road not taken a narrative poem?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: A case is approved when when review value is between 11 and 21, including both endpoints the requirement is that the condition \"either account value is less than 25, or else shipment signer is the same person as shipment designated approver\" fails, and when it is not the requirement is that the condition \"package value is at most 23\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The shipment designated approver is Mira. The routing reference is 612. The package value is 23. The shipment signer is Kofi. The account value is 24. The review value is 21.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,not(or(1,2)),not(3))"}
{"request": {"context": "ticket:\n  channel: email\n  body: will no doubt delight plympton 's legion of fans ; others may find 80 minutes of these shenanigans exhausting .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when it is not the case that (at least one of these conditions hold: [(member verified is yes); (package signer is the same person as package designated approver)]). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member verified is no. The package designated approver is Aiko. The routing reference is 232. The package signer is Noah.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(or(0,1))"}
{"request": {"context": "policy: Approve exactly when NOT (NOT ((shipment verified is yes) OR (case value is at least 27))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment verified is no. The case value is 26. The routing reference is 464.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(not(or(0,1)))"}
{"request": {"context": "document: includes too much obvious padding .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "ticket:\n  channel: web form\n  body: How do you rate a protection plan if you haven't had to use it.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "The Big Thrill released in 1993 is the third full-length album by the German hard rock band Axxis. It was recorded in the USA with producer Joey Balin (Warlock Doro).", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "album", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Look around the Greek islands today and there is little cause to think that much has changed.", "question": "Hypothesis: \"There is evidence that everything has changed around the Greek islands.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "A completely mediocre chain restaurant that is starting to show its age. The burgers are parodies of extremes. For example, today I ordered a \\\"Whiskey River BBQ\\\" hamburger. The only river involved was the river of onion fries that were stacked an inch thick on the sandwich. I know, I can take them off, but honestly who makes a sandwich like this?\\n\\nSince my family and I have been here before, I knew not to expect good service, but this location in particular just continues to dissapoint. I think this trip was my final \\\"never again\\\" realization. They are not getting any better. Move along, nothing to see here.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "A completely mediocre chain restaurant that is starting to show its age. The burgers are parodies of extremes. For example, today I ordered a \\\"Whiskey River BBQ\\\" hamburger. The only river involved was the river of onion fries that were stacked an inch thick on the sandwich. I know, I can take them off, but honestly who makes a sandwich like this?\\n\\nSince my family and I have been here before, I knew not to expect good service, but this location in particular just continues to dissapoint. I think this trip was my final \\\"never again\\\" realization. They are not getting any better. Move along, nothing to see here.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "document: Not too sticky. I am not loving the smell. Arrived on time!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: DECISION RULE [ANY of: [entry value is at least 20 UNLESS request value is at most 36] | [package value is between 15 and 25, including both endpoints UNLESS case signer is the same person as case designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case signer is Aiko. The routing reference is 600. The entry value is 21. The case designated approver is Elin. The package value is 15. The request value is 35.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(unless(0,1),unless(2,3))"}
{"request": {"context": "I've been meaning to review this for a while now! We were in town for a tradeshow and all the buffets seemed to be closed, so we ended up here.\\n\\nThe calamari appetizer was delicious, you can't go wrong with that. Unfortunately my entree was awful. I got the Chicken Saltimbocca with lemon butter sauce, sage, prosciutto, artichoke hearts, lemon and capers. Sounds awesome right? Well, unfortunately everything was totally flavorless. Again it was one of those extremely fattening meals that tasted like nothing. \\n\\nMy bf got the Salmon Sorrento and loved it, so I was happy that he got to enjoy his meal. I forgot how expensive this place is, I'd rather go to Maggiano's.", "question": "question: How many stars did this reviewer give?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "I've been meaning to review this for a while now! We were in town for a tradeshow and all the buffets seemed to be closed, so we ended up here.\\n\\nThe calamari appetizer was delicious, you can't go wrong with that. Unfortunately my entree was awful. I got the Chicken Saltimbocca with lemon butter sauce, sage, prosciutto, artichoke hearts, lemon and capers. Sounds awesome right? Well, unfortunately everything was totally flavorless. Again it was one of those extremely fattening meals that tasted like nothing. \\n\\nMy bf got the Salmon Sorrento and loved it, so I was happy that he got to enjoy his meal. I forgot how expensive this place is, I'd rather go to Maggiano's.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: Netflix Girds for DVD Rental Price War The future suddenly looks shaky for online DVD rental pioneer Netflix Inc. - a plucky home entertainment upstart that managed to hold its ground after retailing giants Wal-Mart Stores Inc. and Blockbuster Inc. invaded its turf.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: email\n  body: Netflix Girds for DVD Rental Price War The future suddenly looks shaky for online DVD rental pioneer Netflix Inc. - a plucky home entertainment upstart that managed to hold its ground after retailing giants Wal-Mart Stores Inc. and Blockbuster Inc. invaded its turf.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: Netflix Girds for DVD Rental Price War The future suddenly looks shaky for online DVD rental pioneer Netflix Inc. - a plucky home entertainment upstart that managed to hold its ground after retailing giants Wal-Mart Stores Inc. and Blockbuster Inc. invaded its turf.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: In 2012, Tyson announced that he would appear in a YouTube series based on his radio show StarTalk. A premiere date for the show has not been announced, but it will be distributed on the Nerdist YouTube Channel. On February 28, 2014, Tyson was a celebrity guest at the White House Student Film Festival. In 2014, he helped revive Carl Sagan's Cosmos: A Personal Voyage television series, presenting Cosmos: A Spacetime Odyssey on both FOX and the National Geographic Channel. Thirteen episodes were aired in the first season, and Tyson has stated that if a second season were produced, he would pass the role of host to someone else in the science world. On April 20, 2015, he began hosting a late-night talk show entitled StarTalk on the National Geographic Channel, where Tyson interviews pop culture celebrities and asks them about their life experiences with science.", "question": "does neil degrasse tyson have a tv show?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "This particular Joe McDoakes short subject was obviously inspired by the all star Warner Brothers spectacular Thank Your Lucky Stars, one of those all star wartime morale boosters of the period. In that one Eddie Cantor played both himself and a would be comedian who'd like to break into films except for his resemblance to Cantor. George O'Hanlon who starred in the McDoakes shorts is both himself and McDoakes who's just trying to get a break in film. Like Thank Your Lucky Stars a few Warner Brothers contract players with a free moment strolled through this film. O'Hanlon's been sent by central casting for a small one line role in a World War I film, but lookalike McDoakes gets the message. The poor guy is so nervous about his big moment, he starts thinking of ways to deliver his one line. Maybe sounding like a real movie star would help. 86 takes later to the exasperation of director Ralph Sanford and the patient Clyde Cook who plays a British cockney soldier they do find a niche in the film business for poor McDoakes. It's worth seeing this very funny short subject which was nominated for an Oscar to find out what happens to O'Hanlon/McDoakes. Both of them.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: In the 1960s, a revisionist school of economic historians proved that limiting wheat exports would have made only a puny dent in the calamity.", "question": "Hypothesis: \" limiting wheat exports would have made only a puny dent in the calamity.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "ticket:\n  channel: web form\n  body: and it marks him as one of the most interesting writer\\/directors working today .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "When Mighty Mouse was conceived , what was his original name ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Customers may order at most 2 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Delivery was requested to a residential address. The order quantity is 2. Noah placed an order for a wireless keyboard.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: Warranty claims made within 365 days of purchase are covered in full. Claims made after that but within 1095 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on November 20, 2029. The claim describes a defect in the stitching. Jonas purchased a backpack on June 3, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Approve exactly when (entry value is less than 39) AND (shipment verified is yes). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 38. The shipment verified is yes. The routing reference is 322.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(0,1)"}
{"request": {"context": "policy: Warranty claims made within 180 days of purchase are covered in full. Claims made after that but within 730 days are covered at half cost. Later claims are not covered.\ncase: Priya purchased a wireless keyboard on May 25, 2026. A warranty claim for it was filed on July 20, 2026. The claim describes a defect in the zipper.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(it is not the case that (at least one of these conditions hold: [(package value is greater than 38); (the elapsed days from request start date to request end date are at most 54)])); (review value is between 45 and 55, including both endpoints)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 498. The package value is 49. The review value is 45. The request end date is 2027-07-18. The request start date is 2027-05-24.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,not(or(1,2)))"}
{"request": {"context": "I am decidedly not in the target audience for this film. I am a man nearly 50 who has only recently stumbled across the world of independent film. This happened quite by accident, with the discovery of a movie called Clerks late one night on television. The first two things I noticed about that film were that it was 1) technically amateurish and 2) brilliantly written. When I read an interview with the director in the local paper and he said that one of his influences was Clerks, I started to get interesting. When he said his main influence was The Station Agent, a movie I'd seen on DVD a week prior, I decided I had to go and check it out. The result could be described along the same lines as Clerks, although the two films are nothing alike content wise. Both films suffer from technical gaffes that are overcome through amazing writing. Whereas Clerks is a day in the life of a man who has nothing in his life at all and is afraid to ask tough questions about himself and his situation, Less Like Me is about a man who seemingly forces himself to be constantly busy, he's always running one way or another, filling his life with little things so that he will never have to", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The ticket was routed through the email queue. Ravi opened a priority ticket about a login failure. The first response arrived 2 hours after the ticket was opened.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "document: Kamaraj (Tamil: காமராஜ்) is a 2004 Tamil biographical film made about the life of the Indian politician K. Kamaraj from Tamil Nadu widely acknowledged as the Kingmaker in Indian politics during the 1960s.He was the chief minister of Tamil Nadu during 1954-1963 and a Member of Parliament during 1952-1954 and 1969-1975. He was known for his simplicity and integrity.Music by Ilaiyaraaja and produced by Ramana communications.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "film", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "What 's the No. 1 killer in industrialized countries ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "If you're going to look after a child, make sure they don't live anywhere near a graveyard. Especially if said kid has a habit of drawing gory pictures and disappears at night among the tombstones to see her 'friends'. But, our long haired heroine, oblivious to all the signs, shacks up with her family the Nortons, which include a strict father and a dullard older brother who becomes a love interest for our budding babysitter. Even more spooky than the zombie gang outside is the cast's tendency to talk even when their lips aren't moving, and for the words to not match the movement of their mouths. But enough of that.. domestic animals are being sacrificed, old ladies are having eyeballs torn out and the orchestra won't shut up during any scene, even the quiet ones. Oh, and the editor is having a day off going by the way the film drones on. In fact, it would been better if everybody involved had taken a breather, smelt what they'd signed up for and gone AWOL. Yes, I know it's hard to get into movies these days, but this sort of starter point is not one on your CV you'd want. If would be like a trainee farm labourer having a conviction for chicken molesting. Featuring one of the worst lead", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "Consistent with that view, we need to continue efforts to shift agency accountability-with appropriate safeguards and oversight- to budgeted resources and results and away from other inputs and processes.", "question": "Hypothesis: \"We do not need to shift agency accountability away from other inputs and processes.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "Kelly & Duke was a daily and Sunday comic strip drawn and written by Jack Moore and syndicated by Universal Press Syndicate. It began in 1972 then known as Kelly but changed to Kelly & Duke title in 1974.The strip centers around a boy named Kelly whose pet and best friend is Duke an anthropomorphic dog with a Southern accent.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "writtenwork", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "document: right yeah not just some doctor's office somewhere", "question": "Hypothesis: \"It was a larger compound, not just a doctor's office.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "Whispering is an unvoiced mode of phonation in which the vocal folds (vocal cords) are abducted so that they do not vibrate; air passes between the arytenoid cartilages to create audible turbulence during speech. Supralaryngeal articulation remains the same as in normal speech.", "question": "do you use your vocal cords to whisper?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "but first , you have to give the audience a reason to want to put for that effort", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "James Dunsmuir (July 8 1851 – June 6 1920) was a British Columbian industrialist and politician. Son of Robert Dunsmuir he was heir to his family's coal fortune. The Dunsmuir family dominated the province's economy in the late nineteenth century and were a leading force in opposing organized labour. Dunsmuir managed his family's coal business from 1876 until 1910 increasing profits and resisting efforts to unionize.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "officeholder", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "and uh some of those old ones of course Bob Wills and his Texas Playboys and Ernest Tubbs and", "question": "Hypothesis: \"Some of the new ones are Bob Wills and Ernest Tubbs.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "Szvetiszláv Sasics (born 19 January 1948) is a Hungarian former modern pentathlete. He competed at the 1976 Summer Olympics winning a bronze medal in the team event.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "athlete", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "What country did the Romans call Hibernia ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "- role: customer\n  content: Entered the store to really friendly helpful staff...The store is packed full of fun things to buy. You are able to buy the Tonto pass at this location. I need to go back just to be able to look at everything.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "- role: customer\n  content: Entered the store to really friendly helpful staff...The store is packed full of fun things to buy. You are able to buy the Tonto pass at this location. I need to go back just to be able to look at everything.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "on the granger movie gauge of 1 to 10 , the powerpuff girls is a fast , frenetic , funny , even punny 6 -- aimed specifically at a grade-school audience .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "Quality of product was fine just a bit thrown off when I received it. These nuts and bolts are tiny. Meant for electronics.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 2 hours after the ticket was opened. Priya opened a priority ticket about a duplicate charge. The ticket was routed through the phone queue.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "i think a lot of kids are having a problem with that today i would would i would agree with you and uh", "question": "Hypothesis: \"That is a serious problem and a lot of children are facing it.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [NOT: [IF member value is at least 14 THEN [NOT: shipment verified is yes] ELSE [ANY of: review value is equal to 22 | entry verified is yes]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 100. The member value is 15. The entry verified is yes. The review value is 33. The shipment verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(if(0,not(1),or(2,3)))"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(it is not the case that (member value is at most 23)); (if (the elapsed days from review start date to review end date are at most 5), use (account value is equal to 57); otherwise use (case value is greater than 27))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-04-27. The account value is 57. The member value is 22. The review end date is 2027-05-02. The case value is 26. The routing reference is 871.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(if(0,1,2),not(3))"}
{"request": {"context": "policy: Warranty claims made within 90 days of purchase are covered in full. Claims made after that but within 365 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on March 28, 2026. Elin purchased a rain jacket on February 24, 2026. The claim describes a defect in the stitching.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Approve exactly when ((review value is between 9 and 19, including both endpoints) AND (case value is greater than 57)) holds and the exception (the elapsed days from entry start date to entry end date are at most 18) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 660. The entry start date is 2027-07-04. The entry end date is 2027-08-01. The case value is 58. The review value is 10.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(and(0,1),2)"}
{"request": {"context": "policy: Orders of fewer than 10 units are charged the list price. Orders of 10 to 49 units receive the volume discount. Orders of 50 units or more receive the wholesale rate.\ncase: Kofi placed a business order for a wireless keyboard. The order is for 9 units. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "From June onward, ukai celebrates the ancient use of cormorant birds to catch aiyu, a popular river fish.", "question": "Hypothesis: \"Ukai is the celebration of using cormorant birds to catch fish.  \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when if (member value is at least 48), use (both these conditions hold: [(it is not the case that (case value is equal to 38)); (it is not the case that (package value is between 52 and 62, including both endpoints))]); otherwise use (it is not the case that (it is not the case that (account value is between 24 and 34, including both endpoints))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 63. The account value is 24. The member value is 58. The routing reference is 217. The case value is 38.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,and(not(1),not(2)),not(not(3)))"}
{"request": {"context": "document: U.S. Treasuries on Firm Footing (Reuters) Reuters - U.S. Treasury prices were higher on\\Friday, cementing gains made the previous day, in the face of\\growing speculation that the pace of U.S. interest rate hikes\\could slow.", "question": "question: What is the topic of this article?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "document: U.S. Treasuries on Firm Footing (Reuters) Reuters - U.S. Treasury prices were higher on\\Friday, cementing gains made the previous day, in the face of\\growing speculation that the pace of U.S. interest rate hikes\\could slow.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: U.S. Treasuries on Firm Footing (Reuters) Reuters - U.S. Treasury prices were higher on\\Friday, cementing gains made the previous day, in the face of\\growing speculation that the pace of U.S. interest rate hikes\\could slow.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: My fianc\\u00e9 and I went here for dinner. The place is very cute. \\nThe food was just ok, nothing special about It. But they did have some great food/drink specials till 7pm when we were there. Will I go back for the food? no. The food lacks flavor and that is what makes Mexican food so great. \\nBut I'll go back for margarita. \\nOverall ok.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "- role: customer\n  content: My fianc\\u00e9 and I went here for dinner. The place is very cute. \\nThe food was just ok, nothing special about It. But they did have some great food/drink specials till 7pm when we were there. Will I go back for the food? no. The food lacks flavor and that is what makes Mexican food so great. \\nBut I'll go back for margarita. \\nOverall ok.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Israelis Kill Five Palestinians in Gaza Strip  GAZA (Reuters) - Israeli troops killed five armed  Palestinians in the Gaza Strip on Saturday as it pressed on  with a massive offensive aimed at stopping militants firing  rockets into Israel.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "Israelis Kill Five Palestinians in Gaza Strip  GAZA (Reuters) - Israeli troops killed five armed  Palestinians in the Gaza Strip on Saturday as it pressed on  with a massive offensive aimed at stopping militants firing  rockets into Israel.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Israelis Kill Five Palestinians in Gaza Strip  GAZA (Reuters) - Israeli troops killed five armed  Palestinians in the Gaza Strip on Saturday as it pressed on  with a massive offensive aimed at stopping militants firing  rockets into Israel.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Expense claims of $2,500 or less are approved automatically. Claims above $2,500 require director sign-off.\ncase: Receipts were attached for every line item. Jonas, the account owner, submitted an expense claim. The claim total is $3,650.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "The term aggregate entry age normal refers to an approach whereby costs are determined for the group as a whole rather than for each individual participant separately.", "question": "Hypothesis: \"Aggregate entry age normal means how you determine costs for the group of twenty.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: what: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when NOT (((package value is less than 26) OR (the elapsed days from entry start date to entry end date are at most 57)) AND (request value is between 18 and 28, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 27. The entry end date is 2027-07-03. The request value is 19. The routing reference is 721. The entry start date is 2027-05-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(and(0,or(1,2)))"}
{"request": {"context": "- role: customer\n  content: How do you get a broken cork out of a bottle ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Invoices paid within 10 days after the due date incur no fee. Payments between 11 and 60 days late incur a 2% fee. Payments later than 60 days incur a 10% fee and a hold on the account.\ncase: Mira's invoice for $120 fell due last quarter. The payment was made by card. Payment was received 0 days after the due date.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the rental agreement.\ncase: The application form was complete and signed. Aiko applied to join the rental agreement. Aiko is 30 years old.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "the dominant feeling is something like nostalgia . '", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 14 days of the purchase date.\ncase: The return request was submitted on May 15, 2026. The order was paid by card and shipped to Tomas's home address. Tomas bought a wireless keyboard on May 5, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "ticket:\n  channel: email\n  body: Any true wrestling fan would have to consider this Wrestlemania to be one of, if not the best of all time. It was packed with excitement and surprises. One of the greatest matches of all time was between Shawn Michaels and Steve Austin with special guest ref Mike Tyson. The show that Michaels put on was unbelievable, especially considering the shape that his back was in and that this was his last match.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "Trichogonia is a genus of flowering plants in the daisy family Asteraceae.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "plant", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Just received my 7 - 8 inch salt lamp. Arrived well packaged and in perfect condition. With the light on, it makes a nice accent piece day and night. Not sure if it's going to make me feel better and all that. Might end up being just a fad like pet rocks. But the light makes it look nice, and the dimmer works very well to adjust light level.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Customers may order at most 10 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: The order quantity is 3. Delivery was requested to a residential address. Elin placed an order for a pair of running shoes.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: Approve exactly when (NOT ((shipment signer is the same person as shipment designated approver) OR (review value is less than 22))) holds and the exception (NOT ((case signer is the same person as case designated approver) AND (entry value is greater than 12))) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 22. The case designated approver is Mira. The routing reference is 548. The shipment designated approver is Ravi. The shipment signer is Ravi. The review value is 22. The case signer is Mira.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(not(or(0,1)),not(and(2,3)))"}
{"request": {"context": "What 's the term for an organism that lives on or in another ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "document: Tovdalselva or Topdalselva (literally: the Tov valley river) is 143 kilometres (89 mi) long and is one of the longest rivers in Southern Norway. The river flows from the northeast to the southwest and reaches the sea at the Topdalsfjorden between Hamresanden and Kjevik. The upper reaches of the river include many lakes such as the Herefossfjorden and the Straumsfjorden (the largest lake on the river).", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "naturalplace", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "What is the name of the inventor of silly putty ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: Aiko, the account owner, submitted an expense claim. Receipts were attached for every line item. The claim total is $378.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "ticket:\n  channel: email\n  body: Went for lunch today and ordered Kung Pao chicken. Possibly the worst I have ever had. No taste, too much celery, and in a brownish sauce. Fried rice and hot n sour soup were ok. Won't do this again.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "ticket:\n  channel: email\n  body: Went for lunch today and ordered Kung Pao chicken. Possibly the worst I have ever had. No taste, too much celery, and in a brownish sauce. Fried rice and hot n sour soup were ok. Won't do this again.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: The wallet itself is okay. However, the inside fabric rubs off on my cards, and has turned most of my cards red. I think it's a great area of improvement given the main purpose of the wallet.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when ((review value is between 9 and 19, including both endpoints) AND (case value is greater than 57)) holds and the exception (the elapsed days from entry start date to entry end date are at most 18) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 660. The entry start date is 2027-07-04. The entry end date is 2027-08-01. The case value is 56. The review value is 10.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),2)"}
{"request": {"context": "ticket:\n  channel: web form\n  body: David Downs, president of Univision Sports, was executive director of the bid. The United States previously hosted the FIFA World Cup in 1994, as well as the FIFA Women's World Cup in 1999 and 2003.", "question": "will the world cup ever be in the united states?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "I can't even articulate the disastrous and horrific legal aspects; what I have to focus on is the SPIRIT of Deborah, her two lawyers, the film makers, and the private investigator. There is NO happy ending here, but what there was...revealed true compassion and support for TOTAL injustice to Deborah and the many others on earth who are incarcerated from a survival response to their abuser. Hopefully every tiny light shines brighter with our awareness. Hopefully... {tears}", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "In September 2017, actor Sean Astin announced that he had spent the last ten years attempting to get a film adaptation greenlit.", "question": "is there a movie based on the book number the stars?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "Every time this film is on the BBC somebody in the Radio Times says how it is a satire against the post war world of rationing and the welfare state. I do not think this is the point of the film at all. The film parodies the spivs(small time criminals who ran the blackmarket) and the housewives league who campaigned against government restrictions but were really a Tory front organisation. Yes of course the film sends up the political/social situation but in the end the people realise that they need all the controls to ensure a fair society,they want to be British and muddle through rather than foreign. But I don't think they go back to being exactly like they were before.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: DECISION RULE [IF [ALL of: entry value is at most 17 | account verified is yes] THEN package value is at least 9 ELSE case signer is the same person as case designated approver] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 959. The case signer is Ravi. The entry value is 18. The account verified is no. The case designated approver is Tomas. The package value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(and(0,1),2,3)"}
{"request": {"context": "The show attracts a variety of participants, from across the United States and abroad, to take part and who possess some form of talents, with acts ranging from singing, dancing, comedy, magic, stunts, variety, and other genres. Each participant who auditions attempts to secure a place in the live episodes of a season by impressing a panel of judges - the current line-up consists of Cowell, Howie Mandel, Mel B, and Heidi Klum. Those that make it into the live episodes compete against each other for both the judges' and public's vote in order to reach the live final, where the winner receives a large cash prize, paid over a period of time, and, since the third season, a chance to headline a show on the Las Vegas Strip.", "question": "do you have to live in america to be on americas got talent?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "document: What are Britain 's two longest rivers ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "document: Beatty is meticulous, even anal.", "question": "Hypothesis: \"Beatty is laid back and easy-going.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "The Souls series (ソウルシリーズ, Sōru shirīzu) is a series of action role-playing video games created and developed by FromSoftware. The series began with the release of Demon's Souls for the PlayStation 3 in 2009. It was followed by Dark Souls in 2011, and its sequels, Dark Souls II and Dark Souls III, in 2014 and 2016, respectively. The series' creator was Hidetaka Miyazaki, who directed all of them with the exception of Dark Souls II.", "question": "is demon's souls part of dark souls?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when (at least one of these conditions hold: [((case value is greater than 27) holds and the exception (the elapsed days from member start date to member end date are at most 8) does not hold); (it is not the case that (shipment value is at most 41))]) holds and the exception (entry value is greater than 40) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member start date is 2027-05-23. The case value is 37. The shipment value is 51. The member end date is 2027-05-30. The entry value is 41. The routing reference is 845.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(not(0),unless(1,2)),3)"}
{"request": {"context": "policy: DECISION RULE [ANY of: [entry value is at least 20 UNLESS request value is at most 36] | [package value is between 15 and 25, including both endpoints UNLESS case signer is the same person as case designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case signer is Aiko. The routing reference is 600. The entry value is 21. The case designated approver is Elin. The package value is 15. The request value is 35.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(unless(0,1),unless(2,3))"}
{"request": {"context": "policy: DECISION RULE [ANY of: request value is between 26 and 36, including both endpoints | [IF package value is equal to 49 THEN entry verified is yes ELSE review value is greater than 24]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 403. The review value is 23. The entry verified is no. The package value is 50. The request value is 25.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,if(1,2,3))"}
{"request": {"context": "it 's affecting , amusing , sad and reflective .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(at least one of these conditions hold: [(entry value is at most 53); (member value is equal to 45)]); (account value is equal to 13)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 44. The entry value is 63. The routing reference is 492. The account value is 23.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(1,2))"}
{"request": {"context": "When did the Jurassic Period end ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: A case is approved when either case signer is the same person as case designated approver, or else either package value is at most 53, and also review value is equal to 53, or else the condition \"entry value is greater than 38\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 54. The entry value is 37. The case signer is Sana. The case designated approver is Aiko. The package value is 64. The routing reference is 425.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(and(1,2),not(3)))"}
{"request": {"context": "policy: Approve exactly when (((case value is between 20 and 30, including both endpoints) OR (member value is less than 55)) AND (entry value is at least 45)) AND (shipment value is at least 7). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 56. The shipment value is 7. The case value is 19. The routing reference is 810. The member value is 65.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,and(1,or(2,3)))"}
{"request": {"context": "policy: DECISION RULE [ALL of: case value is greater than 31 | [NOT: [IF member value is at least 21 THEN the elapsed days from request start date to request end date are at most 55 ELSE review value is less than 59]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 41. The member value is 20. The request start date is 2027-04-02. The routing reference is 448. The review value is 59. The request end date is 2027-05-28.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(0,not(if(1,2,3)))"}
{"request": {"context": "Blue ice, in the context of aviation, is frozen sewage material that has leaked mid-flight from commercial aircraft lavatory waste systems. It is a mixture of human biowaste and liquid disinfectant that freezes at high altitude. The name comes from the blue color of the disinfectant. Airlines are not allowed to dump their waste tanks in mid-flight, and pilots have no mechanism by which to do so; however, leaks sometimes do occur.", "question": "do airplanes dump their waste in the air?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: DECISION RULE [ALL of: case value is greater than 31 | [NOT: [IF member value is at least 21 THEN the elapsed days from request start date to request end date are at most 55 ELSE review value is less than 59]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 41. The member value is 20. The request start date is 2027-04-02. The routing reference is 518. The review value is 58. The request end date is 2027-05-28.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,not(if(1,2,3)))"}
{"request": {"context": "policy: DECISION RULE [IF [ALL of: entry value is at most 17 | account verified is yes] THEN package value is at least 9 ELSE case signer is the same person as case designated approver] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 194. The case signer is Ravi. The entry value is 18. The account verified is no. The case designated approver is Tomas. The package value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(and(0,1),2,3)"}
{"request": {"context": "policy: Expense claims of $2,500 or less are approved automatically. Claims above $2,500 require director sign-off.\ncase: Receipts were attached for every line item. Jonas, the account owner, submitted an expense claim. The claim total is $1,824.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "no , even that 's too committed .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "Excellent hand cleaner for the money, really cuts through grease and grime. The Zep dispenser is very well built, I've been using two of them for over 10 years now.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "What animals do you find in the stock market ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "document: EMI upbeat on music business A WEAK dollar and a lack of major album releases saw music giant EMI #39;s sales slide again in the past six months. But chairman Eric Nicoli sang an uncharacteristically upbeat tune today when he said sales in", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "document: EMI upbeat on music business A WEAK dollar and a lack of major album releases saw music giant EMI #39;s sales slide again in the past six months. But chairman Eric Nicoli sang an uncharacteristically upbeat tune today when he said sales in", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: EMI upbeat on music business A WEAK dollar and a lack of major album releases saw music giant EMI #39;s sales slide again in the past six months. But chairman Eric Nicoli sang an uncharacteristically upbeat tune today when he said sales in", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "It doesn't work and I didn't like it.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Orders of fewer than 5 units are charged the list price. Orders of 5 to 19 units receive the volume discount. Orders of 20 units or more receive the wholesale rate.\ncase: The order is for 20 units. Tomas placed a business order for a pair of running shoes. Delivery is to a branch office.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "policy: A case is approved when member value is between 30 and 40, including both endpoints, except when account verified is yes, and also package signer is the same person as package designated approver, and also review value is less than 14; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 31. The review value is 15. The account verified is yes. The routing reference is 421. The package signer is Aiko. The package designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(1,and(2,3)))"}
{"request": {"context": "policy: DECISION RULE [[case value is less than 29 UNLESS [ANY of: request value is greater than 49 | entry verified is yes]] UNLESS the elapsed days from account start date to account end date are at most 13] -> approve; otherwise reject. Ignore the routing reference.\ncase: The request value is 48. The routing reference is 271. The account end date is 2027-04-17. The account start date is 2027-03-25. The case value is 28. The entry verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(unless(0,or(1,2)),3)"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Xstrata Offers A\\$7.4 Billion for WMC Resources to Add Nickel Xstrata Plc, the world #39;s biggest exporter of power-station coal, offered A\\$7.4 billion (\\$5.5 billion) for WMC Resources Ltd. to gain nickel and copper mines.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Xstrata Offers A\\$7.4 Billion for WMC Resources to Add Nickel Xstrata Plc, the world #39;s biggest exporter of power-station coal, offered A\\$7.4 billion (\\$5.5 billion) for WMC Resources Ltd. to gain nickel and copper mines.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Xstrata Offers A\\$7.4 Billion for WMC Resources to Add Nickel Xstrata Plc, the world #39;s biggest exporter of power-station coal, offered A\\$7.4 billion (\\$5.5 billion) for WMC Resources Ltd. to gain nickel and copper mines.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Birmingham has generally been regarded as the second city of the United Kingdom in terms of populace and GDP while Edinburgh has been promoted as the second city by virtue as the capital of Scotland. Less authoratitive claims have been made on behalf of Cardiff and Belfast due to their status as the respective capital cities of Wales and Northern Ireland. Furthermore, a number of opinion polls have been conducted since 2000 and complicate the issue further, with public response showing a consistent view amongst the British population that Manchester is the second city. It should be noted that Birmingham is technically the United Kingdom's most populous city; the City of London has fewer than 10,000 inhabitants, and Greater London is not recognized as a city.", "question": "is birmingham the 2nd largest city in uk?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "- role: customer\n  content: Lloyds to spend nearly \\$1 billion on broadband network European banking giant Lloyds TSB has tapped IBM to assemble a voice and high-speed Internet network connecting branch branches, call centers and cash machines, the two companies said Monday.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: what: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "- role: customer\n  content: Lloyds to spend nearly \\$1 billion on broadband network European banking giant Lloyds TSB has tapped IBM to assemble a voice and high-speed Internet network connecting branch branches, call centers and cash machines, the two companies said Monday.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: Lloyds to spend nearly \\$1 billion on broadband network European banking giant Lloyds TSB has tapped IBM to assemble a voice and high-speed Internet network connecting branch branches, call centers and cash machines, the two companies said Monday.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "This \"movie\" is such a bad work! Nothing seems to even try and be realistic. Plot is weak, acting - miserable, actors wondering around like in a 1st year production, trying very hard to act with no chance at all from the beginning. What a flop! What a waste of time, money and effort to all concerned including the audience. Well, as in any thriller, here too are murders, corpses and blood. Just imagine someone who 5 minutes ago, committed a murder with a knife, and came out calm and smiling, not to mention clean as a whistle, as if slashing one's throat is done by a virtual agent. Also, this murder was supposed to be done by a tiny fragile woman on a high strong male, and she cut his throat!!! Did she ask him, politely, to bend down for her? Much more stupidity of that same kind is going on and on leaving the audience wondering if this meant to be a joke which just turned out to be a bad one. Continuity is another huge problem as for instance: The eager-hungry groom is lying in bed, waiting for his virgin-bride to get out of the bathroom and after a long while, falls asleep(!?!). Next scene opens with the young couple entering the reception-area, asking for guidance to", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "Well the place was clean! The staff was nice and helpful! I stopped in for a fix! I was at hospital across the street and figured I'd give em a try! Let me say it was flavorless! I tried 4 or 5 flavored looking for one that didn't resemble freezer burn taste in my mouth......... I bought a small because I felt obligated! The ice tasted like... Ice very week flavor, tasted fake! I'm almost ashamed of my self for not going to CJs right down the road! But this place was close and it was hot! I don't see the big deal or the hype behind this stuff! It has zero taste! It's grainy and icy like a snow cone the texture is way off of an Italian ice! I grew up in NYC and ate Mariano's for years that was Italian ice this is an over rated Snowcone, snow ones have more flavor!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "Well the place was clean! The staff was nice and helpful! I stopped in for a fix! I was at hospital across the street and figured I'd give em a try! Let me say it was flavorless! I tried 4 or 5 flavored looking for one that didn't resemble freezer burn taste in my mouth......... I bought a small because I felt obligated! The ice tasted like... Ice very week flavor, tasted fake! I'm almost ashamed of my self for not going to CJs right down the road! But this place was close and it was hot! I don't see the big deal or the hype behind this stuff! It has zero taste! It's grainy and icy like a snow cone the texture is way off of an Italian ice! I grew up in NYC and ate Mariano's for years that was Italian ice this is an over rated Snowcone, snow ones have more flavor!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: A case is approved when when review value is between 11 and 21, including both endpoints the requirement is that the condition \"either account value is less than 25, or else shipment signer is the same person as shipment designated approver\" fails, and when it is not the requirement is that the condition \"package value is at most 23\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The shipment designated approver is Mira. The routing reference is 448. The package value is 23. The shipment signer is Kofi. The account value is 24. The review value is 21.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,not(or(1,2)),not(3))"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The ticket was routed through the chat queue. The first response arrived 117 hours after the ticket was opened. Lena opened a priority ticket about a login failure.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "document: the movie is so resolutely cobbled together out of older movies that it even uses a totally unnecessary prologue , just because it seems obligatory .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: DECISION RULE [ANY of: [review value is at most 33 UNLESS member value is at least 10] | [ALL of: case verified is yes | package value is less than 26]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review value is 33. The case verified is no. The package value is 25. The routing reference is 736. The member value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),unless(2,3))"}
{"request": {"context": "policy: Customers may order at most 5 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Mira placed an order for a rain jacket. Delivery was requested to a residential address. The order quantity is 14.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "far_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "daring and beautifully made .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "Typical casino cafe, with great drink prices and very reasonable food prices. $8 for a bottle of wine or $1 for screwdriver, mimosa & Bloody Mary. \\nTried the signature sandwich Reuben, it was good and a decent portion. The potato salad was the standout, however. It was \\\"loaded\\\" potato salad, with bacon & green onions. Very tasty, could've eaten a ton of it. They should do more with it, like a special dish that features it.\\nWill try to return with some friends for more potato salad!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "Typical casino cafe, with great drink prices and very reasonable food prices. $8 for a bottle of wine or $1 for screwdriver, mimosa & Bloody Mary. \\nTried the signature sandwich Reuben, it was good and a decent portion. The potato salad was the standout, however. It was \\\"loaded\\\" potato salad, with bacon & green onions. Very tasty, could've eaten a ton of it. They should do more with it, like a special dish that features it.\\nWill try to return with some friends for more potato salad!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Warranty claims made within 180 days of purchase are covered in full. Claims made after that but within 730 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on April 19, 2027. The claim describes a defect in the switch. Aiko purchased a desk lamp on January 14, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "But this spot is visually arresting, involving, and dramatic.", "question": "Hypothesis: \"The spot on this stage is quite dramatic.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when ((member value is between 21 and 31, including both endpoints) AND (case value is greater than 52)) AND ((shipment value is at most 60) OR (account value is greater than 45)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 21. The account value is 55. The case value is 62. The routing reference is 398. The shipment value is 61.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(and(0,1),or(2,3))"}
{"request": {"context": "In the history of the Triple Crown, 13 horses have won all three races: Sir Barton (1919), Gallant Fox (1930), Omaha (1935), War Admiral (1937), Whirlaway (1941), Count Fleet (1943), Assault (1946), Citation (1948), Secretariat (1973), Seattle Slew (1977), Affirmed (1978), American Pharoah (2015), and Justify (2018). As of 2018, American Pharoah and Justify are the only living Triple Crown winners.", "question": "question: has any horse won triple crown since secretariat?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the night-shift roster.\ncase: Lena is 20 years old. Lena applied to join the night-shift roster. The application form was complete and signed.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "ticket:\n  channel: chat\n  body: How many times can a nickel-cadmium rechargeable battery be recharged ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "A nice wrapped up of the mystery, and I'm happy that I guessed all correct. I guess being an avid RS genre reader paid off. As with many crimes, it's all back to greed and money. While the story itself was great, and the flash back scenes was refreshing, I didn't really feel connected with Tracy and Johan's romantic relationship. I just didn't get the chemistry. And for someone like Johan, that I though a level headed one, some of his actions were disappointing, especially toward Tracy. And for a woman as strong as Tracy pictured to be, I'm surprised she's kind of meek in accepting treatment like that.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "that's right so uh a dollar tax isn't isn't a dollar that can be you know that that makes more bread or anything else it's uh it's lost", "question": "question: Hypothesis: \"A dollar tax is chopped up into many pieces until it loses value.\" How does it relate to the premise?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "I first saw a track from this DVD at a hifi show Nov 2006 in London ( i was not really into cream until now!!).It was through a high end Arcam system,it sounded great with dts.I had to get this DVD and i'll tell you this is by far the most exciting music DVD i have ever watched.The performance of Cream at their age was just mind blowing and sound quality is the best i have heard on a music DVD.It does not matter what type of music you like,this one will definitely grow on you.It's the sheer brilliance of their performance that will make you watch it again and again.Even new musicians don't cut the mustard these days, as these old rockers do.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "In most prisons, inmates are forbidden from possessing mobile phones due to their ability to communicate with the outside world and other security issues. Mobile phones are one of the most smuggled items into prisons. They provide inmates the ability to make and receive unauthorized phone calls, send email and text messages, use social media, and follow news pertaining to their case, among other forbidden uses.", "question": "question: are you allowed to have your cell phone in jail?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: Elin's invoice for $120 fell due last quarter. The payment was made by cheque. Payment was received 14 days after the due date.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "Thomas Dale Rapp (born 8 March 1947 Bottineau North Dakota) is an American singer and songwriter best known as the leader of Pearls Before Swine the psychedelic folk rock group of the 1960s and 1970s. More recently he has practiced as a lawyer.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "el crimen del padre amaro would likely be most effective if used as a tool to rally anti-catholic protestors .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: Mira placed a business order for a coffee grinder. Delivery is to a branch office. The order is for 963 units.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "This tastes exactly like nothing. I ordered it along with some PB&Me Peanut Butter; by combining one part almond butter power to three parts peanut butter powder I was able to use the almond stuff. That was better than throwing it away but not by much.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "Sharapova withdraws from Advanta Championships  quot;I #39;m very disappointed I am not able to continue my play in Philadelphia, quot; Sharapova said.  quot;I was having a really good tournament and I #39;m sure it will still be an exciting final weekend.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Sharapova withdraws from Advanta Championships  quot;I #39;m very disappointed I am not able to continue my play in Philadelphia, quot; Sharapova said.  quot;I was having a really good tournament and I #39;m sure it will still be an exciting final weekend.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Sharapova withdraws from Advanta Championships  quot;I #39;m very disappointed I am not able to continue my play in Philadelphia, quot; Sharapova said.  quot;I was having a really good tournament and I #39;m sure it will still be an exciting final weekend.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(it is not the case that (member value is at most 23)); (if (the elapsed days from review start date to review end date are at most 5), use (account value is equal to 57); otherwise use (case value is greater than 27))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-04-27. The account value is 57. The member value is 22. The review end date is 2027-05-02. The case value is 26. The routing reference is 396.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(if(0,1,2),not(3))"}
{"request": {"context": "ticket:\n  channel: email\n  body: oddly , the film is n't nearly as downbeat as it sounds , but strikes a tone that 's alternately melancholic , hopeful and strangely funny .", "question": "question: What is the sentiment of this review sentence?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "Consistently overbooked. Controlled chaos. Issues with insurance processing. Need to either stop accepting new patients or add dentists.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "Consistently overbooked. Controlled chaos. Issues with insurance processing. Need to either stop accepting new patients or add dentists.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "I see a lot of folks on this site wishing AG would come out on DVD. Well, I bought it on DVD. (From Borders, no less!) While it is great to have this terrific show in a boxed DVD form, I am upset by the fact that they added very few in the way of \"extra's\" (A director commentary from Shaun Cassidy on the Pilot episode) and the episodes are shown in the same order they were put out on TV. The missing episodes that were never shown prior to being run on Sci Fi channel are in the box set, but are tacked on the final DVD. If you buy the DVD set, get the actual order they are to be viewed and you will be happier. (You will need to swap DVD's in and out of your player to see them in order, but you will be glad you did.) S", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: What is the connection between New Krypton and Daxam ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "Still, it was strange that he had sent no word of any kind.", "question": "Hypothesis: \"He sent no word of any kind and so I worried about him.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "What kind of a sports team is the Wisconsin Badgers ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "Alarm was sounded seven years ago on dig #39;s leak problems A group of state, federal, and Bechtel/Parsons Brinckerhoff engineers issued a confidential report in 1997 which concluded that miles of Big Dig tunnel walls were waterproofed with products", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "Alarm was sounded seven years ago on dig #39;s leak problems A group of state, federal, and Bechtel/Parsons Brinckerhoff engineers issued a confidential report in 1997 which concluded that miles of Big Dig tunnel walls were waterproofed with products", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Alarm was sounded seven years ago on dig #39;s leak problems A group of state, federal, and Bechtel/Parsons Brinckerhoff engineers issued a confidential report in 1997 which concluded that miles of Big Dig tunnel walls were waterproofed with products", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when ((account signer is the same person as account designated approver) holds and the exception (review value is equal to 56) does not hold) holds and the exception (the elapsed days from entry start date to entry end date are at most 39) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account designated approver is Kofi. The entry start date is 2027-06-16. The account signer is Aiko. The review value is 57. The entry end date is 2027-07-26. The routing reference is 424.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(unless(0,1),2)"}
{"request": {"context": "document: I haven't had Baja in awhile, but after the gym last night I was craving their yummy fish tacos. I've always been a fan of the fish tacos, because there's so few \\\"fast\\\" Mexican places you can grab decent fish items. Plus I do love their taquitos with the tasty cheese dipping sauce as well.\\n\\nUnfortunately I think Baja has gone down hill in the past few years since its magnificent presence began infesting cities in Nevada and other places. Yes, it may still be \\\"fresh\\\" compared to say, Del Taco, but it is lacking.\\n\\nI ordered the fish tacos, and was surprised they were so bland. I had to drench them in salsa (supposedly \\\"hot\\\" salsa... which was not... hot.). The fish was dry and unseasoned and it was like a long, stick of fish. It literally almost reminded me of putting a fish stick into a taco shell and calling it a day.\\n\\nWhy is the salsa bland now? Why is the fish not fresh? Is there a new CEO cutting corners, or is it just this particular establishment?\\n\\nI don't know, but I DO know I am probably going to make the trip to Rubio's in Henderson from now on if I really want some good fish tacos.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "document: I haven't had Baja in awhile, but after the gym last night I was craving their yummy fish tacos. I've always been a fan of the fish tacos, because there's so few \\\"fast\\\" Mexican places you can grab decent fish items. Plus I do love their taquitos with the tasty cheese dipping sauce as well.\\n\\nUnfortunately I think Baja has gone down hill in the past few years since its magnificent presence began infesting cities in Nevada and other places. Yes, it may still be \\\"fresh\\\" compared to say, Del Taco, but it is lacking.\\n\\nI ordered the fish tacos, and was surprised they were so bland. I had to drench them in salsa (supposedly \\\"hot\\\" salsa... which was not... hot.). The fish was dry and unseasoned and it was like a long, stick of fish. It literally almost reminded me of putting a fish stick into a taco shell and calling it a day.\\n\\nWhy is the salsa bland now? Why is the fish not fresh? Is there a new CEO cutting corners, or is it just this particular establishment?\\n\\nI don't know, but I DO know I am probably going to make the trip to Rubio's in Henderson from now on if I really want some good fish tacos.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Chah-e Ali Kapari (Persian: چاه علي كپري‎ also Romanized as Chāh-e ʿAlī Kaparī) is a village in Jazmurian Rural District Jazmurian District Rudbar-e Jonubi County Kerman Province Iran. At the 2006 census its population was 39 in 8 families.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "village", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Warranty claims made within 365 days of purchase are covered in full. Claims made after that but within 1095 days are covered at half cost. Later claims are not covered.\ncase: The claim describes a defect in the stitching. A warranty claim for it was filed on August 26, 2026. Kofi purchased a backpack on April 21, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The ticket was routed through the chat queue. Tomas opened a priority ticket about a duplicate charge. The first response arrived 11 hours after the ticket was opened.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "Much to small and narrow", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: A case is approved when entry verified is yes, except when review value is greater than 23, and also member value is less than 18, and also the condition \"shipment verified is yes\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry verified is no. The review value is 34. The routing reference is 439. The member value is 17. The shipment verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(0,and(and(1,2),not(3)))"}
{"request": {"context": "these guys seem great to knock back a beer with but they 're simply not funny performers .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: A case is approved when the condition \"either either case signer is the same person as case designated approver, or else package value is equal to 43, or else the elapsed days from entry start date to entry end date are at most 56\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The case signer is Aiko. The case designated approver is Noah. The entry start date is 2027-07-13. The package value is 44. The entry end date is 2027-09-07. The routing reference is 378.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(or(0,or(1,2)))"}
{"request": {"context": "policy: Approve exactly when ((review value is between 9 and 19, including both endpoints) AND (case value is greater than 57)) holds and the exception (the elapsed days from entry start date to entry end date are at most 18) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 113. The entry start date is 2027-07-04. The entry end date is 2027-08-01. The case value is 56. The review value is 10.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),2)"}
{"request": {"context": "an admirable , sometimes exceptional film", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(it is not the case that (at least one of these conditions hold: [(package value is greater than 38); (the elapsed days from request start date to request end date are at most 54)])); (review value is between 45 and 55, including both endpoints)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 498. The package value is 49. The review value is 44. The request end date is 2027-07-18. The request start date is 2027-05-24.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,not(or(1,2)))"}
{"request": {"context": "The analysis indicates that 6,300 firms submitting 42,500 reports annually will be added by the rule to the 360 facilities in the existing manufacturing sector submitting 3,600 reports for a total compliance cost of $226 million for the first year, declining to $143 million in subsequent years.", "question": "Hypothesis: \"Over 6000 companies will be added to the 360 locations already submitting reports.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when (NOT (NOT (entry value is less than 45))) AND (((the elapsed days from request start date to request end date are at most 39) AND (package value is equal to 54)) OR (NOT (account value is at most 39))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 49. The package value is 55. The request end date is 2027-09-17. The entry value is 44. The routing reference is 745. The request start date is 2027-08-08.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(not(not(0)),or(and(1,2),not(3)))"}
{"request": {"context": "policy: Applicants must be at least 21 years old to be eligible for the night-shift roster.\ncase: Kofi is 34 years old. The application form was complete and signed. Kofi applied to join the night-shift roster.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "policy: Invoices paid within 10 days after the due date incur no fee. Payments between 11 and 60 days late incur a 2% fee. Payments later than 60 days incur a 10% fee and a hold on the account.\ncase: Elin's invoice for $450 fell due last quarter. Payment was received 9 days after the due date. The payment was made by bank transfer.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "document: Cricket was included in the 1998 Commonwealth Games in Malaysia. This is the only time cricket was played at a Commonwealth Games. Matches were played over 50 overs, and had List A status rather than full One Day Internationals. As is normal at the Commonwealth Games, the Caribbean islands that entered participated as separate nations, not as the combined West Indies team. Indeed, the Games were the first occasion on which an Antigua and Barbuda side competed at a senior level. Northern Ireland also entered, this occurrence being noteworthy because Irish cricket is usually represented by an all-island Irish cricket team.", "question": "has cricket ever been in the commonwealth games?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "adaptation 's success in engaging the audience in the travails of creating a screenplay is extraordinary .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "The wings were good, but the pork porterhouse was very dry. Also, the sirloin my daughter got was cooked properly and very tough.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "The wings were good, but the pork porterhouse was very dry. Also, the sirloin my daughter got was cooked properly and very tough.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Customers may order at most 3 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Delivery was requested to a residential address. The order quantity is 1. Ravi placed an order for a coffee grinder.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: DECISION RULE [IF review value is equal to 7 THEN case verified is yes ELSE [ALL of: member verified is yes | account value is at most 56]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The member verified is no. The case verified is no. The review value is 7. The routing reference is 441. The account value is 56.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,1,and(2,3))"}
{"request": {"context": "Skating to Europe According to the International Ice Hockey Federation, 276 N.H.L. players have signed with European teams. Could hockey be headed for a new world order?", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Skating to Europe According to the International Ice Hockey Federation, 276 N.H.L. players have signed with European teams. Could hockey be headed for a new world order?", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Skating to Europe According to the International Ice Hockey Federation, 276 N.H.L. players have signed with European teams. Could hockey be headed for a new world order?", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when NOT (((package value is less than 26) OR (the elapsed days from entry start date to entry end date are at most 57)) AND (request value is between 18 and 28, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 27. The entry end date is 2027-07-03. The request value is 19. The routing reference is 721. The entry start date is 2027-05-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(and(0,or(1,2)))"}
{"request": {"context": "policy: DECISION RULE [IF review value is equal to 7 THEN case verified is yes ELSE [ALL of: member verified is yes | account value is at most 56]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The member verified is no. The case verified is yes. The review value is 7. The routing reference is 821. The account value is 56.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,1,and(2,3))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: I watched this movie because I like Nicolas Cage and well, I found it strange and completely pointless... so I decided to poke around a little bit and got my hands on the 70s copy of it. Wow. what a difference. The original one was way better. I'd like you all to know it did originally actually make a statement, it's existence did have a purpose. It was really the Christian public expressing their fear of paganism. If you dig deeper into it it also makes comments on life but I don't want to go into details, just, simply put, if you were disappointed and you'd like to know what it SHOULD look like, feel free to watch the 70s version, a little dated, but A lot better.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: Don’t waste your time. This computer has a faulty screen. Software wasn’t reset. Has other persons info in it; including the ms office key. Spend the extra 40 bucks and get a new one.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 45 days of the purchase date.\ncase: The order was paid by card and shipped to Ravi's home address. Ravi bought a rain jacket on March 19, 2026. The return request was submitted on May 4, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "policy: A case is approved when either case signer is the same person as case designated approver, or else either package value is at most 53, and also review value is equal to 53, or else the condition \"entry value is greater than 38\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 54. The entry value is 49. The case signer is Sana. The case designated approver is Aiko. The package value is 64. The routing reference is 639.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(and(1,2),not(3)))"}
{"request": {"context": "phoned-in business as usual .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "My hair came out orange. Not any way close to light brown.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "Manilow Sings Sinatra is an album by singer-songwriter Barry Manilow released in 1998. The album was a compilation of himself singing songs originally made notable by Frank Sinatra who had recently died. The album also featured two new compositions intended as tributes to Sinatra.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "album", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "American expansion into the Northwest Territory was being obstructed by various Indian tribes since the end of the Revolution, who were supplied and encouraged by the British. Americans on the western frontier demanded that interference be stopped. There is dispute, however, over whether or not the American desire to annex Canada brought on the war. Several historians believe that the capture of Canada was intended only as a means to secure a bargaining chip, which would then be used to force Britain to back down on the maritime issues. It would also cut off food supplies for Britain's West Indian colonies, and temporarily prevent the British from continuing to arm the Indians. However, many historians believe that a desire to annex Canada was a cause of the war. This view was more prevalent before 1940, but remains widely held today. Congressman Richard Mentor Johnson told Congress that the constant Indian atrocities along the Wabash River in Indiana were enabled by supplies from Canada and were proof that ``the war has already commenced... I shall never die contented until I see England's expulsion from North America and her territories incorporated into the United States.''", "question": "was the war of 1812 the american revolution?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Orders of fewer than 5 units are charged the list price. Orders of 5 to 19 units receive the volume discount. Orders of 20 units or more receive the wholesale rate.\ncase: The order is for 12 units. Tomas placed a business order for a pair of running shoes. Delivery is to a branch office.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "- role: customer\n  content: On August 31, 2004, the film was released on DVD, VHS, and later D-VHS in North America by 20th Century Fox Home Entertainment. As with the original theatrical release, the film's release on home video formats proved to be very popular. Early reports indicated that over 2.4 million copies of the film were sold by the middle of the day. The film was available on DVD with English and Spanish subtitles and on VHS tape with English subtitles. The film was released on Blu-ray in North America as a two-disc Definitive Edition set on February 17, 2009. It was also released on Blu-ray in Australia a week before Easter.", "question": "does the passion of the christ have subtitles?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when (NOT (NOT (entry value is less than 45))) AND (((the elapsed days from request start date to request end date are at most 39) AND (package value is equal to 54)) OR (NOT (account value is at most 39))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 49. The package value is 55. The request end date is 2027-09-17. The entry value is 46. The routing reference is 214. The request start date is 2027-08-08.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(not(not(0)),or(and(1,2),not(3)))"}
{"request": {"context": "Lots of room for appointments.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "The .22 cal swag tip would not fit the cleaning rod, it was next to useless", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "- role: customer\n  content: the performances are immaculate , with roussillon providing comic relief .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "a chilling tale of one of the great crimes of 20th century france : the murder of two rich women by their servants in 1933 .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: A case is approved when shipment verified is yes, and also review verified is yes, except when entry value is at most 57, and also package value is at most 51; any other case is denied. Routing references play no part in the decision.\ncase: The package value is 61. The entry value is 58. The shipment verified is yes. The review verified is no. The routing reference is 499.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),and(2,3))"}
{"request": {"context": "policy: A case is approved when the condition \"member value is equal to 46, and also account value is equal to 12\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 46. The account value is 13. The routing reference is 238.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(and(0,1))"}
{"request": {"context": "Russia successfully launches new type of rocket carrier The first test launch of a new rocket carrier, the Soyuz-2 (Rus), was carried out successfully today from the Plesetsk cosmodrome in Arkhangelsk Region at 2130 Moscow time, 1830 gmt.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Russia successfully launches new type of rocket carrier The first test launch of a new rocket carrier, the Soyuz-2 (Rus), was carried out successfully today from the Plesetsk cosmodrome in Arkhangelsk Region at 2130 Moscow time, 1830 gmt.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Russia successfully launches new type of rocket carrier The first test launch of a new rocket carrier, the Soyuz-2 (Rus), was carried out successfully today from the Plesetsk cosmodrome in Arkhangelsk Region at 2130 Moscow time, 1830 gmt.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: A case is approved when either either member value is less than 51, except when shipment value is at most 32, or else package value is greater than 27, or else the elapsed days from account start date to account end date are at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The account end date is 2027-03-01. The shipment value is 32. The package value is 26. The routing reference is 605. The member value is 51. The account start date is 2027-01-25.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(1,unless(2,3)))"}
{"request": {"context": "Very cute but it's way too big for my girl. I'm afraid the small will be too big too.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "ticket:\n  channel: web form\n  body: There were items were missing, and I could not get anyone to speak with about the missing things. I was supposed to have seven clamps. Instead, I was given three clamps and four metal paper clips. I was given an inventory list and not actual instructions. I am very dissatisfied with the service of the company.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Applicants must be at least 21 years old to be eligible for the apprenticeship.\ncase: Omar is 16 years old. Omar applied to join the apprenticeship. The application form was complete and signed.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "- role: customer\n  content: half the Normal Packs.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "Mistrials are generally not covered by the double jeopardy clause. If a judge dismisses the case or concludes the trial without deciding the facts in the defendant's favor (for example, by dismissing the case on procedural grounds), the case is a mistrial and may normally be retried. Furthermore, if a jury cannot reach a verdict, the judge may declare a mistrial and order a retrial as was addressed in United States v. Josef Perez, 22 U.S. 579 (1824). When the defendant moves for a mistrial, there is no bar to retrial, even if the prosecutor or judge caused the error that forms the basis of the motion. An exception exists, however, where the prosecutor or judge has acted in bad faith. In Oregon v. Kennedy, 456 U.S. 667 (1982), the Supreme Court held that ``only where the governmental conduct in question is intended to 'goad' the defendant into moving for a mistrial may a defendant raise the bar of double jeopardy to a second trial after having succeeded in aborting the first on his own motion.''", "question": "does double jeopardy apply to a hung jury?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "ticket:\n  channel: email\n  body: A Supposedly Fun Thing I'll Never Do  Essays and Arguments , by David Foster Wallace (Little, Brown and Co.).", "question": "Hypothesis: \"\"A supposedly Fun Thing I'll Never Do : Essays and Arguements.\" is a novel by David Foster Wallace. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [[ANY of: entry value is between 39 and 49, including both endpoints | case value is at most 31] UNLESS [ANY of: the elapsed days from shipment start date to shipment end date are at most 33 | package signer is the same person as package designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment start date is 2027-02-15. The shipment end date is 2027-03-21. The package signer is Sana. The entry value is 50. The routing reference is 385. The case value is 31. The package designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),or(2,3))"}
{"request": {"context": "policy: A case is approved when when the condition \"the elapsed days from package start date to package end date are at most 58\" fails the requirement is that shipment value is less than 46, and when it is not the requirement is that request value is greater than 32; any other case is denied. Routing references play no part in the decision.\ncase: The shipment value is 46. The package end date is 2027-03-17. The package start date is 2027-01-17. The routing reference is 489. The request value is 31.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(not(0),1,2)"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: The order is for 40 units. Ravi placed a business order for a rain jacket. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "but i would really check that out because i'm concerned in coming years it's going to be you know just a horrendous burden to especially the younger folks", "question": "question: Hypothesis: \"I would look into that because it might be able to benefit the youth greatly. \" How does it relate to the premise?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 30 days of the purchase date.\ncase: The return request was submitted on February 18, 2026. The order was paid by card and shipped to Noah's home address. Noah bought a backpack on January 25, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(it is not the case that (member value is at most 23)); (if (the elapsed days from review start date to review end date are at most 5), use (account value is equal to 57); otherwise use (case value is greater than 27))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-04-27. The account value is 57. The member value is 22. The review end date is 2027-05-02. The case value is 26. The routing reference is 871.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(if(0,1,2),not(3))"}
{"request": {"context": "policy: Warranty claims made within 180 days of purchase are covered in full. Claims made after that but within 730 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on March 26, 2026. The claim describes a defect in the switch. Aiko purchased a desk lamp on January 14, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: DECISION RULE [IF review value is equal to 7 THEN case verified is yes ELSE [ALL of: member verified is yes | account value is at most 56]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The member verified is no. The case verified is yes. The review value is 7. The routing reference is 441. The account value is 56.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,1,and(2,3))"}
{"request": {"context": "ticket:\n  channel: web form\n  body: nothing here seems as funny as it did in analyze this , not even joe viterelli as de niro 's right-hand goombah .", "question": "question: What is the sentiment of this review sentence?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "- role: customer\n  content: Watch Lists Glance (AP) AP - Other measures related to transportation security, besides expanded use of government watch lists on flights, cruises and at airports, included in the intelligence overhaul legislation:", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "- role: customer\n  content: Watch Lists Glance (AP) AP - Other measures related to transportation security, besides expanded use of government watch lists on flights, cruises and at airports, included in the intelligence overhaul legislation:", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: Watch Lists Glance (AP) AP - Other measures related to transportation security, besides expanded use of government watch lists on flights, cruises and at airports, included in the intelligence overhaul legislation:", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "For some easy hiking, stop off at the lower station of Plan de l'Aiguille (2,310 m/7,580 ft).", "question": "Hypothesis: \"One of the best parts of hiking at the lower station is the beautiful view.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "I mean...seriously? \\nI should have known better given that it only had one positive review. My 600th yelp review, you think I'd've learned by now!\\nI wandered in here looking for a pedicure. My first hint should have been the old crusty polishes on the walls - no Essie etc here. But whatever, I picked out my color and headed to the dilapidated pedicure chair. (I put my bag on the adjacent tray and it couldn't support the weight). They don't use razors here, which is fine, but I expected a thorough scrubbing of my callouses, and when she gave me a tiny little once over then unceremoniously threw my feet back into the water I thought to myself, OK there's no way on earth this is more than a $10 pedicure right? My heels still looked as bad as they did when I walked in. \\nThen she painted my nails and sat me down at the \\\"dryer\\\" which consisted of two tiny fans pointed at my feet. For real. I had to sit there for over ten minutes, she proclaimed my toes dry, then I paid - $25?!!!! For $25 in Boston where I'm from I could get a legit pedicure. I was dumbfounded. I paid, walked home, and lo and behold two of my toes already had smudged polish.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "I mean...seriously? \\nI should have known better given that it only had one positive review. My 600th yelp review, you think I'd've learned by now!\\nI wandered in here looking for a pedicure. My first hint should have been the old crusty polishes on the walls - no Essie etc here. But whatever, I picked out my color and headed to the dilapidated pedicure chair. (I put my bag on the adjacent tray and it couldn't support the weight). They don't use razors here, which is fine, but I expected a thorough scrubbing of my callouses, and when she gave me a tiny little once over then unceremoniously threw my feet back into the water I thought to myself, OK there's no way on earth this is more than a $10 pedicure right? My heels still looked as bad as they did when I walked in. \\nThen she painted my nails and sat me down at the \\\"dryer\\\" which consisted of two tiny fans pointed at my feet. For real. I had to sit there for over ten minutes, she proclaimed my toes dry, then I paid - $25?!!!! For $25 in Boston where I'm from I could get a legit pedicure. I was dumbfounded. I paid, walked home, and lo and behold two of my toes already had smudged polish.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The ticket was routed through the chat queue. Tomas opened a priority ticket about a duplicate charge. The first response arrived 58 hours after the ticket was opened.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "What happened in the Long March ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ANY of: account verified is yes | entry verified is yes] | [ANY of: case value is at least 38 | shipment verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment verified is no. The account verified is yes. The routing reference is 879. The case value is 37. The entry verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(or(0,1),or(2,3))"}
{"request": {"context": "ticket:\n  channel: email\n  body: you know she might call me during a halftime or after the game or something but i just don't call her and now they're starting up with new football", "question": "Hypothesis: \"Sometimes she will call me after a football game.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [ANY of: request value is between 26 and 36, including both endpoints | [IF package value is equal to 49 THEN entry verified is yes ELSE review value is greater than 24]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 403. The review value is 23. The entry verified is no. The package value is 50. The request value is 26.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,if(1,2,3))"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: The claim total is $361. Receipts were attached for every line item. Jonas, the account owner, submitted an expense claim.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "SEC Bars Some Mutual Fund Fees (Reuters) Reuters - The U.S. Securities and Exchange\\Commission barred U.S. mutual funds on Wednesday from\\channeling brokerage commissions toward Wall Street firms based\\on their promotion of the funds' shares.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: what: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "SEC Bars Some Mutual Fund Fees (Reuters) Reuters - The U.S. Securities and Exchange\\Commission barred U.S. mutual funds on Wednesday from\\channeling brokerage commissions toward Wall Street firms based\\on their promotion of the funds' shares.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "SEC Bars Some Mutual Fund Fees (Reuters) Reuters - The U.S. Securities and Exchange\\Commission barred U.S. mutual funds on Wednesday from\\channeling brokerage commissions toward Wall Street firms based\\on their promotion of the funds' shares.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: What color are tennis balls ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "- role: customer\n  content: Product is exactly how its described. Nice feel, nice weight perfect size...", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "as quiet , patient and tenacious as mr. lopez himself , who approaches his difficult , endless work with remarkable serenity and discipline .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Expense claims of $2,500 or less are approved automatically. Claims above $2,500 require director sign-off.\ncase: Receipts were attached for every line item. Ravi, the billing manager, submitted an expense claim. The claim total is $1,980.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "ticket:\n  channel: email\n  body: This show was amazing and full of energy ,I recommend anyone coming to Las Vegas see this show. The cast of celebrity entertainer's song selection really wowed the audience I was on my feet dancing , thank you \\\"Raiding The Vault\\\" show at the LVH for making my rock -n-roll experience one I would never forget! I'm your biggest fan.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "ticket:\n  channel: email\n  body: This show was amazing and full of energy ,I recommend anyone coming to Las Vegas see this show. The cast of celebrity entertainer's song selection really wowed the audience I was on my feet dancing , thank you \\\"Raiding The Vault\\\" show at the LVH for making my rock -n-roll experience one I would never forget! I'm your biggest fan.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Horrible quality. Size is way off. Runs very small.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "document: AL Wrap: Schilling Shines as Red Sox Romp Past Yankees (Reuters) Reuters - David Ortiz's two-run double launched\\a four-run first inning and Curt Schilling allowed one hit over\\seven frames to power the Boston Red Sox to an 11-4 win over\\the New York Yankees in the American League Sunday.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "document: AL Wrap: Schilling Shines as Red Sox Romp Past Yankees (Reuters) Reuters - David Ortiz's two-run double launched\\a four-run first inning and Curt Schilling allowed one hit over\\seven frames to power the Boston Red Sox to an 11-4 win over\\the New York Yankees in the American League Sunday.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: AL Wrap: Schilling Shines as Red Sox Romp Past Yankees (Reuters) Reuters - David Ortiz's two-run double launched\\a four-run first inning and Curt Schilling allowed one hit over\\seven frames to power the Boston Red Sox to an 11-4 win over\\the New York Yankees in the American League Sunday.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when it is not the case that (if (package value is less than 50), use (request value is at least 39); otherwise use (the elapsed days from entry start date to entry end date are at most 15)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry start date is 2027-04-21. The routing reference is 213. The package value is 60. The entry end date is 2027-05-05. The request value is 39.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(if(0,1,2))"}
{"request": {"context": "policy: Approve exactly when NOT (NOT ((shipment verified is yes) OR (case value is at least 27))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment verified is no. The case value is 26. The routing reference is 464.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(not(or(0,1)))"}
{"request": {"context": "policy: A case is approved when entry verified is yes, except when review value is greater than 23, and also member value is less than 18, and also the condition \"shipment verified is yes\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry verified is yes. The review value is 34. The routing reference is 884. The member value is 17. The shipment verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(and(1,2),not(3)))"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 8 hours after the ticket was opened. Lena opened a priority ticket about a login failure. The ticket was routed through the chat queue.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((entry signer is the same person as entry designated approver) holds and the exception (it is not the case that (member value is equal to 50)) does not hold); (account value is equal to 28)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry signer is Kofi. The member value is 50. The routing reference is 842. The account value is 28. The entry designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(0,unless(1,not(2)))"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 14 days of the purchase date.\ncase: The return request was submitted on September 4, 2026. The order was paid by card and shipped to Priya's home address. Priya bought a wireless keyboard on September 1, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "Kodak Java Suit Moves To Penalty Phase Kodak #39;s claim that Sun #39;s use of Java violates three patents it holds could cast a cloud over Java #39;s future direction. By Charles Babcock.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Kodak Java Suit Moves To Penalty Phase Kodak #39;s claim that Sun #39;s use of Java violates three patents it holds could cast a cloud over Java #39;s future direction. By Charles Babcock.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Kodak Java Suit Moves To Penalty Phase Kodak #39;s claim that Sun #39;s use of Java violates three patents it holds could cast a cloud over Java #39;s future direction. By Charles Babcock.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: DECISION RULE [[ANY of: entry value is between 39 and 49, including both endpoints | case value is at most 31] UNLESS [ANY of: the elapsed days from shipment start date to shipment end date are at most 33 | package signer is the same person as package designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment start date is 2027-02-15. The shipment end date is 2027-03-21. The package signer is Sana. The entry value is 50. The routing reference is 510. The case value is 31. The package designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),or(2,3))"}
{"request": {"context": "After several occassions of enjoying expensive wine on their patio, we were asked to leave after objecting to the blond Matre d's ecouragagement of smoking by females outside the restaurant doors. His attempt at flirting was more important to him than keeping customers and unfortunately we have many other fabulous options in the Kierland area for enjoying dinner and wine.", "question": "question: How many stars did this reviewer give?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "After several occassions of enjoying expensive wine on their patio, we were asked to leave after objecting to the blond Matre d's ecouragagement of smoking by females outside the restaurant doors. His attempt at flirting was more important to him than keeping customers and unfortunately we have many other fabulous options in the Kierland area for enjoying dinner and wine.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "What shrubs can be planted that will be safe from deer ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Invoices paid within 10 days after the due date incur no fee. Payments between 11 and 60 days late incur a 2% fee. Payments later than 60 days incur a 10% fee and a hold on the account.\ncase: Payment was received 47 days after the due date. The payment was made by bank transfer. Lena's invoice for $120 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "despite an overwrought ending , the film works as well as it does because of the performances .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Warranty claims made within 365 days of purchase are covered in full. Claims made after that but within 1095 days are covered at half cost. Later claims are not covered.\ncase: The claim describes a defect in the stitching. A warranty claim for it was filed on July 3, 2028. Kofi purchased a backpack on April 21, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Customers may order at most 3 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Delivery was requested to a residential address. The order quantity is 11. Ravi placed an order for a coffee grinder.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "far_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: The claim total is $217. Receipts were attached for every line item. Jonas, the account owner, submitted an expense claim.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "policy: Customers may order at most 10 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: The order quantity is 4. Delivery was requested to a residential address. Omar placed an order for a desk lamp.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: Customers may order at most 5 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Delivery was requested to a residential address. Elin placed an order for a coffee grinder. The order quantity is 13.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "far_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: Approve exactly when it is not the case that (at least one of these conditions hold: [(member verified is yes); (package signer is the same person as package designated approver)]). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member verified is no. The package designated approver is Aiko. The routing reference is 794. The package signer is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(or(0,1))"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 30 days of the purchase date.\ncase: The order was paid by card and shipped to Jonas's home address. Jonas bought a desk lamp on January 10, 2026. The return request was submitted on February 19, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "policy: Approve exactly when ((entry value is equal to 27) AND (NOT (shipment value is at most 46))) AND ((package verified is yes) AND (NOT (member verified is yes))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 364. The member verified is no. The package verified is yes. The shipment value is 47. The entry value is 37.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,not(1)),and(2,not(3)))"}
{"request": {"context": "policy: A case is approved when the condition \"either either case signer is the same person as case designated approver, or else package value is equal to 43, or else the elapsed days from entry start date to entry end date are at most 56\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The case signer is Aiko. The case designated approver is Noah. The entry start date is 2027-07-13. The package value is 44. The entry end date is 2027-09-07. The routing reference is 378.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(or(0,or(1,2)))"}
{"request": {"context": "ticket:\n  channel: email\n  body: Paul Allen to Help Open N.M. Computer Exhibit ALBUQUERQUE, N.M. (AP) -- Microsoft co-founder Paul G...", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: email\n  body: Paul Allen to Help Open N.M. Computer Exhibit ALBUQUERQUE, N.M. (AP) -- Microsoft co-founder Paul G...", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: Paul Allen to Help Open N.M. Computer Exhibit ALBUQUERQUE, N.M. (AP) -- Microsoft co-founder Paul G...", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: paxton 's uneven directorial debut fails to unlock the full potential of what is in many ways a fresh and dramatically substantial spin on the genre .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "Don't fight over small issues.", "question": "Hypothesis: \"Small issues aren't to be fought over. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: The payment was made by card. Payment was received 29 days after the due date. Sana's invoice for $2,300 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "- role: customer\n  content: This wifi dongle is good for casual use but I would not reccomend for gaming since it only supports 2.4 GHZ and does not support 5 GHZ.ive had it for around 6 months and it started to disconnect randomly.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "starts out with tremendous promise , introducing an intriguing and alluring premise , only to fall prey to a boatload of screenwriting cliches that sink it faster than a leaky freighter .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "ticket:\n  channel: web form\n  body: just a kiss wants desperately to come off as a fanciful film about the typical problems of average people .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "They shrank from the starvation and misery a general strike would entail, and were willing to meet the Government half-way.", "question": "Hypothesis: \"They wanted to meet the Government half-way to avoid starvation and misery.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "We don't get around to this part of town so this bookmark has been sitting there for a long while. I was dying hungry as I've sat at the hospital all day so I ordered a extra dish for the fiance and I to share. \\n\\nFood came out shockingly fast and piping hot off the woks I can hear in the kitchen. \\n\\nPork Larb - it was labeled something along the lines of \\\"spicy special salad (larb) on the menu so I missed it at first, disappointed they didn't make it until I saw a Yelp review that ordered it. I really liked the way they do it here. In general, most places are light on the herbal aspects and I appreciated the abundance of mint, cilantro and thin slices of kefir. I normally add condiments to tweak larb at places to my liking but I didn't touch this at all. Love! \\n\\nGreen curry with beef - this is a more watery style of curry, but the flavor was definitely there. Lots of carrot, bamboo shoot,bell pepper and basil came along with it so not as much meat, but it's still enough to go around.\\n\\nPad kee mao/drunken noodles - the flavors of this was lovely, noodles were perfectly cooked and not gummy and the pork was tender.\\n\\nThe Fiance tends to", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "We don't get around to this part of town so this bookmark has been sitting there for a long while. I was dying hungry as I've sat at the hospital all day so I ordered a extra dish for the fiance and I to share. \\n\\nFood came out shockingly fast and piping hot off the woks I can hear in the kitchen. \\n\\nPork Larb - it was labeled something along the lines of \\\"spicy special salad (larb) on the menu so I missed it at first, disappointed they didn't make it until I saw a Yelp review that ordered it. I really liked the way they do it here. In general, most places are light on the herbal aspects and I appreciated the abundance of mint, cilantro and thin slices of kefir. I normally add condiments to tweak larb at places to my liking but I didn't touch this at all. Love! \\n\\nGreen curry with beef - this is a more watery style of curry, but the flavor was definitely there. Lots of carrot, bamboo shoot,bell pepper and basil came along with it so not as much meat, but it's still enough to go around.\\n\\nPad kee mao/drunken noodles - the flavors of this was lovely, noodles were perfectly cooked and not gummy and the pork was tender.\\n\\nThe Fiance tends to", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "MV Dartmouth Castle is a passenger ship operating on the River Dart for the Dartmouth Steam Railway and River Boat Company. She is listed on the National Register of Historic Ships", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "meanoftransportation", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "an original and highly cerebral examination of the psychopathic mind", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when NOT (((package value is less than 26) OR (the elapsed days from entry start date to entry end date are at most 57)) AND (request value is between 18 and 28, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 27. The entry end date is 2027-07-04. The request value is 19. The routing reference is 721. The entry start date is 2027-05-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(and(0,or(1,2)))"}
{"request": {"context": "policy: A case is approved when either case value is greater than 25, except when entry value is equal to 16, or else the condition \"account value is at least 11\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The account value is 22. The entry value is 26. The case value is 36. The routing reference is 139.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(not(0),unless(1,2))"}
{"request": {"context": "document: the real star of this movie is the score , as in the songs translate well to film , and it 's really well directed .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: A case is approved when either either package value is at most 40, or else account value is at least 44, or else request value is at most 23, except when shipment value is at most 46; any other case is denied. Routing references play no part in the decision.\ncase: The request value is 24. The routing reference is 995. The shipment value is 45. The package value is 41. The account value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(or(0,1),unless(2,3))"}
{"request": {"context": "document: Surely You Joust was Ray Stevens' twenty-third studio album and his third for MCA Records. The album's front cover shows Stevens disguised as a medieval knight with a horse standing next to him. The album's back cover shows Stevens in the same costume but in a junk pile with two junk men.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "album", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: The payment was made by card. Payment was received 1 days after the due date. Sana's invoice for $2,300 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "Pitru Paksha is considered by Hindus to be inauspicious, given the death rite performed during the ceremony, known as Shraddha or Tarpan. In southern and western India, it falls in the 2nd paksha (fortnight) Hindu lunar month of Bhadrapada (September) and follows the fortnight immediately after the Ganesh festival. It begins on the Pratipada (first day of the fortnight) ending with the no moon day known as Sarvapitri amavasya, Pitru Amavasya, Peddala Amavasya, Mahalaya amavasya or simply Mahalaya. Most years, the autumnal equinox falls within this period, i.e. the Sun transitions from the northern to the southern hemisphere during this period. In North India and Nepal, and cultures following the purnimanta calendar or the solar calendar, this period may correspond to the waning fortnight of the luni-solar month Ashvin, instead of Bhadrapada.", "question": "is it good to die in pitru paksha?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "I went to see \"Evening\" because of the cast. I'd gone to see \"Norman's Room\" for that reason -- that movie offering Diane Keaton, Leonardo De Caprio and, also, Meryl Streep -- and had loved every minute of it. Same for \"The Notebook\" even though it was chick-flit lite. And my feeling was, anything offering performances by Vanessa Redgrave, Meryl Streep, Patrick Wilson and Glenn Close would be at least as good. Instead, I found sometimes even the greatest actors cannot overcome trite, simplistic and -- on one occasion -- truly offensive material. Now I had no problem with the way the film was structured. I actually enjoy movies that cut back and forth in time to tell a story...so long as one era illuminates the other and vise verse. But while Vanessa's character being on her deathbed and recalling a past event she felt \"was a mistake\" was riveting, at times, the part actually showing what that \"past mistake\" was does nothing to clarify the matter. In fact, it makes it seem meaningless in the silliest \"girl meets boy, girl gets boy, girl loses boy\" fashion, and in the most unbelievable, clichéd, wrong-headed way possible. And from here be spoilers, so bear that in mind should you continue reading. First of all, Claire Danes was brutally miscast. Not only", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "-lrb- a -rrb- poorly executed comedy .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "Ring of Bright Water is a book by Gavin Maxwell about his life in a remote house in coastal Scotland where he kept several wild otters as pets. First published in 1960, it became a best seller and is considered a literary masterpiece, eventually selling over two million copies. A fictionalised film of the same name was made from it and released in 1969.", "question": "is ring of bright water a true story?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when NOT ((request value is at least 60) holds and the exception (review value is between 5 and 15, including both endpoints) does not hold). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 70. The review value is 4. The routing reference is 392.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(unless(0,1))"}
{"request": {"context": "Ya know, I believe this gondola ride looks several billion times cooler than it actually is. Walking about the Venetian you can't help but notice the meandering waterways (both indoor and outdoor) filled with gondolas being pushed by singing Italians. I hand it to the Venetian for the awesome ambiance---however, I probably won't fork over $16/person for a gondola ride again west of the Mississippi.\\n\\nThey offer an option to pay $64 for a private gondola. My date and I just paid the $16/person and ended up with a private gondola by chance. I have to admit we felt victorious given we watched people in line pay the $64. Our gondolier was Italian and nice enough, however I did not get the romantic feeling I would expect because of his incessant small talk. It really was 12 minutes of chatting with a complete stranger and 1 minute of him singing under the outdoor tunnel. My date and I exchanged a peck because making out would have felt like a private soft porn video for the Italian gondolier. \\n\\nThe cool part? Being in a gondola on a man-made body of water. Somehow floating around never ceases to be awesome. I did enjoy the city lights (if you do end up trying this out, I recommend doing the outdoor ride at night) but", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "Ya know, I believe this gondola ride looks several billion times cooler than it actually is. Walking about the Venetian you can't help but notice the meandering waterways (both indoor and outdoor) filled with gondolas being pushed by singing Italians. I hand it to the Venetian for the awesome ambiance---however, I probably won't fork over $16/person for a gondola ride again west of the Mississippi.\\n\\nThey offer an option to pay $64 for a private gondola. My date and I just paid the $16/person and ended up with a private gondola by chance. I have to admit we felt victorious given we watched people in line pay the $64. Our gondolier was Italian and nice enough, however I did not get the romantic feeling I would expect because of his incessant small talk. It really was 12 minutes of chatting with a complete stranger and 1 minute of him singing under the outdoor tunnel. My date and I exchanged a peck because making out would have felt like a private soft porn video for the Italian gondolier. \\n\\nThe cool part? Being in a gondola on a man-made body of water. Somehow floating around never ceases to be awesome. I did enjoy the city lights (if you do end up trying this out, I recommend doing the outdoor ride at night) but", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(entry value is at most 6); (it is not the case that (the elapsed days from case start date to case end date are at most 21))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 909. The case start date is 2027-05-03. The case end date is 2027-06-03. The entry value is 16.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,not(1))"}
{"request": {"context": "ticket:\n  channel: email\n  body: Visually speaking, this film is stunning. It has some delightful black comedic moments. But on the whole, the plot is very clichéd, as is its seeming message. If you're a fan of over-the-top violence in mainstream movies like hostel or saw, you'll love it. If you're looking for something at all high-brow, steer away. I saw it as part of the edinburgh film festival 06, and I only chose it because I was looking for something disturbing. Ultimately, it isn't disturbing. Just grinding and unpleasant to sit through. If you genuinely want to be challenged, go see something like The Lost. If you want to be grossed out, or tell your friends about a really messed up film, then this is for you.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((entry signer is the same person as entry designated approver) holds and the exception (it is not the case that (member value is equal to 50)) does not hold); (account value is equal to 28)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry signer is Kofi. The member value is 60. The routing reference is 842. The account value is 28. The entry designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,unless(1,not(2)))"}
{"request": {"context": "CMS is responsible for the overall management of Medicaid; however, each state is responsible for managing its own program.", "question": "Hypothesis: \"Each state is responsible for managing its own program, though CMS is responsible for the overall management of Medicaid.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: A case is approved when either either member value is less than 51, except when shipment value is at most 32, or else package value is greater than 27, or else the elapsed days from account start date to account end date are at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The account end date is 2027-03-01. The shipment value is 32. The package value is 26. The routing reference is 605. The member value is 51. The account start date is 2027-01-25.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(1,unless(2,3)))"}
{"request": {"context": "- role: customer\n  content: Josef Anton Bellesini was a mayor of Vienna.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "officeholder", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: Elin placed a business order for a backpack. Delivery is to a branch office. The order is for 389 units.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "The states that have legislatively adopted stand-your-ground laws are Alabama, Alaska, Arizona, Florida, Georgia, Idaho, Indiana, Iowa, Kansas, Kentucky, Louisiana, Michigan, Mississippi, Missouri, Montana, Nevada, New Hampshire, North Carolina, Oklahoma, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Utah, West Virginia, and Wyoming.", "question": "does indiana have a stand your ground law?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((request value is greater than 43) holds and the exception (account value is at most 52) does not hold); (at least one of these conditions hold: [(shipment value is at most 57); (member value is between 39 and 49, including both endpoints)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 53. The shipment value is 58. The routing reference is 251. The member value is 39. The request value is 43.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(or(0,1),unless(2,3))"}
{"request": {"context": "policy: Customers may order at most 10 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: The order quantity is 24. Delivery was requested to a residential address. Omar placed an order for a desk lamp.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "far_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "What five-time winner of the Kentucky Derby lost his first 25 races ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "- role: customer\n  content: An anime television series adaptation by Madhouse titled Parasyte -the maxim- (寄生獣 セイの格率, Kiseijū Sei no Kakuritsu) aired in Japan on NTV between October 9, 2014 and March 26, 2015. The series follows Shinichi Izumi, a high school boy whose right hand becomes possessed by an alien Parasite named Migi, finding himself in a battle against other Parasites who feast on other humans. The series was simulcast by Crunchyroll outside of Asia and by Animax Asia in Southeast Asia and South Asia. Sentai Filmworks has licensed the anime for North America, South America, United Kingdom, Australia, and New Zealand release. UK Distributor Animatsu Entertainment will release the series in the UK. In Australia and New Zealand, Hanabee has acquired the series and will release it within the region. The opening theme song is ``Let Me Hear'' performed by Fear, and loathing in Las Vegas. The ending theme is ``It's the Right Time'' performed by Daichi Miura. At Anime Expo 2015, Sentai Filmworks announced that the anime would run on Adult Swim's Toonami block on October 3, 2015. The English dub, provided and distributed by Sentai Filmworks, has been released on DVD and Blu-ray in two parts, with Part 1 (Episodes 1--12) on April 5, 2016 and Part 2 (Episodes 13--24) on July 5, 2016.", "question": "question: will there be another season of parasyte the maxim?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "document: Timur Savcı, who was in the original series, is a producer of Game of Silence. Other producers are David Hudgins, Carol Mendelsohn, Julie Weitz, Niels Arden Oplev, and Tariq Jalil. The series premiered as a ``preview'' on April 12, 2016. It then debuted in its regular Thursday at 10:00 PM timeslot on April 14, 2016, and aired until June 5, 2016. On May 13, 2016, NBC cancelled the series after one season.", "question": "is there a season 2 of game of silence?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "if you like peace , you 'll like promises .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "How big is a baby bald eagle ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "This is by far the worst film I have seen in my entire life. The acting is poor and the storyline is almost incomprehensible. Whether or not you like lightships or any ships for that matter is irrelevant. As for special effects the film has none. The whole film crew were probably on the boat out in rough seas rather than in a studio and when some of the men are \"stabbed\" (if you can even call it that) their reactions are totally unreal. The guns are more quiet than a mute. How this film could have one two awards puts serious questions to the state of the human mind. Well thats about it. This review is probably more fun to read than the film is to watch. If anyone is considering watching it or buying it I would seriously advise you against it for obvious reasons. I have said that it includes a spoiler. If the fact that some people get stabbed and a gun gets fired is a plot giveaway. I suppose it is because they are the only good parts of the film.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: I thought this move was very good. There were a few things that were less than perfect, but overall, I was quite surprised. The courtroom scene in the end seemed a little unrealistic, but was real enough to be entertaining. I found that the movie communicated the hardships of going though military training and the sacrifices that go along with it. Being a military pilot I could relate to many of these parts.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(at least one of these conditions hold: [(entry value is at most 53); (member value is equal to 45)]); (account value is equal to 13)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 45. The entry value is 63. The routing reference is 492. The account value is 23.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,2))"}
{"request": {"context": "The principal bridesmaid, if one is so designated, may be called the chief bridesmaid or maid of honor if she is unmarried, or the matron of honor if she is married. A junior bridesmaid is a girl who is clearly too young to be married, but who is included as an honorary bridesmaid. In the United States, typically only the maid/matron of honor and the best man are the official witnesses for the wedding license.", "question": "is there a difference between maid of honour and chief bridesmaid?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "Penquis Valley High School is a four-year coeducational high school located in Milo Maine. Founded in 1968 (combining Brownville and Milo High Schools) the school serves students from Milo Atkinson Brownville LaGrange and Medford and Orneville Township. Matt Hackett is the Principal and Micah Grant is the Assistant Principal. The school is accredited by NEASC.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: A case is approved when entry verified is yes, except when review value is greater than 23, and also member value is less than 18, and also the condition \"shipment verified is yes\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry verified is no. The review value is 34. The routing reference is 884. The member value is 17. The shipment verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(0,and(and(1,2),not(3)))"}
{"request": {"context": "The acids in wine are an important component in both winemaking and the finished product of wine. They are present in both grapes and wine, having direct influences on the color, balance and taste of the wine as well as the growth and vitality of yeast during fermentation and protecting the wine from bacteria. The measure of the amount of acidity in wine is known as the ``titratable acidity'' or ``total acidity'', which refers to the test that yields the total of all acids present, while strength of acidity is measured according to pH, with most wines having a pH between 2.9 and 3.9. Generally, the lower the pH, the higher the acidity in the wine. However, there is no direct connection between total acidity and pH (it is possible to find wines with a high pH for wine and high acidity). In wine tasting, the term ``acidity'' refers to the fresh, tart and sour attributes of the wine which are evaluated in relation to how well the acidity balances out the sweetness and bitter components of the wine such as tannins. Three primary acids are found in wine grapes: tartaric, malic and citric acids. During the course of winemaking and in the finished wines, acetic, butyric, lactic and succinic acids can play significant roles. Most of the acids involved with wine are fixed acids with the notable exception of acetic acid, mostly found in vinegar, which is volatile and can contribute to the wine fault known as volatile acidity. Sometimes, additional acids, such as ascorbic, sorbic and sulfurous acids, are used in winemaking.", "question": "is tartaric acid the same as citric acid?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "Visconti's first film has all his trademark visual flair and immaculate technique, accompanied by compelling performances from Massimo Girotti as the handsome drifter and, best of all, Clara Calamai as the fabulous, frantic Giovanna. Remade several times as 'The Postman Rings Twice' but never bettered. Can't believe this was the man's first film! It shows the confidence of someone at the zenith of their career.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "Their price was $18,000.", "question": "Hypothesis: \"Their price was $1.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Orders of fewer than 5 units are charged the list price. Orders of 5 to 19 units receive the volume discount. Orders of 20 units or more receive the wholesale rate.\ncase: The order is for 7 units. Noah placed a business order for a coffee grinder. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: Elin placed a business order for a backpack. Delivery is to a branch office. The order is for 63 units.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Here's the deal. The one we went to was under the Coke bottle on the Strip. You wait in a line outside the office cause they don't want people to block the sidewalk or the lobby. Luckily, that line moves somewhat quickly. When you get to the end of that line, you get to go inside and wait in another line. One several screens will be showing what tickets are available that night. You can also see it online under sneak peeks. We got our Ka tickets here and also got a discount voucher for a restaurant in our hotel. The Ka tickets worked out, the voucher not so much as it was 30% off your main entree. They also promise to let you come back tomorrow to buy more tickets but promise to let you stand in the VIP line which means a shorter wait.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Here's the deal. The one we went to was under the Coke bottle on the Strip. You wait in a line outside the office cause they don't want people to block the sidewalk or the lobby. Luckily, that line moves somewhat quickly. When you get to the end of that line, you get to go inside and wait in another line. One several screens will be showing what tickets are available that night. You can also see it online under sneak peeks. We got our Ka tickets here and also got a discount voucher for a restaurant in our hotel. The Ka tickets worked out, the voucher not so much as it was 30% off your main entree. They also promise to let you come back tomorrow to buy more tickets but promise to let you stand in the VIP line which means a shorter wait.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: DECISION RULE [[case value is less than 29 UNLESS [ANY of: request value is greater than 49 | entry verified is yes]] UNLESS the elapsed days from account start date to account end date are at most 13] -> approve; otherwise reject. Ignore the routing reference.\ncase: The request value is 48. The routing reference is 601. The account end date is 2027-04-17. The account start date is 2027-03-25. The case value is 28. The entry verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(unless(0,or(1,2)),3)"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Online Award to Times Writer The New York Times on the Web won an award from the Online News Association for commentary by Nicholas Kristof, an Op-Ed columnist for The New York Times.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Online Award to Times Writer The New York Times on the Web won an award from the Online News Association for commentary by Nicholas Kristof, an Op-Ed columnist for The New York Times.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Online Award to Times Writer The New York Times on the Web won an award from the Online News Association for commentary by Nicholas Kristof, an Op-Ed columnist for The New York Times.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Long Island is a densely populated island off the East Coast of the United States, beginning at New York Harbor just 0.35 miles (0.56 km) from Manhattan Island and extending eastward into the Atlantic Ocean. The island comprises four counties in the U.S. state of New York: Kings and Queens counties (which comprise the New York City boroughs of Brooklyn and Queens, respectively) in the west, and Nassau and Suffolk counties in the east. The majority of New York City residents now live on Long Island. However, many people in the New York metropolitan area (including those in Brooklyn and Queens) colloquially use the term ``Long Island'' to refer exclusively to Nassau and Suffolk counties, which are mainly suburban in character.", "question": "is brooklyn and queens part of long island?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(it is not the case that (member value is at most 23)); (if (the elapsed days from review start date to review end date are at most 5), use (account value is equal to 57); otherwise use (case value is greater than 27))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-04-27. The account value is 57. The member value is 24. The review end date is 2027-05-02. The case value is 26. The routing reference is 871.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(if(0,1,2),not(3))"}
{"request": {"context": "Aenigma is a South African spider genus of the Salticidae family (jumping spiders) with only one described species A. incognita.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "animal", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "The IANA time zone database contains one zone for Ireland in the file zone.tab, named Europe/Dublin.", "question": "is all of ireland in the same time zone?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Bandits and bounty hunters, thought Ca'daan.", "question": "Hypothesis: \"Ca'daan expected the bounty hunters to catch the bandits.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "Wonderful meal - I enjoyed everything that I tasted! My friends raved about this place and I agree with them! I had the grilled catfish and it was wonderfully seasoned and cooked perfectly. The 3 sides were very generous: I had the yams, greens and mac and cheese - I thought I was back at my Grandma's table in Mississippi! Don't miss the corn bread, sooo good! I tasted the smothered pork chips, fried catfish and the ox tails & they were also delicious. Everyone working was very nice and accommodating. The service was fast and attentive.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "Wonderful meal - I enjoyed everything that I tasted! My friends raved about this place and I agree with them! I had the grilled catfish and it was wonderfully seasoned and cooked perfectly. The 3 sides were very generous: I had the yams, greens and mac and cheese - I thought I was back at my Grandma's table in Mississippi! Don't miss the corn bread, sooo good! I tasted the smothered pork chips, fried catfish and the ox tails & they were also delicious. Everyone working was very nice and accommodating. The service was fast and attentive.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "document: Ambidexterity is the state of being equally adapted in the use of both the left and the right hand. When referring to objects, the concept indicates that the object is equally suitable for right-handed and left-handed people. When referring to humans, it indicates that a person has no marked preference for the use of the right or left hand.", "question": "can someone be both left and right handed?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Only used one of them a few times, it gets VERY warm when charging. I'm planning to take them on a vacation, but may not leave them charging when I'm gone or sleeping. They don't retract very well each time. Cool idea but not perfect!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "How is bubble wrap made ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "yes when i first started teaching i remember i went up to my job and i said okay i want to see a curriculum guide you know so i'll know what i'm suppose to teach and they go we don't have one of we don't even know where one is", "question": "Hypothesis: \"I had to start my curriculum from scratch.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 30 days of the purchase date.\ncase: The return request was submitted on August 31, 2026. The order was paid by card and shipped to Sana's home address. Sana bought a wireless keyboard on August 27, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "- role: customer\n  content: Scotland: Rangers down rival Celtic Nacho Novo and Dado Psro each scored as Rangers defeated rival Celtic 2-0 in the Scottish premier league Saturday. Novo scored on a 15th-minute penalty and Prso added the second in the 36th", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "- role: customer\n  content: Scotland: Rangers down rival Celtic Nacho Novo and Dado Psro each scored as Rangers defeated rival Celtic 2-0 in the Scottish premier league Saturday. Novo scored on a 15th-minute penalty and Prso added the second in the 36th", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: Scotland: Rangers down rival Celtic Nacho Novo and Dado Psro each scored as Rangers defeated rival Celtic 2-0 in the Scottish premier league Saturday. Novo scored on a 15th-minute penalty and Prso added the second in the 36th", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "this was the best bonnie and clyde movie i have seen. it has more accurate accounts of what happened and while it doesnt glorify their crimes it casts the pair in a normal light. i give this movie a 10. it has great actors,realistic scenes and excellent writers.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when it is not the case that (if (package value is less than 50), use (request value is at least 39); otherwise use (the elapsed days from entry start date to entry end date are at most 15)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry start date is 2027-04-21. The routing reference is 213. The package value is 60. The entry end date is 2027-05-05. The request value is 39.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(if(0,1,2))"}
{"request": {"context": "Filming took place on location at, among other venues, Wrigley Field (including in between games of a doubleheader between the Cubs and the rival St. Louis Cardinals) and O'Hare Airport. However, the road game against the Dodgers was filmed at Comiskey Park", "question": "was rookie of the year filmed at wrigley?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Preston Xavier Burke, M.D. is a fictional character from the medical drama television series Grey's Anatomy, which airs on ABC in the United States. The character was created by series producer Shonda Rhimes, and was portrayed by actor Isaiah Washington from 2005 to 2007. Introduced as an attending cardiothoracic surgeon at the fictional Seattle Grace Hospital, Burke's romantic relationship with intern Cristina Yang formed one of the main storylines in the shows' first three seasons.", "question": "does dr. burke die in grey's anatomy?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: The claim total is $244. Lena, the warehouse lead, submitted an expense claim. Receipts were attached for every line item.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 14 days of the purchase date.\ncase: The return request was submitted on September 28, 2026. The order was paid by card and shipped to Priya's home address. Priya bought a wireless keyboard on September 1, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "But if they just gave the magazine away, advertisers would lose interest.", "question": "Hypothesis: \"Advertisers would lose interest if magazine were free because that would make them less money.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "document: Best in the city, sushi at a affordable price. Owner gives a 20% military/veteran discount. \\nThey can make sushi the way you like it. We love the tuna lover, and spicy tuna rolls. The place gives a welcoming feeling, and has a fun atmosphere. Kids are welcome, its a chill environment, I don't have to wear a tux and go broke to enjoy, fresh delicious sushi.", "question": "question: How many stars did this reviewer give?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "document: Best in the city, sushi at a affordable price. Owner gives a 20% military/veteran discount. \\nThey can make sushi the way you like it. We love the tuna lover, and spicy tuna rolls. The place gives a welcoming feeling, and has a fun atmosphere. Kids are welcome, its a chill environment, I don't have to wear a tux and go broke to enjoy, fresh delicious sushi.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "I purchased in November for a Christmas gift for my daughter. She went to open it and the spoon broke. I went to try to return/replace and there's a no return policy.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "and more and more people without i have several friends without jobs now that in this last riff went out looking for jobs and jobs are really hard to come by now really hard to come by", "question": "Hypothesis: \"It's so easy to get a job. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when (((case value is between 20 and 30, including both endpoints) OR (member value is less than 55)) AND (entry value is at least 45)) AND (shipment value is at least 7). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 56. The shipment value is 7. The case value is 19. The routing reference is 464. The member value is 65.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,and(1,or(2,3)))"}
{"request": {"context": "ticket:\n  channel: web form\n  body: it 's hard to say who might enjoy this , are there tolstoy groupies out there ?", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "Pretty standard Pho besides my bowl being packed with a tons of protein (meat). It was actually a huge bowl that I could barely make half a dent in.\\n\\nHad a chance to try the 7 course beef as well.\\n\\nI'd visit it again.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "Pretty standard Pho besides my bowl being packed with a tons of protein (meat). It was actually a huge bowl that I could barely make half a dent in.\\n\\nHad a chance to try the 7 course beef as well.\\n\\nI'd visit it again.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "document: William Lustig's followup to \"Maniac\" proves conclusively that, without Tom Savini's spectacular effects and Spinell's convincing performance, \"Maniac\" would never have become the cult hit that it did. \"Vigilante\" is badly directed, with a simple-minded script that spells everything out for you and is predictable at every turn, and also mediocre performances by all the actors. Judging from the sense of \"deja vu\" this film gave me, Lustig had watched \"Death Wish\" several times too many before making this! (*1/2)", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when (review value is at most 37) holds and the exception (at least one of these conditions hold: [(case value is less than 42); (account value is less than 40)]) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 393. The review value is 37. The case value is 41. The account value is 41.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(0,or(1,2))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: What work would be so satisfying that he'd do it for free?", "question": "Hypothesis: \"He does work for free just to pad his resume.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: DECISION RULE [IF [ALL of: entry value is at most 17 | account verified is yes] THEN package value is at least 9 ELSE case signer is the same person as case designated approver] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 194. The case signer is Ravi. The entry value is 18. The account verified is no. The case designated approver is Tomas. The package value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(and(0,1),2,3)"}
{"request": {"context": "such as Internet use,", "question": "Hypothesis: \"Internet use can corrupt a child's mind if the child uses it without supervision.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "ticket:\n  channel: email\n  body: Cheap made. Strings hanging out. Will return", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when ((member value is between 21 and 31, including both endpoints) AND (case value is greater than 52)) AND ((shipment value is at most 60) OR (account value is greater than 45)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 21. The account value is 55. The case value is 51. The routing reference is 398. The shipment value is 61.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),or(2,3))"}
{"request": {"context": "policy: Support responses within 1 hours meet the service level. Responses after 1 but within 8 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 1 hours after the ticket was opened. Lena opened a priority ticket about a login failure. The ticket was routed through the phone queue.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: A case is approved when when the elapsed days from entry start date to entry end date are at most 44 the requirement is that the elapsed days from case start date to case end date are at most 41, and also shipment value is equal to 59, and when it is not the requirement is that the condition \"request value is between 57 and 67, including both endpoints\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry end date is 2027-05-02. The entry start date is 2027-03-19. The case start date is 2027-05-26. The request value is 67. The shipment value is 69. The routing reference is 309. The case end date is 2027-07-05.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,and(1,2),not(3))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Why Tommy?", "question": "question: Hypothesis: \"Why would Tommy let this happen?\" How does it relate to the premise?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "- role: customer\n  content: What does the name Melissa mean ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "document: 3/5/11: we met some other families here for our first visit. Separate play area for under 2 years made it a lot easier for me to relax & enjoy watching my son play on his own & with other little ones. We will be back. :)", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "document: 3/5/11: we met some other families here for our first visit. Separate play area for under 2 years made it a lot easier for me to relax & enjoy watching my son play on his own & with other little ones. We will be back. :)", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "This Mesa Grill does not live up to the one in NY City. I've eaten here in two occasions: dinner and brunch, and was very disappointed both times. Almost all the dishes I had here were drawn with sauces, I think the sauce should enhance the dish not over powering it. The only dish I would recommend is the Spicy Chicken+ Sweet Potato Hash on their brunch menu@ $16.50(with Avocado, Tomato, Maytag Blue Cheese + Buttermilk Serrano Dressing), the dish was very nicely presented and was also hearty and filling. The other good thing here is their complimentary mini blue corn breads.\\n\\nI sincerely feel that the chefs at this location should step up to the plate, and really refine their dishes and cooking skills, inorder to meet the standard of the one in NYC.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "This Mesa Grill does not live up to the one in NY City. I've eaten here in two occasions: dinner and brunch, and was very disappointed both times. Almost all the dishes I had here were drawn with sauces, I think the sauce should enhance the dish not over powering it. The only dish I would recommend is the Spicy Chicken+ Sweet Potato Hash on their brunch menu@ $16.50(with Avocado, Tomato, Maytag Blue Cheese + Buttermilk Serrano Dressing), the dish was very nicely presented and was also hearty and filling. The other good thing here is their complimentary mini blue corn breads.\\n\\nI sincerely feel that the chefs at this location should step up to the plate, and really refine their dishes and cooking skills, inorder to meet the standard of the one in NYC.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Indonesia's Megawati faces former general in landmark presidential polls (AFP) AFP - From the remote jungles of Papua to the high rises of Jakarta, Indonesia was staging landmark elections widely expected to see a former general unseat President Megawati Sukarnoputri.", "question": "question: What is the topic of this article?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "Indonesia's Megawati faces former general in landmark presidential polls (AFP) AFP - From the remote jungles of Papua to the high rises of Jakarta, Indonesia was staging landmark elections widely expected to see a former general unseat President Megawati Sukarnoputri.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Indonesia's Megawati faces former general in landmark presidential polls (AFP) AFP - From the remote jungles of Papua to the high rises of Jakarta, Indonesia was staging landmark elections widely expected to see a former general unseat President Megawati Sukarnoputri.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: TiVo backer Alsop to leave VC firm Prominent venture capitalist at Silicon Valley's New Enterprise Associates wants to \"see if I can design a working life for myself.\"", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: email\n  body: TiVo backer Alsop to leave VC firm Prominent venture capitalist at Silicon Valley's New Enterprise Associates wants to \"see if I can design a working life for myself.\"", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: TiVo backer Alsop to leave VC firm Prominent venture capitalist at Silicon Valley's New Enterprise Associates wants to \"see if I can design a working life for myself.\"", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: Does not even work. wasted my money.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "ticket:\n  channel: web form\n  body: AMP Returns to Profit After Spinoff Australian financial services group AMP Ltd. on Wednesday reported a return to profit in the first half of the year after cutting loose its loss-making British insurance operations.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: web form\n  body: AMP Returns to Profit After Spinoff Australian financial services group AMP Ltd. on Wednesday reported a return to profit in the first half of the year after cutting loose its loss-making British insurance operations.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: AMP Returns to Profit After Spinoff Australian financial services group AMP Ltd. on Wednesday reported a return to profit in the first half of the year after cutting loose its loss-making British insurance operations.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: The payment was made by cheque. Payment was received 10 days after the due date. Ravi's invoice for $2,300 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "Kiichi Nakai (中井 貴一 Nakai Kiichi born 18 September 1961) is a Japanese actor. His father Keiji Sada also a movie actor died when Nakai was only two years old. Nakai started his acting career while he was still in university and was awarded the Rookie of the Year of Japan Academy Awards in 1981.He won the Japan Academy Best Supporting Actor award in 1994 (for 47 Ronin) and Best Actor award in 2003 (for Mibu gishi den (When the Last Sword is Drawn).", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: A case is approved when the condition \"member value is equal to 46, and also account value is equal to 12\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 46. The account value is 13. The routing reference is 238.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(and(0,1))"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(the elapsed days from account start date to account end date are at most 7); (at least one of these conditions hold: [(at least one of these conditions hold: [(case value is equal to 24); (shipment value is between 22 and 32, including both endpoints)]); (entry value is greater than 35)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account start date is 2027-04-19. The routing reference is 105. The case value is 25. The entry value is 34. The shipment value is 22. The account end date is 2027-05-06.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,or(2,3)))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Another curious enigma about Columbus that has baffled historians is that no reliable likeness of him has ever survived.", "question": "Hypothesis: \"Numerous reliable depictions of Columbus have survived over the years.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: what: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "Who was Buffalo Bill ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Support responses within 4 hours meet the service level. Responses after 4 but within 24 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 7 hours after the ticket was opened. The ticket was routed through the phone queue. Mira opened a priority ticket about a login failure.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Approve exactly when (both these conditions hold: [(account value is equal to 32); (the elapsed days from case start date to case end date are at most 7)]) holds and the exception ((request signer is the same person as request designated approver) holds and the exception (member value is at least 38) does not hold) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 794. The request designated approver is Noah. The account value is 31. The member value is 39. The request signer is Ravi. The case start date is 2027-04-07. The case end date is 2027-04-14.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),unless(2,3))"}
{"request": {"context": "Triviella franziskae is a species of small sea snail a marine gastropod mollusk in the family Triviidae the false cowries or trivias.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "animal", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Packaging came totally destroyed and unable to return. Rather pay more for undamaged product.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: DECISION RULE [ANY of: [review value is at most 33 UNLESS member value is at least 10] | [ALL of: case verified is yes | package value is less than 26]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review value is 33. The case verified is yes. The package value is 25. The routing reference is 212. The member value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(and(0,1),unless(2,3))"}
{"request": {"context": "The sequel to the ever popular Cinderella story reminded me somewhat of what they did with one of the Beauty & the Beast movies. It's basically three short stories rolled into 1. OK, the mice are adorable (I love Gus! He's sooo cute!), and Lucifer's awesome (as usual). I liked some of the newer characters as well, (Pom Pom was adorable and I did like Prudence.). Still, the storyline was somewhat limited, but still very cute. So, I vote 7/10.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Customers may order at most 3 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Priya placed an order for a backpack. Delivery was requested to a residential address. The order quantity is 2.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "What World War I battle saw 6 , 0 troops killed in one day ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "document: Ryū Murakami (村上 龍 Murakami Ryū born February 19 1952 in Sasebo Nagasaki) is a Japanese novelist and filmmaker.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "\"Stairway to Heaven\" is a outstanding invention of movie making, probably never duplicated. I rank it with \"The Wizard of Oz\" and \"African Queen,\" although it is a totally different type of movie than \"African Queen.\" \"Stairway to Heaven\" is a psycho-drama that uses performance concepts and technical effects that, to my knowledge, are totally unique. For example, there is the combination of B&W and color footage - as in \"Oz,\" but the significance of the contrast goes way beyond the simple - but beautiful - effect achieved in \"Oz.\" In \"Stairway\" the purpose and effect of the contrast can only be described as powerful. Another brilliant aspect of \"Stairway\" is the concept of \"time\" and how it is used here. How could anybody have conceived of a better way to make time stand still – literally? And then there is the Stairway itself! If you have any imagination at all, you will agree with me. \"Stairway to Heaven\" is a true gem.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: Sanderson joins Iowa State coaching staff Ames, IA (Sports Network) - Former Iowa State standout wrestler Cael Sanderson has joined the school as an assistant for the upcoming season.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "document: Sanderson joins Iowa State coaching staff Ames, IA (Sports Network) - Former Iowa State standout wrestler Cael Sanderson has joined the school as an assistant for the upcoming season.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: Sanderson joins Iowa State coaching staff Ames, IA (Sports Network) - Former Iowa State standout wrestler Cael Sanderson has joined the school as an assistant for the upcoming season.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Got a little long at times but a good story. Enjoyed the cooking tips. Interesting about not making tomato sauce in an iron pot. I never did, but now I have a reason not to.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the rental agreement.\ncase: The application form was complete and signed. Omar is 32 years old. Omar applied to join the rental agreement.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "This is a list of current and former locks and dams of the Upper Mississippi River which ends at the Mississippi River's confluence with the Ohio River at Cairo, Illinois.", "question": "is there a dam on the mississippi river?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "ticket:\n  channel: chat\n  body: a thoroughly awful movie -- dumb , narratively chaotic , visually sloppy ... a weird amalgam of ` the thing ' and a geriatric ` scream . '", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "document: I had to register for IMDb just to post a comment on just how awful this movie is...my cats and a ball of string have a better storyline than this. Not the worst acting I've ever seen, but when you wipe out almost the entire cast of the movie within 5 minutes, it leaves a bit to be desired. There wasn't a single 'scare' moment in the movie, with the exception of when they were watching the movie 'Halloween' on the TV. All around, it seems like it could've been a good story, rolling the credits and saying that Chasey Lain was in it was a bit of a loss as I didn't recognize her right away and her scene was already over before I could've said 'oh yeah, there she is'. I'm so glad I saw this in a hotel and didn't pay for it as I'd be real ticked if I had payed a cent to see this. I normally like or can at least find a redeeming factor in a movie, but this one is an exception. It's so bad that it's not even that amusing so-good-it's-bad....it's just plain bad.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "Seriously? Chicken wings came out in 3 minutes. That's amazing! How can you cook chicken in 3 minutes?\\nChorizo sliders were soggy, had to send them back. If you're looking for pre-made food this is your place. If you like good, quality food I don't recommend Bahamas Breeze.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "Seriously? Chicken wings came out in 3 minutes. That's amazing! How can you cook chicken in 3 minutes?\\nChorizo sliders were soggy, had to send them back. If you're looking for pre-made food this is your place. If you like good, quality food I don't recommend Bahamas Breeze.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Customers may order at most 5 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Tomas placed an order for a coffee grinder. The order quantity is 3. Delivery was requested to a residential address.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "within_limit", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Anthony Pratt and Celia Bobak were nominated for the Academy Award for Best Art Direction, as was John Mathieson for Cinematography. However, both categories were awarded to The Aviator. Andrew Lloyd Webber and lyricist Charles Hart were nominated for the Academy Award for Best Original Song (``Learn to Be Lonely'') but lost to ``Al otro lado del río'' from The Motorcycle Diaries. The song was also nominated for the Golden Globe but it lost to Alfie's ``Old Habits Die Hard''. In the same ceremony, Emmy Rossum was nominated for Best Actress in a Motion Picture Musical or Comedy, losing to Annette Bening in Being Julia. At the Saturn Awards, Rossum won for Best Performance by a Younger Actor, while The Phantom of the Opera was nominated for Best Action/Adventure/Thriller Film and Alexandra Byrne was nominated for Costume Design.", "question": "did the phantom of the opera movie win any awards?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: A case is approved when entry verified is yes, except when review value is greater than 23, and also member value is less than 18, and also the condition \"shipment verified is yes\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry verified is no. The review value is 34. The routing reference is 884. The member value is 17. The shipment verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(0,and(and(1,2),not(3)))"}
{"request": {"context": "policy: Approve exactly when ((member value is between 21 and 31, including both endpoints) AND (case value is greater than 52)) AND ((shipment value is at most 60) OR (account value is greater than 45)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 21. The account value is 55. The case value is 62. The routing reference is 927. The shipment value is 61.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(and(0,1),or(2,3))"}
{"request": {"context": "Rodman could have a place here, after all There are exactly 18.7 reasons why Dennis Rodman thinks he can play in the NBA again. The Worm is much older now, less energetic and physically not what he used to be.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: what: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Rodman could have a place here, after all There are exactly 18.7 reasons why Dennis Rodman thinks he can play in the NBA again. The Worm is much older now, less energetic and physically not what he used to be.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Rodman could have a place here, after all There are exactly 18.7 reasons why Dennis Rodman thinks he can play in the NBA again. The Worm is much older now, less energetic and physically not what he used to be.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: DECISION RULE [ANY of: package value is between 7 and 17, including both endpoints | [ALL of: entry value is greater than 50 | member value is greater than 7]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 414. The package value is 6. The entry value is 51. The member value is 7.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,and(1,2))"}
{"request": {"context": "After our drive to Vegas, I was starving! Since we were staying at the Palazzo, we decided to try it. I had the crab stuffed salmon and the other girls had mac n cheese and sandwiches. My food was just alright...not worth the price. I kept hearing about their so called famous mac n cheese with Doritos. Meh nothing special. Just regular old pasta, with runny sauce, sprinkled with some sad Doritos. The sandwiches everyone else got looked pretty good though..but way to rich and heavy. The service was alright but considering we were one of only 5 tables there at the time, we could have received some more attention and faster service. Overall, not a big fan.", "question": "question: How many stars did this reviewer give?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "After our drive to Vegas, I was starving! Since we were staying at the Palazzo, we decided to try it. I had the crab stuffed salmon and the other girls had mac n cheese and sandwiches. My food was just alright...not worth the price. I kept hearing about their so called famous mac n cheese with Doritos. Meh nothing special. Just regular old pasta, with runny sauce, sprinkled with some sad Doritos. The sandwiches everyone else got looked pretty good though..but way to rich and heavy. The service was alright but considering we were one of only 5 tables there at the time, we could have received some more attention and faster service. Overall, not a big fan.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "document: Parappur is a village in Malappuram district in the state of Kerala India situated near to Kottakkal. The village has an area of 15.11 km2 (4473 sq mi) and is bordered by Vengara Ooragam and Othukkungal villages to the north and Othukkungal village to the east Edarikkod and Kottakkal villages to the south and the Edarikkod and Vengara towards the west.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "village", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "ticket:\n  channel: chat\n  body: disturbing .", "question": "question: What is the sentiment of this review sentence?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Royal Dutch Shell plc (LSE: RDSA RDSB) commonly known as Shell is an Anglo–Dutch multinational oil and gas company headquartered in the Netherlands and incorporated in the United Kingdom. Created by the merger of Royal Dutch Petroleum and UK-based Shell Transport & Trading it is the largest company in the world in terms of revenue and one of the six oil and gas supermajors.Shell is also one of the world's most valuable companies.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "company", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "document: 52-Pick Up never got the respect it should have. It works on many levels, and has a complicated but followable plot. The actors involved give some of their finest performances. Ann-Margret, Roy Scheider, and John Glover are perfectly cast and provide deep character portrayals. Notable too are Vanity, who should have parlayed this into a serious acting career given the unexpected ability she shows, and Kelly Preston, who's character will haunt you for a few days. Anyone who likes action combined with a gritty complicated story will enjoy this.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: Instead, the Romans built a gigantic ramp of rock and earth extending from the nearest adjacent mountain so they could simply march up to the level of the walls.", "question": "Hypothesis: \"It took 2 years for the Romans to build a ramp.  \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "I have been on the hunt for a perfect coffee spot. I work from home, and love mixing it up scenery. Let's hit the basics:\\n\\n1. The location is right off the main drag making it a pretty straight shot to get to. \\n\\n2. Convenience Factor: There is a drive-thru which is perfect if you are on the go. If you are here to stay, there is ample parking!\\n\\n3. Ambiance: There is the cutest outdoor seating area that makes you forget that you are essentially surrounded by a parking lot. This is a \\\"Don't Judge a Book by its Cover\\\" kind of shop. What I mean by that is from the outside, you are not sure it is going to be anything special, especially if you don't see the outdoor seating. When you walk in, you immediately feel at home. It is spacious and offers plenty of seating. The lighting is dim and relaxing and the music is not played so loud that you cannot hear yourself think. It is the perfect spot to work, study, or just meet a friend. Don't believe, go check it out and you can see people doing all of the above!\\n\\n4. The good stuff: I was excited to hear that all of the coffee was organic! They only roast a medium blend which at first", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "I have been on the hunt for a perfect coffee spot. I work from home, and love mixing it up scenery. Let's hit the basics:\\n\\n1. The location is right off the main drag making it a pretty straight shot to get to. \\n\\n2. Convenience Factor: There is a drive-thru which is perfect if you are on the go. If you are here to stay, there is ample parking!\\n\\n3. Ambiance: There is the cutest outdoor seating area that makes you forget that you are essentially surrounded by a parking lot. This is a \\\"Don't Judge a Book by its Cover\\\" kind of shop. What I mean by that is from the outside, you are not sure it is going to be anything special, especially if you don't see the outdoor seating. When you walk in, you immediately feel at home. It is spacious and offers plenty of seating. The lighting is dim and relaxing and the music is not played so loud that you cannot hear yourself think. It is the perfect spot to work, study, or just meet a friend. Don't believe, go check it out and you can see people doing all of the above!\\n\\n4. The good stuff: I was excited to hear that all of the coffee was organic! They only roast a medium blend which at first", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(at least one of these conditions hold: [(entry value is at most 53); (member value is equal to 45)]); (account value is equal to 13)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 45. The entry value is 63. The routing reference is 701. The account value is 23.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,2))"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(it is not the case that (at least one of these conditions hold: [(package value is greater than 38); (the elapsed days from request start date to request end date are at most 54)])); (review value is between 45 and 55, including both endpoints)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 498. The package value is 49. The review value is 45. The request end date is 2027-07-18. The request start date is 2027-05-24.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,not(or(1,2)))"}
{"request": {"context": "Troy Olsen (born July 12 1973) is an American country music singer-songwriter signed to EMI Nashville a sister label to Capitol Records Nashville. He co-wrote Blake Shelton's 2009 single I'll Just Hold On and Tim McGraw's Ghost Town Train a cut from the album Southern Voice.Olsen released his debut single Summer Thing in April 2010 followed by a digital Self-titled extended play on May 11. Summer Thing debuted at No.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "An average airport with an attempt at Vegas glitz that is actually annoying more than interesting.\\n\\nComing into McCarran is fairly easy, but the taxi time is usually substantial and the walk to the baggage area can be long. Signage is mediocre and sometimes hard to follow...especially when trying to find the taxi stand.\\n\\nSpeaking of the taxi stand, it is massively long, but well-run and quick for the amount of people being shuffled through. There are little-to-no smoking restrictions outside of the airport, so persons waiting for a taxi are often subjected to second hand smoke. The people watching is amusing, though, so maybe it's worth it. Watch out for cabbies that take you the long way to the Strip, though. Vegas needs to get control of that.\\n\\nThe food offered at McCarran is chain-based and underwhelming. Bathrooms are surprising large and relatively clean. There are charging stations in a few places, so that's nice, but the chairs are uncomfortable. The chairs are fine, but you are subjected to the noise from the many slot machines in just about all areas of the airport. I have never liked this.\\n\\nIf I am not going to Vegas, I try to avoid connections through McCarran. There are definitely better airports in the U.S. and definitely on the West Coast.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "An average airport with an attempt at Vegas glitz that is actually annoying more than interesting.\\n\\nComing into McCarran is fairly easy, but the taxi time is usually substantial and the walk to the baggage area can be long. Signage is mediocre and sometimes hard to follow...especially when trying to find the taxi stand.\\n\\nSpeaking of the taxi stand, it is massively long, but well-run and quick for the amount of people being shuffled through. There are little-to-no smoking restrictions outside of the airport, so persons waiting for a taxi are often subjected to second hand smoke. The people watching is amusing, though, so maybe it's worth it. Watch out for cabbies that take you the long way to the Strip, though. Vegas needs to get control of that.\\n\\nThe food offered at McCarran is chain-based and underwhelming. Bathrooms are surprising large and relatively clean. There are charging stations in a few places, so that's nice, but the chairs are uncomfortable. The chairs are fine, but you are subjected to the noise from the many slot machines in just about all areas of the airport. I have never liked this.\\n\\nIf I am not going to Vegas, I try to avoid connections through McCarran. There are definitely better airports in the U.S. and definitely on the West Coast.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 14 days of the purchase date.\ncase: The return request was submitted on June 30, 2026. The order was paid by card and shipped to Tomas's home address. Tomas bought a backpack on June 8, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "document: Who runs Andy Capp 's favorite pub ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "Although not officially confirmed by the networks, series co-creator Linda Schuyler has stated that brainstorming is underway for seasons five and six of Next Class. New series leads and recurring characters were also cast to join the series.", "question": "question: will there be more seasons of degrassi next class?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when (entry value is less than 39) AND (shipment verified is yes). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 38. The shipment verified is no. The routing reference is 322.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,1)"}
{"request": {"context": "Jon watched him run to the men and the berserk horse.", "question": "Hypothesis: \"He ran to the calm horse as Jon ran beside him. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when (((case value is between 20 and 30, including both endpoints) OR (member value is less than 55)) AND (entry value is at least 45)) AND (shipment value is at least 7). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 56. The shipment value is 7. The case value is 20. The routing reference is 810. The member value is 65.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(0,and(1,or(2,3)))"}
{"request": {"context": "ticket:\n  channel: web form\n  body: David Tavaré (born on December 20 1984) is a Spanish singer and House Music DJ .", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: A case is approved when shipment signer is the same person as shipment designated approver, except when request value is at most 54, and also review value is at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 34. The shipment signer is Aiko. The request value is 53. The routing reference is 521. The shipment designated approver is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(0,and(1,2))"}
{"request": {"context": "policy: Approve exactly when if (member value is at least 48), use (both these conditions hold: [(it is not the case that (case value is equal to 38)); (it is not the case that (package value is between 52 and 62, including both endpoints))]); otherwise use (it is not the case that (it is not the case that (account value is between 24 and 34, including both endpoints))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 63. The account value is 24. The member value is 58. The routing reference is 553. The case value is 38.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,and(not(1),not(2)),not(not(3)))"}
{"request": {"context": "ticket:\n  channel: web form\n  body: What is the correct way to use `` may '' vs. `` might '' ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The ticket was routed through the chat queue. The first response arrived 40 hours after the ticket was opened. Lena opened a priority ticket about a login failure.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((entry signer is the same person as entry designated approver) holds and the exception (it is not the case that (member value is equal to 50)) does not hold); (account value is equal to 28)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry signer is Kofi. The member value is 60. The routing reference is 434. The account value is 28. The entry designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,unless(1,not(2)))"}
{"request": {"context": "there 's no getting around the fact that this is revenge of the nerds revisited -- again .", "question": "question: What is the sentiment of this review sentence?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: DECISION RULE [IF review value is equal to 7 THEN case verified is yes ELSE [ALL of: member verified is yes | account value is at most 56]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The member verified is no. The case verified is yes. The review value is 7. The routing reference is 441. The account value is 56.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,1,and(2,3))"}
{"request": {"context": "policy: A case is approved when case verified is yes, except when request value is greater than 47, and also account signer is the same person as account designated approver; any other case is denied. Routing references play no part in the decision.\ncase: The case verified is no. The account signer is Kofi. The request value is 46. The account designated approver is Kofi. The routing reference is 865.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,unless(1,2))"}
{"request": {"context": "This movie is AWFUL! I don't even know where to begin, I'm speechless I can't even describe how awful this is. The blood is flourescent first of all, and the acting is AWFUL! The only good part was the biker chick that saves the day. This movie was rediculous, I don't see how it could even get a vote of 1 its so bad. It looks like it was made by highschool students.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: A case is approved when either either member value is less than 51, except when shipment value is at most 32, or else package value is greater than 27, or else the elapsed days from account start date to account end date are at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The account end date is 2027-02-27. The shipment value is 32. The package value is 26. The routing reference is 605. The member value is 51. The account start date is 2027-01-25.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,unless(2,3)))"}
{"request": {"context": "policy: A case is approved when shipment signer is the same person as shipment designated approver, except when request value is at most 54, and also review value is at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 34. The shipment signer is Aiko. The request value is 64. The routing reference is 521. The shipment designated approver is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(1,2))"}
{"request": {"context": "policy: Warranty claims made within 180 days of purchase are covered in full. Claims made after that but within 730 days are covered at half cost. Later claims are not covered.\ncase: The claim describes a defect in the switch. Ravi purchased a backpack on February 12, 2026. A warranty claim for it was filed on April 23, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "- role: customer\n  content: Particularly interesting is the 11th-century donjon (keep) that formed part of the town's southern defenses.", "question": "Hypothesis: \"The donjon lied isolated from other settlements, a lone watchtower among the foothills. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Warranty claims made within 90 days of purchase are covered in full. Claims made after that but within 365 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on February 23, 2026. Jonas purchased a desk lamp on January 26, 2026. The claim describes a defect in the zipper.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: DECISION RULE [NOT: [IF member value is at least 14 THEN [NOT: shipment verified is yes] ELSE [ANY of: review value is equal to 22 | entry verified is yes]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 692. The member value is 15. The entry verified is yes. The review value is 33. The shipment verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(if(0,not(1),or(2,3)))"}
{"request": {"context": "Not all sweet for Lou Before last night's game against the Red Sox, Tampa Bay's Lou Piniella didn't want to talk about speculation that he will", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Not all sweet for Lou Before last night's game against the Red Sox, Tampa Bay's Lou Piniella didn't want to talk about speculation that he will", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Not all sweet for Lou Before last night's game against the Red Sox, Tampa Bay's Lou Piniella didn't want to talk about speculation that he will", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Anthony Paul Kennedy Shriver (born July 20 1965 in Boston Massachusetts) is an American activist for people with intellectual disabilities. In 1989 he founded Best Buddies International an international organization that helps people with intellectual disabilities to find employment and social opportunities. Through his mother he is a nephew of World War II casualty Joseph P. Kennedy Jr. President John F. Kennedy Senator Robert F. Kennedy and Senator Ted Kennedy.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "officeholder", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ALL of: package value is greater than 16 | request value is at least 27] | [case value is at most 47 UNLESS shipment value is at least 38]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 47. The package value is 17. The shipment value is 48. The request value is 28. The routing reference is 358.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),unless(2,3))"}
{"request": {"context": "policy: A case is approved when when review value is between 11 and 21, including both endpoints the requirement is that the condition \"either account value is less than 25, or else shipment signer is the same person as shipment designated approver\" fails, and when it is not the requirement is that the condition \"package value is at most 23\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The shipment designated approver is Mira. The routing reference is 612. The package value is 23. The shipment signer is Kofi. The account value is 25. The review value is 21.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,not(or(1,2)),not(3))"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((entry signer is the same person as entry designated approver) holds and the exception (it is not the case that (member value is equal to 50)) does not hold); (account value is equal to 28)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry signer is Kofi. The member value is 60. The routing reference is 842. The account value is 28. The entry designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,unless(1,not(2)))"}
{"request": {"context": "Hooton confident, Stoddart certain over British GP After weeks of negotiations that, at last viewing, appeared to be forcing the British Grand Prix off the Formula One calendar, Tuesday #39;s teams #39; meeting in London has once again resurrected hopes that the Silverstone race will take place in 2005.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Hooton confident, Stoddart certain over British GP After weeks of negotiations that, at last viewing, appeared to be forcing the British Grand Prix off the Formula One calendar, Tuesday #39;s teams #39; meeting in London has once again resurrected hopes that the Silverstone race will take place in 2005.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Hooton confident, Stoddart certain over British GP After weeks of negotiations that, at last viewing, appeared to be forcing the British Grand Prix off the Formula One calendar, Tuesday #39;s teams #39; meeting in London has once again resurrected hopes that the Silverstone race will take place in 2005.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: 47th--50th Streets--Rockefeller Center is an express station on the IND Sixth Avenue Line of the New York City Subway. It is located along Sixth Avenue (Avenue of the Americas) between 47th and 50th Streets, on the west side of Rockefeller Center. The station is served by the D and F trains at all times, and the B and M trains on weekdays.", "question": "question: does the r train go to rockefeller center?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "What South Korean city is served by Kimpo International Airport ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "I would like to bring to your attention, that twice now I have placed an order for two packages of this item, and in these two instances only one package is being shipped.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when NOT (NOT ((shipment verified is yes) OR (case value is at least 27))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment verified is yes. The case value is 26. The routing reference is 464.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(not(or(0,1)))"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 30 days of the purchase date.\ncase: The return request was submitted on October 8, 2026. The order was paid by card and shipped to Sana's home address. Sana bought a wireless keyboard on August 27, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "policy: Support responses within 4 hours meet the service level. Responses after 4 but within 24 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 1 hours after the ticket was opened. The ticket was routed through the phone queue. Mira opened a priority ticket about a login failure.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Approve exactly when ((entry value is equal to 27) AND (NOT (shipment value is at most 46))) AND ((package verified is yes) AND (NOT (member verified is yes))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 364. The member verified is no. The package verified is yes. The shipment value is 47. The entry value is 37.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,not(1)),and(2,not(3)))"}
{"request": {"context": "ticket:\n  channel: email\n  body: Nintendo DS breaks new ground NEW YORK (CNN/Money) - In June 2001, Nintendo launched the Game Boy Advance -nd gamers cheered. Twenty-one months later, the company rolled out the Game Boy Advance SP  - that cheering turned into an ovation.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: email\n  body: Nintendo DS breaks new ground NEW YORK (CNN/Money) - In June 2001, Nintendo launched the Game Boy Advance -nd gamers cheered. Twenty-one months later, the company rolled out the Game Boy Advance SP  - that cheering turned into an ovation.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: Nintendo DS breaks new ground NEW YORK (CNN/Money) - In June 2001, Nintendo launched the Game Boy Advance -nd gamers cheered. Twenty-one months later, the company rolled out the Game Boy Advance SP  - that cheering turned into an ovation.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "there 's not one decent performance from the cast and not one clever line of dialogue .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "It's a true hole in the wall, that's for sure, but you must check it out!\\nReally great Italian dinners with all you can drink wines, and it even includes dessert and a cappuccino. How are they even making money on that kind of deal! If you go, prepare to stuff yourself with great food and a great time!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "It's a true hole in the wall, that's for sure, but you must check it out!\\nReally great Italian dinners with all you can drink wines, and it even includes dessert and a cappuccino. How are they even making money on that kind of deal! If you go, prepare to stuff yourself with great food and a great time!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: DECISION RULE [ANY of: request value is between 26 and 36, including both endpoints | [IF package value is equal to 49 THEN entry verified is yes ELSE review value is greater than 24]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 403. The review value is 23. The entry verified is no. The package value is 50. The request value is 26.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,if(1,2,3))"}
{"request": {"context": "document: nothing about this movie works .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "This movie provided NOTHING new or worthwhile. After seeing it, my wife and I both agreed that the studio simply churned this out and could have cared less if it was entertaining. This is a good example of a \"concept only\" film--they have a concept about a film and the other details are unimportant because execs KNOW it will make $$ just based on the initial concept. The movie starts with Cruella getting out of prison and going on parole. She no longer hates puppies but has been programmed to adore them--she simply couldn't hurt a flea. This doesn't last too long after her release and she's back to her old ways. Period. The most annoying aspects of the movie were the supporting characters. Eric Idle as the voice of Waddlesworth the bird made me HATE him--and that is TOUGH considering I am a die-hard Python fan. It was obvious he did this because they gave him lots of money (there can't be any other reason). Cruella's low self-esteem servant, Tim McInnerny, was funny in the Black Adder shows but here he is totally wasted and unfunny. And it must have cost a few bucks to get Gérard Depardieu but he was utterly wasted as well. There were some other supporting actors as well but given how poorly written the", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: A case is approved when shipment verified is yes, and also review verified is yes, except when entry value is at most 57, and also package value is at most 51; any other case is denied. Routing references play no part in the decision.\ncase: The package value is 61. The entry value is 58. The shipment verified is yes. The review verified is no. The routing reference is 937.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),and(2,3))"}
{"request": {"context": "Coming from good Minneapolis independent coffee and tea, Coffee Bean & Tea Leaf was mediocre at best. Coffee was watery and bitter. The tea selection was impressive, but the workers were very resistant to make any of it iced. This business seemed to be very corporate flaunting itself as independent with board games, books to read and free water.", "question": "question: How many stars did this reviewer give?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "Coming from good Minneapolis independent coffee and tea, Coffee Bean & Tea Leaf was mediocre at best. Coffee was watery and bitter. The tea selection was impressive, but the workers were very resistant to make any of it iced. This business seemed to be very corporate flaunting itself as independent with board games, books to read and free water.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "What has to be produced in a writ of habeas corpus ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "How can I get someone 's email address ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "What U.S. state are the Finger Lakes in ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Approve exactly when (entry value is less than 39) AND (shipment verified is yes). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 38. The shipment verified is no. The routing reference is 322.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,1)"}
{"request": {"context": "Bands Debut Dual-Sided CD/DVD Hybrid (AP) AP - A new hybrid CD format that offers standard CD audio on one side and the enhanced media capabilities of a DVD on the other made its nationwide debut this week.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Bands Debut Dual-Sided CD/DVD Hybrid (AP) AP - A new hybrid CD format that offers standard CD audio on one side and the enhanced media capabilities of a DVD on the other made its nationwide debut this week.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Bands Debut Dual-Sided CD/DVD Hybrid (AP) AP - A new hybrid CD format that offers standard CD audio on one side and the enhanced media capabilities of a DVD on the other made its nationwide debut this week.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when NOT (NOT ((shipment verified is yes) OR (case value is at least 27))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment verified is no. The case value is 26. The routing reference is 701.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(not(or(0,1)))"}
{"request": {"context": "Stopped in for lunch last week. The pizza is gross, SO GREASY it was not funny. They do have large pieces but the quality is horrible.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "Stopped in for lunch last week. The pizza is gross, SO GREASY it was not funny. They do have large pieces but the quality is horrible.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "despite its flaws ... belinsky is still able to create an engaging story that keeps you guessing at almost every turn .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "They're at head of their class The game will be meaningful on so many levels. When Boston College hosts Syracuse tomorrow at Alumni Stadium, the Eagles will not only be playing for their first Big East title in their last game as a conference member, they will also be looking to send off their 14 departing seniors in grand style by clinching a Bowl Championship Series ...", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "They're at head of their class The game will be meaningful on so many levels. When Boston College hosts Syracuse tomorrow at Alumni Stadium, the Eagles will not only be playing for their first Big East title in their last game as a conference member, they will also be looking to send off their 14 departing seniors in grand style by clinching a Bowl Championship Series ...", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "They're at head of their class The game will be meaningful on so many levels. When Boston College hosts Syracuse tomorrow at Alumni Stadium, the Eagles will not only be playing for their first Big East title in their last game as a conference member, they will also be looking to send off their 14 departing seniors in grand style by clinching a Bowl Championship Series ...", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: Walter Leslie Whitey Farrant (August 12 1912 – October 2 1977) was a Canadian professional ice hockey right winger who played in one National Hockey League game for the Chicago Black Hawks during the 1943–44 NHL season.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "athlete", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "I ordered a sleeping mermaid that is laying down and I got a sleeping mermaid that is curled up. Clearly not the same mold. I haven't used it yet and I also do not have time to reorder the item because my daughter's birthday is in 2 days. I like it, but it is not what I wanted. I would ha e rather had the laying one for the project that I am doing.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Nels Ackerson is the founding partner of the law firm of Ackerson Kauffman Fex and was a 2008 candidate for the United States House of Representatives in Indiana’s 4th Congressional District. He lives in Maryland and practices law in Washington DC.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "officeholder", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Approve exactly when if (account value is greater than 16), use (NOT (shipment value is between 11 and 21, including both endpoints)); otherwise use ((the elapsed days from review start date to review end date are at most 40) AND (package value is at most 48)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-02-27. The routing reference is 615. The shipment value is 12. The review end date is 2027-04-07. The account value is 27. The package value is 58.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,not(1),and(2,3))"}
{"request": {"context": "It's a fast charger but it doesn't charge the phone as fast as my plug in. It's nice but a fast cord charger is better.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "document: it 's a testament to the film 's considerable charm that it succeeds in entertaining , despite playing out like a feature-length sitcom replete with stereotypical familial quandaries .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: A case is approved when either either member value is less than 51, except when shipment value is at most 32, or else package value is greater than 27, or else the elapsed days from account start date to account end date are at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The account end date is 2027-03-01. The shipment value is 32. The package value is 26. The routing reference is 234. The member value is 51. The account start date is 2027-01-25.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(1,unless(2,3)))"}
{"request": {"context": "document: Paired to my bluetooth speaker sometimes, my headphones = never. Once connected, it showed that it was playing , but no sound came from my speaker. Did work with wired headphones, but defeated the purpose of buying a bluetooth mp3 player. Bad feature I noticed was that every time I turned the device off it turned the bluetooth off, so re-pairing required after every shut down. As stated by others, screen resolution poor for anything but menu functions.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "My two year old has a new favorite backpack!! He wears it around the house roaring like a Dino. So glad it is plush bc he wrestles with his daddy wearing it bc that is what a Dino does :)", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "Birding Column: Moments of Photographic Rapture Birdman of Belair Mathew Tekulsky waxes on the serendipitous moments when birds and timing alight together for memorable photographs.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Birding Column: Moments of Photographic Rapture Birdman of Belair Mathew Tekulsky waxes on the serendipitous moments when birds and timing alight together for memorable photographs.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Birding Column: Moments of Photographic Rapture Birdman of Belair Mathew Tekulsky waxes on the serendipitous moments when birds and timing alight together for memorable photographs.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "On tha Cool is the second studio album by the American rapper Baby Beesh released on June 11 2002 on Dope House Records. It was produced by Happy P and Johnny Z. The album has guest performances by SPM Jay Tee Russell Lee Don Cisco Mr. Shadow and DJ Kane.The songs Vamanos and They Don't Even Know also appeared on the album Velvetism.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "album", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "How old is Jeremy Piven ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "Why are there letters on the telephone ? Why are there no Q or Z ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "Not what I expected but are peak proof and customer service is excellent!!!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "the problem with the mayhem in formula 51 is not that it 's offensive , but that it 's boring .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(at least one of these conditions hold: [(entry value is at most 53); (member value is equal to 45)]); (account value is equal to 13)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member value is 45. The entry value is 63. The routing reference is 492. The account value is 23.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,2))"}
{"request": {"context": "Hanson clutched at the scrap he had pocketed, but it showed no sign of leaving, and the tiny blob of sun-stuff remained fixed to the awl.", "question": "Hypothesis: \"There was an important item in Hanson's pocket.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Applicants must be at least 21 years old to be eligible for the volunteer driver program.\ncase: Sana is 30 years old. Sana applied to join the volunteer driver program. The application form was complete and signed.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "policy: Approve exactly when (NOT ((shipment signer is the same person as shipment designated approver) OR (review value is less than 22))) holds and the exception (NOT ((case signer is the same person as case designated approver) AND (entry value is greater than 12))) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 22. The case designated approver is Mira. The routing reference is 346. The shipment designated approver is Ravi. The shipment signer is Tomas. The review value is 22. The case signer is Mira.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(not(or(0,1)),not(and(2,3)))"}
{"request": {"context": "yeah well the Cowboys were America's football team there's no question about that so", "question": "Hypothesis: \"in the old days the Cowboys were a small team with few fans\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "document: I watched 'Ice Age'in the movie theater and I liked the movie. Spite of the fact that 'Ice Age'has many flaws and scientific errors,like humans,sabers,dinosaurs and mammoths living at the same period, and even the location of where the story passes(looks North America,but has some characteristics from Iceland for example) we can have fun even so.(unless you are very severe!) The planet is entering an ICE AGE, and many animals are immigrating to the south where is warmer. Sid is a stupid Sloth that is left behind by his own family, that can't stand him any longer.Walking in his way, he meets Manfred,or how he calls '' Manny'' a moody mammoth who does not care about extinction or immigration and is going to the north. Worried that he can easily be captured, Sid decides to follow Manfred, and in the middle of their journey, they found a human mother with her baby. The mother dies but Manfred and Sid decides to take him and return the baby for the humans. Diego, one of the sabers, decides to follow and help them to go to a shortcut to the human's camp. What Manfred and Sid does not know, is that Diego is from a saber clan who hates humans and wants to kill the baby, and also pretend to betray they", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "Ryo Hiraide (平出涼 Hiraide Ryo born July 18 1991 in Yamanashi) is a Japanese football player.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "athlete", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Mike is honest and dependable. He showed up early and fixed my door fast. I thought the door needed to be replaced, but he explained that he could repair it for now, didn't need replacing yet. He also explained the problem thoroughly, and made some other adjustments. Good deal, glad I didn't try to fix it on my own!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "Mike is honest and dependable. He showed up early and fixed my door fast. I thought the door needed to be replaced, but he explained that he could repair it for now, didn't need replacing yet. He also explained the problem thoroughly, and made some other adjustments. Good deal, glad I didn't try to fix it on my own!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Las 13 Rosas (13 Roses) is a 2007 Spanish film directed by Emilio Martínez Lázaro. It stars Pilar Lopez de Ayala Verónica Sánchez and Marta Etura. The plot based on a true story follows the tragic fate of thirteen young women fighting for their ideals in the aftermath of the Spanish Civil War.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "film", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "If i could give zero stars i would. calling what i ate a pizza would be an insult to pizza. hands down worst pizza i've ever had.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "If i could give zero stars i would. calling what i ate a pizza would be an insult to pizza. hands down worst pizza i've ever had.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Warranty claims made within 90 days of purchase are covered in full. Claims made after that but within 365 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on April 14, 2027. Elin purchased a rain jacket on February 24, 2026. The claim describes a defect in the stitching.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Support responses within 4 hours meet the service level. Responses after 4 but within 24 hours are a minor breach. Anything slower is a major breach.\ncase: Omar opened a priority ticket about an export error. The ticket was routed through the phone queue. The first response arrived 3 hours after the ticket was opened.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ALL of: package value is greater than 16 | request value is at least 27] | [case value is at most 47 UNLESS shipment value is at least 38]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 47. The package value is 17. The shipment value is 48. The request value is 28. The routing reference is 358.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),unless(2,3))"}
{"request": {"context": "Spanish Flu: The Forgotten Fallen is a 2009 television drama. It deals with Dr James Niven's attempts to deal with the 1918 flu pandemic in Manchester. Its screenplay was written by Peter Harness and it starred Bill Paterson as Niven along with Mark Gatiss Kenneth Cranham and Charlotte Riley. It was first broadcast on BBC Four on 5 August 2009.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "film", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Overall, the food was good. Not the best in state, as it's rated, but definitely good. There were 3 of us in our party, me and 2 guys (I was a lucky girl). We ordered the Insalata Grande to start, which was made of Mixed Greens, Fresh Tomatoes, Artichoke Hearts, Roasted Mushrooms, Olives, Pine Nuts, Cracked Pepper, Shaved Parmigiano Reggiano, and Balsamic Vinaigrette. Definitely plenty for the 3 of us to share. Your doctor would approve of the healthy, tasty ingredients in this salad!\\n\\nWe ordered 2 pizzas, Margherita DOC and the Setebello. They're 12\\\" pizzas. 2 was plenty for us but if you had really big eaters, you'd probably need 1 per person. Upon ordering, our server asked us if we wanted our pizza cut in slices. To us that was strange. Don't they always cut pizza in slices? Apparently, here they don't unless you ask. We asked if we didn't get it sliced, how are we supposed to eat it? Do we take a bite and then pass it to the next person at our table? Very strange question we thought.\\n\\nThe Setebello pizza is their signature pizza so we had to try that one. It was delicious. Very flavorful, fresh ingredients. The Margherita DOC pizza was good, but the Setebello blew it away. The Margherita DOC pizza was made", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "Overall, the food was good. Not the best in state, as it's rated, but definitely good. There were 3 of us in our party, me and 2 guys (I was a lucky girl). We ordered the Insalata Grande to start, which was made of Mixed Greens, Fresh Tomatoes, Artichoke Hearts, Roasted Mushrooms, Olives, Pine Nuts, Cracked Pepper, Shaved Parmigiano Reggiano, and Balsamic Vinaigrette. Definitely plenty for the 3 of us to share. Your doctor would approve of the healthy, tasty ingredients in this salad!\\n\\nWe ordered 2 pizzas, Margherita DOC and the Setebello. They're 12\\\" pizzas. 2 was plenty for us but if you had really big eaters, you'd probably need 1 per person. Upon ordering, our server asked us if we wanted our pizza cut in slices. To us that was strange. Don't they always cut pizza in slices? Apparently, here they don't unless you ask. We asked if we didn't get it sliced, how are we supposed to eat it? Do we take a bite and then pass it to the next person at our table? Very strange question we thought.\\n\\nThe Setebello pizza is their signature pizza so we had to try that one. It was delicious. Very flavorful, fresh ingredients. The Margherita DOC pizza was good, but the Setebello blew it away. The Margherita DOC pizza was made", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: The payment was made by cheque. Payment was received 18 days after the due date. Ravi's invoice for $2,300 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "ticket:\n  channel: chat\n  body: The Last Hard Men finds James Coburn an outlaw doing a long sentence breaking free from a chain gang. Do he and his friends head for the Mexican border from jail and safety. No they don't because Coburn has a mission of revenge. To kill the peace officer who brought him in and in the process killed his woman. That peace officer is Charlton Heston who is now retired and he knows what Coburn is after. As he explains it to his daughter, Barbara Hershey, Coburn was holed up in a shack and was involved in a Waco like standoff. His Indian woman was killed in the hail of bullets fired. It's not something he's proud of, she was a collateral casualty in a manhunt. Lest we feel sorry for Coburn he lets us know full well what an evil man he truly is. Heston is his usual stalwart hero, but the acting honors in The Last Hard Men go to James Coburn. He blows everyone else off the screen when he's on. Coburn gets the bright idea of making sure Heston trails him by kidnapping Hershey and taking her to an Indian reservation where the white authorities can't touch him. He knows that Heston has to make it personal then. Coburn's gang includes, Morgan Paull, Thalmus Rasulala, John Quade,", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Warranty claims made within 180 days of purchase are covered in full. Claims made after that but within 730 days are covered at half cost. Later claims are not covered.\ncase: The claim describes a defect in the switch. Ravi purchased a backpack on February 12, 2026. A warranty claim for it was filed on August 7, 2027.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "Tennis: Roddick makes semis Andy Roddick beats Marat Safin in two tie-breaks to reach the Masters Cup semis.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "Tennis: Roddick makes semis Andy Roddick beats Marat Safin in two tie-breaks to reach the Masters Cup semis.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Tennis: Roddick makes semis Andy Roddick beats Marat Safin in two tie-breaks to reach the Masters Cup semis.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "The Stanley Cup playoffs (French: Les séries éliminatoires de la Coupe Stanley) is an elimination tournament in the National Hockey League consisting of four rounds of best-of-seven series. Eight teams from each of the two conferences qualify for the playoffs based on regular season points totals. The final round is commonly known as the Stanley Cup Finals, which sees the two conference champions play for the Stanley Cup.", "question": "is the first round of nhl playoffs best of 7?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when if (member value is at least 48), use (both these conditions hold: [(it is not the case that (case value is equal to 38)); (it is not the case that (package value is between 52 and 62, including both endpoints))]); otherwise use (it is not the case that (it is not the case that (account value is between 24 and 34, including both endpoints))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 63. The account value is 24. The member value is 58. The routing reference is 553. The case value is 38.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,and(not(1),not(2)),not(not(3)))"}
{"request": {"context": "policy: Approve exactly when if (account value is greater than 16), use (NOT (shipment value is between 11 and 21, including both endpoints)); otherwise use ((the elapsed days from review start date to review end date are at most 40) AND (package value is at most 48)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The review start date is 2027-02-27. The routing reference is 615. The shipment value is 10. The review end date is 2027-04-07. The account value is 27. The package value is 58.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,not(1),and(2,3))"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ALL of: request signer is the same person as request designated approver | account value is greater than 29] | [ANY of: package value is greater than 36 | review signer is the same person as review designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 966. The review designated approver is Sana. The account value is 28. The package value is 36. The request signer is Elin. The review signer is Elin. The request designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),or(2,3))"}
{"request": {"context": "Nicolás Monckeberg is a Chilean politician. He is the President of the Chamber of Deputies of Chile.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "officeholder", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "ticket:\n  channel: email\n  body: What did John F. Kennedy consider his greatest blunder in office ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "The Mann Island Buildings are a group of buildings in Liverpool England currently under construction.They comprise three international style mixed use buildings on Mann Island which lies on the waterfront between the Port of Liverpool Building and the Albert Dock.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "building", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "ticket:\n  channel: email\n  body: much credit must be given to the water-camera operating team of don king , sonny miller , and michael stewart .", "question": "question: What is the sentiment of this review sentence?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when it is not the case that (if (package value is less than 50), use (request value is at least 39); otherwise use (the elapsed days from entry start date to entry end date are at most 15)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry start date is 2027-04-21. The routing reference is 695. The package value is 60. The entry end date is 2027-05-05. The request value is 39.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(if(0,1,2))"}
{"request": {"context": "The Plane Driven PD-1 is a modification to the Glasair Sportsman 2+2 to convert it into a practical roadable aircraft. The approach is novel in that it uses a mostly stock aircraft with a modified landing gear pod that carries a separate engine for road propulsion. The PD-1's wings fold along its sides and the main landing gear and road engine pod slide aft along special rails creating a driving configuration.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "meanoftransportation", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Customers may order at most 2 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Delivery was requested to a residential address. The order quantity is 3. Noah placed an order for a wireless keyboard.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "slightly_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the apprenticeship.\ncase: The application form was complete and signed. Ravi applied to join the apprenticeship. Ravi is 29 years old.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: The order is for 21 units. Lena placed a business order for a wireless keyboard. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "policy: Approve exactly when (NOT ((shipment signer is the same person as shipment designated approver) OR (review value is less than 22))) holds and the exception (NOT ((case signer is the same person as case designated approver) AND (entry value is greater than 12))) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 22. The case designated approver is Mira. The routing reference is 548. The shipment designated approver is Ravi. The shipment signer is Tomas. The review value is 22. The case signer is Mira.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(not(or(0,1)),not(and(2,3)))"}
{"request": {"context": "Crockett said the program, funded by a $300,000 grant from the U.S.", "question": "Hypothesis: \"The program received a grant worth over half a million dollars. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 22 hours after the ticket was opened. Priya opened a priority ticket about a duplicate charge. The ticket was routed through the phone queue.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Approve exactly when ((package verified is yes) AND (entry value is at least 36)) OR ((request value is at most 41) AND (shipment value is between 50 and 60, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 472. The entry value is 36. The request value is 51. The package verified is yes. The shipment value is 51.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(and(0,1),and(2,3))"}
{"request": {"context": "Just after the end of WWII Powell & Pressburger were asked to come up with something to try to heal the rift developing between the UK & the USA. At the time there was a lot of \"Overpaid, over sexed and over here\" type of comments. Somehow they came up with this masterpiece. My favourite movie of ALL time. It's got everything. Romance, poetry, emotion, religion, drama and very quirky. I can never explain exactly why, but it hits all the right buttons and although I've seen it hundreds of times (yes, really) I'm still guaranteed to be in tears at many points throughout. Was it the magnificent acting, the wonderful sets, the inspired script ? Who knows. But *DO* watch it and you'll see what I mean.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "document: Roy Rogers stars as Jesse James and his look-alike, gambler man Clint Burns. George \"Gabby\" Hayes is Mr. Rogers's ex-pal, Sheriff Gabby. Gale Storm (Jane Fillmore) and Sally Payne (Polly Morgan) are a noteworthy team, as two reporters on the lookout for stories about the elusive outlaw hero. Of course, mistaken identity / impersonation is a plot development, since Rogers essays a \"dual role\". Rogers is charming, as usual; but, there is nothing really elevating this his performance above the ordinary - any potential to deliver a memorable Jekyll/Hyde performance is done in by poor material. The better pair to watch are Ms. Storm and Ms. Paye as the St. Louis Journal reporters - they are the film's highlight. The songs are fine, though badly synched. *** Jesse James at Bay (1941) Joseph Kane ~ Roy Rogers, George 'Gabby' Hayes, Gale Storm", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "- role: customer\n  content: Tesco aims to take a bite out of Apple #39;s iTunes Supermarket group Tesco is launching itself into the world of digital music with an online download service designed to take on the might of Apple #39;s iTunes.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "- role: customer\n  content: Tesco aims to take a bite out of Apple #39;s iTunes Supermarket group Tesco is launching itself into the world of digital music with an online download service designed to take on the might of Apple #39;s iTunes.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: Tesco aims to take a bite out of Apple #39;s iTunes Supermarket group Tesco is launching itself into the world of digital music with an online download service designed to take on the might of Apple #39;s iTunes.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "The Chosen Family was a Canadian comic strip written and drawn by cartoonist Noreen Stevens. The strip which first appeared in 1988 was based on Stevens' own life revolving around a lesbian couple named Weed and Kenneth-Marie.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "writtenwork", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "The food was god but really could not enjoy it to the fullest because of all the flies..there were so many flies inside and outside.landing on the uncooked food.. The guy sitting next to the meat made no effort to get the flies off the raw meat ... they really need to get that under control..", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "The food was god but really could not enjoy it to the fullest because of all the flies..there were so many flies inside and outside.landing on the uncooked food.. The guy sitting next to the meat made no effort to get the flies off the raw meat ... they really need to get that under control..", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "The Opal-crowned Tanager (Tangara callophrys) is a species of bird in the Thraupidae family the tanagers. It is one of 49 species in the genus Tangara.It is found in the eastern Andes drainages to the western Amazon Basin in southern Colombia eastern Ecuador and Peru and a region of northwestern Bolivia; for Brazil in southwestern-western Amazonas state and Acre.Its natural habitat is subtropical or tropical moist lowland forests.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "animal", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "You gotta wonder how some flics ever get made... this one decided to skip with the why among many other things and just wanders off beyond the moot. And yet you have a number of decent actors doing their best to pump some life into the story. The blue tint throughout the movie overshoots into 'yet again', which on its own would be depressing but here it's overkill. The idea that it's not a medical condition, not some house or gypsy or trinket curse but just something that for no apparent reason starts to happen to our protagonist and then to everyone else around her, just winds up being much like taking a big swig out of an empty mug. Some doppelgangers have super powers but others don't or don't know they do? It seems they're just as clueless as we are. It's a poor man's rip-off of \"Invasion of the Body-Snatchers\" with Keifer Sutherland's \"Mirror\" and \"The Sixth Sense\", were you to seriously botch those three together.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: DECISION RULE [ANY of: package value is between 7 and 17, including both endpoints | [ALL of: entry value is greater than 50 | member value is greater than 7]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 597. The package value is 8. The entry value is 51. The member value is 7.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,and(1,2))"}
{"request": {"context": "absorbing character study by andré turpin .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "California (/ˌkælɪˈfɔːrnjə, -niə/ ( listen) KAL-i-FORN-yə, -FOR-nee-ə) is a state in the Pacific Region of the United States. With 39.3 million residents, California is the most populous state in the United States and the third most extensive by area. The state capital is Sacramento. The Greater Los Angeles Area and the San Francisco Bay Area are the nation's second- and fifth-most populous urban regions, with 18.7 million and 8.8 million residents respectively. Los Angeles is California's most populous city, and the country's second largest after New York City. California also has the nation's most populous county, Los Angeles County; its largest county by area, San Bernardino County; and its fifth most densely populated county, San Francisco.", "question": "is california a part of the united states?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when it is not the case that (if (package value is less than 50), use (request value is at least 39); otherwise use (the elapsed days from entry start date to entry end date are at most 15)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry start date is 2027-04-21. The routing reference is 213. The package value is 60. The entry end date is 2027-05-07. The request value is 39.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(if(0,1,2))"}
{"request": {"context": "Midnight Cowboy is not for everybody. It's raw, painful, and realistic but very entertaining. The lead actors Jon Voight and Dustin Hoffman who would go on to become Oscar winning actors deliver amazing performances. Voight as the Texas hustler, Joe Buck, who migrates from small town Texas to New York City to become a hustler. He does not apologize for his chosen profession but it is not that easy. The New York City women like the rich lady played by Georgeann Johnson and Cass played by Oscar nominated Sylvia Miles are different than Texas women. Sadly, Buck is trying to escape from his past life in Texas. He was raised by his grandmother, Sally Buck, played by the wonderful actress Ruth White who died in 1969 from cancer. The locations in New York City are wonderful to watch as is the relationship between Fatso played by Hoffman and Buck's characters evolve into a moving male to male friendship. The men are struggling to survive the New York City life by not playing by the rules like getting a real job. As the film evolves, Buck's past comes to the surface and it's haunting but not clear. The film is not for children but compared to today's films and television programming, Midnight Cowboy might be more tame. I can't forget a", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: A case is approved when when review value is between 11 and 21, including both endpoints the requirement is that the condition \"either account value is less than 25, or else shipment signer is the same person as shipment designated approver\" fails, and when it is not the requirement is that the condition \"package value is at most 23\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The shipment designated approver is Mira. The routing reference is 612. The package value is 23. The shipment signer is Kofi. The account value is 24. The review value is 21.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,not(or(1,2)),not(3))"}
{"request": {"context": "I thought the food would be good here as the frozen hot chocolate is delicious, but I was wrong. My burger was overcooked and was slightly burnt! The sweet potato fries were average. My friend's pizza was the most expensive cardboard she's ever had and the pancakes were undercooked. It also appeared that our server did not know what she was doing. The only good thing was the fried chicken strips. I will only come back for the frozen hot chocolate. Don't waste your money here for food and only come for dessert!!!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "I thought the food would be good here as the frozen hot chocolate is delicious, but I was wrong. My burger was overcooked and was slightly burnt! The sweet potato fries were average. My friend's pizza was the most expensive cardboard she's ever had and the pancakes were undercooked. It also appeared that our server did not know what she was doing. The only good thing was the fried chicken strips. I will only come back for the frozen hot chocolate. Don't waste your money here for food and only come for dessert!!!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Bareback riding is a form of horseback riding without a saddle. It requires skill, balance, and coordination, as the rider does not have any equipment to compensate for errors of balance or skill.", "question": "do you need a saddle to ride a horse?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "Following the unsuccessful debut solo album, Supernova, Lopes started working on her second solo album. On April 25, 2002, before the album was completed, Lopes died in a car crash while filming a documentary in Honduras, which would later be released as The Last Days of Left Eye in 2007 on VH1. After taking a hiatus following Lopes' death, Watkins, Thomas, and Austin decided that they would complete the remainder of their fourth album, to be called 3D, which also featured production from Rodney Jerkins, The Neptunes, Raphael Saadiq, Missy Elliott and Timbaland. A decision was also made that TLC would retire after the release and promotion of 3D, rather than replace Lopes and continue as a group. However, ultimately they chose to carry on as a duo. Lopes appears vocally on 3D as she had already completed her vocals for two songs for the new album. Another two other songs featured her vocals from previously unreleased songs. Several of the songs eulogized Lopes. 3D was released on November 12, 2002.", "question": "question: are all the members of tlc still alive?\nfocus: Consider the whole message.", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "Just too many incidents of violence. The film goes from one scene to another, and in nearly every one violence erupts. Now I am not one who is shocked by violence, and to me a film without a fight in it has something missing. But, please, not one after another. My reaction was not shock or horror, it was: \"Here we go again.\" There is some semblance of a story in between the scenes of violence, but two thirds of the way through the film I had switched off completely, and couldn't wait for the end. If this is the best the film makers can do, they should find something else to do with their miserable lives, like making shoes or delivering mail.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(account signer is the same person as account designated approver); (it is not the case that (both these conditions hold: [(shipment value is greater than 48); (review value is greater than 28)]))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment value is 59. The review value is 28. The routing reference is 184. The account designated approver is Aiko. The account signer is Tomas.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,not(and(1,2)))"}
{"request": {"context": "The preambles to both the proposed and final rules reflect HUD's finding that the rule will not have a significant impact on the environment.", "question": "Hypothesis: \"The rule will also not have a significant impact on the empire state building.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "Deadly Strike also known in the West as Breakout from Oppression is a 1978 Hong Kong martial arts action film directed Lau Kar-leung and his adopted brother Gordon Liu the latter helming his directorial debut. Film stars Liu alongside Dean Shek.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "film", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "- role: customer\n  content: Siemens to sell mobile phone biz to China - report Chinese distributor lined up as 'potential buyer'", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "- role: customer\n  content: Siemens to sell mobile phone biz to China - report Chinese distributor lined up as 'potential buyer'", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: Siemens to sell mobile phone biz to China - report Chinese distributor lined up as 'potential buyer'", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Oil steady after failure to hit \\$50 Oil prices have steadied after sliding sharply from record highs late last week but the tightening siege on rebels holed up in a shrine in Iraq has heightened uncertainty in the market.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "Oil steady after failure to hit \\$50 Oil prices have steadied after sliding sharply from record highs late last week but the tightening siege on rebels holed up in a shrine in Iraq has heightened uncertainty in the market.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Oil steady after failure to hit \\$50 Oil prices have steadied after sliding sharply from record highs late last week but the tightening siege on rebels holed up in a shrine in Iraq has heightened uncertainty in the market.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Lah Frakh Melleh Namdaran (Persian: له فراخ مله نامداران‎ also Romanized as Lah Frākh Melleh Nāmdārān) is a village in Chin Rural District Ludab District Boyer-Ahmad County Kohgiluyeh and Boyer-Ahmad Province Iran. At the 2006 census its population was 73 in 16 families.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "village", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "This place doesn't have over 2000 reviews and 4 stars for no reason. Service is great. Food is great and location is wonderful. \\n\\nVery romantic and enjoyable dinner was had here!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "This place doesn't have over 2000 reviews and 4 stars for no reason. Service is great. Food is great and location is wonderful. \\n\\nVery romantic and enjoyable dinner was had here!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "What Stephen Sondheim ballad ends : `` Well , maybe next year.. . '' ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: DECISION RULE [ALL of: case value is greater than 31 | [NOT: [IF member value is at least 21 THEN the elapsed days from request start date to request end date are at most 55 ELSE review value is less than 59]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 41. The member value is 20. The request start date is 2027-04-02. The routing reference is 448. The review value is 58. The request end date is 2027-05-28.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,not(if(1,2,3)))"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ALL of: request signer is the same person as request designated approver | account value is greater than 29] | [ANY of: package value is greater than 36 | review signer is the same person as review designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 272. The review designated approver is Sana. The account value is 28. The package value is 36. The request signer is Elin. The review signer is Elin. The request designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),or(2,3))"}
{"request": {"context": "- role: customer\n  content: I thought surely the other reviews saying the product cut their baby were an exception. But bo, this product is defective and cut my baby and caused him to bleed. They shouldn’t be allowed to sell these. Save yourself and your baby and just buy the windi.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "ticket:\n  channel: email\n  body: while it may not add up to the sum of its parts , holofcener 's film offers just enough insight to keep it from being simpleminded , and the ensemble cast is engaging enough to keep you from shifting in your chair too often .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "document: I heard nothing. Mrs. Vandemeyer gazed round her fearfully.", "question": "Hypothesis: \"She said \"I heard nothing\" and looked around.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when ((case value is between 9 and 19, including both endpoints) OR (request value is at most 12)) holds and the exception (review value is at most 42) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The case value is 20. The routing reference is 277. The review value is 43. The request value is 22.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),2)"}
{"request": {"context": "Kansas, Tennessee top NCAA basketball polls Kansas often grabs a spot in the upper half of the Associated Press men's preseason college basketball poll, but the school hasn't been No. 1 since Wilt Chamberlain suited up for the Jayhawks in the 1950s.", "question": "question: What is the topic of this article?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Kansas, Tennessee top NCAA basketball polls Kansas often grabs a spot in the upper half of the Associated Press men's preseason college basketball poll, but the school hasn't been No. 1 since Wilt Chamberlain suited up for the Jayhawks in the 1950s.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Kansas, Tennessee top NCAA basketball polls Kansas often grabs a spot in the upper half of the Associated Press men's preseason college basketball poll, but the school hasn't been No. 1 since Wilt Chamberlain suited up for the Jayhawks in the 1950s.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: This sequel is a total rehash of the first film. A completely pointless movie. It basically just took every single sceanrio of the first film and they redid it in Omen IV except with a female antichrist this time. It even ends the same way as the first one! The music is too busy and interfering, and because its pretty much a copy of Omen I, it's extremely predictable. It's not a horrible movie, it's not terribly made, there is much worse movies out there, this just had absolutely no point in being made. The Omen remake from 2006 is much worse, even more pointless than this, so I guess it has that. If you someone pointed a gun to your head and you had to choose to watch this sequel or the 2006 reamke, I guess I'd choose this.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Orders of fewer than 10 units are charged the list price. Orders of 10 to 49 units receive the volume discount. Orders of 50 units or more receive the wholesale rate.\ncase: Kofi placed a business order for a wireless keyboard. The order is for 82 units. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "ticket:\n  channel: email\n  body: Works very well and shuts tight. I used it everyday to make sure I drink enough water but it cracked and leaked in my bad after about 10 months of use. I would have liked to continue using it for much longer, but it satisfied my needs. I may buy another one in the future.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "amazon"}
{"request": {"context": "What does Lloyd 's Lutine Bell announce ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "Cayoosh Pass (1275 m / 4183 ft) is a mountain pass in the Lillooet Ranges of the Pacific Ranges of the southern Coast Mountains in British Columbia Canada.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "naturalplace", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "there is a subversive element to this disney cartoon , providing unexpected fizzability .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the night-shift roster.\ncase: Lena applied to join the night-shift roster. The application form was complete and signed. Lena is 24 years old.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the apprenticeship.\ncase: The application form was complete and signed. Ravi applied to join the apprenticeship. Ravi is 20 years old.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "Yes! This one actually works! I have purchased 3 other wireless car mounts that said they worked with a case, none of them did. I have a few Seneo wireless chargers at home and work that I love, so I thought I’d try this one out. I’m so happy I finally have a wireless charger for my car that works with my case!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: A case is approved when shipment verified is yes, and also review verified is yes, except when entry value is at most 57, and also package value is at most 51; any other case is denied. Routing references play no part in the decision.\ncase: The package value is 61. The entry value is 58. The shipment verified is yes. The review verified is yes. The routing reference is 499.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(and(0,1),and(2,3))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Poolville Independent School District is a public school district based in the community of Poolville Texas (USA).Located in Parker County a small portion of the district extends into Wise County.In 2009 the school district was rated academically acceptable by the Texas Education Agency.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(account signer is the same person as account designated approver); (it is not the case that (both these conditions hold: [(shipment value is greater than 48); (review value is greater than 28)]))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment value is 59. The review value is 29. The routing reference is 723. The account designated approver is Aiko. The account signer is Tomas.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,not(and(1,2)))"}
{"request": {"context": "The Ancient Magus' Bride (魔法使いの嫁, Mahō Tsukai no Yome) is a Japanese fantasy shōnen manga series written and illustrated by Kore Yamazaki. The series is published in Japan by Mag Garden in its Monthly Comic Garden magazine. The manga is licensed in North America by Seven Seas Entertainment. Wit Studio produced a three-part prequel OVA of the manga and produced an anime television series adaptation that aired from October 2017 to March 2018.", "question": "question: is there going to be a season 2 of the ancient magus bride?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "The richest Americans: Gates No. 1 Microsoft Chairman Bill Gates must be bored of leading the pack on the Forbes list of richest Americans. Gates continues to be the richest man in America for the 11 th consecutive year.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "The richest Americans: Gates No. 1 Microsoft Chairman Bill Gates must be bored of leading the pack on the Forbes list of richest Americans. Gates continues to be the richest man in America for the 11 th consecutive year.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "The richest Americans: Gates No. 1 Microsoft Chairman Bill Gates must be bored of leading the pack on the Forbes list of richest Americans. Gates continues to be the richest man in America for the 11 th consecutive year.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Orders of fewer than 5 units are charged the list price. Orders of 5 to 19 units receive the volume discount. Orders of 20 units or more receive the wholesale rate.\ncase: The order is for 30 units. Noah placed a business order for a coffee grinder. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "policy: DECISION RULE [[ANY of: entry value is between 39 and 49, including both endpoints | case value is at most 31] UNLESS [ANY of: the elapsed days from shipment start date to shipment end date are at most 33 | package signer is the same person as package designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment start date is 2027-02-15. The shipment end date is 2027-03-21. The package signer is Noah. The entry value is 50. The routing reference is 510. The case value is 31. The package designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(or(0,1),or(2,3))"}
{"request": {"context": "How many years did Sleeping Beauty sleep ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "PalmSource unveils  #39;smart phone #39; operating system In an effort to ring up more sales, PalmSource Inc., of Sunnyvale, known for making the operating system for the Palm handheld computer, on Tuesday introduced a new operating system for so-called  quot;smart phones.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "PalmSource unveils  #39;smart phone #39; operating system In an effort to ring up more sales, PalmSource Inc., of Sunnyvale, known for making the operating system for the Palm handheld computer, on Tuesday introduced a new operating system for so-called  quot;smart phones.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "PalmSource unveils  #39;smart phone #39; operating system In an effort to ring up more sales, PalmSource Inc., of Sunnyvale, known for making the operating system for the Palm handheld computer, on Tuesday introduced a new operating system for so-called  quot;smart phones.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ANY of: case value is at most 31 | review verified is yes] | [ANY of: entry signer is the same person as entry designated approver | member verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review verified is yes. The member verified is yes. The routing reference is 975. The entry designated approver is Elin. The case value is 42. The entry signer is Noah.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(or(0,1),or(2,3))"}
{"request": {"context": "Crunch! Oof! Well, That's Physics In Nebraska, a scientist adds the vectors and delta-V's of Newtonian physics to the usual X's and O's of football.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: what: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: what: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Crunch! Oof! Well, That's Physics In Nebraska, a scientist adds the vectors and delta-V's of Newtonian physics to the usual X's and O's of football.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Crunch! Oof! Well, That's Physics In Nebraska, a scientist adds the vectors and delta-V's of Newtonian physics to the usual X's and O's of football.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Applicants must be at least 21 years old to be eligible for the night-shift roster.\ncase: Kofi is 18 years old. The application form was complete and signed. Kofi applied to join the night-shift roster.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "- role: customer\n  content: if you are willing to do this , then you so crazy !", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Warranty claims made within 365 days of purchase are covered in full. Claims made after that but within 1095 days are covered at half cost. Later claims are not covered.\ncase: A warranty claim for it was filed on August 21, 2028. The claim describes a defect in the stitching. Jonas purchased a backpack on June 3, 2026.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the rental agreement.\ncase: The application form was complete and signed. Omar is 20 years old. Omar applied to join the rental agreement.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "What is the S&P 500 used for ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: A case is approved when shipment signer is the same person as shipment designated approver, except when request value is at most 54, and also review value is at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 34. The shipment signer is Aiko. The request value is 64. The routing reference is 177. The shipment designated approver is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(1,2))"}
{"request": {"context": "policy: A case is approved when case verified is yes, except when request value is greater than 47, and also account signer is the same person as account designated approver; any other case is denied. Routing references play no part in the decision.\ncase: The case verified is no. The account signer is Kofi. The request value is 46. The account designated approver is Kofi. The routing reference is 386.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,unless(1,2))"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ALL of: request signer is the same person as request designated approver | account value is greater than 29] | [ANY of: package value is greater than 36 | review signer is the same person as review designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 966. The review designated approver is Sana. The account value is 28. The package value is 36. The request signer is Elin. The review signer is Sana. The request designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(and(0,1),or(2,3))"}
{"request": {"context": "policy: Support responses within 4 hours meet the service level. Responses after 4 but within 24 hours are a minor breach. Anything slower is a major breach.\ncase: Omar opened a priority ticket about an export error. The ticket was routed through the phone queue. The first response arrived 22 hours after the ticket was opened.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: The order is for 379 units. Ravi placed a business order for a rain jacket. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "I love Zonie Baskets! I ordered something for my dad for father's day and had it shipped to Wisconsin. The owner also threw in a free gift!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "I love Zonie Baskets! I ordered something for my dad for father's day and had it shipped to Wisconsin. The owner also threw in a free gift!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: A case is approved when when the condition \"the elapsed days from package start date to package end date are at most 58\" fails the requirement is that shipment value is less than 46, and when it is not the requirement is that request value is greater than 32; any other case is denied. Routing references play no part in the decision.\ncase: The shipment value is 45. The package end date is 2027-03-17. The package start date is 2027-01-17. The routing reference is 489. The request value is 31.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(not(0),1,2)"}
{"request": {"context": "Panama qualified for the FIFA World Cup for the first time for the 2018 tournament in Russia and scored their first goal of the World Cup against England, although they lost the match 6-1.", "question": "has panama ever scored a goal in world cup?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Jason does the best nails I have ever had. It is a beautiful salon and the prices are really reasonable. I hesitated to write a review--only because I don't want the rest of the world to find this gem. It's hard enough to get an appointment now!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "Jason does the best nails I have ever had. It is a beautiful salon and the prices are really reasonable. I hesitated to write a review--only because I don't want the rest of the world to find this gem. It's hard enough to get an appointment now!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ANY of: case value is at most 31 | review verified is yes] | [ANY of: entry signer is the same person as entry designated approver | member verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review verified is no. The member verified is yes. The routing reference is 975. The entry designated approver is Elin. The case value is 42. The entry signer is Noah.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(or(0,1),or(2,3))"}
{"request": {"context": "document: This film can not even be said to be bad for it is sadly, just painfully mediocre. Lacking any real wit or imagination, a thin plot is stretched to the absolute limit and the `jokes' (which are predictable and threadbare) are spun out to such inordinate length that boredom and yawns quickly overtake the viewer. Another notch to mark the sad decline of John Waters and a reminder that what `shocked' or amused us 30 years ago doesn't work quite the same way now. We've seen it all before, and it no longer breaks any taboos because they have long since evaporated. A major miss.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Support responses within 1 hours meet the service level. Responses after 1 but within 8 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 8 hours after the ticket was opened. Lena opened a priority ticket about a login failure. The ticket was routed through the phone queue.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "policy: Approve exactly when (if (request value is between 55 and 65, including both endpoints), use (package value is greater than 46); otherwise use (case value is at most 23)) OR (NOT (the elapsed days from review start date to review end date are at most 51)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 65. The routing reference is 877. The package value is 45. The case value is 34. The review start date is 2027-02-15. The review end date is 2027-04-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(if(0,1,2),not(3))"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(account signer is the same person as account designated approver); (it is not the case that (both these conditions hold: [(shipment value is greater than 48); (review value is greater than 28)]))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment value is 59. The review value is 28. The routing reference is 723. The account designated approver is Aiko. The account signer is Tomas.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,not(and(1,2)))"}
{"request": {"context": "- role: customer\n  content: This little place saved my life every morning I was in Vegas! With no coffee makers in the rooms at the Palazzo- this was a quick jaunt down the elevators to the 2nd level to get my daily caffeine fix.\\n\\nA couple of large, black ,drip coffees will cost you about $8 bucks (Yowza! Wish I was more ambitious as I'd head to McDonald's) but still better than the hour and half room service time quote and heaven only knows that that would have cost!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "- role: customer\n  content: This little place saved my life every morning I was in Vegas! With no coffee makers in the rooms at the Palazzo- this was a quick jaunt down the elevators to the 2nd level to get my daily caffeine fix.\\n\\nA couple of large, black ,drip coffees will cost you about $8 bucks (Yowza! Wish I was more ambitious as I'd head to McDonald's) but still better than the hour and half room service time quote and heaven only knows that that would have cost!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Turkish hostages freed, firm says Iraqi militants have freed 10 employees of a Turkish construction company taken captive in Iraq in last month, the company #39;s general manager says.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "Turkish hostages freed, firm says Iraqi militants have freed 10 employees of a Turkish construction company taken captive in Iraq in last month, the company #39;s general manager says.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Turkish hostages freed, firm says Iraqi militants have freed 10 employees of a Turkish construction company taken captive in Iraq in last month, the company #39;s general manager says.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "- role: customer\n  content: I love this powder. I use it in my smoothies and my husbands protein shakes. It doesn’t leave a taste in it at all. Will be ordering more when I run out.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "They sent a completely different pair of pants. They sent gray jeans. I wanted to return them but it would cost me over 30 dollars to send them back! Terrible!!!", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when ((package verified is yes) AND (entry value is at least 36)) OR ((request value is at most 41) AND (shipment value is between 50 and 60, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 472. The entry value is 35. The request value is 51. The package verified is yes. The shipment value is 51.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),and(2,3))"}
{"request": {"context": "high and the goal cannot be achieved on time, decision makers assess trade-offs between new and existing components to reduce the risks to a more manageable level.", "question": "Hypothesis: \"Risks should not be high because they must achieve their goals.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(the elapsed days from account start date to account end date are at most 7); (at least one of these conditions hold: [(at least one of these conditions hold: [(case value is equal to 24); (shipment value is between 22 and 32, including both endpoints)]); (entry value is greater than 35)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account start date is 2027-04-19. The routing reference is 105. The case value is 25. The entry value is 34. The shipment value is 21. The account end date is 2027-05-06.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(1,or(2,3)))"}
{"request": {"context": "policy: A case is approved when when the elapsed days from entry start date to entry end date are at most 44 the requirement is that the elapsed days from case start date to case end date are at most 41, and also shipment value is equal to 59, and when it is not the requirement is that the condition \"request value is between 57 and 67, including both endpoints\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry end date is 2027-05-02. The entry start date is 2027-03-19. The case start date is 2027-05-26. The request value is 67. The shipment value is 69. The routing reference is 900. The case end date is 2027-07-05.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:if(0,and(1,2),not(3))"}
{"request": {"context": "From here, you could walk to the lovely Hama Rikyu Detached Palace Garden.", "question": "Hypothesis: \"You could reach the Hama Rikyu Detached Palace Garden if you walk from here. \" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: The order is for 681 units. Lena placed a business order for a wireless keyboard. Delivery is to a warehouse.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "policy: Approve exactly when ((entry value is equal to 27) AND (NOT (shipment value is at most 46))) AND ((package verified is yes) AND (NOT (member verified is yes))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 558. The member verified is no. The package verified is yes. The shipment value is 47. The entry value is 37.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,not(1)),and(2,not(3)))"}
{"request": {"context": "- role: customer\n  content: although fairly involving as far as it goes , the film does n't end up having much that is fresh to say about growing up catholic or , really , anything .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when (review value is at most 37) holds and the exception (at least one of these conditions hold: [(case value is less than 42); (account value is less than 40)]) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 393. The review value is 37. The case value is 43. The account value is 41.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,or(1,2))"}
{"request": {"context": "My friends and I have always loved the Happy Hour in the bar but I was disappointed last night to find out the menu has changed and our favorite beef and horseradish sandwich is no longer offered. My friend and I sat at the bar and we were surprised when the bartenders ignored us after they had served our beers. We finally had to ask for some food and although we both tried different items from the new menu, they did not compare.\\n\\nMaybe it was because it was Friday night, but it was extremely noisy with people standing in a group or leaning across us to order at the bar. I have sat there many times before and have not had that experience but it seemed this group of young people were very loud.\\n\\nSorry to say I will have to mark this off my list of places to take my friends and to stop extolling its virtues to the people I meet.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "My friends and I have always loved the Happy Hour in the bar but I was disappointed last night to find out the menu has changed and our favorite beef and horseradish sandwich is no longer offered. My friend and I sat at the bar and we were surprised when the bartenders ignored us after they had served our beers. We finally had to ask for some food and although we both tried different items from the new menu, they did not compare.\\n\\nMaybe it was because it was Friday night, but it was extremely noisy with people standing in a group or leaning across us to order at the bar. I have sat there many times before and have not had that experience but it seemed this group of young people were very loud.\\n\\nSorry to say I will have to mark this off my list of places to take my friends and to stop extolling its virtues to the people I meet.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Approve exactly when NOT ((request value is at least 60) holds and the exception (review value is between 5 and 15, including both endpoints) does not hold). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 70. The review value is 4. The routing reference is 938.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(unless(0,1))"}
{"request": {"context": "Wrong item for the truck", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: Aiko, the account owner, submitted an expense claim. Receipts were attached for every line item. The claim total is $145.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "auto_approved", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "policy: Support responses within 8 hours meet the service level. Responses after 8 but within 48 hours are a minor breach. Anything slower is a major breach.\ncase: The first response arrived 41 hours after the ticket was opened. Lena opened a priority ticket about a login failure. The ticket was routed through the chat queue.", "question": "How does this response time rate against the service level?", "criteria": "", "options": [{"id": "0", "text": "Met"}, {"id": "1", "text": "Minor breach"}, {"id": "2", "text": "Major breach"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_sla_response"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Newsweek rehashes last week's Mars cover story with a long feature about the ingenuity and spiritedness of the NASA team.", "question": "Hypothesis: \"Newsweek has never written a story about NASA.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "document: Valley wins point on spyware Technology companies in Silicon Valley have beaten back what they saw as an overreaching attempt in the California legislature to control computer spyware and hope that a new more palatable", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "document: Valley wins point on spyware Technology companies in Silicon Valley have beaten back what they saw as an overreaching attempt in the California legislature to control computer spyware and hope that a new more palatable", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: Valley wins point on spyware Technology companies in Silicon Valley have beaten back what they saw as an overreaching attempt in the California legislature to control computer spyware and hope that a new more palatable", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [(entry value is at most 6); (it is not the case that (the elapsed days from case start date to case end date are at most 21))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 432. The case start date is 2027-05-03. The case end date is 2027-06-03. The entry value is 16.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,not(1))"}
{"request": {"context": "Small college town coed OD's? (Why do we care?) Acting sheriff investigates the incident. (Why do we care?) The interviews show us the comatose subject (Kirshner) as different as the opinions of the subjects being interviewed. (Why do we care?) Result? A mess of flashbacks in this mess of a movie featuring a handful of one-hit wonders and B-flick divas which begs the question...Why do we care?", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(the elapsed days from account start date to account end date are at most 7); (at least one of these conditions hold: [(at least one of these conditions hold: [(case value is equal to 24); (shipment value is between 22 and 32, including both endpoints)]); (entry value is greater than 35)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account start date is 2027-04-19. The routing reference is 555. The case value is 25. The entry value is 34. The shipment value is 22. The account end date is 2027-05-06.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,or(1,or(2,3)))"}
{"request": {"context": "policy: DECISION RULE [[ANY of: package signer is the same person as package designated approver | case verified is yes] UNLESS [ALL of: the elapsed days from review start date to review end date are at most 7 | account value is between 11 and 21, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review end date is 2027-05-20. The review start date is 2027-05-13. The package designated approver is Elin. The account value is 21. The routing reference is 426. The case verified is no. The package signer is Elin.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),and(2,3))"}
{"request": {"context": "policy: DECISION RULE [ANY of: package value is between 7 and 17, including both endpoints | [ALL of: entry value is greater than 50 | member value is greater than 7]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 414. The package value is 8. The entry value is 51. The member value is 7.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,and(1,2))"}
{"request": {"context": "policy: Approve exactly when it is not the case that (at least one of these conditions hold: [(member verified is yes); (package signer is the same person as package designated approver)]). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member verified is no. The package designated approver is Aiko. The routing reference is 232. The package signer is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(or(0,1))"}
{"request": {"context": "What is D.B. Cooper known for ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "document: Seemed super watery compared to when I used it years ago. Didn't work nearly as well as I remember it did.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when (at least one of these conditions hold: [((case value is greater than 27) holds and the exception (the elapsed days from member start date to member end date are at most 8) does not hold); (it is not the case that (shipment value is at most 41))]) holds and the exception (entry value is greater than 40) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member start date is 2027-05-23. The case value is 37. The shipment value is 51. The member end date is 2027-05-30. The entry value is 41. The routing reference is 242.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(not(0),unless(1,2)),3)"}
{"request": {"context": "`` abandon '' will leave you wanting to abandon the theater .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Approve exactly when both these conditions hold: [((request value is greater than 43) holds and the exception (account value is at most 52) does not hold); (at least one of these conditions hold: [(shipment value is at most 57); (member value is between 39 and 49, including both endpoints)])]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 53. The shipment value is 58. The routing reference is 251. The member value is 39. The request value is 44.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(or(0,1),unless(2,3))"}
{"request": {"context": "This is a unique and bold film. It's energetic, with highly developed characters. Very good performances. I love directors who are not afraid to ask the audience to think... Bret Carr dares to look inward, ask questions, and dig underneath the surface. By the nature of the film, it practically demands that the viewer look inward as well. It can take some courage to do this. I can see how some close-minded people might find this quite disturbing. However, I found it thought-provoking, much more than just entertainment. Bret Carr is truly an original filmmaker, with vision. I am curious to see what he will do next. Watch him.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: DECISION RULE [ANY of: [entry value is at least 20 UNLESS request value is at most 36] | [package value is between 15 and 25, including both endpoints UNLESS case signer is the same person as case designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case signer is Aiko. The routing reference is 600. The entry value is 21. The case designated approver is Aiko. The package value is 15. The request value is 35.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(unless(0,1),unless(2,3))"}
{"request": {"context": "document: Five Deadly Venoms is not as bloody and violent as Story Of Ricky or Super Ninjas, but it features some of the best hand-to-hand fight sequences in Hong Kong film history. Director Chang Cheh creates what is considered by many to be his masterpiece. This movie launched the careers of the five men who play the venoms. Meng Lo plays yet another bad ass. He would go on to be in Super Ninjas. Kuo Chui who is Philip Kwok would go on to Story Of Ricky and Hard Boiled. Any chop socky fan can apperciate this. But I still think it ain't as good as Super Ninjas (also made by Chang Cheh). But all chop socky is good and this is one of the very best.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 45 days of the purchase date.\ncase: Sana bought a pair of running shoes on February 16, 2026. The return request was submitted on February 27, 2026. The order was paid by card and shipped to Sana's home address.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "policy: Returns are accepted only if the return request is submitted within 14 days of the purchase date.\ncase: The return request was submitted on May 28, 2026. The order was paid by card and shipped to Tomas's home address. Tomas bought a wireless keyboard on May 5, 2026.", "question": "Is this return request within the policy window?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_return_window"}
{"request": {"context": "This restaurant may always have a huge lineup but I can promise you that it's worth the wait.\\nI have been to Kazu a few times now and tried quite a few dishes as well as sharing what my girlfriend orders and I can honestly say that I have yet to taste something that I wasn't totally in love with.\\nKazu is a must for any true foodie in the city or visiting the city.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "This restaurant may always have a huge lineup but I can promise you that it's worth the wait.\\nI have been to Kazu a few times now and tried quite a few dishes as well as sharing what my girlfriend orders and I can honestly say that I have yet to taste something that I wasn't totally in love with.\\nKazu is a must for any true foodie in the city or visiting the city.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "that's not too old", "question": "Hypothesis: \"That isn't too old.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: A case is approved when either case signer is the same person as case designated approver, or else either package value is at most 53, and also review value is equal to 53, or else the condition \"entry value is greater than 38\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 54. The entry value is 49. The case signer is Sana. The case designated approver is Aiko. The package value is 64. The routing reference is 425.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(0,or(and(1,2),not(3)))"}
{"request": {"context": "policy: DECISION RULE [ALL of: case value is greater than 31 | [NOT: [IF member value is at least 21 THEN the elapsed days from request start date to request end date are at most 55 ELSE review value is less than 59]]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 41. The member value is 20. The request start date is 2027-04-02. The routing reference is 448. The review value is 58. The request end date is 2027-05-28.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,not(if(1,2,3)))"}
{"request": {"context": "policy: Invoices paid within 10 days after the due date incur no fee. Payments between 11 and 60 days late incur a 2% fee. Payments later than 60 days incur a 10% fee and a hold on the account.\ncase: Elin's invoice for $450 fell due last quarter. Payment was received 71 days after the due date. The payment was made by bank transfer.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "policy: Approve exactly when (both these conditions hold: [(account value is equal to 32); (the elapsed days from case start date to case end date are at most 7)]) holds and the exception ((request signer is the same person as request designated approver) holds and the exception (member value is at least 38) does not hold) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 376. The request designated approver is Noah. The account value is 31. The member value is 39. The request signer is Ravi. The case start date is 2027-04-07. The case end date is 2027-04-14.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),unless(2,3))"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Thailand Decides Against Bird Flu Vaccine (AP) AP - Thailand, faced with a second outbreak of bird flu, said Wednesday its poultry would not be inoculated against the disease because vaccines offer no guarantee of safety.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Thailand Decides Against Bird Flu Vaccine (AP) AP - Thailand, faced with a second outbreak of bird flu, said Wednesday its poultry would not be inoculated against the disease because vaccines offer no guarantee of safety.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Thailand Decides Against Bird Flu Vaccine (AP) AP - Thailand, faced with a second outbreak of bird flu, said Wednesday its poultry would not be inoculated against the disease because vaccines offer no guarantee of safety.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Silers Bald is a mountain in the western Great Smoky Mountains located in the Southeastern United States. Its proximity to Clingmans Dome and its location alongthe Appalachian Trail make it a popular hiking destination. Silers Bald is located on the crest of the Smokies with Thunderhead Mountain to the west and Clingmans Dome to the east.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "naturalplace", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Approve exactly when ((package verified is yes) AND (entry value is at least 36)) OR ((request value is at most 41) AND (shipment value is between 50 and 60, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 591. The entry value is 35. The request value is 51. The package verified is yes. The shipment value is 51.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),and(2,3))"}
{"request": {"context": "policy: A case is approved when case verified is yes, except when request value is greater than 47, and also account signer is the same person as account designated approver; any other case is denied. Routing references play no part in the decision.\ncase: The case verified is no. The account signer is Kofi. The request value is 46. The account designated approver is Kofi. The routing reference is 865.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,unless(1,2))"}
{"request": {"context": "How many member states are in the UN ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Approve exactly when (at least one of these conditions hold: [((case value is greater than 27) holds and the exception (the elapsed days from member start date to member end date are at most 8) does not hold); (it is not the case that (shipment value is at most 41))]) holds and the exception (entry value is greater than 40) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The member start date is 2027-05-23. The case value is 37. The shipment value is 51. The member end date is 2027-05-30. The entry value is 41. The routing reference is 242.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(not(0),unless(1,2)),3)"}
{"request": {"context": "document: It's Chinese alright. \\n\\nI meant its alright Chinese. \\n\\nBut must admit, it wasn't greasy. Nice, clean flavors. \\n\\nWe had the \\n\\n(Entree) Honey Walnut Shrimp - Nothing exciting. Better than Panda, not as good as Chinese Restaurant in Chinatown (PhoKim Long is good). Came with good portioned white rice.\\n\\n(Entree) Tofu Veggies - Good. Came with good portioned white rice. \\n\\n(Side) Veggie Roll (2) - Was ok. The norm, bean sprout, carrots. \\n\\nFor the price we paid for different entree. We will try the Family Value which is a dollar more, but had more of the sides.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "2", "ordinal": true, "source": "yelp"}
{"request": {"context": "document: It's Chinese alright. \\n\\nI meant its alright Chinese. \\n\\nBut must admit, it wasn't greasy. Nice, clean flavors. \\n\\nWe had the \\n\\n(Entree) Honey Walnut Shrimp - Nothing exciting. Better than Panda, not as good as Chinese Restaurant in Chinatown (PhoKim Long is good). Came with good portioned white rice.\\n\\n(Entree) Tofu Veggies - Good. Came with good portioned white rice. \\n\\n(Side) Veggie Roll (2) - Was ok. The norm, bean sprout, carrots. \\n\\nFor the price we paid for different entree. We will try the Family Value which is a dollar more, but had more of the sides.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: A case is approved when shipment signer is the same person as shipment designated approver, except when request value is at most 54, and also review value is at most 34; any other case is denied. Routing references play no part in the decision.\ncase: The review value is 34. The shipment signer is Aiko. The request value is 64. The routing reference is 521. The shipment designated approver is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(1,2))"}
{"request": {"context": "document: The look is really nice. Not a lot of arch support. these shoes are made for narrower feet.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: A case is approved when the condition \"member value is equal to 46, and also account value is equal to 12\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 46. The account value is 13. The routing reference is 719.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(and(0,1))"}
{"request": {"context": "policy: DECISION RULE [ANY of: request value is between 26 and 36, including both endpoints | [IF package value is equal to 49 THEN entry verified is yes ELSE review value is greater than 24]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 971. The review value is 23. The entry verified is no. The package value is 50. The request value is 26.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,if(1,2,3))"}
{"request": {"context": "policy: DECISION RULE [[NOT: entry value is between 12 and 22, including both endpoints] UNLESS [request value is at most 42 UNLESS review value is between 55 and 65, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 891. The review value is 56. The request value is 41. The entry value is 13.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(not(0),unless(1,2))"}
{"request": {"context": "I haven't read through all the comments, but at least one poster mentioned that the 30 minute version might possibly be abridged. I'm curious about that myself because the later parts of the film just didn't make much sense to me even when I rewatched them. 30 minutes seems really short for a movie in 1917 also. \"Poor Little Rich Girl\" which was Tourneur's next film is 65 minutes long and \"Pride of the Clan\" which was his previous feature was 84 minutes long. So I'm relieved to see that I wasn't crazy, there must be part of this film missing and that's why the resolution didn't make much sense. It's hard to review or comment on a movie that you're only able to see half of... but I would recommend this film anyway because of the really fascinating view that it provides us of the insides of an East Coast movie studio of the time. It's the earliest film I've personally seen that's based on the movie industry itself. The main character is a movie star played by Robert Warwick, who was later a mainstay Hollywood character actor and appeared in almost all of Preston Sturges' films. He plays a western actor perhaps vaguely modeled on William S. Hart, who Warwick does resemble somewhat. After the really fascinating sequences", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when ((case value is between 9 and 19, including both endpoints) OR (request value is at most 12)) holds and the exception (review value is at most 42) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The case value is 20. The routing reference is 277. The review value is 43. The request value is 11.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(or(0,1),2)"}
{"request": {"context": "- role: customer\n  content: Tillandsia fresnilloensis is a species of the genus Tillandsia. This species is endemic to Mexico.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "plant", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ALL of: package value is greater than 16 | request value is at least 27] | [case value is at most 47 UNLESS shipment value is at least 38]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 47. The package value is 17. The shipment value is 37. The request value is 28. The routing reference is 358.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(and(0,1),unless(2,3))"}
{"request": {"context": "policy: Warranty claims made within 365 days of purchase are covered in full. Claims made after that but within 1095 days are covered at half cost. Later claims are not covered.\ncase: The claim describes a defect in the stitching. Jonas purchased a rain jacket on May 28, 2026. A warranty claim for it was filed on July 25, 2029.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ALL of: request signer is the same person as request designated approver | account value is greater than 29] | [ANY of: package value is greater than 36 | review signer is the same person as review designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 966. The review designated approver is Sana. The account value is 28. The package value is 36. The request signer is Elin. The review signer is Elin. The request designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),or(2,3))"}
{"request": {"context": "-lrb- shyamalan -rrb- turns the goose-pimple genre on its empty head and fills it with spirit , purpose and emotionally bruised characters who add up to more than body count .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "3", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Expense claims of $250 or less are approved automatically. Claims above $250 require director sign-off.\ncase: Kofi, the billing manager, submitted an expense claim. The claim total is $386. Receipts were attached for every line item.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "ticket:\n  channel: web form\n  body: What are people on here talking about? I must have seen a different movie than you guys. I was so bored that I walked out of the theater. What is up with this stuff. I only laughed once and not even that hard. Whoever greenlit this movie should be taken out into the street and beaten. I usually always find something I like in a movie, but this is crazy. \"Slackers\" makes \"Not Another Teen Movie\" look like a classic. It ranks up there with such trash as \"Body Shots\", \"Con Air\", and \"Conspiracy Theory\" as one of the worst of all time. AVOID AT ALL COSTS! I thought the studio would be handing out money to people as they left the theater as a sign of their apologies, but to my dismay that wasn't happening. Looks like I will never see that money again or the two hours.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "it 's another retelling of alexandre dumas ' classic .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "Of the 71 words in this list, 67 are nouns, and most would generally be considered loanwords; the only modern-English words that contain Q not followed by U and are not borrowed from another language are qiana, qwerty, and tranq. However, all of the loanwords on this list are considered to be naturalised in English according to at least one major dictionary (see References), often because they refer to concepts or societal roles that do not have an accurate equivalent in English. For words to appear here, they must appear in their own entry in a dictionary; words which occur only as part of a longer phrase are not included.", "question": "is there a word that has a q without a u?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "Among the birds, you will see the flash of a kingfisher or hornbill.", "question": "question: Hypothesis: \"You will see a flash of a kingfisher or hornbill among the birds.\" How does it relate to the premise?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral"}, {"id": "contradiction", "text": "contradiction: what: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "How many quarters equal a pound ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "document: What is the origin of the expression `` four-letter word '' ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: Mira's invoice for $980 fell due last quarter. The payment was made by card. Payment was received 88 days after the due date.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "policy: Orders of fewer than 10 units are charged the list price. Orders of 10 to 49 units receive the volume discount. Orders of 50 units or more receive the wholesale rate.\ncase: Sana placed a business order for a coffee grinder. Delivery is to a storefront. The order is for 3 units.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "document: How do you clean an LCD monitor screen ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "description", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(account signer is the same person as account designated approver); (it is not the case that (both these conditions hold: [(shipment value is greater than 48); (review value is greater than 28)]))]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The shipment value is 59. The review value is 28. The routing reference is 723. The account designated approver is Aiko. The account signer is Tomas.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,not(and(1,2)))"}
{"request": {"context": "policy: DECISION RULE [[ANY of: package signer is the same person as package designated approver | case verified is yes] UNLESS [ALL of: the elapsed days from review start date to review end date are at most 7 | account value is between 11 and 21, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review end date is 2027-05-21. The review start date is 2027-05-13. The package designated approver is Elin. The account value is 21. The routing reference is 426. The case verified is no. The package signer is Elin.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(or(0,1),and(2,3))"}
{"request": {"context": "document: Outside Canada, the rights to the A&W brand are owned by Dr Pepper Snapple Group, which in turn licenses the brand to the similarly named U.S.-based restaurant chain; A&W products are distributed via various U.S. bottlers. A&W Food Services of Canada, which is independent of both DPSG and the U.S. chain, is responsible for the restaurants and the marketing of root beer products in that country, with retail products bottled and distributed by The Coca-Cola Company. The U.S. variant is also sold as an import drink in Southeast Asia and Italy (where A&W has restaurants), as well as Australia, the United Kingdom, and Chile, among other countries.", "question": "is a&w root beer a 7 up product?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "ticket:\n  channel: email\n  body: Disturbingly long waits. While waiting in line, the EZ Car Rental right next store, went through 5 people while I stood there. Actually was probably more, but it only dawned on me to start counting when i got really bored. I probably waited in line for a good 20 mins, maybe closer to 25. If time is something you value, this may not be your best choice.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "ticket:\n  channel: email\n  body: Disturbingly long waits. While waiting in line, the EZ Car Rental right next store, went through 5 people while I stood there. Actually was probably more, but it only dawned on me to start counting when i got really bored. I probably waited in line for a good 20 mins, maybe closer to 25. If time is something you value, this may not be your best choice.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "In 2006, Sikma was voted as one of the 100 Legends of the IHSA Boys Basketball Tournament, a group of former players and coaches in honor of the 100 anniversary of the IHSA boys basketball tournament. On June 27, 2017, Sikma was inducted into the Small College Basketball Hall of Fame as part of their second class . Inducted alongside Sikma were Zelmo Beaty, Walt Frazier, Bob Love, Elmore Smith, Jim Spivey, Rico Swanson, George Tinsley, and Al Tucker.", "question": "is jack sikma in the hall of fame?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "I saw this movie the other night. I can't even begin to express how much this movie sucked. The writing, the voice acting, even the claymation. Terrible, Terrible, Terrible. It's like watching 24 hours of C-Span for the sake of comedy. It just doesn't work. It literally falls flat at about every spot possible. Also, the movie's animation is very poor quality. I know that this is an movie made by one person, but to think that he could make 97 minutes worth of crap, maybe he could at least make 1 second worth of funny. This show may take the cake for being the worst film of all time. Yikes. It really was that bad. If you're looking for a movie that will make you laugh, steer clear from this abomination. My advice: Don't even buy it, or look it up for that matter. Your brain will than you.", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: DECISION RULE [ANY of: [ANY of: account verified is yes | entry verified is yes] | [ANY of: case value is at least 38 | shipment verified is yes]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment verified is no. The account verified is yes. The routing reference is 879. The case value is 37. The entry verified is no.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(or(0,1),or(2,3))"}
{"request": {"context": "policy: A case is approved when the condition \"either either case signer is the same person as case designated approver, or else package value is equal to 43, or else the elapsed days from entry start date to entry end date are at most 56\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The case signer is Aiko. The case designated approver is Noah. The entry start date is 2027-07-13. The package value is 44. The entry end date is 2027-09-08. The routing reference is 378.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(or(0,or(1,2)))"}
{"request": {"context": "document: what a concept , what an idea , what a thrill ride .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "sst5"}
{"request": {"context": "Name a film in which Jude Law acted .", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "entity", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: A case is approved when member value is between 30 and 40, including both endpoints, except when account verified is yes, and also package signer is the same person as package designated approver, and also review value is less than 14; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 29. The review value is 15. The account verified is yes. The routing reference is 421. The package signer is Aiko. The package designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(0,and(1,and(2,3)))"}
{"request": {"context": "I didn't like it , but it was the affordable for me option stick to the original if you can afford 100.00 Plus. I could have DIY'd this and it would have been better more to my expectations easier to handle ...", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Expense claims of $1,000 or less are approved automatically. Claims above $1,000 require director sign-off.\ncase: The claim total is $1,855. Receipts were attached for every line item. Tomas, the account owner, submitted an expense claim.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the night-shift roster.\ncase: Lena is 33 years old. Lena applied to join the night-shift roster. The application form was complete and signed.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "The subject of that memo was Bosnia, and while Ambassador Albright emerged as the administration's leading hawk on a number of fronts, arguing for intervention in Haiti and in Rwanda, she made the case for how the United States should take the lead--by threatening and, if necessary, using force--with special passion about the American role in the Balkans.", "question": "Hypothesis: \"The memo was about Bosnia.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "entailment", "ordinal": false, "source": "mnli"}
{"request": {"context": "My husband and I were married at Har Zion in 2003, when the membership was flourishing. Today, their membership is really low, and not many young families are members. It would be great if younger families at least attended services. Babysitting is available the first and third Shabbat morning. My son and I attend fairly regularly. They are the only synagogue in the area that offers daily minyans twice daily.. More people need to be present for that though. Please help by attending minyan at least once a month. Hopefully the incoming new rabbi will help bring in younger, newer members to help grow this synagogue.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "3", "ordinal": true, "source": "yelp"}
{"request": {"context": "My husband and I were married at Har Zion in 2003, when the membership was flourishing. Today, their membership is really low, and not many young families are members. It would be great if younger families at least attended services. Babysitting is available the first and third Shabbat morning. My son and I attend fairly regularly. They are the only synagogue in the area that offers daily minyans twice daily.. More people need to be present for that though. Please help by attending minyan at least once a month. Hopefully the incoming new rabbi will help bring in younger, newer members to help grow this synagogue.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "mushes the college-friends genre -lrb- the big chill -rrb- together with the contrivances and overwrought emotion of soap operas .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "How long does it take the typical American to eat 23 quarts of ice cream ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: DECISION RULE [ANY of: [entry value is at least 20 UNLESS request value is at most 36] | [package value is between 15 and 25, including both endpoints UNLESS case signer is the same person as case designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case signer is Aiko. The routing reference is 121. The entry value is 21. The case designated approver is Elin. The package value is 15. The request value is 35.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(unless(0,1),unless(2,3))"}
{"request": {"context": "policy: DECISION RULE [ANY of: [review value is at most 33 UNLESS member value is at least 10] | [ALL of: case verified is yes | package value is less than 26]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review value is 33. The case verified is no. The package value is 25. The routing reference is 212. The member value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(and(0,1),unless(2,3))"}
{"request": {"context": "she'd be big i don't know  what are you saying Holly oh my little girl is in the second grade and told me to tell you that shboom means good-bye in the Israeli language", "question": "Hypothesis: \"My daughter says shboom means goodbye in Isreali, but I don't believe her.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "neutral", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when (if (request value is between 55 and 65, including both endpoints), use (package value is greater than 46); otherwise use (case value is at most 23)) OR (NOT (the elapsed days from review start date to review end date are at most 51)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 65. The routing reference is 877. The package value is 45. The case value is 34. The review start date is 2027-02-15. The review end date is 2027-04-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:or(if(0,1,2),not(3))"}
{"request": {"context": "policy: Orders of fewer than 10 units are charged the list price. Orders of 10 to 49 units receive the volume discount. Orders of 50 units or more receive the wholesale rate.\ncase: Sana placed a business order for a coffee grinder. Delivery is to a storefront. The order is for 53 units.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "policy: Approve exactly when NOT (((package value is less than 26) OR (the elapsed days from entry start date to entry end date are at most 57)) AND (request value is between 18 and 28, including both endpoints)). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 27. The entry end date is 2027-07-03. The request value is 19. The routing reference is 410. The entry start date is 2027-05-07.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(and(0,or(1,2)))"}
{"request": {"context": "Bishop Unified School District is a school district in Bishop California. It represents the amalgamation of two school districts:Bishop Joint Union High School District which operatedBishop Union High School Palisade Glacier High SchoolBishop Union Elementary School District which operated Elm Street Elementary School Pine Street Elementary SchoolHome Street Middle Schools", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "educationalinstitution", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: DECISION RULE [[ANY of: package signer is the same person as package designated approver | case verified is yes] UNLESS [ALL of: the elapsed days from review start date to review end date are at most 7 | account value is between 11 and 21, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The review end date is 2027-05-20. The review start date is 2027-05-13. The package designated approver is Elin. The account value is 21. The routing reference is 426. The case verified is no. The package signer is Elin.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),and(2,3))"}
{"request": {"context": "policy: Customers may order at most 5 units of any single item per order. Orders up to double the limit are held for review; larger orders are cancelled.\ncase: Tomas placed an order for a coffee grinder. The order quantity is 7. Delivery was requested to a residential address.", "question": "What happens to this order?", "criteria": "", "options": [{"id": "within_limit", "text": "within_limit: Processed normally"}, {"id": "slightly_over", "text": "slightly_over: Held for review"}, {"id": "far_over", "text": "far_over: Cancelled"}]}, "expected_option_id": "slightly_over", "ordinal": false, "source": "contrastive_quantity_limit"}
{"request": {"context": "Honestly this place is overhyped. I got Andy's sage fried chicken with mashed potatoes,eggs,biscuit, and bacon. My food had no flavor whatsoever. I don't like my food too salty but this was just too bland. The only thing I probably finished was the bacon and biscuit lol. The wait service is excellent the food is just not up my alley.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "0", "ordinal": true, "source": "yelp"}
{"request": {"context": "Honestly this place is overhyped. I got Andy's sage fried chicken with mashed potatoes,eggs,biscuit, and bacon. My food had no flavor whatsoever. I don't like my food too salty but this was just too bland. The only thing I probably finished was the bacon and biscuit lol. The wait service is excellent the food is just not up my alley.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "only for young children , if them .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "2", "ordinal": true, "source": "sst5"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Really great food st a great price. The shakes were so good they weren't too sweet just the way I like them. The burgers were delicious. I recommend the onion rings over the fries. The cheese on the cheese fries wasn't that great. My second visit I asked for a neopolitan shake which wasn't on the menu and they made it no problem. I would say 4.5 stars if I could.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "ticket:\n  channel: web form\n  body: Really great food st a great price. The shakes were so good they weren't too sweet just the way I like them. The burgers were delicious. I recommend the onion rings over the fries. The cheese on the cheese fries wasn't that great. My second visit I asked for a neopolitan shake which wasn't on the menu and they made it no problem. I would say 4.5 stars if I could.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: China Building Next-Generation Space Rocket BEIJING (AFP) -- China's space program is expecting government approval this year to build a new and more powerful rocket that will serve as the nation's vehicle to explore the moon, state media said reported.       According to Luan Enjie, director-in-chief of China's lunar exploration program, the new-generation carrier rocket will be developed over the next eight years, Xinhua news agency said...", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: web form\n  body: China Building Next-Generation Space Rocket BEIJING (AFP) -- China's space program is expecting government approval this year to build a new and more powerful rocket that will serve as the nation's vehicle to explore the moon, state media said reported.       According to Luan Enjie, director-in-chief of China's lunar exploration program, the new-generation carrier rocket will be developed over the next eight years, Xinhua news agency said...", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: web form\n  body: China Building Next-Generation Space Rocket BEIJING (AFP) -- China's space program is expecting government approval this year to build a new and more powerful rocket that will serve as the nation's vehicle to explore the moon, state media said reported.       According to Luan Enjie, director-in-chief of China's lunar exploration program, the new-generation carrier rocket will be developed over the next eight years, Xinhua news agency said...", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: email\n  body: What is the best place to live in the world , considering climate , civilization ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "location", "ordinal": false, "source": "trec"}
{"request": {"context": "In American football and Canadian football, a lateral pass or lateral (officially backward pass in American football and onside pass in Canadian football) occurs when the ball carrier throws the football to a teammate in a direction parallel to or away from the opponents' goal line. A lateral pass is distinguished from a forward pass, in which the ball is thrown forward, towards the opposition's end zone. In a lateral pass the ball is not advanced, but unlike a forward pass a lateral may be attempted from anywhere on the field by any player to any player at any time.", "question": "can you pass the ball forward in american football?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "document: What Scottish poet penned To a Mouse and To a Louse ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "It's Quiznos. It's mediocre food. It's conveniently downtown. Convenient mediocrity gets eaten.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "1", "ordinal": true, "source": "yelp"}
{"request": {"context": "It's Quiznos. It's mediocre food. It's conveniently downtown. Convenient mediocrity gets eaten.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "no", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "Frederica von Stade – Mahler Songs is a single-disc LP album of orchestral songs by Gustav Mahler sung by mezzo-soprano Frederica von Stade with the London Philharmonic Orchestra conducted by Andrew Davis. The album was released by CBS Masterworks in 1979 and includes Songs of a Wayfarer two songs from Des Knaben Wunderhorn and the Five Songs on Poems by Friedrich Rückert. These recordings are also now available for download in digital format.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "album", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "Nor did they like his astounding knack for amassing a vast personal fortune while managing, very efficiently, the affairs of state.", "question": "Hypothesis: \"The man shared his wealth with the country.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: what: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: what: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "What people make up half the Soviet Union 's population ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "human", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the rental agreement.\ncase: The application form was complete and signed. Aiko applied to join the rental agreement. Aiko is 20 years old.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "A sea of green tranquility in the heart of Scotland's capital, Princes St gardens is magical all year round.\\n\\nWith its glorious blooming flower beds, freshly cut grass and the central ornate, classical gold fountain (arguably its crowning glory), Prices St gardens offers an oasis of calm in the middle of Edinburgh's hustling high streets.\\n\\nThe gardens were erected in the Victorian era and still exude the air of wealth and showiness that encompassed the old grand tour culture. They are pretty, well maintained and thus play hosts to many Fringe and Festival events, not to mention the world famous Hogmanay stage. The gardens are especially stormed during these seasons, and performers are glorified in the gorgeous surroundings.\\n\\nDuring the winter the gardens host a winter wonderland which is absolutely magical. You can expect to see a busy marketplace selling food and candy canes, not to mention Icelandic reindeer! But even though the gardens are remarkable on these occasions, they still offer a wonderful space to lounge around on during the quieter months, not to mention hosting a cafe. They even have the permanent fixture of a carousel should you get carried away with yourself after a wee coffee!\\n\\nBeautiful all year round, Princes St gardens truly is a park for all seasons.", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "A sea of green tranquility in the heart of Scotland's capital, Princes St gardens is magical all year round.\\n\\nWith its glorious blooming flower beds, freshly cut grass and the central ornate, classical gold fountain (arguably its crowning glory), Prices St gardens offers an oasis of calm in the middle of Edinburgh's hustling high streets.\\n\\nThe gardens were erected in the Victorian era and still exude the air of wealth and showiness that encompassed the old grand tour culture. They are pretty, well maintained and thus play hosts to many Fringe and Festival events, not to mention the world famous Hogmanay stage. The gardens are especially stormed during these seasons, and performers are glorified in the gorgeous surroundings.\\n\\nDuring the winter the gardens host a winter wonderland which is absolutely magical. You can expect to see a busy marketplace selling food and candy canes, not to mention Icelandic reindeer! But even though the gardens are remarkable on these occasions, they still offer a wonderful space to lounge around on during the quieter months, not to mention hosting a cafe. They even have the permanent fixture of a carousel should you get carried away with yourself after a wee coffee!\\n\\nBeautiful all year round, Princes St gardens truly is a park for all seasons.", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "policy: Applicants must be at least 25 years old to be eligible for the night-shift roster.\ncase: Lena applied to join the night-shift roster. The application form was complete and signed. Lena is 42 years old.", "question": "Is the applicant eligible?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "contrastive_age_eligibility"}
{"request": {"context": "document: Mayflies (also known as Canadian soldiers in the United States, and as shadflies or fishflies in Canada and the upper Midwestern U.S.; also up-winged flies in the United Kingdom) are aquatic insects belonging to the order Ephemeroptera. This order is part of an ancient group of insects termed the Palaeoptera, which also contains dragonflies and damselflies. Over 3,000 species of mayfly are known worldwide, grouped into over 400 genera in 42 families.", "question": "are mayflies and fish flies the same thing?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: A case is approved when member value is between 30 and 40, including both endpoints, except when account verified is yes, and also package signer is the same person as package designated approver, and also review value is less than 14; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 31. The review value is 15. The account verified is yes. The routing reference is 928. The package signer is Aiko. The package designated approver is Kofi.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,and(1,and(2,3)))"}
{"request": {"context": "document: The Root Cellar is a children's historical novel by Janet Lunn that is set in the 1980s although much of the action takes place in the 1860s. It follows Rose Larkin an orphan who travels temporally back and forth between Ontario Canada of the 1980s and various settings of U.S. Civil War in the 1860s. This time travel is done through the root cellar of the title.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "writtenwork", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "ENGLISH CRICKETS NEW TV DEAL The England and Wales Cricket Board (ECB) has agreed a deal with pay-TV operator Sky Sports for all home Test matches, one-day internationals, Twenty20 and county action.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "ENGLISH CRICKETS NEW TV DEAL The England and Wales Cricket Board (ECB) has agreed a deal with pay-TV operator Sky Sports for all home Test matches, one-day internationals, Twenty20 and county action.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ENGLISH CRICKETS NEW TV DEAL The England and Wales Cricket Board (ECB) has agreed a deal with pay-TV operator Sky Sports for all home Test matches, one-day internationals, Twenty20 and county action.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "S&S is a Pittsburgh Legend! Not even losing an entire wall of the store can keep them down. It's true-Google it! \\n\\nThe have an awesome selection of cigars ( not the largest- that distinction belongs to Blooms on 12th street ), but their prices are the best around. I can't say they 'know their stuff' when it comes to cigars but face it- you're not going in there to ask the difference between a Macanudo and a Monticristo... The cigars are always fresh and they are always 'over-the-top' friendly. I buy here every time I need to stock up.\\n\\nAnd when it comes to candy & snacks- this place can't be beat. They have ALL you need to add to the kids lunches with their multi-packs and bundles...\\n\\nTheir 'Candy by the pound' store will make you think you're still in the 50's...\\n\\nCheck this place out, then write a review...\\n\\nKeep on, keepin on, S&S...!", "question": "How many stars did this reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: terrible experience"}, {"id": "1", "text": "2 stars: poor"}, {"id": "2", "text": "3 stars: average"}, {"id": "3", "text": "4 stars: good"}, {"id": "4", "text": "5 stars: excellent"}]}, "expected_option_id": "4", "ordinal": true, "source": "yelp"}
{"request": {"context": "S&S is a Pittsburgh Legend! Not even losing an entire wall of the store can keep them down. It's true-Google it! \\n\\nThe have an awesome selection of cigars ( not the largest- that distinction belongs to Blooms on 12th street ), but their prices are the best around. I can't say they 'know their stuff' when it comes to cigars but face it- you're not going in there to ask the difference between a Macanudo and a Monticristo... The cigars are always fresh and they are always 'over-the-top' friendly. I buy here every time I need to stock up.\\n\\nAnd when it comes to candy & snacks- this place can't be beat. They have ALL you need to add to the kids lunches with their multi-packs and bundles...\\n\\nTheir 'Candy by the pound' store will make you think you're still in the 50's...\\n\\nCheck this place out, then write a review...\\n\\nKeep on, keepin on, S&S...!", "question": "Would this reviewer recommend the business?", "criteria": "", "options": [{"id": "yes", "text": "Yes: Clearly positive overall"}, {"id": "no", "text": "No: Negative or mixed"}]}, "expected_option_id": "yes", "ordinal": false, "source": "yelp_yn"}
{"request": {"context": "document: Shayna Steele (born September 23 1975) is an American singer and songwriter and a Broadway stage actress. Her Broadway credits include Rent at the Nederlander Theatre the revival of Jesus Christ Superstar at the Foxwoods Theatre (formerly the Ford Center for the Performing Arts) and the original Broadway cast of Hairspray in which she played a member of the girl group The Dynamites.Steele sings background vocals for Bette Midler in The Showgirl Must Go On in Las Vegas.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: DECISION RULE [[NOT: entry value is between 12 and 22, including both endpoints] UNLESS [request value is at most 42 UNLESS review value is between 55 and 65, including both endpoints]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 891. The review value is 56. The request value is 41. The entry value is 13.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(not(0),unless(1,2))"}
{"request": {"context": "policy: Approve exactly when at least one of these conditions hold: [(it is not the case that (at least one of these conditions hold: [(package value is greater than 38); (the elapsed days from request start date to request end date are at most 54)])); (review value is between 45 and 55, including both endpoints)]. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 517. The package value is 49. The review value is 45. The request end date is 2027-07-18. The request start date is 2027-05-24.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:or(0,not(or(1,2)))"}
{"request": {"context": "policy: Invoices paid within 5 days after the due date incur no fee. Payments between 6 and 30 days late incur a 2% fee. Payments later than 30 days incur a 10% fee and a hold on the account.\ncase: Payment was received 74 days after the due date. The payment was made by cheque. Noah's invoice for $2,300 fell due last quarter.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "a big meal of cliches that the talented cast generally chokes on .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: DECISION RULE [[ANY of: entry value is between 39 and 49, including both endpoints | case value is at most 31] UNLESS [ANY of: the elapsed days from shipment start date to shipment end date are at most 33 | package signer is the same person as package designated approver]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The shipment start date is 2027-02-15. The shipment end date is 2027-03-21. The package signer is Sana. The entry value is 50. The routing reference is 510. The case value is 31. The package designated approver is Sana.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(or(0,1),or(2,3))"}
{"request": {"context": "document: As Most Off You Might off Seen Star Wars: Return Off The Jedi You May Knows Its A Good Movie But As You Might Have Seen On Video They M|might have a party At The end And They Just Probably End The Movie with the party with no a spirits or anything But on the original one (Live TV) When they are Partying But before i say more when Ben obi-wan dies in the Imperial Ship Or Death Star They Saw him Disappear And Yoda Dies From Either Old Age Or Internal Illness But because Luke killed Darth Vader (Real Name: Anakin Skywalker) When They All Are Partying At The end when Luke Or Someone Stops the Spirits Off Ben And Yoda Stands Starring At Him And Smiling While Another Spirit Appears Is its Darth Vader but not as A Sith As The Old Usual Selve off Him And Started Smiling with Ben And Yoda I reckon That made the movie ending a little bit interesting But the Producers or anyone should off made a spirit off Padme And Mace Windu And Other Jedis that got killed with Younglings Under There Arms in the back ground", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when (NOT (NOT (entry value is less than 45))) AND (((the elapsed days from request start date to request end date are at most 39) AND (package value is equal to 54)) OR (NOT (account value is at most 39))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 49. The package value is 55. The request end date is 2027-09-17. The entry value is 46. The routing reference is 745. The request start date is 2027-08-08.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(not(not(0)),or(and(1,2),not(3)))"}
{"request": {"context": "Colpodium is a genus of grass in the Poaceae family. It contains the following species: Colpodium chionogeiton Colpodium hedbergii", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "plant", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Orders of fewer than 100 units are charged the list price. Orders of 100 to 499 units receive the volume discount. Orders of 500 units or more receive the wholesale rate.\ncase: Mira placed a business order for a coffee grinder. Delivery is to a branch office. The order is for 54 units.", "question": "Which pricing tier applies?", "criteria": "", "options": [{"id": "0", "text": "List price"}, {"id": "1", "text": "Volume discount"}, {"id": "2", "text": "Wholesale rate"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_volume_discount"}
{"request": {"context": "document: Mary Cavendish laid her hand upon my arm.", "question": "Hypothesis: \"Mary didn't touch me.\" How does it relate to the premise?", "criteria": "", "options": [{"id": "entailment", "text": "entailment: The hypothesis follows from the premise"}, {"id": "neutral", "text": "neutral: The hypothesis may or may not be true given the premise"}, {"id": "contradiction", "text": "contradiction: The hypothesis contradicts the premise"}]}, "expected_option_id": "contradiction", "ordinal": false, "source": "mnli"}
{"request": {"context": "policy: Approve exactly when (both these conditions hold: [(account value is equal to 32); (the elapsed days from case start date to case end date are at most 7)]) holds and the exception ((request signer is the same person as request designated approver) holds and the exception (member value is at least 38) does not hold) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 794. The request designated approver is Noah. The account value is 32. The member value is 39. The request signer is Ravi. The case start date is 2027-04-07. The case end date is 2027-04-14.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(and(0,1),unless(2,3))"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Fed: High oil constrains spending Reports from Federal Reserve districts say economy is growing but energy prices have cut spending. WASHINGTON (Reuters) - The US economy kept on growing in September and early October, even as higher prices", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Fed: High oil constrains spending Reports from Federal Reserve districts say economy is growing but energy prices have cut spending. WASHINGTON (Reuters) - The US economy kept on growing in September and early October, even as higher prices", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Fed: High oil constrains spending Reports from Federal Reserve districts say economy is growing but energy prices have cut spending. WASHINGTON (Reuters) - The US economy kept on growing in September and early October, even as higher prices", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "I'm sure deep in the recesses of Jack Blacks mind the character of Nacho Libre is absolutely hilarious but no it isn't. You can tell ol Jacks having a whale of a time hammin it up playing a smarmy, slimy Mexican friar with dreams of becoming a wrestler but this movie is a total misfire in just about every single department. I just sat there through most of the movie thinking \"Is this supposed to be funny\" and \"This is the guy from Tenacious D right?\". The truth is this film has NOTHING to offer. AT ALL! It's a lousy script with crappy characters and really naff acting and direction. You'll watch endless moments where you think something funny is surely about to happen but it just doesn't. I was bored stupid about 10 minutes in but though it would surely pick up. It didn't. 90 minutes later I'd barely managed to stave off an aneurism it was that painful. It's like, remember years ago when you'd see anything with your fave actor in it, even some of their really early pap from before they were famous, and you'd be really embarrassed that said actor was actually in such a load of plop. Yeah it's like that. I've enjoyed some of Jack Black's earlier movies like Shallow Hall and I'm", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when (NOT (NOT (entry value is less than 45))) AND (((the elapsed days from request start date to request end date are at most 39) AND (package value is equal to 54)) OR (NOT (account value is at most 39))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The account value is 49. The package value is 55. The request end date is 2027-09-17. The entry value is 46. The routing reference is 745. The request start date is 2027-08-08.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(not(not(0)),or(and(1,2),not(3)))"}
{"request": {"context": "document: Laidback Luke (born Lucas Cornelis van Scheppingen; 22 October 1976) is a Filipino-Dutch DJ and producer. He was voted #46 on DJ Mag's Top 100 DJs list in 2008 before jumping to #27 in 2009 and jumping up 10 places in 2010.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "artist", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "policy: Approve exactly when if (member value is at least 48), use (both these conditions hold: [(it is not the case that (case value is equal to 38)); (it is not the case that (package value is between 52 and 62, including both endpoints))]); otherwise use (it is not the case that (it is not the case that (account value is between 24 and 34, including both endpoints))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The package value is 63. The account value is 24. The member value is 47. The routing reference is 553. The case value is 38.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,and(not(1),not(2)),not(not(3)))"}
{"request": {"context": "policy: Warranty claims made within 180 days of purchase are covered in full. Claims made after that but within 730 days are covered at half cost. Later claims are not covered.\ncase: Priya purchased a wireless keyboard on May 25, 2026. A warranty claim for it was filed on June 21, 2028. The claim describes a defect in the zipper.", "question": "How is this claim covered?", "criteria": "", "options": [{"id": "0", "text": "Covered in full"}, {"id": "1", "text": "Covered at half cost"}, {"id": "2", "text": "Not covered"}]}, "expected_option_id": "2", "ordinal": true, "source": "contrastive_warranty_claim"}
{"request": {"context": "Teen Newsbreak - Getting out the vote A nonpartisan national group is trying to get a record number of voters to the polls this year. By MARK ZALOUDEK. Scientists are unlocking some of the mysteries of the universe as planetary exploration forges ahead on several fronts.", "question": "question: What is the topic of this article?\nfocus: Pick the single best fit.", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "scitech", "ordinal": false, "source": "agnews"}
{"request": {"context": "Teen Newsbreak - Getting out the vote A nonpartisan national group is trying to get a record number of voters to the polls this year. By MARK ZALOUDEK. Scientists are unlocking some of the mysteries of the universe as planetary exploration forges ahead on several fronts.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Teen Newsbreak - Getting out the vote A nonpartisan national group is trying to get a record number of voters to the polls this year. By MARK ZALOUDEK. Scientists are unlocking some of the mysteries of the universe as planetary exploration forges ahead on several fronts.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "NFL memo reminds teams about security The NFL sent a memo to all its teams reminding them about the league #39;s commitment to security following last week #39;s brawl at an NBA game.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "NFL memo reminds teams about security The NFL sent a memo to all its teams reminding them about the league #39;s commitment to security following last week #39;s brawl at an NBA game.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "NFL memo reminds teams about security The NFL sent a memo to all its teams reminding them about the league #39;s commitment to security following last week #39;s brawl at an NBA game.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when (review value is at most 37) holds and the exception (at least one of these conditions hold: [(case value is less than 42); (account value is less than 40)]) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 393. The review value is 37. The case value is 43. The account value is 41.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(0,or(1,2))"}
{"request": {"context": "policy: Approve exactly when ((account signer is the same person as account designated approver) holds and the exception (review value is equal to 56) does not hold) holds and the exception (the elapsed days from entry start date to entry end date are at most 39) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The account designated approver is Aiko. The entry start date is 2027-06-16. The account signer is Aiko. The review value is 57. The entry end date is 2027-07-26. The routing reference is 817.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:unless(unless(0,1),2)"}
{"request": {"context": "policy: Approve exactly when it is not the case that (at least one of these conditions hold: [(member verified is yes); (package signer is the same person as package designated approver)]). Otherwise deny. The routing reference does not affect eligibility.\ncase: The member verified is no. The package designated approver is Aiko. The routing reference is 232. The package signer is Aiko.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(or(0,1))"}
{"request": {"context": "The Nintendo DS Lite is compatible with Game Boy Advance and regular DS games. The DS Lite has a DS slot on top and the Game Boy slot on bottom. It also has a microphone and dual screens.", "question": "question: can nintendo ds lite play gameboy advance games?\nfocus: Use only the information given.", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "yes", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: A case is approved when the condition \"member value is equal to 46, and also account value is equal to 12\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The member value is 46. The account value is 12. The routing reference is 238.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:not(and(0,1))"}
{"request": {"context": "document: Keeps my son busy. He loves digging and learning about each fossil he finds.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: Approve exactly when (entry value is less than 39) AND (shipment verified is yes). Otherwise deny. The routing reference does not affect eligibility.\ncase: The entry value is 38. The shipment verified is no. The routing reference is 512.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(0,1)"}
{"request": {"context": "policy: DECISION RULE [IF [ALL of: entry value is at most 17 | account verified is yes] THEN package value is at least 9 ELSE case signer is the same person as case designated approver] -> approve; otherwise reject. Ignore the routing reference.\ncase: The routing reference is 194. The case signer is Tomas. The entry value is 18. The account verified is no. The case designated approver is Tomas. The package value is 20.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(and(0,1),2,3)"}
{"request": {"context": "ticket:\n  channel: web form\n  body: i searched video store everywhere to find this movie, being the huge elvis fan that i am, and i found it to be a huge disappointment. kurt russel had most of the \"elvis moves\" down and the voice imitation was great, but the dubbed in singing voice of elvis just didnt work for me. the voice didnt always match up with russels mouth, and it was hard for me to get lost in the plot because it bothered me that it was noticeable. also, there were so many freaking discrepancies in the film, people who dont know much about elvis would probably think them to be facts. songs are sung by him earlier than he recorded them in real life, the time when he got his first guitar is wrong, im pretty sure his brother jesse garron was buried in an unmarked grave, not one with a huge headstone reading JESSE GARRON. i know it was just a tv movie, but they skipped over important events, like the come-back-special, and dragged some scenes out for way too long. if you want to see a good movie that shows elvis in his prime rent THATS THE WAY IT IS, or another elvis concert. hearing and seeing the real elvis preform is the only way to truly see his talent. (brilliant statement", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Asian parties hope to deepen exchanges: Senior CPC official All Asian political parties share the hope of further deepening their exchanges and cooperation in the 21st century, said Liu Yunshan, a member of the Political Bureau of the CPC Central Committee on Sunday.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "world", "ordinal": false, "source": "agnews"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Asian parties hope to deepen exchanges: Senior CPC official All Asian political parties share the hope of further deepening their exchanges and cooperation in the 21st century, said Liu Yunshan, a member of the Political Bureau of the CPC Central Committee on Sunday.", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "ticket:\n  channel: chat\n  body: Asian parties hope to deepen exchanges: Senior CPC official All Asian political parties share the hope of further deepening their exchanges and cooperation in the 21st century, said Liu Yunshan, a member of the Political Bureau of the CPC Central Committee on Sunday.", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "The Next Big Cheese Will the next person to fill Michael Eisner's shoes ask for glass slipper shoehorns or odor eaters?", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "The Next Big Cheese Will the next person to fill Michael Eisner's shoes ask for glass slipper shoehorns or odor eaters?", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "The Next Big Cheese Will the next person to fill Michael Eisner's shoes ask for glass slipper shoehorns or odor eaters?", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "one of the worst movies of the year .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: Mira's invoice for $980 fell due last quarter. The payment was made by card. Payment was received 1 days after the due date.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "0", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "document: Oracle makes \\$9.2 billion take-it-or-leave-it PeopleSoft offer The Oracle-PeopleSoft takeover battle may have finally entered its endgame, with Oracle Corp upping its bid and for the first time imposing a deadline for PeopleSoft Inc shareholders to accept it.", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world"}, {"id": "sports", "text": "sports: what: Sports: games, athletes, teams, results"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "business", "ordinal": false, "source": "agnews"}
{"request": {"context": "document: Oracle makes \\$9.2 billion take-it-or-leave-it PeopleSoft offer The Oracle-PeopleSoft takeover battle may have finally entered its endgame, with Oracle Corp upping its bid and for the first time imposing a deadline for PeopleSoft Inc shareholders to accept it.", "question": "Is this article about sports?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: Oracle makes \\$9.2 billion take-it-or-leave-it PeopleSoft offer The Oracle-PeopleSoft takeover battle may have finally entered its endgame, with Oracle Corp upping its bid and for the first time imposing a deadline for PeopleSoft Inc shareholders to accept it.", "question": "Is this article about business?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "yes", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "policy: Approve exactly when ((entry value is equal to 27) AND (NOT (shipment value is at most 46))) AND ((package verified is yes) AND (NOT (member verified is yes))). Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 364. The member verified is no. The package verified is yes. The shipment value is 47. The entry value is 27.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(and(0,not(1)),and(2,not(3)))"}
{"request": {"context": "policy: A case is approved when member value is equal to 8, and also shipment value is less than 13, and also the elapsed days from entry start date to entry end date are at most 44, and also case verified is yes; any other case is denied. Routing references play no part in the decision.\ncase: The routing reference is 382. The entry start date is 2027-02-04. The entry end date is 2027-03-20. The shipment value is 12. The member value is 8. The case verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:and(and(0,1),and(2,3))"}
{"request": {"context": "policy: DECISION RULE [ALL of: [ALL of: package value is greater than 16 | request value is at least 27] | [case value is at most 47 UNLESS shipment value is at least 38]] -> approve; otherwise reject. Ignore the routing reference.\ncase: The case value is 47. The package value is 17. The shipment value is 48. The request value is 28. The routing reference is 974.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:and(and(0,1),unless(2,3))"}
{"request": {"context": "The Princess and the Goblin (Hungarian: A hercegnő és a kobold) is a 1991 British-Hungarian-American animated musical fantasy film directed by József Gémes and written by Robin Lyons, an adaptation of George MacDonald's 1872 novel of the same name.", "question": "is the princess and the goblin a disney movie?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "policy: Approve exactly when ((review value is between 9 and 19, including both endpoints) AND (case value is greater than 57)) holds and the exception (the elapsed days from entry start date to entry end date are at most 18) does not hold. Otherwise deny. The routing reference does not affect eligibility.\ncase: The routing reference is 660. The entry start date is 2027-07-04. The entry end date is 2027-08-01. The case value is 56. The review value is 10.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(and(0,1),2)"}
{"request": {"context": "I don't like using the word \"awful\" to describe any work of the cinema for which a great deal of time, effort, talent and money is spent in its creation but Zefferelli's attempt to adapt Charlotte Brontë's novel 'Jane Eyre' is a total waste of time. The script is lacking in finesse and power, everything explained to the viewer in no uncertain terms, leaving little to the imagination. The lead actors are woefully miscast, clearly hired for their star names, and the musical score drippy and dull. Charlotte Gainsbourg and William Hurt have absolutely no chemistry with one another at all. She is like a wet noodle, worse even than Joan Fontaine, who at least was capable of some modicum of emotional involvement in what should be a story of frustrated passion. And William Hurt acts the entire film on one tone and that tone is flat and devoid of energy. Of course the limp and vapid script does not aid any of these otherwise fine actors in their efforts to bring any whiff of life to this flick. Joan Plowright's Mrs Fairfax is like some Disney creation who keeps popping up to sweeten scenes in which she would have been best left out. There is no mystery surrounding the story of Rochester's first wife. The role of the would-be", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}
{"request": {"context": "policy: Approve exactly when NOT ((request value is at least 60) holds and the exception (review value is between 5 and 15, including both endpoints) does not hold). Otherwise deny. The routing reference does not affect eligibility.\ncase: The request value is 70. The review value is 5. The routing reference is 392.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "accept", "text": "accept: The policy permits this case"}, {"id": "reject", "text": "reject: The policy does not permit this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:not(unless(0,1))"}
{"request": {"context": "policy: A case is approved when when the elapsed days from entry start date to entry end date are at most 44 the requirement is that the elapsed days from case start date to case end date are at most 41, and also shipment value is equal to 59, and when it is not the requirement is that the condition \"request value is between 57 and 67, including both endpoints\" fails; any other case is denied. Routing references play no part in the decision.\ncase: The entry end date is 2027-05-02. The entry start date is 2027-03-19. The case start date is 2027-05-26. The request value is 67. The shipment value is 59. The routing reference is 309. The case end date is 2027-07-05.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "accept", "ordinal": false, "source": "composition_rand:if(0,and(1,2),not(3))"}
{"request": {"context": "policy: Expense claims of $2,500 or less are approved automatically. Claims above $2,500 require director sign-off.\ncase: Receipts were attached for every line item. Ravi, the billing manager, submitted an expense claim. The claim total is $3,432.", "question": "How is this claim handled under the policy?", "criteria": "", "options": [{"id": "auto_approved", "text": "auto_approved: Approved without further review"}, {"id": "director_signoff", "text": "director_signoff: Requires director sign-off"}, {"id": "rejected", "text": "rejected: Rejected outright"}]}, "expected_option_id": "director_signoff", "ordinal": false, "source": "contrastive_spend_threshold"}
{"request": {"context": "Material is very very thin, and the shirt was way smaller than they pants. It looks awful. The shipping took a month just for me to be disappointed.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "policy: DECISION RULE [[case value is less than 29 UNLESS [ANY of: request value is greater than 49 | entry verified is yes]] UNLESS the elapsed days from account start date to account end date are at most 13] -> approve; otherwise reject. Ignore the routing reference.\ncase: The request value is 48. The routing reference is 271. The account end date is 2027-04-17. The account start date is 2027-03-25. The case value is 28. The entry verified is yes.", "question": "Apply the policy to this case.", "criteria": "", "options": [{"id": "reject", "text": "reject: The policy does not permit this case"}, {"id": "accept", "text": "accept: The policy permits this case"}]}, "expected_option_id": "reject", "ordinal": false, "source": "composition_rand:unless(unless(0,or(1,2)),3)"}
{"request": {"context": "ticket:\n  channel: web form\n  body: These did not fit me well- kept sliding off and never felt secure.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "1", "ordinal": true, "source": "amazon"}
{"request": {"context": "Shorthanded Clippers fight off Warriors LOS ANGELES (Ticker) -- Even without leading scorer Corey Maggette, the Los Angeles Clippers were able to complete a sweep. Elton Brand scored 25 points and Bobby Simmons added 22 to lift the Clippers to a", "question": "What is the topic of this article?", "criteria": "", "options": [{"id": "world", "text": "world: World news: politics, international affairs, conflicts"}, {"id": "sports", "text": "sports"}, {"id": "business", "text": "business: Business: companies, markets, economy, finance"}, {"id": "scitech", "text": "scitech: Science and technology: research, gadgets, software, space"}]}, "expected_option_id": "sports", "ordinal": false, "source": "agnews"}
{"request": {"context": "Shorthanded Clippers fight off Warriors LOS ANGELES (Ticker) -- Even without leading scorer Corey Maggette, the Los Angeles Clippers were able to complete a sweep. Elton Brand scored 25 points and Bobby Simmons added 22 to lift the Clippers to a", "question": "Is this article about science and technology?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "Shorthanded Clippers fight off Warriors LOS ANGELES (Ticker) -- Even without leading scorer Corey Maggette, the Los Angeles Clippers were able to complete a sweep. Elton Brand scored 25 points and Bobby Simmons added 22 to lift the Clippers to a", "question": "Is this article about world news?", "criteria": "", "options": [{"id": "yes", "text": "Yes"}, {"id": "no", "text": "No"}]}, "expected_option_id": "no", "ordinal": false, "source": "agnews_yn"}
{"request": {"context": "document: Difficult Loves (Italian: Gli amori difficili) is a 1970 short story collection by Italo Calvino. It concerns love and the difficulty of communication.Some published versions of the English translation by William Weaver omit a number of the stories and also include other Calvino stories about the Second World War and postwar period including those from The Crow Comes Last; some of these were translated by Archibald Colquhoun and Peggy Wright.", "question": "Which category does the subject of this encyclopedia text belong to?", "criteria": "", "options": [{"id": "company", "text": "company"}, {"id": "educationalinstitution", "text": "educationalinstitution"}, {"id": "artist", "text": "artist"}, {"id": "athlete", "text": "athlete"}, {"id": "officeholder", "text": "officeholder"}, {"id": "meanoftransportation", "text": "meanoftransportation"}, {"id": "building", "text": "building"}, {"id": "naturalplace", "text": "naturalplace"}, {"id": "village", "text": "village"}, {"id": "animal", "text": "animal"}, {"id": "plant", "text": "plant"}, {"id": "album", "text": "album"}, {"id": "film", "text": "film"}, {"id": "writtenwork", "text": "writtenwork"}]}, "expected_option_id": "writtenwork", "ordinal": false, "source": "dbpedia14"}
{"request": {"context": "ticket:\n  channel: email\n  body: FORGET CREDIBILITY You must not expect credibility with action movies where the superhero has to perform an endless string of unbelievable feats, being trodden upon in the process but recovering at lightning speed, and transforming innocuous gadgets in lethal weapons... especially when Renny Harlin is directing. \"CLIFFHANGER \" is no exception. But the movie has numerous assets : breathtaking scenery gorgeously photographed, stunning special and visual effects ( the first five minutes are gripping and give the tone of the film ), excellent musical score, welcome attempts at levity to relieve some of the tension, and a solid cast : two heroes ( Stallone, star and cowriter, has the lion's share of the footage, but the excellent Michael Rooker more than stands his ground ), a charming heroin ( Janine Turner ), and one of the most darstardy bunch of villains ever ( priceless John Lithgow and deceivingly feminine Caroline Goodall, but also Rex Linn - in a longer than usual part and who makes the most of it, Leon, Craig Fairbrass ) Good, solid entertainment then , if no credibility.As Roger Ebert wrote ( about another film )\"It's the kind of movie you can sit back and enjoy as long as you don't make the mistake of thinking too much.\"", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "yes", "ordinal": false, "source": "imdb"}
{"request": {"context": "A riveting true story that keeps you guessing as the characters involved come more and more to life. The author is a terrific story-teller as well as a brilliant attorney.", "question": "How many stars did this product reviewer give?", "criteria": "", "options": [{"id": "0", "text": "1 star: very negative"}, {"id": "1", "text": "2 stars: negative"}, {"id": "2", "text": "3 stars: mixed"}, {"id": "3", "text": "4 stars: positive"}, {"id": "4", "text": "5 stars: very positive"}]}, "expected_option_id": "4", "ordinal": true, "source": "amazon"}
{"request": {"context": "document: What is the per-capita income of Colombia , South America ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "policy: Invoices paid within 15 days after the due date incur no fee. Payments between 16 and 45 days late incur a 2% fee. Payments later than 45 days incur a 10% fee and a hold on the account.\ncase: Elin's invoice for $120 fell due last quarter. The payment was made by cheque. Payment was received 26 days after the due date.", "question": "Which fee tier applies?", "criteria": "", "options": [{"id": "0", "text": "No fee"}, {"id": "1", "text": "2% fee"}, {"id": "2", "text": "10% fee and account hold"}]}, "expected_option_id": "1", "ordinal": true, "source": "contrastive_late_fee"}
{"request": {"context": "When was the Bill of Rights ratified ?", "question": "What kind of answer does this question ask for?", "criteria": "", "options": [{"id": "abbreviation", "text": "abbreviation: Asks what an abbreviation stands for"}, {"id": "entity", "text": "entity: Asks about a thing, object, animal, product, or creative work"}, {"id": "description", "text": "description: Asks for a definition, description, reason, or manner"}, {"id": "human", "text": "human: Asks about a person, group, or organisation"}, {"id": "location", "text": "location: Asks about a place"}, {"id": "number", "text": "number: Asks for a number, date, count, or other numeric value"}]}, "expected_option_id": "number", "ordinal": false, "source": "trec"}
{"request": {"context": "Straight Outta Compton is a 2015 American biographical film directed by F. Gary Gray, depicting the career of gangsta rap group N.W.A. Titled after N.W.A's 1988 debut studio album, the film focuses on members Eazy-E, Ice Cube, and Dr. Dre, and the rise and fall of the group. Members of N.W.A were involved in making the film, including Ice Cube and Dr. Dre as producers, as was Eazy-E's widow, Tomica Woods-Wright, while MC Ren and DJ Yella served as creative consultants. Ice Cube is portrayed by his son, O'Shea Jackson Jr., with Corey Hawkins as Dr. Dre and Jason Mitchell as Eazy-E. Paul Giamatti also stars as N.W.A's manager Jerry Heller.", "question": "did eazy e's son play in straight outta compton?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The passage supports a yes answer"}, {"id": "no", "text": "No: The passage supports a no answer or does not say"}]}, "expected_option_id": "no", "ordinal": false, "source": "boolq"}
{"request": {"context": "below is well below expectations .", "question": "What is the sentiment of this review sentence?", "criteria": "", "options": [{"id": "0", "text": "very negative"}, {"id": "1", "text": "negative"}, {"id": "2", "text": "neutral"}, {"id": "3", "text": "positive"}, {"id": "4", "text": "very positive"}]}, "expected_option_id": "0", "ordinal": true, "source": "sst5"}
{"request": {"context": "WARNING:I advise anyone who has not seen the film yet to not read this comment. Although I haven't seen them all,The Hamiltons sure did deliver one lowsy piece of entertainment,which it did not entertain me at all!!!!I thought that in common with the semi-bad acting,stupid plot scheme,and the twist at the end of the movie,which was very retarded,this movie sucked!!!Okay,so supposively these are people who eat other people.Yeah......notice I said people not humans,not because they aren't human,or wait I think they are,OH NO WAIT,THEY DIDN'T TELL YOU!!!!!So okay,are these people cannibals,or are they imbreds,or what are THEY!!!!I mean,maybe they're just \"THINGS\" that came here to see what people taste like or,are they cannibals who have eaten people for a long time now,or maybe this movie was HHOORRIIBBLLEE!!WHICH IT WAS!!!So if you think The Hamiltons is good,I ask of you,why,why,WHY,WHY,why was it so awesome,because to me it was just flat out terrible!!!One big BOOOOOOO for The Hamiltons!!!Go see The Gravedancers,Tooth & Nail,or Borderland for a piece of entertainment!!!!!", "question": "Is this movie review positive?", "criteria": "", "options": [{"id": "yes", "text": "Yes: The reviewer liked the film overall"}, {"id": "no", "text": "No: The reviewer disliked the film overall"}]}, "expected_option_id": "no", "ordinal": false, "source": "imdb"}