complai / data /eval /eval_set.jsonl
aliceberthale's picture
Fix #1: retrieval-grounded confidence calibration (honest fit, LOO-CV) + 3-guard review gate + grounding floor; in-scope answers now read ~85%
6999dbb
Raw History Blame Contribute Delete
6.92 kB
{"id": "cbam-001", "question": "If I import 50 tonnes of steel into the EU in 2026, do I have CBAM obligations?", "gold": "Yes. Steel is a covered sector under CBAM (Regulation EU 2023/956). Importing 50 tonnes above the €150 de minimis threshold in 2026 requires you to be an Authorised CBAM Declarant, report embedded emissions, and surrender CBAM certificates by 31 May 2027.", "disposition": "required"}
{"id": "cbam-002", "question": "Is there a minimum tonnage below which CBAM does not apply to steel imports?", "gold": "No minimum tonnage exemption exists. CBAM applies to any commercial quantity above the €150 intrinsic value threshold. A 1-tonne shipment worth more than €150 is in scope.", "disposition": "factual"}
{"id": "cbam-003", "question": "When does the CBAM transitional period end?", "gold": "The transitional period ends on 31 December 2025. The definitive regime with certificate purchase obligations begins 1 January 2026.", "disposition": "factual"}
{"id": "cbam-004", "question": "Can I deduct a carbon price already paid in the exporting country from my CBAM certificates?", "gold": "Yes. Article 9 allows deduction of carbon prices effectively paid in the country of origin for embedded carbon, provided the mechanism is legally binding and covers equivalent scope to the EU ETS.", "disposition": "required"}
{"id": "cbam-005", "question": "Are imports from Norway subject to CBAM?", "gold": "No. Norway is listed in Annex III as a country integrated into the EU ETS and is therefore exempt from CBAM certificate obligations.", "disposition": "factual"}
{"id": "csrd-001", "question": "Does a company with 300 employees and €50M turnover fall under CSRD scope?", "gold": "Yes. The company meets two of the three large-company criteria (>250 employees and >€40M turnover) and is in scope under Phase 2. First reporting year is FY 2025, with the sustainability statement published in 2026.", "disposition": "required"}
{"id": "csrd-002", "question": "Is a micro-enterprise with 8 employees exempt from CSRD?", "gold": "Yes. Companies with fewer than 10 employees and turnover ≤ €900k or balance sheet ≤ €450k are explicitly excluded from CSRD obligations.", "disposition": "factual"}
{"id": "csrd-003", "question": "What does double materiality mean under CSRD?", "gold": "Double materiality requires assessing both impact materiality (how the company affects people and the environment) and financial materiality (how sustainability matters affect the company's finances). A topic is reportable if it is material from either perspective.", "disposition": "factual"}
{"id": "csrd-004", "question": "Is ESRS 2 mandatory for all in-scope CSRD companies?", "gold": "Yes. ESRS 2 (General Disclosures) is mandatory for all in-scope companies regardless of the outcome of the materiality assessment.", "disposition": "required"}
{"id": "csrd-005", "question": "When must a large non-EU parent company start CSRD reporting?", "gold": "Non-EU parent companies with net EU turnover exceeding €150 million and an EU subsidiary or qualifying branch must start reporting for financial year 2028, with the first report published in 2029.", "disposition": "required"}
{"id": "cbam-006", "question": "Does CBAM cover aluminium imports?", "gold": "Yes. Aluminium is one of the covered sectors under CBAM, alongside cement, iron and steel, fertilisers, electricity and hydrogen.", "disposition": "factual"}
{"id": "cbam-007", "question": "What is the deadline for surrendering CBAM certificates each year?", "gold": "CBAM certificates must be surrendered by 31 May of the year following the import year, in the annual CBAM declaration.", "disposition": "required"}
{"id": "cbam-008", "question": "Can a non-EU company act as the Authorised CBAM Declarant?", "gold": "No. Only an importer established in the EU (or an indirect customs representative established in the EU) can hold Authorised CBAM Declarant status.", "disposition": "prohibited"}
{"id": "cbam-009", "question": "Does CBAM cover imports of finished passenger cars?", "gold": "No. Passenger cars are not a covered CBAM good. CBAM covers cement, iron and steel, aluminium, fertilisers, electricity and hydrogen, not finished vehicles.", "disposition": "factual"}
{"id": "cbam-010", "question": "Does CBAM apply to imported textiles and clothing?", "gold": "No. Textiles and clothing are not covered CBAM goods.", "disposition": "factual"}
{"id": "csrd-006", "question": "Does CSRD require reporting of Scope 3 greenhouse gas emissions?", "gold": "Yes. Under ESRS E1, in-scope companies must disclose material Scope 1, 2 and 3 greenhouse gas emissions.", "disposition": "required"}
{"id": "csrd-007", "question": "Can the CSRD sustainability statement be published only on the company website?", "gold": "No. The sustainability statement must be included in a dedicated section of the management report in ESEF/XHTML format, not only on the website.", "disposition": "prohibited"}
{"id": "tax-001", "question": "What are the six environmental objectives of the EU Taxonomy?", "gold": "Climate change mitigation, climate change adaptation, sustainable use of water and marine resources, transition to a circular economy, pollution prevention and control, and protection of biodiversity and ecosystems.", "disposition": "factual"}
{"id": "tax-002", "question": "What four conditions must an activity meet to be Taxonomy-aligned?", "gold": "It must substantially contribute to at least one environmental objective, do no significant harm to the others, meet minimum social safeguards, and comply with the technical screening criteria.", "disposition": "factual"}
{"id": "oos-001", "question": "What is the capital of France?", "gold": "OUT_OF_SCOPE: Not an EU compliance question; the corpus does not address it and the system should abstain or route to human review.", "disposition": "needs_review"}
{"id": "oos-002", "question": "How do I bake sourdough bread?", "gold": "OUT_OF_SCOPE: Not a compliance question; the system should abstain.", "disposition": "needs_review"}
{"id": "oos-003", "question": "What import VAT do I pay on goods ordered from Spain?", "gold": "OUT_OF_SCOPE: VAT is outside CBAM/CSRD/Taxonomy scope; the system should abstain or route to human review.", "disposition": "needs_review"}
{"id": "hard-001", "question": "Does CBAM apply to steel screws embedded inside imported machinery?", "gold": "AMBIGUOUS: Steel components inside non-covered finished products are generally outside direct CBAM scope, but this is a complex classification question that warrants expert review.", "disposition": "needs_review"}
{"id": "hard-002", "question": "Is a gas-fired power plant Taxonomy-aligned?", "gold": "AMBIGUOUS: Under the Complementary Climate Delegated Act, certain gas activities can qualify transitionally under strict conditions; this needs case-by-case assessment.", "disposition": "needs_review"}