Part 3: retrain on legally-grounded data, document the more-data-is-not-better regression and the macro-F1 fix
dd318c1 verified Download test_v2.csv from Devseis/eu-ai-act-classifier-experiment: direct link, hf CLI and curl.
- Browser
- Download file 25.8 kB
-
https://huggingface.co/Devseis/eu-ai-act-classifier-experiment/resolve/main/test_v2.csv
- Command line
-
hf download hf://Devseis/eu-ai-act-classifier-experiment/test_v2.csv
-
curl -L -o test_v2.csv https://huggingface.co/Devseis/eu-ai-act-classifier-experiment/resolve/main/test_v2.csv
25.8 kB
| text,label | |
| "Proposed use case: an AI system designed to an AI tool verifying whether a submitted health-insurance claim matches the policy terms, after the policy is already in force.",minimal_risk | |
| "Software flagging a group of arriving passengers for a routine health check based on flight-level epidemiological data, applied equally to everyone on the flight.",minimal_risk | |
| "Our client wants to deploy an AI tool to a classroom camera system inferring students' emotional states (boredom, frustration) to flag them to teachers, with no safety justification.",prohibited | |
| "Our client wants to deploy an AI tool to an AI tool verifying whether a submitted health-insurance claim matches the policy terms, after the policy is already in force.",minimal_risk | |
| We are developing an AI system that will software that crawls public CCTV footage feeds to continuously expand a facial-recognition matching database.,prohibited | |
| "We're planning to use machine learning to a city-run app that combines school attendance, grocery purchases and social media activity into one score used to deny families public housing.",prohibited | |
| A startup is building a product that uses AI to software analysing biometric data from photos to guess a person's sexual orientation for a dating-safety product.,prohibited | |
| "Our client wants to deploy an AI tool to an AI system drafting judicial decisions by analysing case facts, submissions and applicable case law.",high_risk | |
| We are developing an AI system that will a dating-safety startup inferring users' sexual orientation from uploaded selfies without consent.,prohibited | |
| "We're planning to use machine learning to an AI speech-to-text tool transcribing court hearing audio into a written record, without any legal analysis.",minimal_risk | |
| Hospital triage software automatically classifying incoming emergency patients by urgency.,high_risk | |
| Software determining whether a student should be placed in a remedial or advanced programme based on an automated assessment.,high_risk | |
| "A startup is building a product that uses AI to a courier company's AI flagging shipping-label mistakes for an employee to fix, not evaluating the employee.",minimal_risk | |
| Our client wants to deploy an AI tool to software that crawls public CCTV footage feeds to continuously expand a facial-recognition matching database.,prohibited | |
| We're planning to use machine learning to a telecom company's automated phone line handling billing questions in natural conversation.,limited_risk | |
| "We are developing an AI system that will software assessing human-trafficking vulnerability from indicators like age, documentation status and travel patterns.",high_risk | |
| A startup is building a product that uses AI to a financial newsletter auto-generating market commentary articles published directly to subscribers.,limited_risk | |
| We are developing an AI system that will an AI concierge on a hotel's website chatting directly with guests about bookings.,limited_risk | |
| We are developing an AI system that will software assigning primary-school children to public schools based on home address and catchment-area rules.,high_risk | |
| We are developing an AI system that will software classifying students with special educational needs into appropriate support programmes.,high_risk | |
| A startup is building a product that uses AI to an AI system generating individualised suspect risk profiles from neighbourhood and registry data during a missing-child case.,high_risk | |
| Proposed use case: an AI system designed to hospital triage software automatically classifying incoming emergency patients by urgency.,high_risk | |
| "A startup is building a product that uses AI to a maritime AI system used solely to detect obstacles and avoid collisions, with no link to border enforcement.",minimal_risk | |
| Software setting a person's health-insurance premium based on an automated risk assessment of their lifestyle data.,high_risk | |
| We are developing an AI system that will a telecom company's automated phone line handling billing questions in natural conversation.,limited_risk | |
| "A startup is building a product that uses AI to a public square's live biometric camera network identifying any pedestrian in real time, used for general crime monitoring rather than one of the narrow permitted purposes.",prohibited | |
| Proposed use case: an AI system designed to software that searches social media and job boards to source and shortlist candidates against recruiter criteria.,high_risk | |
| We're planning to use machine learning to a prison system predicting an offender's reoffending likelihood from criminal record and in-facility behaviour.,high_risk | |
| "A startup is building a product that uses AI to a cybersecurity monitoring tool watching for anomalies in energy-grid network traffic, independent of physical controls.",minimal_risk | |
| "Call-centre software analysing agents' voice tone to infer stress or frustration levels purely for management review, unrelated to safety.",prohibited | |
| We're planning to use machine learning to a tool detecting duplicate rows in an imported spreadsheet.,minimal_risk | |
| "Our client wants to deploy an AI tool to a public square's live biometric camera network identifying any pedestrian in real time, used for general crime monitoring rather than one of the narrow permitted purposes.",prohibited | |
| "We are developing an AI system that will an AI speech-to-text tool transcribing court hearing audio into a written record, without any legal analysis.",minimal_risk | |
| Proposed use case: an AI system designed to software analysing biometric data from photos to guess a person's sexual orientation for a dating-safety product.,prohibited | |
| Our client wants to deploy an AI tool to a meeting-notes app summarising a call for the people who were already on it.,minimal_risk | |
| "Proposed use case: an AI system designed to a maritime AI system used solely to detect obstacles and avoid collisions, with no link to border enforcement.",minimal_risk | |
| A therapy app using AI-guided cognitive behavioural therapy techniques with a patient's explicit informed consent.,minimal_risk | |
| Proposed use case: an AI system designed to marketing software that exploits subconscious psychological triggers to get users to sign away data rights they would normally refuse.,prohibited | |
| We are developing an AI system that will a therapy app using AI-guided cognitive behavioural therapy techniques with a patient's explicit informed consent.,minimal_risk | |
| "We're planning to use machine learning to a wildfire-spread forecasting tool used for general disaster planning, not linked to any specific dispatch decision.",minimal_risk | |
| A tool detecting duplicate rows in an imported spreadsheet.,minimal_risk | |
| "We're planning to use machine learning to an AI system drafting judicial decisions by analysing case facts, submissions and applicable case law.",high_risk | |
| We are developing an AI system that will an AI tool that analyses video interview recordings to score candidates before a hiring manager sees the footage.,high_risk | |
| "Proposed use case: an AI system designed to an AI system drafting judicial decisions by analysing case facts, submissions and applicable case law.",high_risk | |
| We are developing an AI system that will software that searches social media and job boards to source and shortlist candidates against recruiter criteria.,high_risk | |
| A startup is building a product that uses AI to a political chatbot conversing with voters to persuade them toward a specific candidate.,high_risk | |
| A startup is building a product that uses AI to a system monitoring webcam footage during standardised testing to detect prohibited behaviour like unauthorised device use.,high_risk | |
| A startup is building a product that uses AI to an AI concierge on a hotel's website chatting directly with guests about bookings.,limited_risk | |
| Marketing software that exploits subconscious psychological triggers to get users to sign away data rights they would normally refuse.,prohibited | |
| We're planning to use machine learning to a phone-based voice assistant taking restaurant orders in conversation with callers.,limited_risk | |
| We're planning to use machine learning to an app recommending recipes based on ingredients a user already has.,minimal_risk | |
| An app suggesting further study options to a student based on interests and career goals they entered themselves.,minimal_risk | |
| We are developing an AI system that will hospital triage software automatically classifying incoming emergency patients by urgency.,high_risk | |
| Software analysing biometric data from photos to guess a person's sexual orientation for a dating-safety product.,prohibited | |
| "A startup is building a product that uses AI to software flagging a group of arriving passengers for a routine health check based on flight-level epidemiological data, applied equally to everyone on the flight.",minimal_risk | |
| We're planning to use machine learning to software comparing a residence-permit applicant's current statements against prior submissions to flag discrepancies.,high_risk | |
| Proposed use case: an AI system designed to a dating-safety startup inferring users' sexual orientation from uploaded selfies without consent.,prohibited | |
| Proposed use case: an AI system designed to AI-based exam proctoring software flagging students for suspected cheating during an online test.,high_risk | |
| "We're planning to use machine learning to a public square's live biometric camera network identifying any pedestrian in real time, used for general crime monitoring rather than one of the narrow permitted purposes.",prohibited | |
| "Our client wants to deploy an AI tool to an AI tool transcribing a panicked emergency caller's words into text to save the call-taker time, without judging urgency.",minimal_risk | |
| Our client wants to deploy an AI tool to a political chatbot conversing with voters to persuade them toward a specific candidate.,high_risk | |
| We are developing an AI system that will software setting a person's health-insurance premium based on an automated risk assessment of their lifestyle data.,high_risk | |
| A startup is building a product that uses AI to a stadium's live biometric camera system identifying any attendee in real time for general crowd management.,prohibited | |
| "Our client wants to deploy an AI tool to a courier company's AI flagging shipping-label mistakes for an employee to fix, not evaluating the employee.",minimal_risk | |
| A startup is building a product that uses AI to an app suggesting further study options to a student based on interests and career goals they entered themselves.,minimal_risk | |
| A startup is building a product that uses AI to a domestic-violence risk-assessment tool analysing police reports and prior incidents to score a potential victim's risk.,high_risk | |
| Proposed use case: an AI system designed to workplace software continuously monitoring and scoring employees' on-the-job behaviour for management decisions.,high_risk | |
| Proposed use case: an AI system designed to an AI concierge on a hotel's website chatting directly with guests about bookings.,limited_risk | |
| Our client wants to deploy an AI tool to a telecom company's automated phone line handling billing questions in natural conversation.,limited_risk | |
| "Software assessing human-trafficking vulnerability from indicators like age, documentation status and travel patterns.",high_risk | |
| We are developing an AI system that will software determining whether a student should be placed in a remedial or advanced programme based on an automated assessment.,high_risk | |
| Our client wants to deploy an AI tool to an app recommending recipes based on ingredients a user already has.,minimal_risk | |
| "We're planning to use machine learning to call-centre software analysing agents' voice tone to infer stress or frustration levels purely for management review, unrelated to safety.",prohibited | |
| A startup is building a product that uses AI to an AI enemy character's combat behaviour in a video game.,minimal_risk | |
| Our client wants to deploy an AI tool to a platform that shows voters individually tailored persuasive political messaging intended to shift their vote.,high_risk | |
| We are developing an AI system that will a platform that shows voters individually tailored persuasive political messaging intended to shift their vote.,high_risk | |
| "A city-run app that combines school attendance, grocery purchases and social media activity into one score used to deny families public housing.",prohibited | |
| Our client wants to deploy an AI tool to software comparing a residence-permit applicant's current statements against prior submissions to flag discrepancies.,high_risk | |
| A startup is building a product that uses AI to a police profiling tool used during an active criminal investigation to narrow down a suspect pool.,high_risk | |
| We're planning to use machine learning to an AI system generating individualised suspect risk profiles from neighbourhood and registry data during a missing-child case.,high_risk | |
| Our client wants to deploy an AI tool to software classifying students with special educational needs into appropriate support programmes.,high_risk | |
| A dating-safety startup inferring users' sexual orientation from uploaded selfies without consent.,prohibited | |
| We are developing an AI system that will marketing software that exploits subconscious psychological triggers to get users to sign away data rights they would normally refuse.,prohibited | |
| A startup is building a product that uses AI to a meeting-notes app summarising a call for the people who were already on it.,minimal_risk | |
| Proposed use case: an AI system designed to a prison system predicting an offender's reoffending likelihood from criminal record and in-facility behaviour.,high_risk | |
| A startup is building a product that uses AI to software setting a person's health-insurance premium based on an automated risk assessment of their lifestyle data.,high_risk | |
| "We're planning to use machine learning to an AI tool verifying whether a submitted health-insurance claim matches the policy terms, after the policy is already in force.",minimal_risk | |
| A startup is building a product that uses AI to a fitness app suggesting a workout plan based on a user's stated goals.,minimal_risk | |
| "We're planning to use machine learning to a classroom camera system inferring students' emotional states (boredom, frustration) to flag them to teachers, with no safety justification.",prohibited | |
| A startup is building a product that uses AI to workplace software continuously monitoring and scoring employees' on-the-job behaviour for management decisions.,high_risk | |
| We are developing an AI system that will an AI tool assessing how reliable a piece of forensic evidence is during a criminal prosecution.,high_risk | |
| We're planning to use machine learning to software evaluating whether a signature or document was forged as part of a criminal investigation.,high_risk | |
| We are developing an AI system that will a prison system predicting an offender's reoffending likelihood from criminal record and in-facility behaviour.,high_risk | |
| Proposed use case: an AI system designed to an app suggesting further study options to a student based on interests and career goals they entered themselves.,minimal_risk | |
| We're planning to use machine learning to a financial newsletter auto-generating market commentary articles published directly to subscribers.,limited_risk | |
| A startup is building a product that uses AI to an AI tool that analyses video interview recordings to score candidates before a hiring manager sees the footage.,high_risk | |
| We are developing an AI system that will a phone-based voice assistant taking restaurant orders in conversation with callers.,limited_risk | |
| A startup is building a product that uses AI to a system that targets people going through bankruptcy with manipulative loan offers timed to their financial desperation.,prohibited | |
| We're planning to use machine learning to a fitness app suggesting a workout plan based on a user's stated goals.,minimal_risk | |
| A startup is building a product that uses AI to a platform that shows voters individually tailored persuasive political messaging intended to shift their vote.,high_risk | |
| We are developing an AI system that will an AI tool categorising support tickets as 'billing' or 'technical' for internal routing only.,minimal_risk | |
| Proposed use case: an AI system designed to a fitness app suggesting a workout plan based on a user's stated goals.,minimal_risk | |
| "Proposed use case: an AI system designed to call-centre software analysing agents' voice tone to infer stress or frustration levels purely for management review, unrelated to safety.",prohibited | |
| We are developing an AI system that will a system that targets people going through bankruptcy with manipulative loan offers timed to their financial desperation.,prohibited | |
| "Proposed use case: an AI system designed to software flagging a group of arriving passengers for a routine health check based on flight-level epidemiological data, applied equally to everyone on the flight.",minimal_risk | |
| "We are developing an AI system that will a cybersecurity monitoring tool watching for anomalies in energy-grid network traffic, independent of physical controls.",minimal_risk | |
| We're planning to use machine learning to software that searches social media and job boards to source and shortlist candidates against recruiter criteria.,high_risk | |
| We are developing an AI system that will software controlling the safety shut-off logic for a regional electricity substation.,high_risk | |
| "A startup is building a product that uses AI to a classroom camera system inferring students' emotional states (boredom, frustration) to flag them to teachers, with no safety justification.",prohibited | |
| We're planning to use machine learning to a domestic-violence risk-assessment tool analysing police reports and prior incidents to score a potential victim's risk.,high_risk | |
| Proposed use case: an AI system designed to software determining whether a student should be placed in a remedial or advanced programme based on an automated assessment.,high_risk | |
| A phone-based voice assistant taking restaurant orders in conversation with callers.,limited_risk | |
| Proposed use case: an AI system designed to a financial newsletter auto-generating market commentary articles published directly to subscribers.,limited_risk | |
| We are developing an AI system that will workplace software continuously monitoring and scoring employees' on-the-job behaviour for management decisions.,high_risk | |
| A system that targets people going through bankruptcy with manipulative loan offers timed to their financial desperation.,prohibited | |
| We're planning to use machine learning to software that crawls public CCTV footage feeds to continuously expand a facial-recognition matching database.,prohibited | |
| "A startup is building a product that uses AI to a wildfire-spread forecasting tool used for general disaster planning, not linked to any specific dispatch decision.",minimal_risk | |
| We're planning to use machine learning to an AI tool categorising support tickets as 'billing' or 'technical' for internal routing only.,minimal_risk | |
| We are developing an AI system that will an AI scheduling assistant coordinating interview time slots between candidates and recruiters.,minimal_risk | |
| An AI enemy character's combat behaviour in a video game.,minimal_risk | |
| We are developing an AI system that will an app recommending recipes based on ingredients a user already has.,minimal_risk | |
| "Proposed use case: an AI system designed to an AI speech-to-text tool transcribing court hearing audio into a written record, without any legal analysis.",minimal_risk | |
| "A courier company's AI flagging shipping-label mistakes for an employee to fix, not evaluating the employee.",minimal_risk | |
| We're planning to use machine learning to an AI tool assessing how reliable a piece of forensic evidence is during a criminal prosecution.,high_risk | |
| "We are developing an AI system that will a city-run app that combines school attendance, grocery purchases and social media activity into one score used to deny families public housing.",prohibited | |
| A stadium's live biometric camera system identifying any attendee in real time for general crowd management.,prohibited | |
| We are developing an AI system that will software comparing a residence-permit applicant's current statements against prior submissions to flag discrepancies.,high_risk | |
| Proposed use case: an AI system designed to an AI scheduling assistant coordinating interview time slots between candidates and recruiters.,minimal_risk | |
| Proposed use case: an AI system designed to software evaluating whether a signature or document was forged as part of a criminal investigation.,high_risk | |
| "Proposed use case: an AI system designed to a wildfire-spread forecasting tool used for general disaster planning, not linked to any specific dispatch decision.",minimal_risk | |
| We're planning to use machine learning to a system monitoring webcam footage during standardised testing to detect prohibited behaviour like unauthorised device use.,high_risk | |
| Proposed use case: an AI system designed to a system monitoring webcam footage during standardised testing to detect prohibited behaviour like unauthorised device use.,high_risk | |
| Proposed use case: an AI system designed to an AI system generating individualised suspect risk profiles from neighbourhood and registry data during a missing-child case.,high_risk | |
| "Our client wants to deploy an AI tool to a language-learning app's AI that decides a learner's official proficiency level, steering their curriculum path.",high_risk | |
| "Our client wants to deploy an AI tool to a maritime AI system used solely to detect obstacles and avoid collisions, with no link to border enforcement.",minimal_risk | |
| Software assigning primary-school children to public schools based on home address and catchment-area rules.,high_risk | |
| Our client wants to deploy an AI tool to a stadium's live biometric camera system identifying any attendee in real time for general crowd management.,prohibited | |
| A startup is building a product that uses AI to AI-based exam proctoring software flagging students for suspected cheating during an online test.,high_risk | |
| "We are developing an AI system that will a language-learning app's AI that decides a learner's official proficiency level, steering their curriculum path.",high_risk | |
| Proposed use case: an AI system designed to a tool detecting duplicate rows in an imported spreadsheet.,minimal_risk | |
| Proposed use case: an AI system designed to an AI tool categorising support tickets as 'billing' or 'technical' for internal routing only.,minimal_risk | |
| We're planning to use machine learning to a police profiling tool used during an active criminal investigation to narrow down a suspect pool.,high_risk | |
| A domestic-violence risk-assessment tool analysing police reports and prior incidents to score a potential victim's risk.,high_risk | |
| Proposed use case: an AI system designed to an AI enemy character's combat behaviour in a video game.,minimal_risk | |
| Proposed use case: an AI system designed to an AI tool assessing how reliable a piece of forensic evidence is during a criminal prosecution.,high_risk | |
| Software controlling the safety shut-off logic for a regional electricity substation.,high_risk | |
| "We're planning to use machine learning to an AI tool transcribing a panicked emergency caller's words into text to save the call-taker time, without judging urgency.",minimal_risk | |
| Our client wants to deploy an AI tool to a police profiling tool used during an active criminal investigation to narrow down a suspect pool.,high_risk | |
| Software evaluating whether a signature or document was forged as part of a criminal investigation.,high_risk | |
| We're planning to use machine learning to AI-based exam proctoring software flagging students for suspected cheating during an online test.,high_risk | |
| "A startup is building a product that uses AI to a language-learning app's AI that decides a learner's official proficiency level, steering their curriculum path.",high_risk | |
| "Our client wants to deploy an AI tool to a cybersecurity monitoring tool watching for anomalies in energy-grid network traffic, independent of physical controls.",minimal_risk | |
| "We're planning to use machine learning to software assessing human-trafficking vulnerability from indicators like age, documentation status and travel patterns.",high_risk | |
| We're planning to use machine learning to a meeting-notes app summarising a call for the people who were already on it.,minimal_risk | |
| A political chatbot conversing with voters to persuade them toward a specific candidate.,high_risk | |
| We're planning to use machine learning to an AI tool that analyses video interview recordings to score candidates before a hiring manager sees the footage.,high_risk | |
| We're planning to use machine learning to software classifying students with special educational needs into appropriate support programmes.,high_risk | |
| "We are developing an AI system that will an AI tool transcribing a panicked emergency caller's words into text to save the call-taker time, without judging urgency.",minimal_risk | |
| A startup is building a product that uses AI to software controlling the safety shut-off logic for a regional electricity substation.,high_risk | |
| Our client wants to deploy an AI tool to an AI scheduling assistant coordinating interview time slots between candidates and recruiters.,minimal_risk | |
| Our client wants to deploy an AI tool to a therapy app using AI-guided cognitive behavioural therapy techniques with a patient's explicit informed consent.,minimal_risk | |
| A startup is building a product that uses AI to software assigning primary-school children to public schools based on home address and catchment-area rules.,high_risk | |