| --- |
| language: fr |
| license: mit |
| base_model: camembert/camembert-base |
| tags: |
| - text-classification |
| - french |
| --- |
| |
| # bert-rubriques |
|
|
| CamemBERT-base fine-tuned to annotate the section of French news article. 6 categories are allowed: politics, society, international, sport, culture and economics. |
| An article relating to political, social or economic events outside France should be classified in international - so that politics contains only French politics, etc. Culture has no borders, so a Bob Dylan album should end up in the culture section. Economics includes both business news (e.g., some company making profit) or macroeconomics news (e.g. a recession). |
| Politics is strictly about political actors -- even though everything is political, of course. |
|
|
|
|
| This has been developed for the [Gallicagram](gallicagram.com) project, to avoid 'structure effects': a word could be more present in a corpus or a time period because the section where it tends to appear |
| is larger, and we might want to control for that. More generally, press corpora are really meta-corpora, and lexicometrics/cultural analytics shouldn't mix sports with politics. |
|
|
| This model has been trained on 960 articles of Le Monde archives (1945-...) with the wonderful [ActiveTigger](https://activetigger.com/) (s/o to Émilien Schultz and his colleagues), using active learning to speed up learning. |
| It is meant to work more broadly on any post-war French news articles - and in our experience, it does work. It is probably not as accurate outside of high-brow printed press - I would not use it on tabloid, social media or TV transcript. |
|
|
| ## Training details |
|
|
| - Base model: camembert/camembert-base |
| - Epochs: 5 |
| - Batch size: 4 (gradient accumulation: 4) |
| - Learning rate: 3e-5 |
| - Weight decay: 0.01 |
| - Max sequence length: 512 |
| - Training examples: 960 |
|
|
| ## Evaluation |
| On a test-set of 241 examples, stratified for time (between 1945 and 2024), overall accuracy is 81.7%, F1 macro 0.813, F1 weighted 0.818. Sports and international are, understandably, better classified (see below). |
| Note that the categories are quite porous, so most errors are ambiguous cases, and this level of accuracy should be understood as a lower bound. |
|
|
| Confusion matrix: |
| | true \ pred | culture | international | politique | société | sport | économie | |
| |---|---|---|---|---|---|---| |
| | **culture** | 34 | 2 | 1 | 4 | 0 | 1 | |
| | **international** | 3 | 66 | 2 | 0 | 0 | 3 | |
| | **politique** | 0 | 3 | 23 | 5 | 0 | 2 | |
| | **société** | 5 | 1 | 4 | 39 | 0 | 2 | |
| | **sport** | 0 | 0 | 0 | 0 | 18 | 0 | |
| | **économie** | 0 | 0 | 0 | 5 | 1 | 17 | |
|
|
|
|
