Commit History

  1. Update requirements.txt⁠

    1fe38fd

  1. Update requirements.txt

    1fcbb8d

  1. add pdf

    31d64cc
  2. Delete explanation_filtering_pipeline.pdf

    Hugo Laurençon
    3ae24a3
  3. Update app.py

    Hugo Laurençon
    b881ada
  4. Upload explanation_filtering_pipeline.pdf

    Hugo Laurençon
    2be3583
  5. Delete explanation_filtering_pipeline.pdf

    Hugo Laurençon
    3327a22

  1. remove arabic and viet models

    836c1b3
  2. Add Portuguese

    ba331e0
  3. delete unused models

    34eca0f
  4. Merge branch 'main' of https://huggingface.co/spaces/huggingface/text-data-filtering

    f6058aa
  5. back to before portuguese

    091dbe4

  1. Update app.py

    Hugo Laurençon
    c4be503
  2. Update app.py

    Hugo Laurençon
    72d8e96
  3. update visu for Portuguese

    2b811ac

  1. 7 languages supported

    ea01f38

  1. new kenlm models

    b37a555

  1. add register information

    061d2e4

  1. new filter on word repetition ratio

    4809033

  1. visualization: small step for the slider on flagged words ratio

    fa81556
  2. visualization: choose between several languages

    0610f9d
  3. fix bug

    0319ee2
  4. distributions for the filters on words and discarded words by filter

    da13b29

  1. visualization: upload our own stop words and flagged words list

    5d56c36
  2. quick fix

    1bc0c1e
  3. everything in expanders

    2c2527f
  4. display distributions in sidebar and filtering parameters in expanders

    5d485e5

  1. rename badwords to flagged words + new flagged words list of 68 words

    f217a73

  1. button to download parameters

    bfbcd60

  1. add warning message

    649ea6a

  1. better visualization

    8f0da78
  2. fix division by 0 in compute_special_characters_ratio

    b607b76
  3. new tool to analyse our own doc

    6f25c5c
  4. fix requirements

    d463071
  5. fix packages

    924da6e
  6. test

    d1e3e7b

  1. correction of bug

    22701ae
  2. delete app_2

    c340078
  3. merge

    6ddbf7d
  4. filter on repetition removal

    693f997

  1. Update app.py

    Hugo Laurençon
    189d6aa
  2. chinese visu

    611e98e
  3. Delete en_examples_with_stats_no_small_docs.json

    Hugo Laurençon
    58d483d
  4. Delete en_examples_with_stats_ldnoob.json

    Hugo Laurençon
    b190ef8
  5. Delete en_examples_with_stats.json

    Hugo Laurençon
    0376199

  1. remove zipf's law and update of the doc

    3fd19c1
  2. visu with discarded documents by filter

    14574d7
  3. new visu

    6303415

  1. faster visu (less documents)

    07c617e
  2. new visu

    0add2d4