Ariful1904129 commited on
Commit
cd6eef4
·
verified ·
1 Parent(s): d2dfc53

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -13
README.md CHANGED
@@ -12,7 +12,7 @@ tags:
12
  - reproduction
13
  ---
14
 
15
- # CodeBERT for Flaky Test Categorisation (FlakeBench, fold 2)
16
 
17
  Classifies a Java/Kotlin test method into one of six categories: five kinds of flaky
18
  test plus non-flaky.
@@ -58,18 +58,8 @@ paper's method, which trains on the raw distribution (97% non-flaky).
58
  | **Macro F1** | **65.24%** | **69.19%** | **69.89%** | **58.49%** | **65.79%** |
59
 
60
  The **Balanced** configuration (uploaded weights) achieves the best macro-F1 of
61
- **69.89%** on this fold, outperforming the paper's reported 65.79% overall, but at
62
- the cost of a **0.00% F1 on Concurrency** across every configuration tried — this
63
- category collapses entirely regardless of rebalancing strategy, likely due to its
64
- very small sample size (37 tests total across the whole dataset) and single-fold
65
- evaluation rather than the paper's 4-fold average. The **Augmented** configuration
66
- underperforms the paper on most categories, suggesting the synthetic variants used
67
- for minority-class expansion may not be adding genuinely useful signal.
68
-
69
- **Caveat**: these numbers come from a **single fold**, not the paper's 4-fold
70
- average, so they are not directly comparable to the paper's reported scores without
71
- that caveat in mind — a single lucky/unlucky project split can swing per-category
72
- F1 substantially, especially for categories with as few as 33–41 test examples.
73
 
74
  ## Usage
75
 
 
12
  - reproduction
13
  ---
14
 
15
+ # CodeBERT for Flaky Test Categorisation (FlakeBench)
16
 
17
  Classifies a Java/Kotlin test method into one of six categories: five kinds of flaky
18
  test plus non-flaky.
 
58
  | **Macro F1** | **65.24%** | **69.19%** | **69.89%** | **58.49%** | **65.79%** |
59
 
60
  The **Balanced** configuration (uploaded weights) achieves the best macro-F1 of
61
+ **69.89%** .
62
+
 
 
 
 
 
 
 
 
 
 
63
 
64
  ## Usage
65