Update README.md

README.md

@@ -22,7 +22,6 @@ test plus non-flaky.
 A fine-tune of `microsoft/codebert-base` on the FlakeBench dataset from
 [*Understanding and Improving Flaky Test Classification*](https://utexas.app.box.com/v/august-shi-OOPSLA2025)
 (OOPSLA 2025), trained as a reproduction exercise on a single 8 GB consumer GPU.
-
 The uploaded weights are the "Balanced" configuration below.
 
 ## Training configurations
@@ -46,6 +45,32 @@ The uploaded weights are the "Balanced" configuration below.
 Hardware: 1× RTX 4060 Laptop (8 GB). Class rebalancing is the one deviation from the
 paper's method, which trains on the raw distribution (97% non-flaky).
 
+## Results (per-category F1, fold 2 test set)
+
+| Category         |   Baseline |    lr 2e-5 |   Balanced |  Augmented |      Paper |
+| ---------------- | ---------: | ---------: | ---------: | ---------: | ---------: |
+| Async Wait        |     76.92% |     78.26% |     74.07% |     64.52% |     58.37% |
+| Concurrency       |      0.00% |      0.00% |      0.00% |      0.00% |     35.92% |
+| Time              |     57.14% |     66.67% |     66.67% |     40.00% |     72.73% |
+| Unordered Coll.   |     75.00% |     83.33% |     83.33% |     72.73% |     73.63% |
+| Order Dep.        |     82.35% |     86.96% |     95.24% |     73.68% |     64.35% |
+| Non-flaky         |    100.00% |     99.92% |     99.51% |    100.00% |    100.00% |
+| **Macro F1**      | **65.24%** | **69.19%** | **69.89%** | **58.49%** | **65.79%** |
+
+The **Balanced** configuration (uploaded weights) achieves the best macro-F1 of
+**69.89%** on this fold, outperforming the paper's reported 65.79% overall, but at
+the cost of a **0.00% F1 on Concurrency** across every configuration tried — this
+category collapses entirely regardless of rebalancing strategy, likely due to its
+very small sample size (37 tests total across the whole dataset) and single-fold
+evaluation rather than the paper's 4-fold average. The **Augmented** configuration
+underperforms the paper on most categories, suggesting the synthetic variants used
+for minority-class expansion may not be adding genuinely useful signal.
+
+**Caveat**: these numbers come from a **single fold**, not the paper's 4-fold
+average, so they are not directly comparable to the paper's reported scores without
+that caveat in mind — a single lucky/unlucky project split can swing per-category
+F1 substantially, especially for categories with as few as 33–41 test examples.
+
 ## Usage
 
 ```python
@@ -75,6 +100,9 @@ Labels: `0` async wait, `1` concurrency, `2` time, `3` unordered collections,
 `4` test order dependency, `5` non-flaky.
 
 Scope: Java/Kotlin test methods; inputs longer than 512 tokens are truncated.
+**Known limitation**: this checkpoint does not predict Concurrency (label `1`) at
+all on the fold-2 test set — treat any Concurrency-relevant use case with caution
+until retrained with more Concurrency examples or evaluated across all 4 folds.
 
 ## Citation
 
@@ -89,4 +117,4 @@ endorsed by its authors.
 }
 ```
 
-Dataset and method: [UT-SE-Research/FlakyLens](https://github.com/UT-SE-Research/FlakyLens).
+Dataset and method: [UT-SE-Research/FlakyLens](https://github.com/UT-SE-Research/FlakyLens).