Update README.md
Browse files
README.md
CHANGED
|
@@ -155,7 +155,7 @@ Folate is evaluated on the three standard generalization splits of **Mind2Web**
|
|
| 155 |
|
| 156 |
- **Task Formulation:** At each step, the model receives the natural-language task instruction, action history context, and up to **50 candidate DOM elements** (`Avg Cands ≈ 50.0`) extracted from the page snapshot, and outputs a relevance logit per candidate.
|
| 157 |
- **Metrics:**
|
| 158 |
-
- **Top-1 / Top-3 / Top-5:** Percentage of steps where a ground-truth target node (`pos_candidates`) appears within the top
|
| 159 |
- **MRR (Mean Reciprocal Rank):** Average for the highest-ranked ground-truth element across all evaluated steps.
|
| 160 |
- **Hardware & Runtime:** Latency is measured end-to-end per step on **2 vCPUs** with **no GPU acceleration**, using the exported INT8 quantized ONNX artifact (`onnxruntime`). Variance between `test_website` (`4,328.2 ms/step`) and `test_domain` (`2,431.0 ms/step`) reflects longer tokenized DOM attribute sequences on unseen site templates rather than candidate count.
|
| 161 |
|
|
|
|
| 155 |
|
| 156 |
- **Task Formulation:** At each step, the model receives the natural-language task instruction, action history context, and up to **50 candidate DOM elements** (`Avg Cands ≈ 50.0`) extracted from the page snapshot, and outputs a relevance logit per candidate.
|
| 157 |
- **Metrics:**
|
| 158 |
+
- **Top-1 / Top-3 / Top-5:** Percentage of steps where a ground-truth target node (`pos_candidates`) appears within the top ranked elements.
|
| 159 |
- **MRR (Mean Reciprocal Rank):** Average for the highest-ranked ground-truth element across all evaluated steps.
|
| 160 |
- **Hardware & Runtime:** Latency is measured end-to-end per step on **2 vCPUs** with **no GPU acceleration**, using the exported INT8 quantized ONNX artifact (`onnxruntime`). Variance between `test_website` (`4,328.2 ms/step`) and `test_domain` (`2,431.0 ms/step`) reflects longer tokenized DOM attribute sequences on unseen site templates rather than candidate count.
|
| 161 |
|