asermnasr commited on
Commit
b02fd60
·
verified ·
1 Parent(s): 62c88f2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -155,7 +155,7 @@ Folate is evaluated on the three standard generalization splits of **Mind2Web**
155
 
156
  - **Task Formulation:** At each step, the model receives the natural-language task instruction, action history context, and up to **50 candidate DOM elements** (`Avg Cands ≈ 50.0`) extracted from the page snapshot, and outputs a relevance logit per candidate.
157
  - **Metrics:**
158
- - **Top-1 / Top-3 / Top-5:** Percentage of steps where a ground-truth target node (`pos_candidates`) appears within the top $k$ ranked elements.
159
  - **MRR (Mean Reciprocal Rank):** Average for the highest-ranked ground-truth element across all evaluated steps.
160
  - **Hardware & Runtime:** Latency is measured end-to-end per step on **2 vCPUs** with **no GPU acceleration**, using the exported INT8 quantized ONNX artifact (`onnxruntime`). Variance between `test_website` (`4,328.2 ms/step`) and `test_domain` (`2,431.0 ms/step`) reflects longer tokenized DOM attribute sequences on unseen site templates rather than candidate count.
161
 
 
155
 
156
  - **Task Formulation:** At each step, the model receives the natural-language task instruction, action history context, and up to **50 candidate DOM elements** (`Avg Cands ≈ 50.0`) extracted from the page snapshot, and outputs a relevance logit per candidate.
157
  - **Metrics:**
158
+ - **Top-1 / Top-3 / Top-5:** Percentage of steps where a ground-truth target node (`pos_candidates`) appears within the top ranked elements.
159
  - **MRR (Mean Reciprocal Rank):** Average for the highest-ranked ground-truth element across all evaluated steps.
160
  - **Hardware & Runtime:** Latency is measured end-to-end per step on **2 vCPUs** with **no GPU acceleration**, using the exported INT8 quantized ONNX artifact (`onnxruntime`). Variance between `test_website` (`4,328.2 ms/step`) and `test_domain` (`2,431.0 ms/step`) reflects longer tokenized DOM attribute sequences on unseen site templates rather than candidate count.
161