Instructions to use apus-ailab/APUS-Reflex with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use apus-ailab/APUS-Reflex with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("apus-ailab/APUS-Reflex", device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs: document held-out 1000-question benchmark design
#7
by gump2049 - opened
- README.md +1 -1
- README.zh-CN.md +1 -1
README.md
CHANGED
|
@@ -40,7 +40,7 @@ We introduce **APUS-OpenJev-v1**, a family of decision models for browser agents
|
|
| 40 |
|
| 41 |
**Decision-oriented inference.** The runtime scores candidate labels and maps them back to application actions. This removes the need to generate a structured answer token by token for bounded-choice tasks. Choose `effort="low"` or `effort="high"` to balance compute cost and decision quality.
|
| 42 |
|
| 43 |
-
On
|
| 44 |
|
| 45 |

|
| 46 |
|
|
|
|
| 40 |
|
| 41 |
**Decision-oriented inference.** The runtime scores candidate labels and maps them back to application actions. This removes the need to generate a structured answer token by token for bounded-choice tasks. Choose `effort="low"` or `effort="high"` to balance compute cost and decision quality.
|
| 42 |
|
| 43 |
+
On the held-out **1,000-question expanded development panel**, **APUS-OpenJev 35B-A3B achieves 82.20%, 9B 81.10%, and 4B 80.50% accuracy**, compared with **77.00% for the Jev API**. The panel is stratified across browser action selection, principle judgment, BoolQ, MNLI, and score-level decisions, with frozen identities and reference labels kept separate from training data. All scores use the same 1,000-question denominator; invalid or unavailable responses remain in that denominator. The optimized 9B decision service reports **25.58 ms median HTTP latency** in the run snapshot below. The chart uses historical checkpoint-5949 full-depth evaluation, while the published 9B quality result uses `9B-3000`; latency uses `9B-5949`. The 80-question Frozen80 panel remains the release regression suite and is reported separately below.
|
| 44 |
|
| 45 |

|
| 46 |
|
README.zh-CN.md
CHANGED
|
@@ -42,7 +42,7 @@ tags:
|
|
| 42 |
|
| 43 |
**面向决策的推理。** runtime对候选标签评分,再映射回应用动作。对于有界选择任务,无需逐token生成结构化答案。通过选择`effort="low"`或`effort="high"`,在计算成本与决策质量之间取得平衡。
|
| 44 |
|
| 45 |
-
在
|
| 46 |
|
| 47 |

|
| 48 |
|
|
|
|
| 42 |
|
| 43 |
**面向决策的推理。** runtime对候选标签评分,再映射回应用动作。对于有界选择任务,无需逐token生成结构化答案。通过选择`effort="low"`或`effort="high"`,在计算成本与决策质量之间取得平衡。
|
| 44 |
|
| 45 |
+
在留出的**1000题扩展开发面板**上,**APUS-OpenJev 35B-A3B准确率为82.20%、9B为81.10%、4B为80.50%**,**Jev API为77.00%**。该面板围绕浏览器动作选择、原则判断、BoolQ、MNLI和情绪分级决策进行分层,题目身份与参考标签冻结,并与训练数据隔离。所有结果使用相同的1000题分母;无效或不可用响应仍计入分母。优化后的9B决策服务在下方运行快照中达到**25.58 ms HTTP中位延迟**。图表采用历史checkpoint-5949完整深度评测;当前发布的9B质量结果对应`9B-3000`,延迟测量对应`9B-5949`。Frozen80 80题面板继续作为发布回归套件,单独在下方报告。
|
| 46 |
|
| 47 |

|
| 48 |
|