Spaces:
Sleeping
Sleeping
Upload folder using huggingface_hub
Browse files- CHANGELOG.md +6 -0
- CLAUDE.md +1 -1
- README.md +9 -9
- openenv_verirl_env.egg-info/PKG-INFO +1 -1
- pyproject.toml +1 -1
CHANGELOG.md
CHANGED
|
@@ -1,3 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
## [0.3.8] - 2026-04-08
|
| 2 |
|
| 3 |
Bug Fixes
|
|
|
|
| 1 |
+
## [0.3.9] - 2026-04-08
|
| 2 |
+
|
| 3 |
+
Bug Fixes
|
| 4 |
+
~~~~~~~~~
|
| 5 |
+
|
| 6 |
+
- Add score= field to [END] output line so autograder can extract per-task final scores (end-score-field)
|
| 7 |
## [0.3.8] - 2026-04-08
|
| 8 |
|
| 9 |
Bug Fixes
|
CLAUDE.md
CHANGED
|
@@ -258,7 +258,7 @@ The inference script MUST output ONLY these three line types to stdout:
|
|
| 258 |
```
|
| 259 |
[START] task=<task> env=<benchmark> model=<model>
|
| 260 |
[STEP] step=<n> action=<action> reward=<0.00> done=<true|false> error=<msg|null>
|
| 261 |
-
[END] success=<true|false> steps=<n> rewards=<r1,r2,...,rn>
|
| 262 |
```
|
| 263 |
|
| 264 |
NO other output to stdout (no comments, no validation messages, no summaries).
|
|
|
|
| 258 |
```
|
| 259 |
[START] task=<task> env=<benchmark> model=<model>
|
| 260 |
[STEP] step=<n> action=<action> reward=<0.00> done=<true|false> error=<msg|null>
|
| 261 |
+
[END] success=<true|false> steps=<n> score=<score> rewards=<r1,r2,...,rn>
|
| 262 |
```
|
| 263 |
|
| 264 |
NO other output to stdout (no comments, no validation messages, no summaries).
|
README.md
CHANGED
|
@@ -23,7 +23,7 @@ VeriRL frames this as a multi-step agentic task. The three tasks span the core b
|
|
| 23 |
|
| 24 |
1. **MAC Unit** — a 2-stage pipelined multiply-accumulate unit, the arithmetic kernel inside every convolution and matrix-multiply accelerator
|
| 25 |
2. **AXI-Stream FIFO** — a 4-entry FIFO with correct handshaking and backpressure, the standard interconnect protocol between datapath blocks
|
| 26 |
-
3. **4×4 Systolic Array** — a weight-stationary matrix multiply engine with diagonal skewing and a
|
| 27 |
|
| 28 |
Each task includes a detailed Verilog interface specification and a testbench that exercises correctness, edge cases, and (for tasks 1 and 3) timing behavior. Scores are computed by compiling agent submissions with `iverilog`, running functional simulation with `vvp`, and estimating area with `yosys` — the same open-source EDA toolchain used in academic and hobbyist chip design.
|
| 29 |
|
|
@@ -58,7 +58,7 @@ sequenceDiagram
|
|
| 58 |
|
| 59 |
LLM->>Env: submit (or turn budget exhausted)
|
| 60 |
Env->>EDA: Final complete evaluation
|
| 61 |
-
EDA-->>Env: Weighted Score [0,
|
| 62 |
Env-->>LLM: Final Score & Episode Done
|
| 63 |
```
|
| 64 |
|
|
@@ -93,7 +93,7 @@ Only one action is executed per step. Actions that have prerequisites (e.g., `ru
|
|
| 93 |
| `turn_number` | `int` | Current turn (1-indexed after first step) |
|
| 94 |
| `turns_remaining` | `int` | Steps left in the episode |
|
| 95 |
| `current_verilog` | `str \| None` | The Verilog source currently on file |
|
| 96 |
-
| `final_score` | `float \| None` | Final score in [0,
|
| 97 |
| `score_breakdown` | `dict \| None` | Per-dimension scores: `compile`, `sim`, `timing`, `area` |
|
| 98 |
| `reward` | `float` | Per-step reward (see Reward Function below) |
|
| 99 |
| `done` | `bool` | Whether the episode has ended |
|
|
@@ -120,7 +120,7 @@ A 4-entry synchronous FIFO implementing the AXI4-Stream handshake on both slave
|
|
| 120 |
|
| 121 |
**Testbench:** 34 assertions covering reset state, single push/pop, fill-to-full, drain-in-order, simultaneous enqueue/dequeue, downstream stall, partial drain with refill, and rapid push/pop interleaving.
|
| 122 |
|
| 123 |
-
**Scoring weights:** compile 10% · functional correctness
|
| 124 |
|
| 125 |
---
|
| 126 |
|
|
@@ -128,11 +128,11 @@ A 4-entry synchronous FIFO implementing the AXI4-Stream handshake on both slave
|
|
| 128 |
|
| 129 |
**Turn budget:** 12 | **Module:** `systolic_array`
|
| 130 |
|
| 131 |
-
A 4×4 grid of processing elements computing C = A × B for INT8 activations and INT4 weights. Weights are preloaded and held stationary; activations flow left-to-right with diagonal skewing so that PE[i][j] receives its activation at cycle `i+j` from the start pulse. The `done` signal must assert within **
|
| 132 |
|
| 133 |
**Testbench:** 7 test cases with 76 individual output assertions, covering identity weights, zero weights, known-value matrix products, powers-of-two weights, single-column weights, uniform weights, and diagonal weights.
|
| 134 |
|
| 135 |
-
**Scoring weights:** compile 5% · functional correctness 50% · timing (done ≤
|
| 136 |
|
| 137 |
---
|
| 138 |
|
|
@@ -146,7 +146,7 @@ reward = +0.02 (any Verilog is on file)
|
|
| 146 |
+ 0.10 × (tests_passed / tests_total) (absolute test ratio)
|
| 147 |
+ 0.15 × (Δ test ratio vs previous sim run) (improvement bonus)
|
| 148 |
- min(0.01 × turn_number, 0.05) (time penalty, capped at 0.05)
|
| 149 |
-
clamped to [0.
|
| 150 |
```
|
| 151 |
|
| 152 |
The improvement bonus rewards each incremental fix: fixing 5 failing tests in one step earns more than fixing 1. `write_file` resets the compile and simulation state so that all progress signals are tied to the current code on file.
|
|
@@ -257,8 +257,8 @@ Baseline agent: `openai/gpt-oss-120b` via Hugging Face inference router.
|
|
| 257 |
The inference script includes task enumeration and grader validation (run before inference):
|
| 258 |
- All 3 tasks discovered: `mac_unit`, `axi_fifo`, `systolic_array`
|
| 259 |
- All graders tested with empty submission → all score 0.0 (valid)
|
| 260 |
-
- All per-step rewards in valid range [
|
| 261 |
-
- All final scores in valid range [0.
|
| 262 |
|
| 263 |
**Interpretation:**
|
| 264 |
- MAC unit (easy): 0.154 — some compilation/simulation credit, but test failures
|
|
|
|
| 23 |
|
| 24 |
1. **MAC Unit** — a 2-stage pipelined multiply-accumulate unit, the arithmetic kernel inside every convolution and matrix-multiply accelerator
|
| 25 |
2. **AXI-Stream FIFO** — a 4-entry FIFO with correct handshaking and backpressure, the standard interconnect protocol between datapath blocks
|
| 26 |
+
3. **4×4 Systolic Array** — a weight-stationary matrix multiply engine with diagonal skewing and a 10-cycle latency requirement, structurally similar to the core of a TPU
|
| 27 |
|
| 28 |
Each task includes a detailed Verilog interface specification and a testbench that exercises correctness, edge cases, and (for tasks 1 and 3) timing behavior. Scores are computed by compiling agent submissions with `iverilog`, running functional simulation with `vvp`, and estimating area with `yosys` — the same open-source EDA toolchain used in academic and hobbyist chip design.
|
| 29 |
|
|
|
|
| 58 |
|
| 59 |
LLM->>Env: submit (or turn budget exhausted)
|
| 60 |
Env->>EDA: Final complete evaluation
|
| 61 |
+
EDA-->>Env: Weighted Score [0.01, 0.99]
|
| 62 |
Env-->>LLM: Final Score & Episode Done
|
| 63 |
```
|
| 64 |
|
|
|
|
| 93 |
| `turn_number` | `int` | Current turn (1-indexed after first step) |
|
| 94 |
| `turns_remaining` | `int` | Steps left in the episode |
|
| 95 |
| `current_verilog` | `str \| None` | The Verilog source currently on file |
|
| 96 |
+
| `final_score` | `float \| None` | Final score in [0.01, 0.99] — set on `submit` or episode expiry |
|
| 97 |
| `score_breakdown` | `dict \| None` | Per-dimension scores: `compile`, `sim`, `timing`, `area` |
|
| 98 |
| `reward` | `float` | Per-step reward (see Reward Function below) |
|
| 99 |
| `done` | `bool` | Whether the episode has ended |
|
|
|
|
| 120 |
|
| 121 |
**Testbench:** 34 assertions covering reset state, single push/pop, fill-to-full, drain-in-order, simultaneous enqueue/dequeue, downstream stall, partial drain with refill, and rapid push/pop interleaving.
|
| 122 |
|
| 123 |
+
**Scoring weights:** compile 10% · simulation (functional correctness + protocol compliance) 70% · area efficiency 20%
|
| 124 |
|
| 125 |
---
|
| 126 |
|
|
|
|
| 128 |
|
| 129 |
**Turn budget:** 12 | **Module:** `systolic_array`
|
| 130 |
|
| 131 |
+
A 4×4 grid of processing elements computing C = A × B for INT8 activations and INT4 weights. Weights are preloaded and held stationary; activations flow left-to-right with diagonal skewing so that PE[i][j] receives its activation at cycle `i+j` from the start pulse. The `done` signal must assert within **10 clock cycles** of `start` — this requires implementing shift-register delay lines of depth `i` on each activation row. Designs that assert `done` after 10 but within 13 cycles receive partial timing credit.
|
| 132 |
|
| 133 |
**Testbench:** 7 test cases with 76 individual output assertions, covering identity weights, zero weights, known-value matrix products, powers-of-two weights, single-column weights, uniform weights, and diagonal weights.
|
| 134 |
|
| 135 |
+
**Scoring weights:** compile 5% · functional correctness 50% · timing (done ≤ 10 cycles) 30% · area efficiency 15%
|
| 136 |
|
| 137 |
---
|
| 138 |
|
|
|
|
| 146 |
+ 0.10 × (tests_passed / tests_total) (absolute test ratio)
|
| 147 |
+ 0.15 × (Δ test ratio vs previous sim run) (improvement bonus)
|
| 148 |
- min(0.01 × turn_number, 0.05) (time penalty, capped at 0.05)
|
| 149 |
+
clamped to [0.01, 0.99]
|
| 150 |
```
|
| 151 |
|
| 152 |
The improvement bonus rewards each incremental fix: fixing 5 failing tests in one step earns more than fixing 1. `write_file` resets the compile and simulation state so that all progress signals are tied to the current code on file.
|
|
|
|
| 257 |
The inference script includes task enumeration and grader validation (run before inference):
|
| 258 |
- All 3 tasks discovered: `mac_unit`, `axi_fifo`, `systolic_array`
|
| 259 |
- All graders tested with empty submission → all score 0.0 (valid)
|
| 260 |
+
- All per-step rewards in valid range [0.01, 0.99]
|
| 261 |
+
- All final scores in valid range [0.01, 0.99]
|
| 262 |
|
| 263 |
**Interpretation:**
|
| 264 |
- MAC unit (easy): 0.154 — some compilation/simulation credit, but test failures
|
openenv_verirl_env.egg-info/PKG-INFO
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
Metadata-Version: 2.4
|
| 2 |
Name: openenv-verirl_env
|
| 3 |
-
Version: 0.3.
|
| 4 |
Summary: Verilog hardware design environment for OpenEnv — trains LLMs to write RTL for AI-accelerator primitives
|
| 5 |
Requires-Python: >=3.10
|
| 6 |
Requires-Dist: openai>=2.30.0
|
|
|
|
| 1 |
Metadata-Version: 2.4
|
| 2 |
Name: openenv-verirl_env
|
| 3 |
+
Version: 0.3.9
|
| 4 |
Summary: Verilog hardware design environment for OpenEnv — trains LLMs to write RTL for AI-accelerator primitives
|
| 5 |
Requires-Python: >=3.10
|
| 6 |
Requires-Dist: openai>=2.30.0
|
pyproject.toml
CHANGED
|
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
|
|
| 10 |
|
| 11 |
[project]
|
| 12 |
name = "openenv-verirl_env"
|
| 13 |
-
version = "0.3.
|
| 14 |
description = "Verilog hardware design environment for OpenEnv — trains LLMs to write RTL for AI-accelerator primitives"
|
| 15 |
requires-python = ">=3.10"
|
| 16 |
dependencies = [
|
|
|
|
| 10 |
|
| 11 |
[project]
|
| 12 |
name = "openenv-verirl_env"
|
| 13 |
+
version = "0.3.9"
|
| 14 |
description = "Verilog hardware design environment for OpenEnv — trains LLMs to write RTL for AI-accelerator primitives"
|
| 15 |
requires-python = ">=3.10"
|
| 16 |
dependencies = [
|