Vidhikoul commited on
Commit
3b33bb3
·
verified ·
1 Parent(s): c87e3fb

Upload folder using huggingface_hub

Browse files
Dockerfile ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ # Multi-stage build using openenv-base
8
+ # This Dockerfile is flexible and works for both:
9
+ # - In-repo environments (with local OpenEnv sources)
10
+ # - Standalone environments (with openenv from PyPI/Git)
11
+ # The build script (openenv build) handles context detection and sets appropriate build args.
12
+
13
+ ARG BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest
14
+ FROM ${BASE_IMAGE} AS builder
15
+
16
+ WORKDIR /app
17
+
18
+ # Ensure git is available (required for installing dependencies from VCS)
19
+ RUN apt-get update && \
20
+ apt-get install -y --no-install-recommends git wget curl && \
21
+ rm -rf /var/lib/apt/lists/*
22
+
23
+
24
+ RUN wget -q https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.9.3/junit-platform-console-standalone-1.9.3.jar \
25
+ -O /app/junit-platform-console-standalone.jar
26
+
27
+ # Build argument to control whether we're building standalone or in-repo
28
+ ARG BUILD_MODE=in-repo
29
+ ARG ENV_NAME=UnitTestCaseGenerator
30
+
31
+ # Copy environment code (always at root of build context)
32
+ COPY . /app/env
33
+
34
+ # For in-repo builds, openenv is already vendored in the build context
35
+ # For standalone builds, openenv will be installed via pyproject.toml
36
+ WORKDIR /app/env
37
+
38
+ # Ensure uv is available (for local builds where base image lacks it)
39
+ RUN if ! command -v uv >/dev/null 2>&1; then \
40
+ curl -LsSf https://astral.sh/uv/install.sh | sh && \
41
+ mv /root/.local/bin/uv /usr/local/bin/uv && \
42
+ mv /root/.local/bin/uvx /usr/local/bin/uvx; \
43
+ fi
44
+
45
+ # Install dependencies using uv sync
46
+ # If uv.lock exists, use it; otherwise resolve on the fly
47
+
48
+ # RUN apt-get update && apt-get install -y wget
49
+ RUN --mount=type=cache,target=/root/.cache/uv \
50
+ if [ -f uv.lock ]; then \
51
+ uv sync --frozen --no-install-project --no-editable; \
52
+ else \
53
+ uv sync --no-install-project --no-editable; \
54
+ fi
55
+
56
+ RUN --mount=type=cache,target=/root/.cache/uv \
57
+ if [ -f uv.lock ]; then \
58
+ uv sync --frozen --no-editable; \
59
+ else \
60
+ uv sync --no-editable; \
61
+ fi
62
+
63
+ # Final runtime stage
64
+ FROM ${BASE_IMAGE}
65
+
66
+ WORKDIR /app
67
+
68
+ # Java final stage mein install karo
69
+ RUN apt-get update && \
70
+ apt-get install -y --no-install-recommends default-jdk && \
71
+ rm -rf /var/lib/apt/lists/*
72
+
73
+ # JUnit jar builder se copy karo
74
+ COPY --from=builder /app/junit-platform-console-standalone.jar \
75
+ /app/junit-platform-console-standalone.jar
76
+ ENV JUNIT_JAR=/app/junit-platform-console-standalone.jar
77
+
78
+ # Copy the virtual environment from builder
79
+ COPY --from=builder /app/env/.venv /app/.venv
80
+
81
+ # Copy the environment code
82
+ COPY --from=builder /app/env /app/env
83
+
84
+ # Set PATH to use the virtual environment
85
+ ENV PATH="/app/.venv/bin:$PATH"
86
+
87
+ # Set PYTHONPATH so imports work correctly
88
+ ENV PYTHONPATH="/app/env:$PYTHONPATH"
89
+ ENV ENABLE_WEB_INTERFACE=true
90
+ ENV JAVA_HOME=/usr/lib/jvm/default-java
91
+ ENV PATH="${JAVA_HOME}/bin:/app/.venv/bin:$PATH"
92
+
93
+ # Health check
94
+ HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
95
+ CMD curl -f http://localhost:8000/health || exit 1
96
+
97
+ # Run the FastAPI server
98
+ # The module path is constructed to work with the /app/env structure
99
+ CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]
README.md CHANGED
@@ -1,10 +1,282 @@
 
 
 
 
 
 
 
 
1
  ---
2
- title: TestCaseGenerator Env
3
- emoji: 👀
4
- colorFrom: green
5
- colorTo: blue
6
  sdk: docker
7
  pinned: false
 
 
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+
3
+
4
+ # JUnit TestCase Generator
5
+
6
+ An RL environment where an agent generates JUnit 5 test cases for Java code.
7
+
8
+
9
  ---
10
+ title: Tcgenerator Environment Server
11
+ emoji: 🖱️
12
+ colorFrom: blue
13
+ colorTo: gray
14
  sdk: docker
15
  pinned: false
16
+ app_port: 8000
17
+ base_path: /web
18
+ tags:
19
+ - openenv
20
  ---
21
 
22
+
23
+ ## Key Features
24
+ - Executes generated tests in a sandboxed Java subprocess
25
+ - Reward based on:
26
+ - Pass rate (70%)
27
+ - Test quantity (10%)
28
+ - Edge case coverage (10%)
29
+ - Exception handling (10%)
30
+
31
+ ## Tasks
32
+ - Calculator (easy)
33
+ - SafeMath (medium)
34
+ - BankAccount (hard)
35
+
36
+ ## How it works
37
+ 1. Agent generates JUnit test code
38
+ 2. Environment compiles + runs tests
39
+ 3. Reward computed based on execution results
40
+
41
+ # Tcgenerator Environment
42
+
43
+ A simple test environment that echoes back messages. Perfect for testing the env APIs as well as demonstrating environment usage patterns.
44
+
45
+ ## Quick Start
46
+
47
+ The simplest way to use the Tcgenerator environment is through the `TcgeneratorEnv` class:
48
+
49
+ ```python
50
+ from tcgenerator import TcgeneratorAction, TcgeneratorEnv
51
+
52
+ try:
53
+ # Create environment from Docker image
54
+ tcgeneratorenv = TcgeneratorEnv.from_docker_image("tcgenerator-env:latest")
55
+
56
+ # Reset
57
+ result = tcgeneratorenv.reset()
58
+ print(f"Reset: {result.observation.echoed_message}")
59
+
60
+ # Send multiple messages
61
+ messages = ["Hello, World!", "Testing echo", "Final message"]
62
+
63
+ for msg in messages:
64
+ result = tcgeneratorenv.step(TcgeneratorAction(message=msg))
65
+ print(f"Sent: '{msg}'")
66
+ print(f" → Echoed: '{result.observation.echoed_message}'")
67
+ print(f" → Length: {result.observation.message_length}")
68
+ print(f" → Reward: {result.reward}")
69
+
70
+ finally:
71
+ # Always clean up
72
+ tcgeneratorenv.close()
73
+ ```
74
+
75
+ That's it! The `TcgeneratorEnv.from_docker_image()` method handles:
76
+ - Starting the Docker container
77
+ - Waiting for the server to be ready
78
+ - Connecting to the environment
79
+ - Container cleanup when you call `close()`
80
+
81
+ ## Building the Docker Image
82
+
83
+ Before using the environment, you need to build the Docker image:
84
+
85
+ ```bash
86
+ # From project root
87
+ docker build -t tcgenerator-env:latest -f server/Dockerfile .
88
+ ```
89
+
90
+ ## Deploying to Hugging Face Spaces
91
+
92
+ You can easily deploy your OpenEnv environment to Hugging Face Spaces using the `openenv push` command:
93
+
94
+ ```bash
95
+ # From the environment directory (where openenv.yaml is located)
96
+ openenv push
97
+
98
+ # Or specify options
99
+ openenv push --namespace my-org --private
100
+ ```
101
+
102
+ The `openenv push` command will:
103
+ 1. Validate that the directory is an OpenEnv environment (checks for `openenv.yaml`)
104
+ 2. Prepare a custom build for Hugging Face Docker space (enables web interface)
105
+ 3. Upload to Hugging Face (ensuring you're logged in)
106
+
107
+ ### Prerequisites
108
+
109
+ - Authenticate with Hugging Face: The command will prompt for login if not already authenticated
110
+
111
+ ### Options
112
+
113
+ - `--directory`, `-d`: Directory containing the OpenEnv environment (defaults to current directory)
114
+ - `--repo-id`, `-r`: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml)
115
+ - `--base-image`, `-b`: Base Docker image to use (overrides Dockerfile FROM)
116
+ - `--private`: Deploy the space as private (default: public)
117
+
118
+ ### Examples
119
+
120
+ ```bash
121
+ # Push to your personal namespace (defaults to username/env-name from openenv.yaml)
122
+ openenv push
123
+
124
+ # Push to a specific repository
125
+ openenv push --repo-id my-org/my-env
126
+
127
+ # Push with a custom base image
128
+ openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest
129
+
130
+ # Push as a private space
131
+ openenv push --private
132
+
133
+ # Combine options
134
+ openenv push --repo-id my-org/my-env --base-image custom-base:latest --private
135
+ ```
136
+
137
+ After deployment, your space will be available at:
138
+ `https://huggingface.co/spaces/<repo-id>`
139
+
140
+ The deployed space includes:
141
+ - **Web Interface** at `/web` - Interactive UI for exploring the environment
142
+ - **API Documentation** at `/docs` - Full OpenAPI/Swagger interface
143
+ - **Health Check** at `/health` - Container health monitoring
144
+ - **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions
145
+
146
+ ## Environment Details
147
+
148
+ ### Action
149
+ **TcgeneratorAction**: Contains a single field
150
+ - `message` (str) - The message to echo back
151
+
152
+ ### Observation
153
+ **TcgeneratorObservation**: Contains the echo response and metadata
154
+ - `echoed_message` (str) - The message echoed back
155
+ - `message_length` (int) - Length of the message
156
+ - `reward` (float) - Reward based on message length (length × 0.1)
157
+ - `done` (bool) - Always False for echo environment
158
+ - `metadata` (dict) - Additional info like step count
159
+
160
+ ### Reward
161
+ The reward is calculated as: `message_length × 0.1`
162
+ - "Hi" → reward: 0.2
163
+ - "Hello, World!" → reward: 1.3
164
+ - Empty message → reward: 0.0
165
+
166
+ ## Advanced Usage
167
+
168
+ ### Connecting to an Existing Server
169
+
170
+ If you already have a Tcgenerator environment server running, you can connect directly:
171
+
172
+ ```python
173
+ from tcgenerator import TcgeneratorEnv
174
+
175
+ # Connect to existing server
176
+ tcgeneratorenv = TcgeneratorEnv(base_url="<ENV_HTTP_URL_HERE>")
177
+
178
+ # Use as normal
179
+ result = tcgeneratorenv.reset()
180
+ result = tcgeneratorenv.step(TcgeneratorAction(message="Hello!"))
181
+ ```
182
+
183
+ Note: When connecting to an existing server, `tcgeneratorenv.close()` will NOT stop the server.
184
+
185
+ ### Using the Context Manager
186
+
187
+ The client supports context manager usage for automatic connection management:
188
+
189
+ ```python
190
+ from tcgenerator import TcgeneratorAction, TcgeneratorEnv
191
+
192
+ # Connect with context manager (auto-connects and closes)
193
+ with TcgeneratorEnv(base_url="http://localhost:8000") as env:
194
+ result = env.reset()
195
+ print(f"Reset: {result.observation.echoed_message}")
196
+ # Multiple steps with low latency
197
+ for msg in ["Hello", "World", "!"]:
198
+ result = env.step(TcgeneratorAction(message=msg))
199
+ print(f"Echoed: {result.observation.echoed_message}")
200
+ ```
201
+
202
+ The client uses WebSocket connections for:
203
+ - **Lower latency**: No HTTP connection overhead per request
204
+ - **Persistent session**: Server maintains your environment state
205
+ - **Efficient for episodes**: Better for many sequential steps
206
+
207
+ ### Concurrent WebSocket Sessions
208
+
209
+ The server supports multiple concurrent WebSocket connections. To enable this,
210
+ modify `server/app.py` to use factory mode:
211
+
212
+ ```python
213
+ # In server/app.py - use factory mode for concurrent sessions
214
+ app = create_app(
215
+ TcgeneratorEnvironment, # Pass class, not instance
216
+ TcgeneratorAction,
217
+ TcgeneratorObservation,
218
+ max_concurrent_envs=4, # Allow 4 concurrent sessions
219
+ )
220
+ ```
221
+
222
+ Then multiple clients can connect simultaneously:
223
+
224
+ ```python
225
+ from tcgenerator import TcgeneratorAction, TcgeneratorEnv
226
+ from concurrent.futures import ThreadPoolExecutor
227
+
228
+ def run_episode(client_id: int):
229
+ with TcgeneratorEnv(base_url="http://localhost:8000") as env:
230
+ result = env.reset()
231
+ for i in range(10):
232
+ result = env.step(TcgeneratorAction(message=f"Client {client_id}, step {i}"))
233
+ return client_id, result.observation.message_length
234
+
235
+ # Run 4 episodes concurrently
236
+ with ThreadPoolExecutor(max_workers=4) as executor:
237
+ results = list(executor.map(run_episode, range(4)))
238
+ ```
239
+
240
+ ## Development & Testing
241
+
242
+ ### Direct Environment Testing
243
+
244
+ Test the environment logic directly without starting the HTTP server:
245
+
246
+ ```bash
247
+ # From the server directory
248
+ python3 server/tcgenerator_environment.py
249
+ ```
250
+
251
+ This verifies that:
252
+ - Environment resets correctly
253
+ - Step executes actions properly
254
+ - State tracking works
255
+ - Rewards are calculated correctly
256
+
257
+ ### Running Locally
258
+
259
+ Run the server locally for development:
260
+
261
+ ```bash
262
+ uvicorn server.app:app --reload
263
+ ```
264
+
265
+ ## Project Structure
266
+
267
+ ```
268
+ tcgenerator/
269
+ ├── .dockerignore # Docker build exclusions
270
+ ├── __init__.py # Module exports
271
+ ├── README.md # This file
272
+ ├── openenv.yaml # OpenEnv manifest
273
+ ├── pyproject.toml # Project metadata and dependencies
274
+ ├── uv.lock # Locked dependencies (generated)
275
+ ├── client.py # TcgeneratorEnv client
276
+ ├── models.py # Action and Observation models
277
+ └── server/
278
+ ├── __init__.py # Server module exports
279
+ ├── tcgenerator_environment.py # Core environment logic
280
+ ├── app.py # FastAPI application (HTTP + WebSocket endpoints)
281
+ └── Dockerfile # Container image definition
282
+ ```
__init__.py ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ """Unittestcasegenerator Environment."""
8
+
9
+ from .client import UnittestcasegeneratorEnv
10
+ from .models import UnittestcasegeneratorAction, UnittestcasegeneratorObservation
11
+
12
+ __all__ = [
13
+ "UnittestcasegeneratorAction",
14
+ "UnittestcasegeneratorObservation",
15
+ "UnittestcasegeneratorEnv",
16
+ ]
client.py ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ """Unittestcasegenerator Environment Client."""
8
+
9
+ from typing import Dict
10
+
11
+ from openenv.core import EnvClient
12
+ from openenv.core.client_types import StepResult
13
+ from openenv.core.env_server.types import State
14
+
15
+ from .models import UnittestcasegeneratorAction, UnittestcasegeneratorObservation
16
+
17
+
18
+ class UnittestcasegeneratorEnv(
19
+ EnvClient[UnittestcasegeneratorAction, UnittestcasegeneratorObservation, State]
20
+ ):
21
+ """
22
+ Client for the Unittestcasegenerator Environment.
23
+
24
+ This client maintains a persistent WebSocket connection to the environment server,
25
+ enabling efficient multi-step interactions with lower latency.
26
+ Each client instance has its own dedicated environment session on the server.
27
+
28
+ Example:
29
+ >>> # Connect to a running server
30
+ >>> with UnittestcasegeneratorEnv(base_url="http://localhost:8000") as client:
31
+ ... result = client.reset()
32
+ ... print(result.observation.echoed_message)
33
+ ...
34
+ ... result = client.step(UnittestcasegeneratorAction(message="Hello!"))
35
+ ... print(result.observation.echoed_message)
36
+
37
+ Example with Docker:
38
+ >>> # Automatically start container and connect
39
+ >>> client = UnittestcasegeneratorEnv.from_docker_image("UnitTestCaseGenerator-env:latest")
40
+ >>> try:
41
+ ... result = client.reset()
42
+ ... result = client.step(UnittestcasegeneratorAction(message="Test"))
43
+ ... finally:
44
+ ... client.close()
45
+ """
46
+
47
+ def _step_payload(self, action: UnittestcasegeneratorAction) -> Dict:
48
+ """
49
+ Convert UnittestcasegeneratorAction to JSON payload for step message.
50
+
51
+ Args:
52
+ action: UnittestcasegeneratorAction instance
53
+
54
+ Returns:
55
+ Dictionary representation suitable for JSON encoding
56
+ """
57
+ return {
58
+ "message": action.message,
59
+ }
60
+
61
+ def _parse_result(self, payload: Dict) -> StepResult[UnittestcasegeneratorObservation]:
62
+ """
63
+ Parse server response into StepResult[UnittestcasegeneratorObservation].
64
+
65
+ Args:
66
+ payload: JSON response data from server
67
+
68
+ Returns:
69
+ StepResult with UnittestcasegeneratorObservation
70
+ """
71
+ obs_data = payload.get("observation", {})
72
+ observation = UnittestcasegeneratorObservation(
73
+ echoed_message=obs_data.get("echoed_message", ""),
74
+ message_length=obs_data.get("message_length", 0),
75
+ done=payload.get("done", False),
76
+ reward=payload.get("reward"),
77
+ metadata=obs_data.get("metadata", {}),
78
+ )
79
+
80
+ return StepResult(
81
+ observation=observation,
82
+ reward=payload.get("reward"),
83
+ done=payload.get("done", False),
84
+ )
85
+
86
+ def _parse_state(self, payload: Dict) -> State:
87
+ """
88
+ Parse server response into State object.
89
+
90
+ Args:
91
+ payload: JSON response from state request
92
+
93
+ Returns:
94
+ State object with episode_id and step_count
95
+ """
96
+ return State(
97
+ episode_id=payload.get("episode_id"),
98
+ step_count=payload.get("step_count", 0),
99
+ )
inference.py ADDED
@@ -0,0 +1,182 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import asyncio
2
+ import os
3
+ import textwrap
4
+ from typing import List, Optional
5
+
6
+ from openai import OpenAI
7
+
8
+ from server.UnitTestCaseGenerator_environment import (
9
+ UnittestcasegeneratorEnvironment,
10
+ UnittestcasegeneratorAction,
11
+ )
12
+
13
+ # ── CONFIG ─────────────────────────────────────────────
14
+
15
+ API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
16
+ API_BASE_URL = os.getenv("API_BASE_URL") or "https://router.huggingface.co/v1"
17
+ MODEL_NAME = os.getenv("MODEL_NAME") or "Qwen/Qwen2.5-72B-Instruct"
18
+
19
+ BENCHMARK = "unit_test_env"
20
+ MAX_STEPS = 1
21
+ SUCCESS_SCORE_THRESHOLD = 0.5
22
+
23
+ DIFFICULTIES = ["easy", "medium", "hard"]
24
+
25
+ # ── PROMPT ─────────────────────────────────────────────
26
+
27
+ SYSTEM_PROMPT = textwrap.dedent("""
28
+ You are an expert Java developer. You write JUnit 5 unit tests.
29
+
30
+ Rules:
31
+ - ALWAYS read the source code carefully before writing tests
32
+ - ALWAYS use the exact class name specified in the task hint
33
+ - NEVER write tests for a different class than what is given
34
+ - Use @Test annotation on every test method
35
+ - Always import: import org.junit.jupiter.api.Test;
36
+ - Always import: import static org.junit.jupiter.api.Assertions.*;
37
+ - Use assertEquals, assertTrue, assertFalse, assertThrows
38
+ - Reply with ONLY Java code, no explanation
39
+ """).strip()
40
+
41
+ # ── LOGGING (STRICT FORMAT) ─────────────────────────────
42
+
43
+ def log_start(task: str, env: str, model: str):
44
+ print(f"[START] task={task} env={env} model={model}", flush=True)
45
+
46
+
47
+ def log_step(step: int, action: str, reward: float, done: bool, error: Optional[str]):
48
+ error_val = error if error else "null"
49
+ print(
50
+ f"[STEP] step={step} action={action[:100]} reward={reward:.2f} done={str(done).lower()} error={error_val}",
51
+ flush=True,
52
+ )
53
+
54
+
55
+ def log_end(success: bool, steps: int, score: float, rewards: List[float]):
56
+ rewards_str = ",".join(f"{r:.2f}" for r in rewards)
57
+ print(
58
+ f"[END] success={str(success).lower()} steps={steps} score={score:.2f} rewards={rewards_str}",
59
+ flush=True,
60
+ )
61
+
62
+ # ── MODEL CALL ─────────────────────────────────────────────
63
+
64
+ def get_tests_from_model(
65
+ client: OpenAI,
66
+ source_code: str,
67
+ task_hint: str,
68
+ feedback: Optional[str],
69
+ ) -> str:
70
+
71
+ prompt = f"""
72
+ {task_hint}
73
+
74
+ Source code:
75
+ {source_code}
76
+
77
+ Feedback: {feedback or "None"}
78
+
79
+ Write JUnit 5 tests:
80
+ """
81
+
82
+ try:
83
+ completion = client.chat.completions.create(
84
+ model=MODEL_NAME,
85
+ messages=[
86
+ {"role": "system", "content": SYSTEM_PROMPT},
87
+ {"role": "user", "content": prompt},
88
+ ],
89
+ temperature=0.3,
90
+ max_tokens=800,
91
+ )
92
+
93
+ text = (completion.choices[0].message.content or "").strip()
94
+ return text.replace("```java", "").replace("```", "").strip()
95
+
96
+ except Exception:
97
+ return "public class PlaceholderTest {}"
98
+
99
+ # ── MAIN ─────────────────────────────────────────────
100
+
101
+ async def main():
102
+ client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
103
+ from client import TcgeneratorEnv
104
+
105
+ all_rewards = []
106
+
107
+ for difficulty in DIFFICULTIES:
108
+ env = UnittestcasegeneratorEnvironment()
109
+
110
+ rewards: List[float] = []
111
+ steps_taken = 0
112
+ feedback = None
113
+
114
+ log_start(task=difficulty, env=BENCHMARK, model=MODEL_NAME)
115
+
116
+ try:
117
+ result = env.reset(difficulty=difficulty)
118
+
119
+ source_code = result.source_code
120
+ task_hint = result.task_hint
121
+
122
+ for step in range(1, MAX_STEPS + 1):
123
+ if result.done:
124
+ break
125
+
126
+ action = get_tests_from_model(client, source_code, task_hint, feedback)
127
+
128
+ result = env.step(UnittestcasegeneratorAction(test_code=action))
129
+
130
+ reward = result.reward or 0.0
131
+ done = result.done
132
+ error = getattr(result, "error", None)
133
+
134
+ feedback = f"Passed {result.passed}/{result.total}"
135
+
136
+ rewards.append(reward)
137
+ steps_taken = step
138
+
139
+ log_step(step, action, reward, done, error)
140
+
141
+ if done:
142
+ break
143
+
144
+ score = max(rewards) if rewards else 0.0
145
+ score = min(max(score, 0.0), 1.0)
146
+
147
+ success = score >= SUCCESS_SCORE_THRESHOLD
148
+
149
+ finally:
150
+ try:
151
+ if hasattr(env, "close"):
152
+ env.close()
153
+ except Exception:
154
+ pass
155
+
156
+ log_end(success, steps_taken, score, rewards)
157
+
158
+ all_rewards.append(score)
159
+
160
+ # ── FINAL SUMMARY ─────────────────────────────
161
+
162
+ if len(all_rewards) == 3:
163
+ easy, medium, hard = all_rewards
164
+ avg = (easy + medium + hard) / 3
165
+
166
+ print("\n=== FINAL RESULT ===")
167
+ print(f"Model: {MODEL_NAME}")
168
+ print(f"Benchmark: {BENCHMARK}\n")
169
+
170
+ print(f"Easy : {easy:.2f}")
171
+ print(f"Medium : {medium:.2f}")
172
+ print(f"Hard : {hard:.2f}\n")
173
+
174
+ print(f"Average: {avg:.2f}")
175
+ print(f"Success: {'Yes' if avg >= SUCCESS_SCORE_THRESHOLD else 'No'}")
176
+ print("=====================")
177
+
178
+
179
+ # ── RUN ─────────────────────────────────────────────
180
+
181
+ if __name__ == "__main__":
182
+ asyncio.run(main())
models.py ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ """
8
+ Data models for the Unittestcasegenerator Environment.
9
+
10
+ The UnitTestCaseGenerator environment is a simple test environment that echoes back messages.
11
+ """
12
+ from typing import Optional
13
+ from openenv.core.env_server.types import Action, Observation
14
+ from pydantic import Field
15
+
16
+
17
+ class UnittestcasegeneratorAction(Action):
18
+ """Action for the Unittestcasegenerator environment - just a message to echo."""
19
+
20
+ test_code: str = Field(..., description="Java JUnit test code")
21
+
22
+
23
+ class UnittestcasegeneratorObservation(Observation):
24
+ """Observation from the Unittestcasegenerator environment - the echoed message."""
25
+
26
+ source_code: str = Field(..., description="Java source code to test")
27
+ task_hint: str = Field(..., description="Instructions for writing tests")
28
+ passed: int = Field(default=0, description="Tests passed")
29
+ failed: int = Field(default=0, description="Tests failed")
30
+ total: int = Field(default=0, description="Total tests run")
31
+ error: Optional[str] = Field(default=None, description="Error message if any")
32
+ reward: float = Field(default=0.0, description="Score 0.0 to 1.0")
33
+ done: bool = Field(default=False, description="Episode finished?")
34
+ metadata: Optional[dict] = Field(default=None)
openenv.yaml ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ spec_version: 1
2
+ name: UnitTestCaseGenerator
3
+ type: space
4
+ runtime: fastapi
5
+ app: server.app:app
6
+ port: 8000
7
+
pyproject.toml ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ [build-system]
8
+ requires = ["setuptools>=45", "wheel"]
9
+ build-backend = "setuptools.build_meta"
10
+
11
+ [project]
12
+ name = "openenv-UnitTestCaseGenerator"
13
+ version = "0.1.0"
14
+ description = "Unittestcasegenerator environment for OpenEnv"
15
+ requires-python = ">=3.10"
16
+ dependencies = [
17
+ # Core OpenEnv runtime (provides FastAPI server + HTTP client types)
18
+ # install from github
19
+ # "openenv-core[core] @ git+https://github.com/meta-pytorch/OpenEnv.git",
20
+ "openenv-core[core]>=0.2.2",
21
+ # Environment-specific dependencies
22
+ # Add all dependencies needed for your environment here
23
+ # Examples:
24
+ # "numpy>=1.19.0",
25
+ # "torch>=2.0.0",
26
+ # "gymnasium>=0.29.0",
27
+ # "openspiel>=1.0.0",
28
+ # "smolagents>=1.22.0,<2",
29
+ ]
30
+
31
+ [project.optional-dependencies]
32
+ dev = [
33
+ "pytest>=8.0.0",
34
+ "pytest-cov>=4.0.0",
35
+ ]
36
+
37
+ [project.scripts]
38
+ # Server entry point - enables running via: uv run --project . server
39
+ # or: python -m UnitTestCaseGenerator.server.app
40
+ server = "UnitTestCaseGenerator.server.app:main"
41
+
42
+ [tool.setuptools]
43
+ include-package-data = true
44
+ packages = ["UnitTestCaseGenerator", "UnitTestCaseGenerator.server"]
45
+ package-dir = { "UnitTestCaseGenerator" = ".", "UnitTestCaseGenerator.server" = "server" }
server/UnitTestCaseGenerator_environment.py ADDED
@@ -0,0 +1,458 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+ """
7
+ Unittestcasegenerator Environment Implementation.
8
+
9
+ An RL environment where an LLM agent writes JUnit 5 unit tests
10
+ for given Java classes. Tasks range from simple pure functions
11
+ to stateful classes requiring exception testing.
12
+ """
13
+
14
+ import os
15
+ import re
16
+ import subprocess
17
+ import shutil
18
+ import tempfile
19
+ import textwrap
20
+ import json
21
+ from typing import Any, Optional
22
+ from uuid import uuid4
23
+
24
+ from openenv.core.env_server.interfaces import Environment
25
+ from openenv.core.env_server.types import State
26
+
27
+ try:
28
+ from ..models import UnittestcasegeneratorAction, UnittestcasegeneratorObservation
29
+ except ImportError:
30
+ from models import UnittestcasegeneratorAction, UnittestcasegeneratorObservation
31
+
32
+
33
+ # ─────────────────────────────────────────────
34
+ # TASKS
35
+ # ─────────────────────────────────────────────
36
+
37
+ TASKS = {
38
+ "easy": {
39
+ "source_code": textwrap.dedent("""
40
+ public class Calculator {
41
+ public int add(int a, int b) {
42
+ return a + b;
43
+ }
44
+ public boolean isPalindrome(String s) {
45
+ String clean = s.toLowerCase().replace(" ", "");
46
+ String reversed = new StringBuilder(clean).reverse().toString();
47
+ return clean.equals(reversed);
48
+ }
49
+ public double celsiusToFahrenheit(double c) {
50
+ return c * 9.0 / 5.0 + 32;
51
+ }
52
+ }
53
+ """).strip(),
54
+ "source_class": "Calculator",
55
+ "test_class": "CalculatorTest",
56
+ "task_hint": (
57
+ "Write JUnit 5 tests for Calculator class. "
58
+ "1) Class name MUST be CalculatorTest. "
59
+ "2) Use @Test annotation on every test method. "
60
+ "3) Import: import org.junit.jupiter.api.Test; "
61
+ "4) Import: import static org.junit.jupiter.api.Assertions.*; "
62
+ "5) Write at least 6 test methods. "
63
+ "6) Test add(), isPalindrome(), celsiusToFahrenheit(). "
64
+ "Reply with ONLY the Java code."
65
+ ),
66
+ "expected_min_tests": 6,
67
+ "requires_edge_cases": False,
68
+ "requires_exception": False,
69
+ },
70
+
71
+ "medium": {
72
+ "source_code": textwrap.dedent("""
73
+ public class SafeMath {
74
+ public double safeDivide(double a, double b) {
75
+ if (b == 0)
76
+ throw new IllegalArgumentException("Cannot divide by zero");
77
+ return a / b;
78
+ }
79
+ public int getFirstElement(int[] arr) {
80
+ if (arr.length == 0)
81
+ throw new IndexOutOfBoundsException("Array is empty");
82
+ return arr[0];
83
+ }
84
+ public int parsePositiveInt(String s) {
85
+ int val = Integer.parseInt(s);
86
+ if (val <= 0)
87
+ throw new IllegalArgumentException("Must be positive");
88
+ return val;
89
+ }
90
+ }
91
+ """).strip(),
92
+ "source_class": "SafeMath",
93
+ "test_class": "SafeMathTest",
94
+ "task_hint": (
95
+ "Write JUnit 5 tests for SafeMath class. "
96
+ "1) Class name MUST be SafeMathTest. "
97
+ "2) Use @Test annotation on every test method. "
98
+ "3) Import: import org.junit.jupiter.api.Test; "
99
+ "4) Import: import static org.junit.jupiter.api.Assertions.*; "
100
+ "5) Use assertThrows() for exception testing. "
101
+ "6) Test edge cases: empty array, zero, negative. "
102
+ "7) Write at least 8 test methods. "
103
+ "Reply with ONLY the Java code."
104
+ ),
105
+ "expected_min_tests": 8,
106
+ "requires_edge_cases": True,
107
+ "requires_exception": True,
108
+ },
109
+
110
+ "hard": {
111
+ "source_code": textwrap.dedent("""
112
+ import java.util.ArrayList;
113
+ import java.util.List;
114
+ public class BankAccount {
115
+ private String owner;
116
+ private double balance;
117
+ private List<String> transactions;
118
+ public BankAccount(String owner, double balance) {
119
+ this.owner = owner;
120
+ this.balance = balance;
121
+ this.transactions = new ArrayList<>();
122
+ }
123
+ public double deposit(double amount) {
124
+ if (amount <= 0)
125
+ throw new IllegalArgumentException("Deposit must be positive");
126
+ this.balance += amount;
127
+ this.transactions.add("deposit:" + amount);
128
+ return this.balance;
129
+ }
130
+ public double withdraw(double amount) {
131
+ if (amount <= 0)
132
+ throw new IllegalArgumentException("Withdrawal must be positive");
133
+ if (amount > this.balance)
134
+ throw new IllegalArgumentException("Insufficient funds");
135
+ this.balance -= amount;
136
+ this.transactions.add("withdraw:" + amount);
137
+ return this.balance;
138
+ }
139
+ public int getTransactionCount() { return this.transactions.size(); }
140
+ public double getBalance() { return this.balance; }
141
+ public String getOwner() { return this.owner; }
142
+ }
143
+ """).strip(),
144
+ "source_class": "BankAccount",
145
+ "test_class": "BankAccountTest",
146
+ "task_hint": (
147
+ "Write JUnit 5 tests for BankAccount class. "
148
+ "1) Class name MUST be BankAccountTest. "
149
+ "2) Use @Test annotation on every test method. "
150
+ "3) Import: import org.junit.jupiter.api.Test; "
151
+ "4) Import: import static org.junit.jupiter.api.Assertions.*; "
152
+ "5) Use assertThrows() for exception testing. "
153
+ "6) Test deposit, withdraw, getBalance, getTransactionCount, getOwner. "
154
+ "7) Test exceptions: negative deposit, withdraw, insufficient funds. "
155
+ "8) Write at least 10 test methods. "
156
+ "Reply with ONLY the Java code."
157
+ ),
158
+ "expected_min_tests": 10,
159
+ "requires_edge_cases": True,
160
+ "requires_exception": True,
161
+ },
162
+ }
163
+
164
+ DIFFICULTIES = ("easy", "medium", "hard")
165
+
166
+
167
+ # ─────────────────────────────────────────────
168
+ # JAVA DETECTION
169
+ # ─────────────────────────────────────────────
170
+
171
+ def find_java():
172
+ for cmd in ["java", "/usr/bin/java", "/usr/local/bin/java"]:
173
+ try:
174
+ result = subprocess.run([cmd, "-version"], capture_output=True, text=True)
175
+ if result.returncode == 0:
176
+ print(json.dumps({"event": "debug", "type": "java_found", "path": cmd}), flush=True)
177
+ return cmd
178
+ except FileNotFoundError:
179
+ continue
180
+ return None
181
+
182
+
183
+ def find_javac():
184
+ for cmd in ["javac", "/usr/bin/javac", "/usr/local/bin/javac"]:
185
+ try:
186
+ result = subprocess.run([cmd, "-version"], capture_output=True, text=True)
187
+ if result.returncode == 0:
188
+ print(json.dumps({"event": "debug", "type": "javac_found", "path": cmd}), flush=True)
189
+ return cmd
190
+ except FileNotFoundError:
191
+ continue
192
+ return None
193
+
194
+
195
+ def find_junit_jar():
196
+ env_path = os.getenv("JUNIT_JAR")
197
+ if env_path and os.path.exists(env_path):
198
+ print(json.dumps({"event": "debug", "type": "junit_found", "path": env_path}), flush=True)
199
+ return env_path
200
+
201
+ candidates = [
202
+ "/app/junit-platform-console-standalone.jar",
203
+ "/app/env/junit-platform-console-standalone.jar",
204
+ os.path.join(os.path.dirname(__file__), "..", "junit-platform-console-standalone.jar"),
205
+ os.path.join(os.getcwd(), "junit-platform-console-standalone.jar"),
206
+ os.path.join(os.getcwd(), "libs", "junit-platform-console-standalone.jar"),
207
+ # "/Users/vidhikoul/Desktop/UnitTestCaseGenerator/tcgenerator/junit-platform-console-standalone.jar",
208
+ ]
209
+
210
+ for path in candidates:
211
+ abs_path = os.path.abspath(path)
212
+ if os.path.exists(abs_path):
213
+ print(json.dumps({"event": "debug", "type": "junit_found", "path": abs_path}), flush=True)
214
+ return abs_path
215
+
216
+ print(json.dumps({"event": "debug", "type": "junit_not_found", "tried": candidates}), flush=True)
217
+ return None
218
+
219
+
220
+ # ─────────────────────────────────────────────
221
+ # JUNIT SANDBOX
222
+ # ─────────────────────────────────────────────
223
+
224
+ def run_junit_tests(source_code: str, test_code: str,
225
+ source_class: str, test_class: str,
226
+ timeout: int = 30):
227
+
228
+ java = find_java()
229
+ javac = find_javac()
230
+ junit = find_junit_jar()
231
+
232
+ if not java or not javac:
233
+ return 0, 0, 0, "Java not found!"
234
+ if not junit:
235
+ return 0, 0, 0, "JUnit jar not found!"
236
+
237
+ tmpdir = tempfile.mkdtemp()
238
+
239
+ try:
240
+ source_file = os.path.join(tmpdir, f"{source_class}.java")
241
+ test_file = os.path.join(tmpdir, f"{test_class}.java")
242
+
243
+ with open(source_file, "w") as f:
244
+ f.write(source_code)
245
+ with open(test_file, "w") as f:
246
+ f.write(test_code)
247
+
248
+ compile_result = subprocess.run(
249
+ [javac, "-cp", junit, source_file, test_file],
250
+ capture_output=True, text=True,
251
+ cwd=tmpdir, timeout=30,
252
+ )
253
+
254
+ if compile_result.returncode != 0:
255
+ err = compile_result.stderr[:500]
256
+ print(json.dumps({"event": "debug", "type": "compile_error", "message": err[:200]}), flush=True)
257
+ return 0, 0, 0, f"Compile error: {err}"
258
+
259
+ run_result = subprocess.run(
260
+ [java, "-jar", junit, "-cp", tmpdir,
261
+ "--select-class", test_class, "--details", "summary"],
262
+ capture_output=True, text=True,
263
+ cwd=tmpdir, timeout=timeout,
264
+ )
265
+
266
+ output = run_result.stdout + run_result.stderr
267
+ print(json.dumps({"event": "debug", "type": "junit_output", "message": output[:300]}), flush=True)
268
+
269
+ passed, failed = parse_junit_output(output)
270
+ total = passed + failed
271
+ error = None if passed > 0 else output[-400:]
272
+
273
+ return passed, failed, total, error
274
+
275
+ except subprocess.TimeoutExpired:
276
+ return 0, 0, 0, "Timeout"
277
+ except Exception as e:
278
+ return 0, 0, 0, str(e)
279
+ finally:
280
+ shutil.rmtree(tmpdir, ignore_errors=True)
281
+
282
+
283
+ def parse_junit_output(output: str):
284
+ passed = failed = 0
285
+
286
+ p = re.findall(r'(\d+)\s+tests?\s+successful', output, re.IGNORECASE)
287
+ if p:
288
+ passed = int(p[0])
289
+
290
+ f = re.findall(r'(\d+)\s+tests?\s+failed', output, re.IGNORECASE)
291
+ if f:
292
+ failed = int(f[0])
293
+
294
+ if passed == 0 and failed == 0:
295
+ passed = len(re.findall(r'\[\s*OK\s*\]', output))
296
+ failed = len(re.findall(r'\[\s*FAILED\s*\]', output))
297
+
298
+ return passed, failed
299
+
300
+
301
+ def compute_reward(passed, total, test_code, task_cfg):
302
+ if total == 0:
303
+ return 0.0
304
+
305
+ base = (passed / total) * 0.7
306
+ quantity_bonus = 0.1 if total >= task_cfg["expected_min_tests"] else 0.0
307
+
308
+ edge_bonus = 0.0
309
+ if task_cfg["requires_edge_cases"]:
310
+ edge_keywords = ["empty", "zero", "null", "negative", "0", "[]", '""']
311
+ if any(kw in test_code.lower() for kw in edge_keywords):
312
+ edge_bonus = 0.1
313
+
314
+ exception_bonus = 0.0
315
+ if task_cfg["requires_exception"]:
316
+ if "assertthrows" in test_code.lower():
317
+ exception_bonus = 0.1
318
+
319
+ return round(min(base + quantity_bonus + edge_bonus + exception_bonus, 1.0), 3)
320
+
321
+
322
+ # ─────────────────────────────────────────────
323
+ # ENVIRONMENT CLASS
324
+ # ─────────────────────────────────────────────
325
+
326
+ class UnittestcasegeneratorEnvironment(Environment):
327
+ """
328
+ JUnit Test Case Generator Environment.
329
+
330
+ An RL environment where an LLM agent learns to write JUnit 5 unit
331
+ tests for Java classes. The agent receives Java source code and must
332
+ produce test code that compiles and passes.
333
+
334
+ Difficulty levels:
335
+ easy: Simple pure functions — Calculator class (6+ tests)
336
+ medium: Exception handling — SafeMath class (8+ tests)
337
+ hard: Stateful class — BankAccount class (10+ tests)
338
+
339
+ Episode flow:
340
+ 1. reset(difficulty="easy"|"medium"|"hard")
341
+ 2. agent reads source_code + task_hint from observation
342
+ 3. agent calls step(action) with JUnit test code
343
+ 4. environment compiles + runs tests → returns reward (0.0–1.0)
344
+ 5. repeat up to 6 steps or until reward >= 0.95
345
+ """
346
+
347
+ SUPPORTS_CONCURRENT_SESSIONS: bool = True
348
+
349
+ def __init__(self):
350
+ """Initialize the UnitTestCaseGenerator environment."""
351
+ self._state = State(episode_id=str(uuid4()), step_count=0)
352
+ self._episode_count = 0
353
+ self._step_count = 0
354
+ self._max_steps = 6
355
+ self._best_reward = 0.0
356
+ self._difficulty = "easy"
357
+ self._task_cfg = TASKS["easy"]
358
+
359
+ def reset(
360
+ self,
361
+ difficulty: Optional[str] = None,
362
+ episode_id: Optional[str] = None,
363
+ **kwargs: Any,
364
+ ) -> UnittestcasegeneratorObservation:
365
+ """
366
+ Reset the environment with a new task.
367
+
368
+ Args:
369
+ difficulty: "easy" | "medium" | "hard"
370
+ Auto-cycles easy→medium→hard if not provided.
371
+ episode_id: Optional custom episode ID.
372
+
373
+ Returns:
374
+ UnittestcasegeneratorObservation with source_code and task_hint.
375
+ """
376
+ if difficulty not in DIFFICULTIES:
377
+ difficulty = DIFFICULTIES[self._episode_count % 3]
378
+
379
+ self._episode_count += 1
380
+ self._difficulty = difficulty
381
+ self._task_cfg = TASKS[difficulty]
382
+ self._step_count = 0
383
+ self._best_reward = 0.0
384
+ self._state = State(
385
+ episode_id=episode_id or str(uuid4()),
386
+ step_count=0
387
+ )
388
+
389
+ print(json.dumps({
390
+ "event": "debug",
391
+ "type": "reset",
392
+ "difficulty": difficulty,
393
+ "source_class": self._task_cfg["source_class"],
394
+ }), flush=True)
395
+
396
+ return UnittestcasegeneratorObservation(
397
+ source_code = self._task_cfg["source_code"],
398
+ task_hint = self._task_cfg["task_hint"],
399
+ passed=0, failed=0, total=0,
400
+ error=None, reward=0.0, done=False,
401
+ metadata={
402
+ "difficulty": difficulty,
403
+ "episode_id": self._state.episode_id,
404
+ "message": (
405
+ f"New {difficulty} task loaded. "
406
+ f"Write JUnit 5 tests for {self._task_cfg['source_class']} class."
407
+ ),
408
+ }
409
+ )
410
+
411
+ def step(self, action: UnittestcasegeneratorAction) -> UnittestcasegeneratorObservation:
412
+ """
413
+ Execute one step — compile and run the agent's JUnit test code.
414
+
415
+ Args:
416
+ action: UnittestcasegeneratorAction with test_code field.
417
+
418
+ Returns:
419
+ UnittestcasegeneratorObservation with passed/failed counts and reward.
420
+ """
421
+ self._state.step_count += 1
422
+ self._step_count += 1
423
+ done = self._step_count >= self._max_steps
424
+
425
+ passed, failed, total, error = run_junit_tests(
426
+ source_code = self._task_cfg["source_code"],
427
+ test_code = action.test_code,
428
+ source_class = self._task_cfg["source_class"],
429
+ test_class = self._task_cfg["test_class"],
430
+ )
431
+
432
+ reward = compute_reward(passed, total, action.test_code, self._task_cfg)
433
+ self._best_reward = max(self._best_reward, reward)
434
+
435
+ if reward >= 0.95:
436
+ done = True
437
+
438
+ print(json.dumps({
439
+ "event": "debug",
440
+ "type": "step_result",
441
+ "passed": passed,
442
+ "failed": failed,
443
+ "total": total,
444
+ "reward": reward,
445
+ "done": done,
446
+ }), flush=True)
447
+
448
+ return UnittestcasegeneratorObservation(
449
+ source_code = self._task_cfg["source_code"],
450
+ task_hint = self._task_cfg["task_hint"],
451
+ passed=passed, failed=failed, total=total,
452
+ error=error, reward=reward, done=done,
453
+ )
454
+
455
+ @property
456
+ def state(self) -> State:
457
+ """Get current episode state."""
458
+ return self._state
server/__init__.py ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ """Unittestcasegenerator environment server components."""
8
+
9
+ from .UnitTestCaseGenerator_environment import UnittestcasegeneratorEnvironment
10
+
11
+ __all__ = ["UnittestcasegeneratorEnvironment"]
server/app.py ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copyright (c) Meta Platforms, Inc. and affiliates.
2
+ # All rights reserved.
3
+ #
4
+ # This source code is licensed under the BSD-style license found in the
5
+ # LICENSE file in the root directory of this source tree.
6
+
7
+ """
8
+ FastAPI application for the Unittestcasegenerator Environment.
9
+
10
+ This module creates an HTTP server that exposes the UnittestcasegeneratorEnvironment
11
+ over HTTP and WebSocket endpoints, compatible with EnvClient.
12
+
13
+ Endpoints:
14
+ - POST /reset: Reset the environment
15
+ - POST /step: Execute an action
16
+ - GET /state: Get current environment state
17
+ - GET /schema: Get action/observation schemas
18
+ - WS /ws: WebSocket endpoint for persistent sessions
19
+
20
+ Usage:
21
+ # Development (with auto-reload):
22
+ uvicorn server.app:app --reload --host 0.0.0.0 --port 8000
23
+
24
+ # Production:
25
+ uvicorn server.app:app --host 0.0.0.0 --port 8000 --workers 4
26
+
27
+ # Or run directly:
28
+ python -m server.app
29
+ """
30
+
31
+ try:
32
+ from openenv.core.env_server.http_server import create_app
33
+ except Exception as e: # pragma: no cover
34
+ raise ImportError(
35
+ "openenv is required for the web interface. Install dependencies with '\n uv sync\n'"
36
+ ) from e
37
+
38
+ try:
39
+ from models import UnittestcasegeneratorAction, UnittestcasegeneratorObservation
40
+ from .UnitTestCaseGenerator_environment import UnittestcasegeneratorEnvironment
41
+ except ModuleNotFoundError:
42
+ from models import UnittestcasegeneratorAction, UnittestcasegeneratorObservation
43
+ from server.UnitTestCaseGenerator_environment import UnittestcasegeneratorEnvironment
44
+
45
+
46
+ # Create the app with web interface and README integration
47
+ app = create_app(
48
+ UnittestcasegeneratorEnvironment,
49
+ UnittestcasegeneratorAction,
50
+ UnittestcasegeneratorObservation,
51
+ env_name="UnitTestCaseGenerator",
52
+ max_concurrent_envs=1, # increase this number to allow more concurrent WebSocket sessions
53
+ )
54
+
55
+
56
+ def main(host: str = "0.0.0.0", port: int = 8000):
57
+ """
58
+ Entry point for direct execution via uv run or python -m.
59
+
60
+ This function enables running the server without Docker:
61
+ uv run --project . server
62
+ uv run --project . server --port 8001
63
+ python -m UnitTestCaseGenerator.server.app
64
+
65
+ Args:
66
+ host: Host address to bind to (default: "0.0.0.0")
67
+ port: Port number to listen on (default: 8000)
68
+
69
+ For production deployments, consider using uvicorn directly with
70
+ multiple workers:
71
+ uvicorn UnitTestCaseGenerator.server.app:app --workers 4
72
+ """
73
+ import uvicorn
74
+
75
+ uvicorn.run(app, host=host, port=port)
76
+
77
+
78
+ if __name__ == "__main__":
79
+ import argparse
80
+
81
+ parser = argparse.ArgumentParser()
82
+ parser.add_argument("--port", type=int, default=8000)
83
+ args = parser.parse_args()
84
+ main(port=args.port)
server/requirements.txt ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ openenv[core]>=0.2.0
2
+ fastapi>=0.115.0
3
+ uvicorn>=0.24.0
4
+
5
+
6
+
uv.lock ADDED
The diff for this file is too large to render. See raw diff