File size: 2,631 Bytes
f0dfeb7
520edbd
 
 
 
 
 
 
 
f0dfeb7
520edbd
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
---
license: apache-2.0
base_model: convaiinnovations/laya
tags:
- reinforcement-learning
- highway-env
- autonomous-driving
- safety-shield
- laya
---

# CS654 Final Project: model weights

These are the final weights for a CS 654 (Reinforcement Learning) course project. The project compares DQN, Laya, and a Laya-shielded DQN on [highway-env](https://highway-env.farama.org/) (`highway-fast-v0`).

| Folder | Contents |
|---|---|
| `laya-highway/` | [Laya](https://huggingface.co/convaiinnovations/laya), fine-tuned to judge driving actions as safe or unsafe and to choose actions |

## Download

```bash
hf download adamlizerbram/cs654-final-project --include "laya-highway/*" --local-dir models/final  # -> models/final/laya-highway
```

```python
import laya
agent = laya.load("adamlizerbram/cs654-final-project", subfolder="laya-highway")
```

## laya-highway

One checkpoint answers two questions about a text description of the highway around the ego car:

- **Safety** (`noul`): will a proposed action lead to a collision or a gap under 5 m within 3 s? This yes/no answer drives the safety shield.
- **Action** (`choice`): which legal action is safest while still making good progress? This is the Laya-alone driving policy.

**Data.** Labels come from simulation, not humans. For each visited state and each legal action, a copy of the simulator applies the action and then keeps lane and speed for 2 more steps.
- An action is **unsafe** if the copy crashes or the same-lane gap drops below 5 m.
- The **action label** is the safe action with the most forward progress.
- Train episodes use seeds 0–299 and validation episodes use seeds 10000–10049. Evaluation seeds (100000+) were never used for training or tuning.

**Training.** `laya-train` from laya 0.4.30 on an A100:
- 4 epochs with the default `rlcd` objective, `--shuffle-options`, and an effective batch of 64
- 16,805 train items and 400 calibration items
- Temperature scaling fitted T = 5.0 for both question types

**Validation results** (before → after fine-tuning):

| Question | Items | Accuracy | ECE |
|---|---|---|---|
| Safety | 2,754 | 0.503 → 0.947 | 0.303 → 0.025 |
| Action | 697 | 0.144 → 0.891 | 0.586 → 0.048 |

The fine-tuned P(unsafe) is nearly binary: 99% of predictions fall below 0.1 or at 0.9 and above, and both clusters are well calibrated. 6.2% of truly unsafe actions are scored below 0.1.

**Limitations.**
- The labels imitate a 3-second lookahead rule, not an optimal driver.
- `highway-fast-v0` uses coarse 5 Hz physics.
- The model reads a fixed text format, so other state descriptions are out of distribution.