Reinforcement Learning
Transformers
Safetensors
qwen3.5
playpen
score
self-correction
dialogue-games
Instructions to use Valhari14/Learning-in-Interaction with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Valhari14/Learning-in-Interaction with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Valhari14/Learning-in-Interaction", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,6 +1,18 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
| 4 |
# Learning in Interaction using ArCHer
|
| 5 |
|
| 6 |
> Interactive fine-tuning and alignment experiments using LoRA, supervised learning, and reinforcement-learning-based adaptation on LLaMA-3.1-8B-Instruct.
|
|
@@ -20,7 +32,7 @@ The goal of this work is to explore interactive learning paradigms for large lan
|
|
| 20 |
- Preference and reward-based optimization
|
| 21 |
- Future GRPO / ArCHer reinforcement learning methods
|
| 22 |
|
| 23 |
-
The experiments are built on top of Meta's **LLaMA-3.1-8B-Instruct** model using the Hugging Face.
|
| 24 |
|
| 25 |
---
|
| 26 |
|
|
@@ -77,7 +89,6 @@ Llama-3.1-8B-Instruct/
|
|
| 77 |
|
| 78 |
---
|
| 79 |
|
| 80 |
-
|
| 81 |
## Loading a Checkpoint
|
| 82 |
|
| 83 |
### Using Transformers + PEFT
|
|
@@ -107,19 +118,49 @@ tokenizer = AutoTokenizer.from_pretrained(base_model)
|
|
| 107 |
|
| 108 |
---
|
| 109 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
## Notes
|
| 111 |
|
| 112 |
- Only selected checkpoints and milestone runs are stored.
|
| 113 |
- Large files are managed using Git LFS.
|
| 114 |
- Intermediate checkpoints are omitted to reduce repository size.
|
|
|
|
| 115 |
|
| 116 |
---
|
| 117 |
|
| 118 |
## Planned Work
|
| 119 |
|
| 120 |
-
|
| 121 |
|
| 122 |
- GRPO-based reinforcement learning
|
|
|
|
| 123 |
- Multi-turn interaction benchmarks
|
| 124 |
- Automated best-checkpoint selection
|
| 125 |
- Evaluation pipelines and benchmarking suites
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: transformers
|
| 4 |
+
base_model: meta-llama/Llama-3.1-8B-Instruct
|
| 5 |
+
tags:
|
| 6 |
+
- llama
|
| 7 |
+
- lora
|
| 8 |
+
- sft
|
| 9 |
+
- alignment
|
| 10 |
+
- reinforcement-learning
|
| 11 |
+
- peft
|
| 12 |
+
- transformers
|
| 13 |
+
- archer
|
| 14 |
---
|
| 15 |
+
|
| 16 |
# Learning in Interaction using ArCHer
|
| 17 |
|
| 18 |
> Interactive fine-tuning and alignment experiments using LoRA, supervised learning, and reinforcement-learning-based adaptation on LLaMA-3.1-8B-Instruct.
|
|
|
|
| 32 |
- Preference and reward-based optimization
|
| 33 |
- Future GRPO / ArCHer reinforcement learning methods
|
| 34 |
|
| 35 |
+
The experiments are built on top of Meta's **LLaMA-3.1-8B-Instruct** model using the Hugging Face ecosystem.
|
| 36 |
|
| 37 |
---
|
| 38 |
|
|
|
|
| 89 |
|
| 90 |
---
|
| 91 |
|
|
|
|
| 92 |
## Loading a Checkpoint
|
| 93 |
|
| 94 |
### Using Transformers + PEFT
|
|
|
|
| 118 |
|
| 119 |
---
|
| 120 |
|
| 121 |
+
## Example Inference
|
| 122 |
+
|
| 123 |
+
```python
|
| 124 |
+
inputs = tokenizer(
|
| 125 |
+
"Explain reinforcement learning in simple terms.",
|
| 126 |
+
return_tensors="pt"
|
| 127 |
+
).to(model.device)
|
| 128 |
+
|
| 129 |
+
outputs = model.generate(
|
| 130 |
+
**inputs,
|
| 131 |
+
max_new_tokens=128
|
| 132 |
+
)
|
| 133 |
+
|
| 134 |
+
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 135 |
+
```
|
| 136 |
+
|
| 137 |
+
---
|
| 138 |
+
|
| 139 |
## Notes
|
| 140 |
|
| 141 |
- Only selected checkpoints and milestone runs are stored.
|
| 142 |
- Large files are managed using Git LFS.
|
| 143 |
- Intermediate checkpoints are omitted to reduce repository size.
|
| 144 |
+
- Additional experiments and evaluation results will be added over time.
|
| 145 |
|
| 146 |
---
|
| 147 |
|
| 148 |
## Planned Work
|
| 149 |
|
| 150 |
+
Future extensions include:
|
| 151 |
|
| 152 |
- GRPO-based reinforcement learning
|
| 153 |
+
- Human feedback integration
|
| 154 |
- Multi-turn interaction benchmarks
|
| 155 |
- Automated best-checkpoint selection
|
| 156 |
- Evaluation pipelines and benchmarking suites
|
| 157 |
+
|
| 158 |
+
---
|
| 159 |
+
|
| 160 |
+
## License & Usage
|
| 161 |
+
|
| 162 |
+
This repository follows the licensing terms of the underlying base model:
|
| 163 |
+
|
| 164 |
+
- `meta-llama/Llama-3.1-8B-Instruct`
|
| 165 |
+
|
| 166 |
+
Users must comply with the original Llama license and usage restrictions when using derived checkpoints.
|