Update README.md

README.md

@@ -1,3 +1,125 @@
 ---
 license: mit
 ---
+# Learning in Interaction using ArCHer
+
+> Interactive fine-tuning and alignment experiments using LoRA, supervised learning, and reinforcement-learning-based adaptation on LLaMA-3.1-8B-Instruct.
+
+---
+
+## Overview
+
+This repository contains checkpoints, training configurations, and experimental artifacts for the project:
+
+**Learning in Interaction using ArCHer**
+
+The goal of this work is to explore interactive learning paradigms for large language models through:
+
+- Supervised Fine-Tuning (SFT)
+- LoRA-based parameter-efficient adaptation
+- Preference and reward-based optimization
+- Future GRPO / ArCHer reinforcement learning methods
+
+The experiments are built on top of Meta's **LLaMA-3.1-8B-Instruct** model using the Hugging Face.
+
+---
+
+## Base Model
+
+- **Model:** `meta-llama/Llama-3.1-8B-Instruct`
+- **Framework:** Transformers + PEFT
+- **Training Infrastructure:** Hugging Face Trainer
+- **Fine-Tuning Method:** LoRA
+
+---
+
+## Repository Structure
+
+```text
+Llama-3.1-8B-Instruct/
+│
+├── sft_lora/
+│   └── checkpoint-1611/
+│       ├── adapter_model.safetensors
+│       ├── adapter_config.json
+│       ├── tokenizer.json
+│       ├── tokenizer_config.json
+│       ├── trainer_state.json
+│       ├── training_args.bin
+│       ├── optimizer.pt
+│       ├── scheduler.pt
+│       ├── rng_state.pth
+│       └── chat_template.jinja
+│
+├── sft_lora_with_val_splits/
+│   └── checkpoint-2016/
+│       ├── adapter_model.safetensors
+│       ├── adapter_config.json
+│       ├── tokenizer.json
+│       ├── tokenizer_config.json
+│       ├── trainer_state.json
+│       ├── training_args.bin
+│       ├── optimizer.pt
+│       ├── scheduler.pt
+│       ├── rng_state.pth
+│       └── chat_template.jinja
+```
+
+---
+
+## Experiment Summary
+
+| Experiment | Checkpoint | Description |
+|---|---|---|
+| SFT + LoRA | `checkpoint-1611` | Initial supervised fine-tuning using LoRA adapters |
+| SFT + LoRA + Validation Splits | `checkpoint-2016` | Validation-aware training with improved monitoring |
+| ArCHer / GRPO (Upcoming) | TBD | Reinforcement-learning-based interactive optimization |
+
+---
+
+
+## Loading a Checkpoint
+
+### Using Transformers + PEFT
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base_model = "meta-llama/Llama-3.1-8B-Instruct"
+
+# Load base model
+model = AutoModelForCausalLM.from_pretrained(
+    base_model,
+    device_map="auto",
+    torch_dtype="auto"
+)
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(
+    model,
+    "path_to_checkpoint"
+)
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained(base_model)
+```
+
+---
+
+## Notes
+
+- Only selected checkpoints and milestone runs are stored.
+- Large files are managed using Git LFS.
+- Intermediate checkpoints are omitted to reduce repository size.
+
+---
+
+## Planned Work
+
+Planned extensions include:
+
+- GRPO-based reinforcement learning
+- Multi-turn interaction benchmarks
+- Automated best-checkpoint selection
+- Evaluation pipelines and benchmarking suites