| --- |
| license: gpl-3.0 |
| datasets: |
| - lmsys/lmsys-chat-1m |
| language: |
| - en |
| --- |
| |
| # V2L (Vector-to-Language) — Beta |
|
|
| V2L is a two-stage architecture for mapping semantic embeddings into natural language. |
| It consists of: |
|
|
| 1. **Semantic Mapper** — aligns source embeddings with target embedding space. |
| 2. **Embedding Decoder** — generates text from target-space embeddings. |
|
|
| This repo provides **example `.pt` files** and training scripts to explore the architecture. |
| All checkpoints included here are *beta examples* — lightweight, but functional. |
|
|
| --- |
|
|
| ## Project Structure |
|
|
| * **`main.py`** — trains the **Semantic Mapper** (x-emb → y-emb) with cosine embedding loss. |
| * **`decoder.py`** — trains the **Embedding Decoder** (y-emb → target text) using strict teacher forcing. |
| * **`formatter.py`** — formats raw chat JSON into `(source, target)` pairs. |
| * **`pre_embed.py`** — generates sentence embeddings and saves them to `chat_embeddings.pt`. |
| * **`test_chat.py`** — runs an interactive CLI chatbot with the trained mapper + decoder. |
| * **`test_embed.py`** — quick embedding throughput benchmark. |
| * **`test_full.py`** — **end-to-end script that trains both the mapper and the decoder in a single run**. |
| * **`testcuda.py`** — checks CUDA device availability. |
|
|
| --- |
|
|
| ## Workflow |
|
|
| 1. **Format dataset** |
|
|
| ```bash |
| python formatter.py |
| ``` |
|
|
| Produces `chat_1turn.csv`. |
|
|
| 2. **Precompute embeddings** |
|
|
| ```bash |
| python pre_embed.py |
| ``` |
|
|
| Produces `chat_embeddings.pt`. |
|
|
| 3. **Train mapper** |
|
|
| ```bash |
| python main.py |
| ``` |
|
|
| Produces `semantic_mapper.pth`. |
|
|
| 4. **Train decoder** |
|
|
| ```bash |
| python decoder.py |
| ``` |
|
|
| Produces `embedding_decoder.pth`. |
|
|
| 5. **Chat interactively** |
|
|
| ```bash |
| python test_chat.py |
| ``` |
|
|
| ➡️ Alternatively, run **`test_full.py`** to train **both mapper and decoder** in one script and then test generation end-to-end. |
| |
| --- |
| |
| ## Checkpoints |
| |
| * **`semantic_mapper.pth`** — maps source embeddings into target embedding space. |
| * **`embedding_decoder.pth`** — GRU-based text generator conditioned on embeddings. |
| * **`chat_embeddings.pth`** — A pre-embedded dict of inputs and outputs |
|
|
| All `.pt` files in this repo are example checkpoints. They do work, but are not trained to full convergence. |
|
|
| --- |
|
|
| ## Dataset Note |
|
|
| Due to technical constraints, the dataset is **not included** in this repo. |
| All scripts assume data derived from **[lmsys/lmsys-chat-1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m)**, but filtered to **English-only** and cleaned of formatting artifacts. |
|
|
| --- |
|
|
| ## Example |
|
|
| ```python |
| from test_chat import chat |
| chat() # Starts interactive loop |
| ``` |
|
|
| Sample run: |
|
|
| ``` |
| User: What does GPT stand for? |
| Bot: The GPT (Generative Pre-trained Transformer is a type of neural network that is used to describe the probability of text and use language that uses a few loop to each other. |
| ``` |
|
|
| --- |
|
|
|
|
