Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: π¨
|
| 4 |
colorFrom: purple
|
| 5 |
colorTo: blue
|
|
@@ -13,3 +13,74 @@ short_description: Finds inflection points in model outputs and displays diffs
|
|
| 13 |
---
|
| 14 |
|
| 15 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: InflectionLM
|
| 3 |
emoji: π¨
|
| 4 |
colorFrom: purple
|
| 5 |
colorTo: blue
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
| 16 |
+
|
| 17 |
+
# InflectionLM: Output and Token Visualization
|
| 18 |
+
|
| 19 |
+
InflectionLM is a tool designed to visualize "inflection points" in Large Language Model (LLM) generation. By leveraging independent sampling, the tool generates multiple alternative output paths and identifies tokens where the model's confidence was low, highlighting these as potential points where the generation could have diverged.
|
| 20 |
+
|
| 21 |
+
## π Features
|
| 22 |
+
|
| 23 |
+
- **Diverse Response Generation**: Generates 3 independent responses for every prompt using the GEMMA 4 model, utilizing Top-P (Nucleus) and Top-K sampling to maximize diversity.
|
| 24 |
+
- **Confidence Highlighting**: Automatically highlights tokens with a probability score below **0.6** in red, marking them as "inflection points."
|
| 25 |
+
- **Interactive Visualization**:
|
| 26 |
+
- A Gradio-based GUI to input prompts and view results.
|
| 27 |
+
- Ability to switch between the 3 generated responses.
|
| 28 |
+
- Toggleable detailed view showing exact probability scores for every token in a response.
|
| 29 |
+
- **Customizable Generation**: Adjustable temperature settings to control the randomness and diversity of the output.
|
| 30 |
+
- **UI Preferences**: Support for both light and dark modes.
|
| 31 |
+
|
| 32 |
+
## π οΈ Tech Stack
|
| 33 |
+
|
| 34 |
+
- **Language**: Python
|
| 35 |
+
- **LLM**: [GEMMA 4](https://huggingface.co/google/gemma-4-31B-it) (via Hugging Face `transformers`)
|
| 36 |
+
- **Deep Learning Framework**: PyTorch
|
| 37 |
+
- **UI Framework**: Gradio
|
| 38 |
+
- **Compute**: Optimized for GPU usage (via `@spaces.GPU` for ZeroGPU environments)
|
| 39 |
+
|
| 40 |
+
## π Project Structure
|
| 41 |
+
|
| 42 |
+
```text
|
| 43 |
+
InflectionLM/
|
| 44 |
+
βββ app.py # Gradio web application and UI logic
|
| 45 |
+
βββ inflections_funcs.py # Core logic for model loading, generation, and scoring
|
| 46 |
+
βββ requirements.txt # Project dependencies
|
| 47 |
+
βββ Stochastic_parrot.JPG # UI asset image
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
## π Getting Started
|
| 51 |
+
|
| 52 |
+
### Prerequisites
|
| 53 |
+
|
| 54 |
+
- Python 3.10+
|
| 55 |
+
- A GPU with sufficient VRAM to load GEMMA 4 (31B)
|
| 56 |
+
- Hugging Face account and access to the GEMMA 4 model
|
| 57 |
+
|
| 58 |
+
### Installation
|
| 59 |
+
|
| 60 |
+
1. Clone the repository:
|
| 61 |
+
```bash
|
| 62 |
+
git clone <repo-url>
|
| 63 |
+
cd InflectionLM
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
2. Install dependencies:
|
| 67 |
+
```bash
|
| 68 |
+
pip install -r requirements.txt
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
### Running the Application
|
| 72 |
+
|
| 73 |
+
Ensure your environment variables for Hugging Face are set (e.g., `HUGGING_FACE_HUB_TOKEN`), then run:
|
| 74 |
+
|
| 75 |
+
```bash
|
| 76 |
+
python app.py
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
The application will start a Gradio server. Open the provided URL in your browser to begin experimenting.
|
| 80 |
+
|
| 81 |
+
## π How it Works
|
| 82 |
+
|
| 83 |
+
The tool uses **Multinomial Sampling** with Top-P and Top-K filtering to generate distinct sequences. Instead of a deterministic beam search, it samples from the probability distribution to find diverse but high-quality responses. After generation, it extracts the softmax probability of each chosen token from the model's output scores.
|
| 84 |
+
|
| 85 |
+
Tokens with a probability $< 0.6$ are considered "unconfident." In linguistic terms, these are the "inflections"βmoments where the model's internal probability distribution was flatter, meaning alternative tokens were almost as likely, leading to potential variations in the final text.
|
| 86 |
+
|