cafierom commited on
Commit
cbead1b
Β·
verified Β·
1 Parent(s): 360bdcd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -1
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: Inflections
3
  emoji: 🐨
4
  colorFrom: purple
5
  colorTo: blue
@@ -13,3 +13,74 @@ short_description: Finds inflection points in model outputs and displays diffs
13
  ---
14
 
15
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: InflectionLM
3
  emoji: 🐨
4
  colorFrom: purple
5
  colorTo: blue
 
13
  ---
14
 
15
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
16
+
17
+ # InflectionLM: Output and Token Visualization
18
+
19
+ InflectionLM is a tool designed to visualize "inflection points" in Large Language Model (LLM) generation. By leveraging independent sampling, the tool generates multiple alternative output paths and identifies tokens where the model's confidence was low, highlighting these as potential points where the generation could have diverged.
20
+
21
+ ## 🌟 Features
22
+
23
+ - **Diverse Response Generation**: Generates 3 independent responses for every prompt using the GEMMA 4 model, utilizing Top-P (Nucleus) and Top-K sampling to maximize diversity.
24
+ - **Confidence Highlighting**: Automatically highlights tokens with a probability score below **0.6** in red, marking them as "inflection points."
25
+ - **Interactive Visualization**:
26
+ - A Gradio-based GUI to input prompts and view results.
27
+ - Ability to switch between the 3 generated responses.
28
+ - Toggleable detailed view showing exact probability scores for every token in a response.
29
+ - **Customizable Generation**: Adjustable temperature settings to control the randomness and diversity of the output.
30
+ - **UI Preferences**: Support for both light and dark modes.
31
+
32
+ ## πŸ› οΈ Tech Stack
33
+
34
+ - **Language**: Python
35
+ - **LLM**: [GEMMA 4](https://huggingface.co/google/gemma-4-31B-it) (via Hugging Face `transformers`)
36
+ - **Deep Learning Framework**: PyTorch
37
+ - **UI Framework**: Gradio
38
+ - **Compute**: Optimized for GPU usage (via `@spaces.GPU` for ZeroGPU environments)
39
+
40
+ ## πŸ“ Project Structure
41
+
42
+ ```text
43
+ InflectionLM/
44
+ β”œβ”€β”€ app.py # Gradio web application and UI logic
45
+ β”œβ”€β”€ inflections_funcs.py # Core logic for model loading, generation, and scoring
46
+ β”œβ”€β”€ requirements.txt # Project dependencies
47
+ └── Stochastic_parrot.JPG # UI asset image
48
+ ```
49
+
50
+ ## πŸš€ Getting Started
51
+
52
+ ### Prerequisites
53
+
54
+ - Python 3.10+
55
+ - A GPU with sufficient VRAM to load GEMMA 4 (31B)
56
+ - Hugging Face account and access to the GEMMA 4 model
57
+
58
+ ### Installation
59
+
60
+ 1. Clone the repository:
61
+ ```bash
62
+ git clone <repo-url>
63
+ cd InflectionLM
64
+ ```
65
+
66
+ 2. Install dependencies:
67
+ ```bash
68
+ pip install -r requirements.txt
69
+ ```
70
+
71
+ ### Running the Application
72
+
73
+ Ensure your environment variables for Hugging Face are set (e.g., `HUGGING_FACE_HUB_TOKEN`), then run:
74
+
75
+ ```bash
76
+ python app.py
77
+ ```
78
+
79
+ The application will start a Gradio server. Open the provided URL in your browser to begin experimenting.
80
+
81
+ ## πŸ“– How it Works
82
+
83
+ The tool uses **Multinomial Sampling** with Top-P and Top-K filtering to generate distinct sequences. Instead of a deterministic beam search, it samples from the probability distribution to find diverse but high-quality responses. After generation, it extracts the softmax probability of each chosen token from the model's output scores.
84
+
85
+ Tokens with a probability $< 0.6$ are considered "unconfident." In linguistic terms, these are the "inflections"β€”moments where the model's internal probability distribution was flatter, meaning alternative tokens were almost as likely, leading to potential variations in the final text.
86
+