Buckets:

imrancoder
/

deepseek-reasoning-blind-spots

Files

xet

imrancoder/deepseek-reasoning-blind-spots / README.md

imrancoder

24 days ago

preview code

download

raw

3.89 kB

# Blind Spots of Frontier Models: Repetitive Reasoning Loops

## 1. Research Question

While working on LLM reasoning and post-training, I became interested in a problem that accuracy benchmarks may miss: **a model can generate a long reasoning trace without making meaningful progress.**

A model may repeat the same idea, equation, or conclusion instead of correcting its approach. Final-answer accuracy alone may not reveal this behavior.

My goal is to investigate repetitive reasoning loops in small reasoning models and explore ways to reduce them without harming useful reasoning.

## 2. Model and Initial Findings

I selected [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B), an open-weight reasoning model that can be studied with limited GPU resources.

In my initial experiment, I generated responses and used an automated detector to identify potential repetitive loops.

| Metric | Result |
|---|---:|
| Generation attempts | 9,920 |
| Potential loops detected | 844 |
| Generations without detected loops | 9,076 |
| Prompts skipped due to length | 80 |
| Failed generations | 0 |

The detector flagged 844 out of 9,920 attempts (8.51%) as potential loops. Excluding the 80 skipped prompts, the detected-loop rate was 8.58%.

These are preliminary results, not proof that every flagged response contains faulty reasoning. Manual inspection and further controlled experiments are needed to validate the detector and understand the behavior.

## 3. Notebook Structure

Due to limited GPU resources and notebook runtime constraints, I divided the experiment into three notebooks rather than running the entire workflow in one notebook. This makes the stages easier to execute, inspect, and reproduce.

- **Notebook 1 — Generation:** Runs the initial DeepSeek-R1-Distill-Qwen-1.5B generation experiment and identifies potential repetitive loops.
- **Notebook 2 — Training:** Contains the training stage for exploring preference-based optimization using examples of repetitive and preferred reasoning behavior.
- **Notebook 3 — Merge and Validation:** Merges the trained adapter with the base model, where applicable, and checks the resulting model.

Each notebook represents a separate stage of the experimental workflow. The three notebooks together document the process from generation to training and validation.

## 4. Proposed Solution

I propose building a carefully checked dataset containing unproductive reasoning loops, successful recovery examples, and useful reasoning trajectories.

A potential solution is preference-based fine-tuning, such as Final Token Preference Optimization (FTPO), to encourage models to recognize unproductive repetition and change their approach when needed.

The original and fine-tuned models should be compared on the same held-out prompts. Evaluation should measure loop frequency, answer correctness, task completion, reasoning length, and recovery from repetitive behavior.

The goal is not simply to shorten responses, but to reduce unproductive repetition while preserving useful reasoning.

## 5. Conclusion

My initial experiment provides a starting point for investigating repetitive reasoning as a potential blind spot in small reasoning models. The next steps are to validate the detector through manual review, evaluate the behavior across different tasks, and determine whether targeted fine-tuning can reduce loops without harming answer quality.

## References and Artifacts

- **Model:** [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
- **Liquid AI.** [Reducing Doom Loops with Final Token Preference Optimization](https://www.liquid.ai/blog/antidoom). Liquid AI Blog, 2026.
- **Artifacts:** Three experimental notebooks covering generation, training, and merge/validation are included in this Hugging Face Bucket.

Xet Storage Details

Size:: 3.89 kB
Xet hash:: c6210a0619be3833010eb50bb78bf459f063131721453c9e6377bfd361f452fc

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# Blind Spots of Frontier Models: Repetitive Reasoning Loops

	## 1. Research Question

	While working on LLM reasoning and post-training, I became interested in a problem that accuracy benchmarks may miss: a model can generate a long reasoning trace without making meaningful progress.

	A model may repeat the same idea, equation, or conclusion instead of correcting its approach. Final-answer accuracy alone may not reveal this behavior.

	My goal is to investigate repetitive reasoning loops in small reasoning models and explore ways to reduce them without harming useful reasoning.

	## 2. Model and Initial Findings

	I selected [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B), an open-weight reasoning model that can be studied with limited GPU resources.

	In my initial experiment, I generated responses and used an automated detector to identify potential repetitive loops.

	\| Metric \| Result \|
	\|---\|---:\|
	\| Generation attempts \| 9,920 \|
	\| Potential loops detected \| 844 \|
	\| Generations without detected loops \| 9,076 \|
	\| Prompts skipped due to length \| 80 \|
	\| Failed generations \| 0 \|

	The detector flagged 844 out of 9,920 attempts (8.51%) as potential loops. Excluding the 80 skipped prompts, the detected-loop rate was 8.58%.

	These are preliminary results, not proof that every flagged response contains faulty reasoning. Manual inspection and further controlled experiments are needed to validate the detector and understand the behavior.

	## 3. Notebook Structure

	Due to limited GPU resources and notebook runtime constraints, I divided the experiment into three notebooks rather than running the entire workflow in one notebook. This makes the stages easier to execute, inspect, and reproduce.

	- Notebook 1 — Generation: Runs the initial DeepSeek-R1-Distill-Qwen-1.5B generation experiment and identifies potential repetitive loops.
	- Notebook 2 — Training: Contains the training stage for exploring preference-based optimization using examples of repetitive and preferred reasoning behavior.
	- Notebook 3 — Merge and Validation: Merges the trained adapter with the base model, where applicable, and checks the resulting model.

	Each notebook represents a separate stage of the experimental workflow. The three notebooks together document the process from generation to training and validation.

	## 4. Proposed Solution

	I propose building a carefully checked dataset containing unproductive reasoning loops, successful recovery examples, and useful reasoning trajectories.

	A potential solution is preference-based fine-tuning, such as Final Token Preference Optimization (FTPO), to encourage models to recognize unproductive repetition and change their approach when needed.

	The original and fine-tuned models should be compared on the same held-out prompts. Evaluation should measure loop frequency, answer correctness, task completion, reasoning length, and recovery from repetitive behavior.

	The goal is not simply to shorten responses, but to reduce unproductive repetition while preserving useful reasoning.

	## 5. Conclusion

	My initial experiment provides a starting point for investigating repetitive reasoning as a potential blind spot in small reasoning models. The next steps are to validate the detector through manual review, evaluate the behavior across different tasks, and determine whether targeted fine-tuning can reduce loops without harming answer quality.

	## References and Artifacts

	- Model: [DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
	- Liquid AI. [Reducing Doom Loops with Final Token Preference Optimization](https://www.liquid.ai/blog/antidoom). Liquid AI Blog, 2026.
	- Artifacts: Three experimental notebooks covering generation, training, and merge/validation are included in this Hugging Face Bucket.