Cooking-Qwen-Merged

A small, specialized cooking language model fine-tuned using synthetic instruction data, knowledge distillation, and Low-Rank Adaptation (LoRA).

Overview

Cooking-Qwen-Merged is a fine-tuned Qwen language model designed to answer cooking questions, explain recipes, identify ingredient quantities, and provide preparation guidance.

The goal of this project was straightforward: build and fine-tune a functional cooking assistant within a single weekend.

Rather than training a language model from scratch, the project uses a pretrained Qwen model and a teacherโ€“student knowledge distillation pipeline to generate specialized training data.

A larger language model was provided with relevant recipe information retrieved from a vector database. Its generated answers were then used to fine-tune the smaller model using LoRA.

The resulting model contains approximately 597 million parameters, with fewer than 0.2% updated during fine-tuning.

Model Details

Property Description
Developer Murphy Amos
Hugging Face burpi
Model type Causal language model
Architecture Qwen3
Base model Qwen3-0.6B
Parameters 597,196,800
Language English
Specialization Cooking and recipe question answering
Fine-tuning method Supervised fine-tuning with LoRA
Training framework PyTorch, Transformers, PEFT
Distribution Merged model weights
License Pending verification of base model and dataset licenses

Project Objective

The objective was to develop a small, domain-specific cooking assistant over a weekend using existing open-source machine learning tools.

The project involved building a data-generation pipeline, generating thousands of cooking-related questions, using a larger teacher model to produce training responses, and fine-tuning a smaller Qwen model.

The focus was on delivering a working model rather than conducting a formal research experiment.

Intended Uses

Cooking-Qwen-Merged is intended for:

  • Answering general cooking questions.
  • Explaining recipe instructions and preparation methods.
  • Identifying ingredients and their quantities.
  • Suggesting ingredient substitutions.
  • Explaining cooking techniques.
  • Assisting with recipe scaling.
  • Providing beginner-friendly cooking guidance.
  • Answering questions using externally retrieved recipe context.

Out-of-Scope Uses

The model is not intended to replace professional advice regarding nutrition, food allergies, or food safety.

Generated cooking instructions should be independently verified when incorrect information could create a safety risk.

How to Use

Install the required dependencies:

pip install torch transformers accelerate

Load the merged model from Hugging Face:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "burpi/Cooking-Qwen-Merged"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.float32
)

model.eval()

question = "How do I cook a steak on the stove?"

messages = [
    {
        "role": "system",
        "content": "You are a cooking assistant. Answer cooking questions clearly and accurately."
    },
    {
        "role": "user",
        "content": question
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)

print(response)

This example assumes the repository contains the merged model weights and tokenizer files.

The model can also be combined with a retrieval system to supply recipe-specific information during inference.

Training Methodology

Dataset and Question Generation

The project began with a structured recipe dataset containing ingredients, quantities, cooking instructions, and other recipe information.

Using ChatGPT Astra, a collection of ~5,000 cooking-related questions was generated based on the dataset.

These questions covered ingredient measurements, cooking techniques, preparation instructions, recipe scaling, substitutions, dietary modifications, and troubleshooting.

Synthetic Response Generation Using Gemini and RAG

The generated questions were then processed through a Retrieval-Augmented Generation (RAG) pipeline using Google Gemini as the teacher language model.

For each question, the pipeline:

  1. Queried a vector database containing recipe information.
  2. Retrieved relevant recipe context.
  3. Supplied the question and retrieved context to Gemini.
  4. Generated a cooking-related answer grounded in the retrieved information.
  5. Stored the resulting questionโ€“answer interaction for fine-tuning.

This produced synthetic conversational training data based on the original recipe dataset.

Student Model Fine-Tuning

The generated questionโ€“answer interactions were used to fine-tune a pretrained Qwen3-0.6B causal language model.

Fine-tuning was performed using Low-Rank Adaptation (LoRA) through Hugging Face PEFT, updating approximately 0.192% of the model's parameters.

The objective was to transfer cooking-specific response behavior from Gemini-generated examples into a smaller Qwen model capable of running independently.

Training Dataset

The training logs reported the following split:

Dataset Examples
Training 4,270
Evaluation 1,068
Total 5,338

The original question-generation collection contained 5,000 questions. The reported example counts are taken from the training logs and have not been reconciled with the question count.

Pipeline Summary

Recipe Dataset โ†’ Astra (Question Generation) โ†’ Gemini + RAG (Response Generation) โ†’ Synthetic Questionโ€“Answer Dataset โ†’ Qwen (LoRA Fine-Tuning) โ†’ Merged Cooking Model

Parameter-Efficient Fine-Tuning

Low-Rank Adaptation (LoRA) was used to update a small subset of the model's parameters.

Training property Value
Total parameters 597,196,800
Trainable parameters 1,146,880
Trainable proportion 0.192%
Initial learning rate Approximately 0.0002
Training steps Approximately 2,150
Training approach LoRA / PEFT
Optimization objective Supervised next-token prediction

LoRA adapters were applied to selected attention projection layers, including query and value projections.

The trained adapter was subsequently merged with the pretrained model weights.

Training Performance

The model was fine-tuned for approximately 2,150 optimization steps using LoRA and a linearly decaying learning-rate schedule.

Training and Evaluation Curves

Training and evaluation loss with learning-rate schedule

The graph shows:

  • Raw training loss.
  • Smoothed training loss.
  • Evaluation loss at recorded checkpoints.
  • Learning-rate decay throughout training.

Training Metrics

Metric Approximate value
Initial recorded training loss 3.17
Final training loss 1.40
Final evaluation loss 1.41
Training-loss reduction 56%
Initial learning rate 0.0002
Final learning rate Approximately 0
Optimization steps Approximately 2,150
Learning-rate schedule Linear decay

Training Observations

Initial convergence: Training loss decreased rapidly during the first 200โ€“300 optimization steps, indicating substantial early adaptation to the cooking instruction dataset.

Continued optimization: Loss decreased more gradually throughout the remaining training process, stabilizing around 1.35โ€“1.45.

Evaluation performance: Evaluation loss remained approximately 1.4 at the recorded evaluation checkpoints. The relatively small difference between training and evaluation loss suggests no obvious severe overfitting at those checkpoints, although further testing is necessary.

Learning-rate behavior: The learning rate decreased approximately linearly from 2 ร— 10โปโด to zero across the training run.

Interpretation

The results demonstrate that the LoRA fine-tuning process reduced the model's next-token prediction loss on the cooking instruction dataset.

These measurements establish training convergence but do not independently verify cooking accuracy, factual reliability, or improvement over the original pretrained model.

Evaluation Status

The model has undergone preliminary interactive testing with cooking-related questions.

A comprehensive independent benchmark comparing the pretrained and fine-tuned models has not yet been completed.

Future evaluations may measure:

  • Ingredient quantity accuracy.
  • Factual answer accuracy.
  • Cooking instruction correctness.
  • Hallucination frequency.
  • Performance on unseen recipes.
  • Standalone versus retrieval-augmented performance.

Limitations

Cooking-Qwen-Merged has several limitations:

  • The model may generate incorrect ingredient quantities, cooking times, or temperatures.
  • It may produce plausible but unsupported cooking instructions.
  • Synthetic training data may contain errors inherited from the teacher model or retrieval pipeline.
  • Some generated training questions contain formatting or measurement inconsistencies.
  • The model has not been comprehensively evaluated on unseen recipes.
  • Fine-tuning does not guarantee memorization of the complete recipe database.
  • Retrieval augmentation may improve performance on questions requiring exact recipe information.

The model should be treated as a cooking assistant rather than an authoritative recipe database.

Technical Architecture

The model is based on a pretrained Qwen causal language model.

Its development used:

  • PyTorch: Model execution and optimization.
  • Hugging Face Transformers: Model loading and training.
  • PEFT: Parameter-efficient fine-tuning.
  • LoRA: Low-rank weight adaptation.
  • Vector database: Recipe retrieval during synthetic data generation.
  • Teacher language model: Generation of supervised training responses.

After fine-tuning, the LoRA adapter was merged into the base model for deployment.

Future Improvements

Potential improvements include:

  1. Benchmarking the fine-tuned model against the original Qwen model.
  2. Improving the quality and consistency of synthetic training examples.
  3. Testing generalization on recipes excluded from training.
  4. Quantizing the model to reduce memory requirements.
  5. Comparing standalone inference with retrieval-augmented inference.
  6. Integrating the model into a cooking assistant application.

Project Background

Cooking-Qwen-Merged was developed as an independent weekend machine learning project.

The project demonstrates a practical workflow for creating a specialized language model using retrieval-grounded synthetic data, teacherโ€“student distillation, and parameter-efficient fine-tuning.

It is an engineering project rather than a peer-reviewed research publication.

Citation

Amos, M. (2026). Cooking-Qwen-Merged: A Small Cooking Language Model Fine-Tuned with Synthetic Instruction Data and LoRA. Hugging Face.

https://huggingface.co/burpi/Cooking-Qwen-Merged

Author

Murphy Amos

AI Software Engineer | Computer Game Development Undergraduate

GitHub | Hugging Face

Downloads last month
78
Safetensors
Model size
0.6B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for burpi/Cooking-Qwen-Merged

Finetuned
Qwen/Qwen3-0.6B
Finetuned
(1,491)
this model
Adapters
1 model