Instructions to use Shell2334/info642-travel-classifier-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Shell2334/info642-travel-classifier-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForSequenceClassification base_model = AutoModelForSequenceClassification.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0") model = PeftModel.from_pretrained(base_model, "Shell2334/info642-travel-classifier-lora") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from Shell2334/info642-travel-classifier-lora: direct link, hf CLI and curl.
- Browser
- Download file 6 kB
-
https://huggingface.co/Shell2334/info642-travel-classifier-lora/resolve/main/README.md
- Command line
-
hf download hf://Shell2334/info642-travel-classifier-lora/README.md
-
curl -L -o README.md https://huggingface.co/Shell2334/info642-travel-classifier-lora/resolve/main/README.md
6 kB
| license: apache-2.0 | |
| base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0 | |
| library_name: peft | |
| pipeline_tag: text-classification | |
| language: | |
| - en | |
| tags: | |
| - peft | |
| - lora | |
| - text-classification | |
| - travel | |
| # INFO 642 - Travel Request Classifier (TinyLlama + LoRA) | |
| ## What the Model Does | |
| This model is a seven-category text classifier developed for our INFO 642 Modular AI Travel Planner project. | |
| The model identifies the user's travel request and helps the n8n workflow select the appropriate module. | |
| It predicts one of seven labels: | |
| 1. NEW_TRIP - Create a new travel itinerary. | |
| 2. CHANGE_FLIGHT - Modify an existing flight. | |
| 3. CHANGE_HOTEL - Modify an existing hotel. | |
| 4. CHANGE_ATTRACTION - Modify an attraction or activity. | |
| 5. CHANGE_RESTAURANT - Modify a restaurant or meal. | |
| 6. MULTI_CHANGE - Modify multiple itinerary components. | |
| 7. OTHER_OR_UNCLEAR - Handle unclear, general, or unsupported requests. | |
| In our workflow, this model is intended to replace the original Groq-based placeholder classifier. The other modules handle itinerary generation, data lookup, budget calculations, and itinerary modifications. | |
| The classifier does not make reservations or payments. | |
| ## Base Model | |
| - Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0 | |
| - Model type: Seven-class text classification | |
| - Fine-tuning method: Low-Rank Adaptation (LoRA) | |
| - Framework: Hugging Face Transformers and PEFT | |
| We followed the Lesson 16 TinyLlama training approach and added a new seven-class classification head. | |
| LoRA allows us to train a small number of parameters instead of updating the entire base model. | |
| Training configuration: | |
| | Parameter | Value | | |
| |---|---| | |
| | LoRA rank | 8 | | |
| | LoRA alpha | 16 | | |
| | LoRA dropout | 0.05 | | |
| | Target modules | q_proj, k_proj, v_proj, o_proj | | |
| | Epochs | 3 | | |
| | Batch size | 8 | | |
| | Learning rate | 0.0002 | | |
| | Trainable parameters | 2,267,136 | | |
| Only approximately 0.22% of the model's parameters were trainable. | |
| ## Training Data | |
| We created a synthetic English dataset of 280 travel-related user requests for this course project. | |
| The dataset includes seven balanced categories, with 40 examples per category. | |
| Data split: | |
| | Dataset | Samples | | |
| |---|---:| | |
| | Training set | 224 | | |
| | Held-out test set | 56 | | |
| | Total | 280 | | |
| Each category contains 32 training examples and 8 held-out test examples. | |
| The training data covers requests such as creating trips, changing flights, finding cheaper hotels, replacing attractions, modifying restaurants, and changing multiple itinerary components. | |
| Some examples also include unclear requests and unsupported booking or payment requests. | |
| The test data was not used for model training. | |
| All examples are synthetic and do not represent real customer conversations. | |
| ## Results | |
| We evaluated the fine-tuned model using 56 held-out test requests. | |
| ### Overall Accuracy | |
| **Held-out test accuracy: 98.2% (55/56 correct)** | |
| Additional model comparisons: | |
| | Model | Accuracy | | |
| |---|---:| | |
| | BART Zero-shot | 46.4% | | |
| | TinyLlama with untrained classification head | 14.3% | | |
| | TinyLlama + LoRA | 98.2% | | |
| The fine-tuned model performed substantially better on our synthetic test set. | |
| ### Training Loss | |
| | Epoch | Loss | | |
| |---|---:| | |
| | 1 | 1.944 | | |
| | 2 | 0.461 | | |
| | 3 | 0.091 | | |
| The training loss decreased across all three epochs. | |
| ### Confusion Matrix Analysis | |
| The fine-tuned model correctly classified 55 out of 56 test examples. | |
| Six categories achieved perfect recall. | |
| The only classification error occurred when a MULTI_CHANGE request was predicted as CHANGE_ATTRACTION. | |
| The relevant category metrics were: | |
| | Category | Precision | Recall | F1-score | | |
| |---|---:|---:|---:| | |
| | CHANGE_ATTRACTION | 0.89 | 1.00 | 0.94 | | |
| | MULTI_CHANGE | 1.00 | 0.88 | 0.93 | | |
| The macro-average F1-score was 0.98. | |
| This indicates that requests involving multiple changes may sometimes be confused with requests involving only one attraction change. | |
| The complete confusion matrix and classification report are included in the training notebook. | |
| ## Limitations | |
| Although the model achieved high accuracy on the held-out dataset, several limitations remain. | |
| 1. **Synthetic training data:** The dataset contains only 280 synthetic examples. Real user requests may be more complex or expressed differently. | |
| 2. **Multi-change confusion:** Requests involving multiple itinerary components may sometimes be classified as a single-category change. | |
| 3. **Ambiguous language:** Short or unclear requests may not provide enough information for reliable classification. | |
| 4. **Limited task coverage:** The model only supports the seven predefined categories. It cannot independently create itineraries, verify live availability, make reservations, or process payments. | |
| 5. **Generalization:** The 98.2% accuracy was measured on a small synthetic held-out dataset. Performance on real travel conversations has not been established. | |
| The model is intended for educational workflow routing and should not be used as an independent booking or payment system. | |
| ## Fresh-Session Model Verification | |
| After training, we uploaded the LoRA adapter, classification head, tokenizer, and label configuration to this public Hugging Face repository. | |
| We then opened a fresh Google Colab notebook and loaded the model directly from Hugging Face. | |
| Test input: | |
| > Please update the afternoon sightseeing activity to the science center. | |
| Expected prediction from the original training notebook: | |
| `CHANGE_ATTRACTION` | |
| Prediction after loading from Hugging Face: | |
| `CHANGE_ATTRACTION` | |
| **Result: PASS - The fresh-session prediction matched the original notebook.** | |
| This verifies that the uploaded model can be loaded in a separate session and reproduce the prediction for this test example. | |
| ### Fresh-Session Test Screenshot | |
|  | |
| ## Project Information | |
| Course: INFO 642 | |
| Project: Modular AI Travel Planner | |
| Model repository: Shell2334/info642-travel-classifier-lora | |
| Purpose: Educational research and demonstration. | |