File size: 1,615 Bytes
99e126d 0780330 99e126d 0780330 99e126d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | # Query Classifier Model
This is a probabilistic multi-label classifier for routing business queries to appropriate systems.
## Model Description
- **Architecture**: Multi-layer neural network with sigmoid outputs
- **Input**: TF-IDF vectorized queries (max 500 features)
- **Output**: Independent probabilities for each system
- **Systems**: accounting, communication, enterprise_resource_planning, knowledge_management, project_management, sales
## Performance
- **Macro F1**: 0.817
- **Micro F1**: 0.803
- **Subset Accuracy**: 0.704
- **Hamming Loss**: 0.071
## Usage
```python
from src.core.services__OLD_REFACTOR.query_classifier_service import classify_query
# Classify a single query
result = classify_query("show total revenue for last quarter")
print(f"Top system: {result['top_system']}")
print(f"Confidence: {result['confidence']:.3f}")
print(f"All systems: {result['predicted_systems']}")
```
## Training Data
- **Total queries**: 621
- **Training samples**: 496
- **Test samples**: 125
- **Sources**: Apache Calcite fine-tuning data (Jira, QuickBooks, HubSpot, Gmail, Notion, SAP)
## Systems Supported
- **accounting**: Accounting
- **communication**: Communication
- **enterprise_resource_planning**: Enterprise Resource Planning
- **knowledge_management**: Knowledge Management
- **project_management**: Project Management
- **sales**: Sales
## Model Files
- `final_model.pth`: PyTorch model weights and configuration
- `vectorizer.pkl`: TF-IDF vectorizer for text preprocessing
- `mlb.pkl`: Multi-label binarizer for label encoding
- `metadata.json`: Model metadata and performance metrics
|