| # Query Classifier Model |
|
|
| This is a probabilistic multi-label classifier for routing business queries to appropriate systems. |
|
|
| ## Model Description |
| - **Architecture**: Multi-layer neural network with sigmoid outputs |
| - **Input**: TF-IDF vectorized queries (max 500 features) |
| - **Output**: Independent probabilities for each system |
| - **Systems**: accounting, communication, enterprise_resource_planning, knowledge_management, project_management, sales |
|
|
| ## Performance |
| - **Macro F1**: 0.817 |
| - **Micro F1**: 0.803 |
| - **Subset Accuracy**: 0.704 |
| - **Hamming Loss**: 0.071 |
|
|
| ## Usage |
|
|
| ```python |
| from src.core.services__OLD_REFACTOR.query_classifier_service import classify_query |
| |
| # Classify a single query |
| result = classify_query("show total revenue for last quarter") |
| print(f"Top system: {result['top_system']}") |
| print(f"Confidence: {result['confidence']:.3f}") |
| print(f"All systems: {result['predicted_systems']}") |
| ``` |
|
|
| ## Training Data |
| - **Total queries**: 621 |
| - **Training samples**: 496 |
| - **Test samples**: 125 |
| - **Sources**: Apache Calcite fine-tuning data (Jira, QuickBooks, HubSpot, Gmail, Notion, SAP) |
|
|
| ## Systems Supported |
| - **accounting**: Accounting |
| - **communication**: Communication |
| - **enterprise_resource_planning**: Enterprise Resource Planning |
| - **knowledge_management**: Knowledge Management |
| - **project_management**: Project Management |
| - **sales**: Sales |
|
|
| ## Model Files |
| - `final_model.pth`: PyTorch model weights and configuration |
| - `vectorizer.pkl`: TF-IDF vectorizer for text preprocessing |
| - `mlb.pkl`: Multi-label binarizer for label encoding |
| - `metadata.json`: Model metadata and performance metrics |
|
|
