README.md

Query Classifier Model

This is a probabilistic multi-label classifier for routing business queries to appropriate systems.

Model Description

• Architecture: Multi-layer neural network with sigmoid outputs
• Input: TF-IDF vectorized queries (max 500 features)
• Output: Independent probabilities for each system
• Systems: accounting, communication, enterprise_resource_planning, knowledge_management, project_management, sales

Performance

• Macro F1: 0.817
• Micro F1: 0.803
• Subset Accuracy: 0.704
• Hamming Loss: 0.071

Usage

from src.core.services__OLD_REFACTOR.query_classifier_service import classify_query

# Classify a single query
result = classify_query("show total revenue for last quarter")
print(f"Top system: {result['top_system']}")
print(f"Confidence: {result['confidence']:.3f}")
print(f"All systems: {result['predicted_systems']}")

Training Data

• Total queries: 621
• Training samples: 496
• Test samples: 125
• Sources: Apache Calcite fine-tuning data (Jira, QuickBooks, HubSpot, Gmail, Notion, SAP)

Systems Supported

• accounting: Accounting
• communication: Communication
• enterprise_resource_planning: Enterprise Resource Planning
• knowledge_management: Knowledge Management
• project_management: Project Management
• sales: Sales

Model Files

• final_model.pth: PyTorch model weights and configuration
• vectorizer.pkl: TF-IDF vectorizer for text preprocessing
• mlb.pkl: Multi-label binarizer for label encoding
• metadata.json: Model metadata and performance metrics