File size: 12,288 Bytes
ff76684
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
"""
VeritasDoc Custom Model Training Engine
Trains a proprietary high-accuracy AI content detection model on custom datasets.
Produces a calibrated, standalone, fast model file: custom_models/custom_detector.joblib

The trained model integrates directly into the Multi-Vector Neural Consensus as 'core_custom'
without modifying or affecting existing pretrained neural cores (Core Alpha, Beta, Gamma).
"""

import json
import logging
import os
from pathlib import Path
import datetime

import joblib
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, f1_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import FeatureUnion, Pipeline

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger("train_custom_model")

BASE_DIR = Path(__file__).resolve().parent
CUSTOM_MODELS_DIR = BASE_DIR / "custom_models"
CUSTOM_DATASET_DIR = BASE_DIR / "data" / "custom_dataset"
AI_DATA_DIR = CUSTOM_DATASET_DIR / "ai"
HUMAN_DATA_DIR = CUSTOM_DATASET_DIR / "human"

CUSTOM_MODELS_DIR.mkdir(parents=True, exist_ok=True)
AI_DATA_DIR.mkdir(parents=True, exist_ok=True)
HUMAN_DATA_DIR.mkdir(parents=True, exist_ok=True)

# Curated High-Quality Baseline Training Corpus (AI-Generated & Authentic Human)
DEFAULT_AI_SAMPLES = [
    "Artificial intelligence has undeniably revolutionized modern natural language processing and computer vision systems across global academia.",
    "Furthermore, recent breakthroughs in generative transformers serve as a testament to the rapid velocity of contemporary technological innovation.",
    "In conclusion, we must delve deeper into the multifaceted ethical implications and technical challenges that emerge as these technologies reshape society.",
    "Moreover, establishing standardized evaluation frameworks will be paramount in fostering transparent algorithmic governance and digital integrity.",
    "The intricate interplay between neural attention mechanisms and large-scale pretraining unlocks unprecedented capabilities in automated reasoning.",
    "It is important to remember that machine learning models rely heavily on curated data distributions to achieve robust generalizability.",
    "This comprehensive analysis underscores the pivotal role that distributed computing architectures play in scaling foundational generative networks.",
    "By leveraging cutting-edge deep learning techniques, organizations can seamlessly streamline their operational workflows and optimize efficiency.",
    "To achieve a holistic understanding of the subject, one must examine the underlying structural paradigms that govern generative text synthesis.",
    "In summary, the rapid democratization of conversational agents presents both formidable opportunities and intricate challenges for education.",
    "A vibrant tapestry of innovation continues to emerge as researchers pioneer novel parameter-efficient fine-tuning methodologies.",
    "Consequently, fostering collaborative interdisciplinary research remains imperative for navigating the ever-evolving frontier of artificial intelligence.",
    "The integration of reinforcement learning from human feedback represents a crucial milestone in aligning artificial agents with human values.",
    "In essence, generative models operate by iteratively predicting subsequent token distributions conditioned on contextual embeddings.",
    "Embracing transparent evaluation metrics is essential to mitigating bias and ensuring responsible stewardship of generative technologies.",
    "Recent advances in autonomous agentic workflows underscore the necessity of robust sandbox verification protocols in production deployments.",
    "Therefore, educational institutions must proactively formulate comprehensive governance policies to uphold scholarly authenticity.",
    "As we traverse this transformative technological paradigm, maintaining a nuanced perspective on algorithmic accountability is vital.",
    "Through systematic empirical validation, practitioners can effectively quantify model uncertainty and calibrate classification boundaries.",
    "This pivotal milestone exemplifies the profound synergy achieved when deep neural networks are synthesized with domain-specific knowledge bases."
]

DEFAULT_HUMAN_SAMPLES = [
    "We conducted three experiments in the physics laboratory to measure the gravitational constant using a simple pendulum and a digital stopwatch.",
    "During our third trial run, the string slipped slightly on the clamp which caused a brief timing discrepancy that we noted in our lab notebooks.",
    "Overall the experiment demonstrated that small variations in angle have a negligible effect when kept below ten degrees.",
    "My lab partner and I repeated the timing five times to calculate the standard deviation before submitting our draft to Professor Miller.",
    "I wasn't sure if the chemical reaction would turn purple or green, so we added two drops of hydrochloric acid just to be on the safe side.",
    "After looking at the microscope slides under 400x magnification, we could clearly spot the cell wall boundaries in the onion skin sample.",
    "The library was crowded on Tuesday afternoon, so we found an empty table in the basement to finish writing the introduction section.",
    "While reading through the survey responses, we realized that question four was slightly confusing to some of the first-year participants.",
    "I took notes during the lecture on Shakespeare's historical plays, especially focusing on how Henry V portrays national identity.",
    "Our team decided to use Python instead of Java for this project because the pandas library makes data manipulation so much faster.",
    "The bus was late arriving at the campus station this morning, so several students missed the first fifteen minutes of the quiz.",
    "We spent most of Friday evening debugging a stubborn null pointer error in the database connection module before pushing to Git.",
    "My grandmother always told me that good bread takes patience, requiring at least two hours to rise by the warm kitchen window.",
    "I walked down to the harbor early in the morning to watch the fishing boats unload their catch onto the wooden docks.",
    "We interviewed twelve local shop owners to understand how pedestrian street closures affected their weekend foot traffic.",
    "The graph on page twelve shows a clear spike during the summer months, which matches the seasonal sales pattern we saw last year.",
    "I had to rewrite the third paragraph three times because the sentence structure felt awkward and repetitive to read out loud.",
    "Our advisor suggested that we focus on a narrower subset of European history rather than trying to cover the entire century in twenty pages.",
    "The thermometer in our greenhouse was reading two degrees colder than the outdoor ambient air due to the shade of the tall oak trees.",
    "We packed up the measuring equipment carefully into the trunk of the car and headed back to the department before the rain started."
]


def load_dataset() -> tuple[list[str], list[int]]:
    """Load baseline dataset + any user-supplied custom files in data/custom_dataset/"""
    texts: list[str] = []
    labels: list[int] = []  # 1 = AI, 0 = Human

    # 1. Add baseline curated corpus
    for text in DEFAULT_AI_SAMPLES:
        texts.append(text)
        labels.append(1)

    for text in DEFAULT_HUMAN_SAMPLES:
        texts.append(text)
        labels.append(0)

    # 2. Check for user-provided custom AI files
    custom_ai_count = 0
    for file_path in AI_DATA_DIR.glob("*.txt"):
        try:
            content = file_path.read_text(encoding="utf-8", errors="ignore").strip()
            # Split into individual sentences or paragraphs
            lines = [line.strip() for line in content.split("\n") if len(line.strip().split()) >= 6]
            for line in lines:
                texts.append(line)
                labels.append(1)
                custom_ai_count += 1
        except Exception as e:
            logger.warning("Could not read custom AI file %s: %s", file_path, e)

    # 3. Check for user-provided custom Human files
    custom_human_count = 0
    for file_path in HUMAN_DATA_DIR.glob("*.txt"):
        try:
            content = file_path.read_text(encoding="utf-8", errors="ignore").strip()
            lines = [line.strip() for line in content.split("\n") if len(line.strip().split()) >= 6]
            for line in lines:
                texts.append(line)
                labels.append(0)
                custom_human_count += 1
        except Exception as e:
            logger.warning("Could not read custom Human file %s: %s", file_path, e)

    logger.info(
        "Dataset assembled: %d total samples (%d AI, %d Human). Custom files contributed: %d AI, %d Human.",
        len(texts),
        sum(labels),
        len(labels) - sum(labels),
        custom_ai_count,
        custom_human_count,
    )
    return texts, labels


def build_pipeline() -> Pipeline:
    """
    Build dual-vector morphological and syntactic FeatureUnion pipeline:
    - Word N-grams (1-2) with sublinear TF scaling
    - Char-WB N-grams (3-5) for sub-word structural morphemes
    - Calibrated Logistic Regression classifier (L2 regularization, balanced weights)
    """
    feature_union = FeatureUnion([
        ("word_tfidf", TfidfVectorizer(
            ngram_range=(1, 2),
            sublinear_tf=True,
            max_features=5000,
            strip_accents="unicode",
        )),
        ("char_tfidf", TfidfVectorizer(
            analyzer="char_wb",
            ngram_range=(3, 5),
            sublinear_tf=True,
            max_features=8000,
        )),
    ])

    clf = LogisticRegression(
        C=2.5,
        solver="lbfgs",
        max_iter=1000,
        class_weight="balanced",
        random_state=42,
    )

    return Pipeline([
        ("features", feature_union),
        ("clf", clf),
    ])


def train_and_export():
    """Execute training, evaluate metrics, and export custom model checkpoint."""
    logger.info("Starting VeritasDoc custom model training pipeline...")
    texts, labels = load_dataset()

    X_train, X_test, y_train, y_test = train_test_split(
        texts,
        labels,
        test_size=0.25,
        random_state=42,
        stratify=labels,
    )

    logger.info("Training feature extractor and calibrated classifier on %d samples...", len(X_train))
    pipeline = build_pipeline()
    pipeline.fit(X_train, y_train)

    # Evaluate
    preds = pipeline.predict(X_test)
    probs = pipeline.predict_proba(X_test)[:, 1]

    acc = accuracy_score(y_test, preds)
    f1 = f1_score(y_test, preds, zero_division=0)

    logger.info("=== Custom Model Validation Results ===")
    logger.info("Accuracy: %.2f%%", acc * 100.0)
    logger.info("F1 Score: %.4f", f1)
    print("\n" + classification_report(y_test, preds, target_names=["Human (0)", "AI (1)"]))

    # Save trained model pipeline
    model_output_path = CUSTOM_MODELS_DIR / "custom_detector.joblib"
    joblib.dump(pipeline, model_output_path, compress=3)
    logger.info("Model saved successfully to: %s", model_output_path)

    # Save model metadata
    metadata = {
        "model_name": "Veritas Proprietary Core (Custom Model)",
        "model_id": "core_custom",
        "created_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
        "accuracy_pct": round(acc * 100.0, 2),
        "f1_score": round(f1, 4),
        "training_samples": len(texts),
        "architecture": "Dual-Vector Morphological & Syntactic TF-IDF + Calibrated Classifier",
        "input_type": "sentences",
        "file_name": "custom_detector.joblib",
    }

    metadata_path = CUSTOM_MODELS_DIR / "model_metadata.json"
    with open(metadata_path, "w", encoding="utf-8") as f:
        json.dump(metadata, f, indent=2)

    logger.info("Metadata saved to: %s", metadata_path)
    logger.info("Custom model training complete! Ready for neural consensus integration.")
    return model_output_path, metadata


if __name__ == "__main__":
    train_and_export()