Spaces:
Sleeping
Sleeping
Commit History
-
Fix regression: restore balanced training settings, add technical manual filter
OliverPerrinTraining configs: - Restored LR: 3e-5 (medium), 2e-5 (full) - Restored weight decay: 0.01 - Restored label smoothing: 0.1 - Restored dropout: 0.1 - Restored task weights: emotion=1.5, topic=0.5 - Restored epochs: 5 (medium), 8 (full) - Restored early stopping patience: 3 Discovery dataset: - Added technical manual filter (pottery recipes, camera specs, etc.) - These non-narrative texts produce garbage summaries9710200 -
Major improvements: real titles, anti-overfitting, better demo UX
OliverPerrinTraining configs (medium.yaml, full.yaml): - Reduced epochs: 5→4, 8→6 (stop before overfitting) - Lower LR: 3e-5→2e-5, 2e-5→1.5e-5 - Increased weight decay: 0.01→0.02 - Increased label smoothing: 0.1→0.15 - Boost emotion weight: 1.5→2.0 (tends to underfit) - Reduce topic weight: 0.5→0.3 (overfits quickly) - Earlier early stopping: patience 3→2 - Increased dropout: 0.1→0.15 Data download (download_data.py): - BookSum: Extract real book titles and chapter names - arXiv: Extract paper titles from abstract first sentence - Gutenberg: Parse JSON metadata for real titles and authors Discovery dataset (build_discovery_dataset.py): - Use real titles from source data - Increased default sizes: 300 books, 200 papers, 100 literary - Better fallback for missing titles - Improved error handling Demo (demo_gradio.py): - Cleaner card format with prominent titles - Confidence badges (green/yellow/red) - Dynamic counts in header - Better table layout for topic/emotion0fe274c
-
Update LexiMind: improved training, model architecture, and evaluation
OliverPerrin076bc18
-
Fix Pylance type errors, add inductor compilation support
OliverPerrin- Add cast() for registered buffer access (pe, cos, sin) in decoder.py, positional_encoding.py, and attention.py to satisfy Pylance type checking - Add cast() for compile_model return types in train.py - Add type: ignore for rouge_score import (no type stubs available) - Add safe_compile.py for torch.compile with inductor backend (default mode) - Add nan_debugger.py for debugging NaN/Inf during training - Update configs: batch_size=12 for medium/full, fix layer counts to match FLAN-T5-base - Add benchmark mode to train.py for speed testing without saving checkpoints - Suppress torch inductor warnings that interfere with tqdm progress bars All checks pass: ruff, mypy, Pylancecf79a6c
-
feat: Add FLAN-T5 compatibility with relative position bias
Major changes: - Implement T5RelativePositionBias for encoder/decoder self-attention - T5 uses unscaled attention (no sqrt(d_k) scaling) - Add float32 softmax path for numerical stability - Switch to aot_eager compile backend (inductor causes NaN in decoder backward) - Add gated-gelu activation support for T5 FFN - Fix vocab size handling (32100 vs 32128) - Update model configs for T5-base architecture - Add dev/medium training configs for faster iteration - Optimize training for ~4 min dev runs on RTX 4070 The model now correctly loads FLAN-T5-base weights and generates coherent summaries with proper encoder-decoder architecture.486475d
-
Refactor: Consolidate dependencies, improve testing, and add CI/CD
- Consolidated project dependencies into pyproject.toml. - Removed requirements.txt, requirements-dev.txt, and setup.py. - Removed scripts/download_data.sh. - Added comprehensive tests for src/data, src/training, and src/utils. - Fixed FutureWarning in Trainer regarding torch.amp.GradScaler. - Integrated mlflow for experiment tracking in Trainer. - Added ruff and mypy for linting and type checking. - Added .pre-commit-config.yaml for git hooks. - Added GitHub Actions CI workflow (.github/workflows/ci.yml).f6d689c
-
Added pretrained BART wights for the encoder/decoder while keeping custom architectiure and task heads
3225a94
-
chore: snapshot current refinements
185b05e
-
Reformatted Project Structure
133ab4d