Spaces:
Sleeping
Sleeping
Commit History
-
Fixed HF build issue
OliverPerrin6553b4f -
Added Bert Baseline comparison training script and ignored unneccessary files
OliverPerrinaca8362
-
Cleaned up code, added multiseed training wrapper, PyTorch profiler training option, updated gradio demo, made changes to research paper to match new changes and new training results from adding new training techniques, architecture.md now explains all designs and decisions
OliverPerrin90a2698
-
Full training results & evaluation with BERTScore
OliverPerrinTraining: - Completed 7 epochs (~6 hours) with early stopping - Optimized config: batch=10, grad_accum=4, lr=3e-5, topic_weight=0.3 - Froze bottom 4 encoder layers for stable transfer learning Evaluation Results: - Summarization: ROUGE-1=0.3064, BERTScore F1=0.8300 - Topic Classification: 85.2% accuracy (7 classes) - Emotion Detection: F1=0.1987 (28 classes, multi-label) Paper (docs/paper.tex): - Added comprehensive experimental results section - Added training dynamics figures - Fixed Unicode characters for LaTeX compatibility - Updated methodology with actual datasets used New scripts: - scripts/evaluate.py: Comprehensive evaluation with BERTScore - Rebuilt discovery dataset with 1000 samples for demo1e95f87
-
Add evaluation_report.json for metrics display
OliverPerrin7e802ad -
Medium training run
OliverPerrinb93250a -
Medium training run
OliverPerrin277b4d1
-
Medium training run
OliverPerrin8a54fec -
Medium training run
OliverPerrine12987d
-
New trianing run on english only data
OliverPerrin40cadca -
Updated Training run, fixed dataset langauge issue
OliverPerrine3422d2 -
Updated the model with new bug fixes
OliverPerrin5c8762c -
Fixed compiling issue, added legnth penalty, and atttempting freezing encoder layers 0-5 to lower parameters and preserve T5's langauge understanding.
OliverPerrinbaf3026
-
Clean up codebase and fix training bugs
OliverPerrin- Fixed total_loss tracking for validation (early stopping works now) - Lowered emotion threshold from 0.5 to 0.3 for multi-label - Optimized full.yaml config for faster training (50k samples cap) - Consolidated utils into core.py - Removed unused modules and scripts - Fixed Gutenberg download key format issue - Updated visualize_training default model name - Switch MLflow to SQLite backend1601799
-
Update Gradio demo, inference factory, and evaluation results
OliverPerrine3d3925
-
Update LexiMind: improved training, model architecture, and evaluation
OliverPerrin076bc18
-
Improve summarization output quality
OliverPerrin- Add _format_summary() for proper capitalization and punctuation - Add repetition_penalty (1.2) to reduce repetitive outputs - Fix period spacing and sentence capitalization - Remove leading special characters from generated text69b8f98 -
Expand datasets: GoEmotions (28 emotions), Yahoo Answers (10 topics), BookSum
OliverPerrin- Updated download_data.py with new dataset downloaders - Updated preprocess_data.py to handle JSONL format - Updated labels.json with 28 emotions and 10 topics - Updated datasets.yaml with new dataset configs - Updated full.yaml training config (warmup_steps: 1000) - Training results: 74% topic accuracy, ROUGE 0.2849becd3c -
Full training run, code cleanup, mypy/ruff fixes
OliverPerrin- Completed 3-epoch full training run (21 hours) - Emotion F1: 94.6%, Topic Accuracy: 94.2%, ROUGE-like: 0.36 - Code simplification and standardized docstrings across all modules - Fixed all mypy type errors (47 files pass) - Fixed all ruff linting errors and reformatted code - All 75 tests passing - Added GPU optimizations: TF32, Flash Attention, memory-efficient SDP - Training configs optimized for RTX 4070 12GB - tqdm progress bars for training and evaluationa47e5cf
-
Training run: dev config 1 epoch results
OliverPerrinResults: - Topic accuracy: 85.76% (Sports 96%, World 87%, Business 81%, Sci/Tech 78%) - Summarization ROUGE-like: 0.343, BLEU: 0.088 - Emotion F1 macro: 0.356 Updated inference pipeline, demo script, and evaluation outputs7bfcb3e -
feat: Add FLAN-T5 compatibility with relative position bias
Major changes: - Implement T5RelativePositionBias for encoder/decoder self-attention - T5 uses unscaled attention (no sqrt(d_k) scaling) - Add float32 softmax path for numerical stability - Switch to aot_eager compile backend (inductor causes NaN in decoder backward) - Add gated-gelu activation support for T5 FFN - Fix vocab size handling (32100 vs 32128) - Update model configs for T5-base architecture - Add dev/medium training configs for faster iteration - Optimize training for ~4 min dev runs on RTX 4070 The model now correctly loads FLAN-T5-base weights and generates coherent summaries with proper encoder-decoder architecture.486475d
-
Refactor: Consolidate dependencies, improve testing, and add CI/CD
- Consolidated project dependencies into pyproject.toml. - Removed requirements.txt, requirements-dev.txt, and setup.py. - Removed scripts/download_data.sh. - Added comprehensive tests for src/data, src/training, and src/utils. - Fixed FutureWarning in Trainer regarding torch.amp.GradScaler. - Integrated mlflow for experiment tracking in Trainer. - Added ruff and mypy for linting and type checking. - Added .pre-commit-config.yaml for git hooks. - Added GitHub Actions CI workflow (.github/workflows/ci.yml).f6d689c
-
refactor: update training code, history and remove large checkpoint to fix storage limit
0ec3f34
-
Summarization fix
bdb7386 -
Refactor: Fix tokenization, pathing, and track visuals with LFS
56880bb