Spaces:
Sleeping
Sleeping
Commit History
-
Updated Research Paper, README, and old gradio about info, along with other docs.
OliverPerrin8f5fea2
-
Fixed compiling issue, added legnth penalty, and atttempting freezing encoder layers 0-5 to lower parameters and preserve T5's langauge understanding.
OliverPerrinbaf3026
-
Clean up codebase and fix training bugs
OliverPerrin- Fixed total_loss tracking for validation (early stopping works now) - Lowered emotion threshold from 0.5 to 0.3 for multi-label - Optimized full.yaml config for faster training (50k samples cap) - Consolidated utils into core.py - Removed unused modules and scripts - Fixed Gutenberg download key format issue - Updated visualize_training default model name - Switch MLflow to SQLite backend1601799
-
Expand datasets: GoEmotions (28 emotions), Yahoo Answers (10 topics), BookSum
OliverPerrin- Updated download_data.py with new dataset downloaders - Updated preprocess_data.py to handle JSONL format - Updated labels.json with 28 emotions and 10 topics - Updated datasets.yaml with new dataset configs - Updated full.yaml training config (warmup_steps: 1000) - Training results: 74% topic accuracy, ROUGE 0.2849becd3c
-
feat: Add FLAN-T5 compatibility with relative position bias
Major changes: - Implement T5RelativePositionBias for encoder/decoder self-attention - T5 uses unscaled attention (no sqrt(d_k) scaling) - Add float32 softmax path for numerical stability - Switch to aot_eager compile backend (inductor causes NaN in decoder backward) - Add gated-gelu activation support for T5 FFN - Fix vocab size handling (32100 vs 32128) - Update model configs for T5-base architecture - Add dev/medium training configs for faster iteration - Optimize training for ~4 min dev runs on RTX 4070 The model now correctly loads FLAN-T5-base weights and generates coherent summaries with proper encoder-decoder architecture.486475d
-
chore: snapshot current refinements
185b05e
-
Reformatted Project Structure
133ab4d