Spaces:
Sleeping
Sleeping
Commit History
-
Fix jiwer crash on post-normalisation empty refs; register SLR106/105 datasets
jefffffff9 +1: Claude Sonnet 4.6compute_metrics: pre-apply lowercasing + punctuation removal before filtering empty references — a label like "?" passes r.strip() but becomes "" after RemovePunctuation, causing jiwer to raise ValueError mid-evaluation. Pass already-normalised strings directly to jiwer.cer / jiwer.wer so no second transform is applied. web_harvester: add commented-in registry entries for OpenSLR SLR106 (10k clean Guinea Pular utterances, 49 speakers, CC BY-SA 4.0) and SLR105 (Guinea radio corpus, Pular-tagged validation split). Both are uncommented once the audio is uploaded to HF — best available Guinea Pular ASR data. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>6682858
-
Add Adlam/Pular Fula integration: transliterator + 3 new datasets + normalisation pipeline
jefffffff9 +1: Claude Sonnet 4.6- src/data/adlam.py: full Adlam↔Latin transliterator for Pular (Guinea Fula) — bidirectional char map (U+1E900–U+1E95F), digraph-aware latin_to_adlam(), contains_adlam(), normalize_pular() (Adlam→Latin + NFC + lowercase) - src/data/web_harvester.py: add 3 Fula training sources to HF_DATASET_REGISTRY: Pullo-Africa-Protagonist/Fula-pular (9,761 audio rows, primary), guizme/adlam_fulfulde (51 Adlam-script rows, auto-converted to Latin) - notebooks/kaggle_master_trainer.ipynb: Cell 11 gets inline _normalize_pular (Adlam→Latin); Cell 12 applies it for lang=='ful' in prepare_dataset - app.py: import normalize_pular; apply in _do_asr and _convo_pipeline for Fula; update Self-Teaching tab help text to list all 3 Fula datasets Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>ced078c
-
Fix: add trust_remote_code=True for datasets 3.x compatibility
Broulaye Doumbia71bb3bc
-
Phase 3: Voice-to-Voice S2S pipeline — F5-TTS, LLM brain, CER metric
jefffffff9 +1: Claude Sonnet 4.6app.py: - Add LLM_MODEL_ID env var (default: Qwen/Qwen2.5-72B-Instruct) - Import GemmaClient + bam_normalize at module level - Add _voice_ref_path / _voice_ref_text / _llm_client state - Add set_voice_reference(): converts MP3->24kHz WAV, auto-transcribes - Add _convo_pipeline(): ASR -> bam_normalize -> LLM (phonetic Bambara system prompt) -> F5-TTS (voice ref) with MMS-TTS fallback - handle_ask() accepts convo_mode=bool, routes to _convo_pipeline or the original sensor pipeline accordingly - Tab 1 UI: Conversation Mode toggle, Voice Reference upload accordion, stop_recording auto-submit for true back-to-back loop src/tts/f5_tts.py (new): - Lazy-loaded F5TTS wrapper; synthesize() with ref_wav + ref_text - to_wav_24k() resampler (F5-TTS needs 24 kHz input) - Graceful fallback (returns None) when f5-tts not installed src/data/bam_normalize.py (new): - _bam_norm(): ou->u, dj->j, gn->ny, ch->c, oo->o-open, ee->e-open - Used at inference (app.py) and training (notebook Cell 11) req…8952fff
-
Fix Self-Teaching tab: float sliders, deduplication, Kaggle API fallback
jefffffff9 +1: Claude Sonnet 4.6- Cast slider values to int (Gradio returns float; broke Wikipedia aplimit and Kaggle ds.take() in notebook Cell 9) - Deduplicate dataset_sources.jsonl on each HF import (avoid loading same dataset multiple times per training run) - Kaggle trigger: try Python API (KaggleApiExtended) first to avoid binary PATH issues; subprocess + shutil.which as fallback - Fix Fula dataset registry: fleurs uses legacy script loader incompatible with datasets>=3.0; switch to google/WaxalNLP ful_asr - Fix UI description: HF import writes to dataset_sources.jsonl (streamed by Kaggle), not to corrections.jsonl Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>24b1617 -
Fix Wikipedia and HF harvest buttons
jefffffff9 +1: Claude Sonnet 4.6Wikipedia fix: - Add User-Agent header to all requests (Wikipedia API returns 403 without it) - Extract shared _upload_jsonl() helper to reduce code duplication HF harvest fix: - Remove audio re-upload approach (500 HTTP requests -> Gradio timeout) - Instead write dataset references to dataset_sources.jsonl on the Hub - One small file upload (~instant) instead of gigabytes of audio - Kaggle notebook Cell 4 now reads dataset_sources.jsonl dynamically: any dataset registered in Space Self-Teaching tab is automatically picked up at training time with no notebook edits needed Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a5737ac -
Add mass training + internet self-teaching features
jefffffff9 +1: Claude Sonnet 4.6New Tab 5 — Bulk Upload: - Upload ZIP of audio files + CSV (filename,transcription) in one operation - Batch-inserts all samples into corrections.jsonl + uploads audio to Hub New Tab 6 — Self-Teaching: - Wikipedia harvest: bm/ff Wikipedia API → sentence text → vocabulary.jsonl (868 Bambara articles, 17k+ Fula articles) - HF dataset import: RobotsMali/jeli-asr (33k Bambara) + google/fleurs ff_sn (Fula) → corrections.jsonl, one click - Auto-training trigger: fires Kaggle kernel via REST API when corrections.jsonl reaches AUTO_TRAIN_THRESHOLD (default 50) entries Requires KAGGLE_USERNAME + KAGGLE_KEY Space secrets New src/data/web_harvester.py: - harvest_wikipedia_text(lang, max_articles) — MediaWiki API, no deps - harvest_hf_audio(lang, token) — generator yielding (wav_bytes, text, path) - _numpy_to_wav_bytes() — stdlib-only WAV encoder Auto-trigger also fires after each Tab 2 correction save. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>eddcaea
-
Add multi-user speaker profiles, collective voice, and mode toggle
jefffffff9 +1: Claude Sonnet 4.6Task 1 — SpeakerProfileManager (src/voice/speaker_profiles.py): SpeechBrain ECAPA-TDNN identifies speakers by cosine similarity of 192-d embeddings (threshold 0.75). Each user gets user_N_sb.npy (running-average SpeechBrain embedding) and user_N_ov.npy (running-average OpenVoice V2 tone-color SE). New utterances update the running average so profiles sharpen over time. Task 2 — get_collective_embedding() (speaker_profiles.py): Loads all user_N_ov.npy files and returns their mean vector — the "Collective Voice" that blends all known speakers into one SE. Task 3 — Voice Mode toggle (app_lab.py + src/tts/voice_cloner.py): Added Individual / Collective radio to the UI. In Individual mode the TTS output is cloned into the last detected speaker's voice via OpenVoice V2 ToneColorConverter. In Collective mode the mean SE is used. Text-input path falls back to the collective SE. Both modes degrade gracefully to base VITS voice if OpenVoice is not ready. requirem…49910a9 -
Add confidence loop, curiosity engine, and lightweight TTS
jefffffff9 +1: Claude Sonnet 4.6Task 1 — TTS refactor (src/tts/waxal_tts.py): Switch Bambara TTS from Qwen2-based MALIBA-AI to ynnov/ekodi-bambara-tts-female (VitsModel + AutoTokenizer) — much lighter on CPU Basic; no trust_remote_code needed. Fula is an explicit generate_pular_tts() placeholder returning None until the model is trained. Task 2 — Active learning / confidence loop (src/engine/stt_processor.py): transcribe_with_confidence() wraps Whisper generate() with output_scores=True and computes avg_logprob via compute_transition_scores(). If avg_logprob < -1.0, app_lab.py replaces the transcript with CONFUSION_PROMPT so the LLM asks the user in English to repeat and explain the word. Task 3 — Proactive gaps (src/engine/curiosity.py): CuriosityEngine.maybe_ask() fires every 5 interactions — sends the last 10 vocabulary entries to Qwen and appends a 🌱 question in the chat asking the user to teach a missing agricultural term. Task 4 — Zero-cost persistence: MemoryManager._push_to_h…3657607 -
Fix Bambara TTS red state + surface detailed errors in UI
jefffffff9 +1: Claude Sonnet 4.6- src/tts/waxal_tts.py: load MALIBA-AI/bambara-tts directly via AutoModelForCausalLM (trust_remote_code=True) — no pip install needed at runtime; HF Spaces blocks GitHub outbound so the old lazy subprocess install was silently failing every time - app_lab.py: wrap process_audio / process_text in try/except so exceptions surface as '❌ Error: ...' in the status box instead of a generic Gradio popup with no message; add logging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>082adaa -
Fix: install maliba-ai lazily at runtime, not at build time
jefffffff9 +1: Claude Sonnet 4.6maliba-ai 2.0.0b0 has strict pins (soundfile==0.13.1, librosa==0.11.0) that conflict with our requirements.txt. Removing it from requirements.txt and installing it via subprocess on the first Bambara TTS call instead. This pattern is standard for HF Spaces when a dependency has irreconcilable version conflicts with the rest of the stack. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c154b17 -
Phase 2: Waxal TTS — Bambara voice output + Fula training notebook
jefffffff9 +1: Claude Sonnet 4.6TTS engine (src/tts/waxal_tts.py): - Bambara: MALIBA-AI/bambara-tts (non-Meta, Mali community, 10 native speakers) Loads via custom maliba-ai package; writes WAV to tempfile, reads back as numpy - Fula: ous-sow/fula-tts (our own model, loads once trained) Lazy-loads; gracefully reports 'not trained yet' until notebook is run - WaxalTTSEngine.audio_to_gradio() converts float32 → int16 for gr.Audio app_lab.py: - Imports WaxalTTSEngine; preloads both models at startup in background - _run_llm_and_tts() shared core: Gemma → memory → TTS → return audio tuple - process_audio() and process_text() now return 4-tuple (adds audio_out) - UI: added gr.Audio output widget with autoplay; status bar shows TTS readiness per language (🟢/🟡/🔴) Training notebook (notebooks/train_fula_tts.ipynb): - 9 cells: GPU check → install → HF login → config → load WaxalNLP ful_tts → prepare dataset (WAV + metadata.csv) → Coqui VITS trainer → push to HF Hub → synthesis test - Runs on Kaggle T4 (~2-3h); …dd38e25 -
Fix: switch LLM to Qwen2.5-72B-Instruct (Gemma not on HF free tier)
jefffffff9 +1: Claude Sonnet 4.6google/gemma-3-4b-it requires a paid provider on HF Serverless Inference. Qwen/Qwen2.5-72B-Instruct is available on the free tier and has better multilingual coverage for Bambara/Fula context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>61e52d7 -
Phase 1: Sahel-Voice-Lab — The Memory Loop
jefffffff9 +1: Claude Sonnet 4.6New project pivot: self-learning voice assistant for Bambara/Fula using 100% non-Meta tech stack. New files: - app_lab.py — Gradio UI: push-to-talk + last 5 words panel - src/memory/memory_manager.py — persists vocabulary.jsonl to HF Hub - src/llm/gemma_client.py — Gemma via HF Serverless Inference API - data/vocabulary.jsonl — empty initial vocabulary store Flow: audio → Whisper STT → Gemma (with vocabulary context) → teaching intent → MemoryManager.add_word_pair() → Hub push question intent → answer from vocabulary conversation → natural reply README.md updated: app_file changed to app_lab.py, stack documented. Stack: openai/whisper-large-v3-turbo (STT), google/gemma-3-4b-it (LLM), Waxal TTS (Phase 2), HF Dataset for memory. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>096b19d
-
Add phrase library + fuzzy matcher for general Bambara/Fula conversation
jefffffff9 +1: Claude Sonnet 4.6Core problem: the app responded to 'I ni ce' with a sensor default message because it had no conversational knowledge. Fixed with a phrase-library approach that requires zero ML training. New files: - data/phrases/bambara.json — 37 curated Bambara phrases with Whisper variants - data/phrases/fula.json — 38 curated Fula/Pular phrases with Whisper variants - src/conversation/phrase_matcher.py — fuzzy matcher using rapidfuzz (falls back to substring match if rapidfuzz unavailable) app.py changes: - Pipeline tries phrase library FIRST; sensor pipeline is fallback - Low-confidence unknown intents return 'I didn't understand' in native lang - New 'Knowledge Base' tab: paste phrase pairs (native|english) — immediate - YouTube import tab: download audio + save transcription to training dataset - User additions persisted to HF Hub (survive Space restarts) - Phrase additions loaded from Hub in background at startup requirements.txt: added rapidfuzz==3.13.0, yt-dlp==2025.3.31 Co-Author…6f4d8d0 -
Improve Bambara/Fula quality + add full correction UI
jefffffff9 +1: Claude Sonnet 4.6voice_responder.py: - Every template now returns (native_text, english_translation) tuple - Added greeting/farewell/thanks/soil_ok/weather_ok/pest_ok templates - Short MMS-TTS-optimised sentences kept (≤6 words) intent_parser.py: - Added greeting, thanks, farewell intents with Bambara/Fula/French/English keywords - Added French and English keywords to agricultural intents app.py: - _run_pipeline returns (transcript, english_translation, response_text, audio) - Tab 1: added English translation box; added "Send to Correction" button - Tab 2: added English translation field + corrected_english field + corrected_response field - "Send to Correction" auto-populates all Tab 2 fields from Tab 1 in one click - _save_feedback_to_hub stores corrected_english and corrected_response Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>d351193 -
Switch training dataset from google/waxal to google/fleurs
jefffffff9 +1: Claude Sonnet 4.6google/waxal was removed from HuggingFace Hub. google/fleurs is the maintained replacement with identical column schema (audio, transcription). Subset mapping: bam → bam_ML, ful → ff_SN. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>d2183cd
-
Initial commit: Sahel-Agri Voice AI
jefffffff976db545