Spaces:
Running
Running
Commit History
-
feat: sync robot preferences to HF, thinking movement + wait/announce on plan
Juan Jimenez Carrero +1: Claude Opus 4.8Settings sync (user noticed prefs didn't sync): the HF dataset only carried AppData. Now a NON-SECRET settings subset (language, robotName, voice, activationMode, allowInterrupt, inventorySort) rides along in a separate cookaiware-settings.json. Secrets (openaiKey, hfToken) and device-local state (lastSyncedAt, hfDatasetSync) are deliberately never uploaded. pushToCloud sends them; "Load from cloud" and the auto-sync "keep cloud" branch pull + validate them (applySyncableSettings keeps the current value for any invalid field). Silent-thinking fix: the menu designer takes ~10-20 s during which the robot sat still and mute. Now: - VoiceController.startThinking/stopThinking pulses a VARIED thinking body language (mostly "thinking", an occasional glance) every 2.6 s; app wraps both designAndCommitPlan and designMeal awaits with it (finally-stopped, also cleared on session stop). Only active during a live voice session. - Prompt now tells Reachy to ask the user to wait BEFORE calling…b5352be
-
feat: pick robot voice, lock/undo by voice, protect locked meals; fix key text
Juan Jimenez Carrero +1: Claude Opus 4.8Voice picker (like the official conversation app): new "Robot voice" setting in Talk to Reachy lists the OpenAI gpt-realtime voices (marin & cedar — the newest, most natural — plus alloy/ash/ballad/coral/echo/sage/shimmer/verse). Threaded settings.voice -> controller -> RealtimeSession.output.voice (was hardcoded shimmer). Default is now marin; applies on the next Talk-to-Reachy connect. Lock + undo by voice (parity with the app buttons): - set_meal_lock tool -> app.setMealLock(day, meal, locked): keep/free one meal. - undo_plan_change tool -> app.undoPlan(): restore the previous menu. - Reused resolveDay so 'today'/'tomorrow'/weekday all work. Locked meals are now protected: set_day_meal (and the app's designMeal) refuse a locked slot, returning a 'locked' error; the prompt tells Reachy to ask the user and only unlock+replace on confirmation. Previously voice could silently overwrite a kept meal. Accuracy fix (user-flagged): the OpenAI key hint wrongly said vision uses gpt-4o-mini …f080bc3
-
feat: voice proactivity/interrupt settings + intelligent matrix-based menus
Juan Jimenez Carrero +1: Claude Opus 4.8Voice settings (apply on the next session): - Activation mode: respond to any speech, or ONLY when addressed by the robot's name (prompt-enforced wake rule; stays fully silent otherwise). - Allow interruptions: maps to server VAD interrupt_response, keeps the mic pump live while the robot speaks, cuts local TTS on speech_started, and skips input_buffer clears that would clip a barge-in. Meal planning overhaul: - Family size is now REQUIRED to plan (gate + CTA to Settings; voice tools return an instructive error that makes the model ask and call set_family). - 'Who eats what' 7×3 matrix (days × breakfast/lunch/dinner): each cell holds adults+children for that meal; tap → steppers, 'not planned' clears; stored as family_profile.plan_matrix (legacy weekday/weekend schedule auto-derives the default matrix, preserving the original app's behavior). - Intelligent generation through gpt-realtime (the key's only scope): * Button: short headless TEXT realtime session (planLLM.ts…cd6a098
-
fix: garbled voice on tool calls, sync safety, edit-not-duplicate, calmer persona
Juan Jimenez Carrero +1: Claude Opus 4.8- Garbled/'buzzing' speech before plan creation: we requested a new response while the previous one was still streaming, so two responses' audio chunks interleaved on one playback timeline. Responses are now gated: track response.created/done and defer response.create until the active one ends. - Auto-sync: enabling the toggle now downloads the cloud copy first and, if it differs, asks which side to keep (cloud vs this device) before the first auto-save — it can no longer silently overwrite. Switch styling fixed (the global input rules were deforming the checkbox). - New update_inventory_item tool: edits an existing item in place (location, expiration, quantity, rename); prompt now forbids add_inventory for changes, which was creating duplicates. - Sort by Location added (groups by location, then name; no-location last). - Robot name setting (default Reachy) fed into the voice persona; persona rewritten to be REACTIVE: do only what was asked, one-two short sentences, …ff39cb2
-
feat: conversation minimize, HF sync UX, inventory editing/sort/drag/tags
Juan Jimenez Carrero +1: Claude Opus 4.8Conversation: - Minimize/expand the transcript panel (chevron + tap-header-to-expand; minimized header shows the message count). HF sync: - Save/Load side by side; each asks for confirmation and states what will be overwritten (item count, plan y/n, shopping count) before replacing. - Auto-sync toggle: debounced upload ~4s after each change (reuses hfDatasetSync); shows 'last synced' relative time. Inventory: - Edit any item in a styled modal (name/qty/unit/expiration/location); new inv.normalizeItemInput replaces the item in place. - Restyled 'Use' modal (was window.prompt) with an amount field + 'use all'. - Delete asks for confirmation. - Expiration date now shown on every card (clock icon + date, red if overdue); sort control: Expiration / Name / Custom. - Custom mode: pointer-based drag-to-reorder (works on touch), persisted. - Storage location shown as a chip; tapping it filters the list to that location; add/edit location field autocompletes from existing locations…a03aea0
-
fix: HF sync save — don't fail on 'repo already exists' + surface real errors
Juan Jimenez Carrero +1: Claude Fable 5'Load from my HF account' worked but 'Save' always failed once the dataset existed: ensureRepo ran createRepo first and only swallowed errors matching /409|exist/i, but the Hub words the already-exists error differently, so every save after the first aborted before uploading. createRepo failures are now never fatal (uploadFiles is the authority and raises the real error if the repo is truly missing), and sync toasts now include the actual error message instead of the generic 'check the token'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>37cc891
-
feat: robot-only voice — remove phone mode, prime robot mic, visible motion
Juan Jimenez Carrero +1: Claude Fable 5Per user decision, the conversation is with the robot, full stop: - Removed the mic-source toggle and all phone-mic/phone-speaker paths. Voice requires the robot: its mic in, its speaker out (replaceTrack). Clear errors if either is unavailable instead of silent fallbacks. - Robot hearing fix: remote WebRTC audio tracks feed SILENCE into WebAudio on Chrome unless also consumed by a media element — prime the robot mic track with a hidden muted <audio> before the PCM pump reads it. This is the prime suspect for 'Reachy can't hear me'. - Motion: keep the 20 Hz head loop but make it clearly visible — speech envelope has a 0.35 floor while speaking (up to ~9° pitch), slow attentive sway while listening, antennas ease up smoothly while talking (slow lerp, never audio-rate). [motion] console logs every 3 s report level/env/ok so a failing send is visible in the console. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>45d3b03
-
feat: voice — talk to Reachy via OpenAI Realtime, wired to the kitchen tools
Juan Jimenez Carrero +1: Claude Fable 5Browser voice loop modeled on Pollen's reachy-stories (static-Space pattern): - OpenAI Realtime (gpt-realtime) over WebSocket with the key in the Sec-WebSocket-Protocol slot; key pasted in Settings, kept on-device. - Audio: getUserMedia patched to a synthetic destination so TTS plays on the robot speaker; robot mic captured via an RTCPeerConnection interceptor in index.html; 24 kHz PCM16 mic pump (AudioWorklet) → input buffer. - Function calling wired to the ported TS domain (add/cook/list inventory, set family, generate plan, get day, generate/list shopping) — voice and touch share state and update live. - Mic-source toggle (Reachy's mic at home / phone mic away); AI voice to the robot speaker with client-side head-wobble, or the phone speaker away. - Floating 'Talk to Reachy' button with connecting/listening/speaking states. - enableMicrophone enabled so the SDK sets up the outbound audio path. Verified in preview: UI, no-key guidance, settings. Live voice needs the user…e6c8aff
-
feat: add mobile-first JS web app (Pollen's new Reachy Mini JS workflow)
Juan Jimenez Carrero +1: Claude Fable 5New web/ TS+Vite app served as this Space's page, alongside the existing Python on-robot app — one Space, two flavours: - Host-shell integration per ts/APP_CREATION_GUIDE.md: dispatch/embed split, @pollen-robotics/reachy-mini-sdk pinned 1.8.1, onLeave -> safelyReturnToPose - Domain logic ported 1:1 from Python (inventory, algorithmic meal planner, shopping shortfalls, family profile); parity-checked against the Python implementation on the repo's real data (scripts/parity-check.ts) - Spanish keywords added to meal categorization (data is in Spanish) - Mobile-first UI: bottom tabs, 44px targets, dark/light via host theme, bilingual ES/EN; localStorage + JSON export/import + opt-in HF dataset sync - Robot gestures: rAF-tweened antenna wiggle, happy dance, concerned tilt, sweep look, greeting - Frontmatter: app_build_command/app_file/hf_oauth + reachy_mini_js_app tag Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>a519434