feat: howdy friend static Space

.gitignore

@@ -0,0 +1 @@
+agents.local.md

README.md

@@ -0,0 +1,42 @@
+---
+title: Howdy Friend
+emoji: 👋
+colorFrom: yellow
+colorTo: green
+sdk: static
+pinned: false
+hf_oauth: true
+short_description: Reachy tracks your face and gets happy when you wave.
+tags:
+  - reachy_mini
+  - reachy_mini_js_app
+---
+
+# Howdy Friend
+
+Reachy Mini watches its camera, keeps its head (and body) pointed at the nearest face, gives a quick nod and chirp when it first notices you, and plays a happy clip from the Reachy Mini emotions library when you wave at it.
+
+Everything runs in the browser. The robot only streams video and receives motion commands over WebRTC, so this works on a Wireless Reachy Mini without loading the onboard computer.
+
+## How it works
+
+- **Face tracking**: MediaPipe Face Detector runs on the robot's video stream in the browser. The largest face drives a smoothed proportional controller on head yaw and pitch, sent with `setTarget()` at ~15 Hz. If the face drifts far to the side, the body turns to follow so the head never hits its yaw limit.
+- **Wave detection**: MediaPipe Hand Landmarker finds an open hand. The wrist's horizontal position is tracked over a 1.2 s window; two or more direction reversals with enough travel counts as a wave. A 5 s cooldown prevents double triggers.
+- **Reactions**: On first sight of a face, a short synthesized nod + chirp. On a wave, a random clip from a curated happy set in `pollen-robotics/reachy-mini-emotions-library`, played daemon-side with `playMove()` so motion and audio stay in sync. The clip is shifted by the current body yaw so the robot keeps facing you during the reaction.
+
+## Deploying
+
+No build step. Push these files to the Space:
+
+```
+git init -b main
+git remote add space https://huggingface.co/spaces/RickDakan/howdy_friend
+git add -A && git commit -m "feat: howdy friend"
+git push -u space main
+```
+
+Open the Space URL, sign in with Hugging Face, pick your robot.
+
+## Tuning
+
+Constants at the top of `app.js`: tracking gain, deadband, smoothing, body-follow threshold, wave sensitivity, cooldowns, and the list of reaction clips. Sign conventions for yaw/pitch are noted there in case your camera mount differs.

app.js

@@ -0,0 +1,637 @@
+// Howdy Friend - Reachy Mini face tracking + wave reaction
+// Bare HTML + CDN JS app using the Reachy Mini host shell (SDK 1.8.0).
+
+import { ReachyMini } from "https://cdn.jsdelivr.net/npm/@pollen-robotics/reachy-mini-sdk@1.8.0/+esm";
+
+const SDK_VERSION = "1.8.0";
+const MP_VERSION = "0.10.14";
+
+// ---------------------------------------------------------------------------
+// Tuning constants
+// ---------------------------------------------------------------------------
+
+// Sign conventions (robot frame, right-hand rule, z up):
+//   yaw   > 0 = head turns to the robot's LEFT
+//   pitch > 0 = nose DOWN
+// A face to the RIGHT of image center (ex > 0) means the robot must turn RIGHT (yaw decreases).
+// A face ABOVE image center (ey < 0) means the robot must look UP (pitch decreases).
+// Flip either sign below if your robot turns the wrong way.
+const YAW_SIGN = -1;
+const PITCH_SIGN = 1;
+
+const TRACK = {
+  hz: 15,                 // vision + command rate
+  gainDegPerSec: 45,      // how fast the head chases a face at full-frame error
+  deadband: 0.05,         // normalized error below which we don't move
+  smoothing: 0.35,        // 0..1, higher = snappier
+  maxPitchDeg: 30,
+  maxHeadYawFromBodyDeg: 55,   // hard SDK limit is 65
+  bodyFollowStartDeg: 22,      // head-body offset that starts the body turning
+  bodyRateDegPerSec: 40,
+  maxBodyYawDeg: 150,          // hard SDK limit is 160
+  noFaceResetSec: 3.0,         // ease back to center after this long without a face
+  noticeAfterAbsentSec: 4.0,   // "new arrival" nod only if no face was seen for this long
+};
+
+const WAVE = {
+  windowSec: 1.2,
+  minReversals: 2,
+  minTravelHandWidths: 1.0,
+  hysteresisHandWidths: 0.15,
+  cooldownSec: 5.0,
+};
+
+// Curated happy clips from pollen-robotics/reachy-mini-emotions-library
+const WAVE_CLIPS = ["welcoming1", "welcoming2", "enthusiastic2", "cheerful1", "success2", "loving1", "helpful1"];
+const EMOTIONS_BASE = "https://huggingface.co/datasets/pollen-robotics/reachy-mini-emotions-library/resolve/main/";
+
+const INIT_ANTENNAS = [-0.1745, 0.1745]; // radians, matches SDK INIT_POSE
+const DEG = Math.PI / 180;
+
+// ---------------------------------------------------------------------------
+// Dispatch: host shell vs embedded app
+// ---------------------------------------------------------------------------
+
+window.ReachyMini = ReachyMini;
+window.dispatchEvent(new Event("reachymini:ready"));
+
+const params = new URLSearchParams(window.location.search);
+const isEmbed = params.get("embedded") === "1" || params.get("embed") === "1";
+
+if (isEmbed) {
+  document.body.classList.add("embed");
+  runEmbed().catch((err) => {
+    console.error("[howdy-friend] boot failed", err);
+    log(`Boot failed: ${err?.message ?? err}`);
+    window.parent.postMessage(
+      { source: "reachy-mini", type: "embed:error", version: 1, message: String(err), fatal: true },
+      window.location.origin,
+    );
+  });
+} else {
+  document.body.classList.add("host");
+  const { mountHost } = await import(
+    `https://cdn.jsdelivr.net/npm/@pollen-robotics/reachy-mini-sdk@${SDK_VERSION}/host/dist/entry/auto.js`
+  );
+  mountHost({
+    appName: "Howdy Friend",
+    appIconUrl: "/public/icon.svg",
+    appEmoji: "👋",
+    enableMicrophone: false,
+  });
+}
+
+// ---------------------------------------------------------------------------
+// Embedded app
+// ---------------------------------------------------------------------------
+
+async function runEmbed() {
+  const { connectToHost } = await import(
+    `https://cdn.jsdelivr.net/npm/@pollen-robotics/reachy-mini-sdk@${SDK_VERSION}/host/dist/entry/embed.js`
+  );
+
+  const ui = bindUi();
+  setStatus("Connecting to robot", "");
+
+  const handle = await connectToHost();
+  const reachy = handle.reachy;
+  log(`Connected as ${handle.userName ?? "user"}`);
+
+  const detachVideo = reachy.attachVideo(ui.video);
+
+  setStatus("Loading vision models", "");
+  const vision = await loadVision();
+  log("Vision models ready");
+
+  // Start fetching emotion clips in the background; not required to start tracking.
+  const clips = new ClipCache();
+  clips.preload(WAVE_CLIPS).then((n) => log(`${n}/${WAVE_CLIPS.length} reaction clips ready`));
+
+  const app = new HowdyApp({ reachy, vision, clips, ui });
+  app.start();
+
+  handle.onLeave(() => app.shutdown(detachVideo));
+}
+
+// ---------------------------------------------------------------------------
+// Core app: tracking loop, wave detection, reactions
+// ---------------------------------------------------------------------------
+
+class HowdyApp {
+  constructor({ reachy, vision, clips, ui }) {
+    this.reachy = reachy;
+    this.vision = vision;
+    this.clips = clips;
+    this.ui = ui;
+
+    // Last COMMANDED pose (the SDK docs say: use your own buffer, not telemetry)
+    this.headYaw = 0;   // degrees, world frame
+    this.headPitch = 0;
+    this.bodyYaw = 0;
+
+    this.trackingEnabled = true;
+    this.bodyFollow = true;
+    this.muted = false;
+    this.busy = false;         // a playMove is running
+    this.running = false;
+
+    this.lastFaceAt = -Infinity;
+    this.faceAbsentSince = 0;
+    this.lastWaveAt = -Infinity;
+    this.lastSendAt = 0;
+    this.lastTickAt = 0;
+
+    this.wave = new WaveDetector();
+
+    ui.btnTrack.onclick = () => {
+      this.trackingEnabled = !this.trackingEnabled;
+      ui.btnTrack.setAttribute("aria-pressed", String(this.trackingEnabled));
+      ui.btnTrack.textContent = this.trackingEnabled ? "Tracking on" : "Tracking off";
+    };
+    ui.btnBody.onclick = () => {
+      this.bodyFollow = !this.bodyFollow;
+      ui.btnBody.setAttribute("aria-pressed", String(this.bodyFollow));
+      ui.btnBody.textContent = this.bodyFollow ? "Body turns" : "Head only";
+    };
+    ui.btnMute.onclick = () => {
+      this.muted = !this.muted;
+      ui.btnMute.setAttribute("aria-pressed", String(this.muted));
+      ui.btnMute.textContent = this.muted ? "Sound off" : "Sound on";
+    };
+    ui.btnWave.onclick = () => this.reactToWave();
+  }
+
+  start() {
+    this.running = true;
+    this.reachy.setTarget({ antennas: INIT_ANTENNAS });
+    setStatus("Looking for faces", "");
+    this.lastTickAt = performance.now();
+    this.scheduleTick();
+  }
+
+  scheduleTick() {
+    if (!this.running) return;
+    setTimeout(() => this.tick().catch((e) => console.error(e)).finally(() => this.scheduleTick()), 1000 / TRACK.hz);
+  }
+
+  async tick() {
+    const now = performance.now();
+    const dt = Math.min(0.2, (now - this.lastTickAt) / 1000);
+    this.lastTickAt = now;
+
+    const v = this.ui.video;
+    if (v.readyState < 2 || v.videoWidth === 0) return;
+    syncOverlaySize(this.ui.overlay, v);
+
+    // Vision
+    const faces = this.vision.face.detectForVideo(v, now).detections ?? [];
+    const hands = this.vision.hand.detectForVideo(v, now).landmarks ?? [];
+    const face = pickLargestFace(faces);
+    drawOverlay(this.ui.overlay, v, face, hands);
+
+    // Wave detection runs even while busy so we don't miss one mid-reaction
+    const waved = this.wave.update(hands, now / 1000, v.videoWidth / v.videoHeight);
+
+    if (this.busy) return;
+
+    // Face bookkeeping
+    if (face) {
+      const absentFor = (now - this.lastFaceAt) / 1000;
+      const isNewArrival = absentFor > TRACK.noticeAfterAbsentSec;
+      this.lastFaceAt = now;
+      if (isNewArrival) {
+        await this.noticeFace();
+        return;
+      }
+    }
+
+    if (waved && now - this.lastWaveAt > WAVE.cooldownSec * 1000) {
+      this.lastWaveAt = now;
+      await this.reactToWave();
+      return;
+    }
+
+    // Tracking
+    if (!this.trackingEnabled) {
+      setStatus("Tracking paused", face ? "face in view" : "");
+      return;
+    }
+
+    if (face) {
+      const cx = (face.boundingBox.originX + face.boundingBox.width / 2) / v.videoWidth;
+      const cy = (face.boundingBox.originY + face.boundingBox.height / 2) / v.videoHeight;
+      const ex = (cx - 0.5) * 2; // -1..1
+      const ey = (cy - 0.5) * 2;
+      this.servo(ex, ey, dt);
+      setStatus("Tracking", `yaw ${this.headYaw.toFixed(0)}° body ${this.bodyYaw.toFixed(0)}°`, "tracking");
+    } else if ((now - this.lastFaceAt) / 1000 > TRACK.noFaceResetSec) {
+      this.easeHome(dt);
+      setStatus("Looking for faces", "");
+    }
+
+    this.sendPose(now);
+  }
+
+  servo(ex, ey, dt) {
+    const step = TRACK.gainDegPerSec * dt;
+    const dYaw = Math.abs(ex) > TRACK.deadband ? YAW_SIGN * ex * step : 0;
+    const dPitch = Math.abs(ey) > TRACK.deadband ? PITCH_SIGN * ey * step : 0;
+
+    let yaw = this.headYaw + dYaw;
+    let pitch = clamp(this.headPitch + dPitch, -TRACK.maxPitchDeg, TRACK.maxPitchDeg);
+
+    // Body follows when the head is far off the body's axis
+    if (this.bodyFollow) {
+      const offset = yaw - this.bodyYaw;
+      if (Math.abs(offset) > TRACK.bodyFollowStartDeg) {
+        const move = Math.sign(offset) * Math.min(Math.abs(offset), TRACK.bodyRateDegPerSec * dt);
+        this.bodyYaw = clamp(this.bodyYaw + move, -TRACK.maxBodyYawDeg, TRACK.maxBodyYawDeg);
+      }
+    }
+    yaw = clamp(yaw, this.bodyYaw - TRACK.maxHeadYawFromBodyDeg, this.bodyYaw + TRACK.maxHeadYawFromBodyDeg);
+
+    this.headYaw = lerp(this.headYaw, yaw, TRACK.smoothing);
+    this.headPitch = lerp(this.headPitch, pitch, TRACK.smoothing);
+  }
+
+  easeHome(dt) {
+    const k = Math.min(1, dt * 1.5);
+    this.headYaw = lerp(this.headYaw, this.bodyYaw, k);
+    this.headPitch = lerp(this.headPitch, 0, k);
+    if (Math.abs(this.bodyYaw) > 0.5) this.bodyYaw = lerp(this.bodyYaw, 0, k * 0.5);
+    else this.bodyYaw = 0;
+  }
+
+  sendPose(now) {
+    if (now - this.lastSendAt < 1000 / TRACK.hz - 5) return;
+    this.lastSendAt = now;
+    this.reachy.setTarget({
+      head: headMatrix(0, this.headPitch * DEG, this.headYaw * DEG),
+      body_yaw: this.bodyYaw * DEG,
+    });
+  }
+
+  // --- Reactions -----------------------------------------------------------
+
+  async noticeFace() {
+    setStatus("Oh, hello", "", "reacting");
+    log("Noticed a face");
+    const motion = buildNodMotion({ yawDeg: this.headYaw, pitchDeg: this.headPitch, bodyYawDeg: this.bodyYaw });
+    const audioBlob = this.muted ? undefined : chirpWav();
+    await this.play(motion, audioBlob, "notice nod");
+  }
+
+  async reactToWave() {
+    if (this.busy) return;
+    const name = WAVE_CLIPS[Math.floor(Math.random() * WAVE_CLIPS.length)];
+    setStatus("Wave detected!", name, "reacting");
+    log(`Wave -> ${name}`);
+    const clip = await this.clips.get(name);
+    if (!clip) { log(`Clip ${name} unavailable`); return; }
+    // Shift the clip so the robot keeps facing the person instead of snapping to body_yaw 0
+    const motion = offsetMotionYaw(clip.motion, this.bodyYaw * DEG);
+    await this.play(motion, this.muted ? undefined : clip.wav, name);
+  }
+
+  async play(motion, audioBlob, description) {
+    if (this.busy) return;
+    this.busy = true;
+    try {
+      const result = await this.reachy.playMove(motion, { audioBlob, description, audioLeadMs: -100 });
+      if (result?.error) log(`playMove error: ${result.error}`);
+      // Re-sync our commanded buffer to where the clip left the robot
+      const last = motion.set_target_data[motion.set_target_data.length - 1];
+      if (last?.head) {
+        const rpy = matrixToRpyDeg(last.head);
+        this.headYaw = rpy.yaw;
+        this.headPitch = rpy.pitch;
+      }
+      if (typeof last?.body_yaw === "number") this.bodyYaw = last.body_yaw / DEG;
+      this.reachy.setTarget({ antennas: INIT_ANTENNAS });
+    } catch (e) {
+      log(`Reaction failed: ${e?.message ?? e}`);
+    } finally {
+      this.busy = false;
+      this.lastFaceAt = performance.now(); // don't immediately re-nod
+    }
+  }
+
+  shutdown(detachVideo) {
+    this.running = false;
+    try { this.reachy.cancelMove(); } catch (_) {}
+    try { detachVideo?.(); } catch (_) {}
+    try { this.vision.face.close(); this.vision.hand.close(); } catch (_) {}
+    safeReturnHome(this.reachy);
+  }
+}
+
+// ---------------------------------------------------------------------------
+// Wave detector
+// ---------------------------------------------------------------------------
+
+class WaveDetector {
+  constructor() { this.samples = []; }
+
+  // hands: array of 21-landmark arrays (normalized coords). Returns true on a detected wave.
+  update(hands, tSec, aspect) {
+    const hand = hands.find(isOpenPalmUp);
+    if (!hand) { this.samples.length = 0; return false; }
+
+    const xs = hand.map((p) => p.x * aspect); // correct for non-square frames
+    const width = Math.max(...xs) - Math.min(...xs);
+    if (width < 1e-3) return false;
+    const wristX = hand[0].x * aspect;
+
+    this.samples.push({ t: tSec, x: wristX, w: width });
+    while (this.samples.length && tSec - this.samples[0].t > WAVE.windowSec) this.samples.shift();
+    if (this.samples.length < 6) return false;
+
+    const w = this.samples.reduce((a, s) => a + s.w, 0) / this.samples.length;
+    const hyst = WAVE.hysteresisHandWidths * w;
+
+    let reversals = 0, travel = 0, dir = 0, anchor = this.samples[0].x;
+    for (const s of this.samples) {
+      const d = s.x - anchor;
+      if (Math.abs(d) < hyst) continue;
+      const nd = Math.sign(d);
+      if (dir !== 0 && nd !== dir) reversals++;
+      dir = nd;
+      travel += Math.abs(d);
+      anchor = s.x;
+    }
+    if (reversals >= WAVE.minReversals && travel >= WAVE.minTravelHandWidths * w) {
+      this.samples.length = 0;
+      return true;
+    }
+    return false;
+  }
+}
+
+// Open palm, fingers pointing up (image y grows downward)
+function isOpenPalmUp(lm) {
+  const wrist = lm[0], midMcp = lm[9];
+  if (wrist.y < midMcp.y) return false; // hand not upright
+  const tips = [8, 12, 16, 20], pips = [6, 10, 14, 18];
+  let extended = 0;
+  for (let i = 0; i < 4; i++) if (lm[tips[i]].y < lm[pips[i]].y) extended++;
+  return extended >= 3;
+}
+
+// ---------------------------------------------------------------------------
+// Vision setup (MediaPipe Tasks Vision, in-browser)
+// ---------------------------------------------------------------------------
+
+async function loadVision() {
+  const mp = await import(`https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@${MP_VERSION}/+esm`);
+  const files = await mp.FilesetResolver.forVisionTasks(
+    `https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@${MP_VERSION}/wasm`,
+  );
+  const [face, hand] = await Promise.all([
+    mp.FaceDetector.createFromOptions(files, {
+      baseOptions: {
+        modelAssetPath: "https://storage.googleapis.com/mediapipe-models/face_detector/blaze_face_short_range/float16/1/blaze_face_short_range.tflite",
+        delegate: "GPU",
+      },
+      runningMode: "VIDEO",
+      minDetectionConfidence: 0.6,
+    }),
+    mp.HandLandmarker.createFromOptions(files, {
+      baseOptions: {
+        modelAssetPath: "https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task",
+        delegate: "GPU",
+      },
+      runningMode: "VIDEO",
+      numHands: 1,
+      minHandDetectionConfidence: 0.6,
+      minTrackingConfidence: 0.5,
+    }),
+  ]);
+  return { face, hand };
+}
+
+function pickLargestFace(detections) {
+  let best = null, bestArea = 0;
+  for (const d of detections) {
+    const a = d.boundingBox.width * d.boundingBox.height;
+    if (a > bestArea) { best = d; bestArea = a; }
+  }
+  return best;
+}
+
+// ---------------------------------------------------------------------------
+// Emotion clips: JSON motion + Opus audio -> 16 kHz mono PCM WAV
+// ---------------------------------------------------------------------------
+
+class ClipCache {
+  constructor() { this.map = new Map(); }
+
+  async preload(names) {
+    const results = await Promise.allSettled(names.map((n) => this.get(n)));
+    return results.filter((r) => r.status === "fulfilled" && r.value).length;
+  }
+
+  async get(name) {
+    if (this.map.has(name)) return this.map.get(name);
+    const p = (async () => {
+      try {
+        const [motionRes, audioRes] = await Promise.all([
+          fetch(EMOTIONS_BASE + name + ".json"),
+          fetch(EMOTIONS_BASE + name + ".ogg"),
+        ]);
+        if (!motionRes.ok) throw new Error(`motion ${motionRes.status}`);
+        const motion = normalizeMotion(await motionRes.json());
+        let wav;
+        if (audioRes.ok) wav = await toWav16kMono(await audioRes.arrayBuffer());
+        return { motion, wav };
+      } catch (e) {
+        log(`Failed to load ${name}: ${e.message}`);
+        this.map.delete(name);
+        return null;
+      }
+    })();
+    this.map.set(name, p);
+    return p;
+  }
+}
+
+function normalizeMotion(raw) {
+  // Flatten nested 4x4 head matrices to 16 floats; leave everything else as-is
+  const set_target_data = raw.set_target_data.map((f) => ({
+    ...f,
+    head: f.head && Array.isArray(f.head[0]) ? f.head.flat() : f.head,
+  }));
+  return { ...raw, set_target_data };
+}
+
+async function toWav16kMono(arrayBuffer) {
+  const ctx = new (window.AudioContext || window.webkitAudioContext)();
+  const decoded = await ctx.decodeAudioData(arrayBuffer);
+  await ctx.close();
+  const targetRate = 16000;
+  const length = Math.ceil(decoded.duration * targetRate);
+  const off = new OfflineAudioContext(1, length, targetRate);
+  const src = off.createBufferSource();
+  src.buffer = decoded;
+  src.connect(off.destination);
+  src.start();
+  const rendered = await off.startRendering();
+  return encodeWav(rendered.getChannelData(0), targetRate);
+}
+
+function encodeWav(samples, sampleRate) {
+  const buf = new ArrayBuffer(44 + samples.length * 2);
+  const v = new DataView(buf);
+  const str = (o, s) => { for (let i = 0; i < s.length; i++) v.setUint8(o + i, s.charCodeAt(i)); };
+  str(0, "RIFF"); v.setUint32(4, 36 + samples.length * 2, true); str(8, "WAVE");
+  str(12, "fmt "); v.setUint32(16, 16, true); v.setUint16(20, 1, true); v.setUint16(22, 1, true);
+  v.setUint32(24, sampleRate, true); v.setUint32(28, sampleRate * 2, true);
+  v.setUint16(32, 2, true); v.setUint16(34, 16, true);
+  str(36, "data"); v.setUint32(40, samples.length * 2, true);
+  let o = 44;
+  for (let i = 0; i < samples.length; i++, o += 2) {
+    const s = Math.max(-1, Math.min(1, samples[i]));
+    v.setInt16(o, s < 0 ? s * 0x8000 : s * 0x7fff, true);
+  }
+  return new Blob([buf], { type: "audio/wav" });
+}
+
+// Two quick rising chirps, ~0.45 s total
+function chirpWav() {
+  const sr = 16000, dur = 0.45, n = Math.floor(sr * dur);
+  const out = new Float32Array(n);
+  const chirp = (t0, t1, f0, f1) => {
+    for (let i = Math.floor(t0 * sr); i < Math.floor(t1 * sr) && i < n; i++) {
+      const t = i / sr - t0, T = t1 - t0, u = t / T;
+      const f = f0 + (f1 - f0) * u;
+      const env = Math.sin(Math.PI * u);
+      out[i] += 0.6 * env * Math.sin(2 * Math.PI * (f0 * t + 0.5 * (f1 - f0) * t * t / T));
+    }
+  };
+  chirp(0.0, 0.18, 900, 1600);
+  chirp(0.22, 0.45, 1100, 2000);
+  return encodeWav(out, sr);
+}
+
+// ---------------------------------------------------------------------------
+// Motion helpers
+// ---------------------------------------------------------------------------
+
+// Short nod from the current pose, antennas perk, 0.9 s at 50 Hz
+function buildNodMotion({ yawDeg, pitchDeg, bodyYawDeg }) {
+  const hz = 50, dur = 0.9, n = Math.round(dur * hz);
+  const time = [], set_target_data = [];
+  for (let i = 0; i <= n; i++) {
+    const t = i / hz, u = t / dur;
+    const dip = u < 0.625 ? 14 * Math.sin(Math.PI * u * 1.6) : 0; // one dip in the first ~0.56 s
+    const perk = 25 * DEG * Math.sin(Math.PI * u);
+    time.push(t);
+    set_target_data.push({
+      head: headMatrix(0, (pitchDeg + dip) * DEG, yawDeg * DEG),
+      antennas: [INIT_ANTENNAS[0] - perk, INIT_ANTENNAS[1] + perk],
+      body_yaw: bodyYawDeg * DEG,
+    });
+  }
+  return { time, set_target_data };
+}
+
+// Rotate a whole recorded move about world z by `yawRad` so it plays relative to the current body yaw
+function offsetMotionYaw(motion, yawRad) {
+  if (Math.abs(yawRad) < 1e-4) return motion;
+  const Rz = headMatrix(0, 0, yawRad);
+  const maxBody = TRACK.maxBodyYawDeg * DEG;
+  return {
+    ...motion,
+    set_target_data: motion.set_target_data.map((f) => ({
+      ...f,
+      head: f.head ? mat4mul(Rz, f.head) : f.head,
+      body_yaw: typeof f.body_yaw === "number" ? clamp(f.body_yaw + yawRad, -maxBody, maxBody) : f.body_yaw,
+    })),
+  };
+}
+
+// Row-major flat 4x4 from roll/pitch/yaw (radians), ZYX convention, zero translation
+function headMatrix(r, p, y) {
+  const cr = Math.cos(r), sr = Math.sin(r), cp = Math.cos(p), sp = Math.sin(p), cy = Math.cos(y), sy = Math.sin(y);
+  return [
+    cy * cp, cy * sp * sr - sy * cr, cy * sp * cr + sy * sr, 0,
+    sy * cp, sy * sp * sr + cy * cr, sy * sp * cr - cy * sr, 0,
+    -sp,     cp * sr,                cp * cr,                0,
+    0, 0, 0, 1,
+  ];
+}
+
+function matrixToRpyDeg(m) {
+  const pitch = Math.asin(clamp(-m[8], -1, 1));
+  const roll = Math.atan2(m[9], m[10]);
+  const yaw = Math.atan2(m[4], m[0]);
+  return { roll: roll / DEG, pitch: pitch / DEG, yaw: yaw / DEG };
+}
+
+function mat4mul(a, b) {
+  const o = new Array(16);
+  for (let r = 0; r < 4; r++) for (let c = 0; c < 4; c++) {
+    let s = 0;
+    for (let k = 0; k < 4; k++) s += a[r * 4 + k] * b[k * 4 + c];
+    o[r * 4 + c] = s;
+  }
+  return o;
+}
+
+function safeReturnHome(reachy) {
+  try { reachy.setMotorMode?.("enabled"); } catch (_) {}
+  const target = { head: headMatrix(0, 0, 0), antennas: INIT_ANTENNAS, body_yaw: 0 };
+  try {
+    if (typeof reachy.gotoTarget === "function") reachy.gotoTarget({ ...target, duration: 1.2 });
+    else reachy.setTarget(target);
+  } catch (_) {}
+}
+
+// ---------------------------------------------------------------------------
+// UI
+// ---------------------------------------------------------------------------
+
+function bindUi() {
+  const $ = (id) => document.getElementById(id);
+  return {
+    video: $("video"), overlay: $("overlay"),
+    btnTrack: $("btnTrack"), btnBody: $("btnBody"), btnMute: $("btnMute"), btnWave: $("btnWave"),
+  };
+}
+
+function setStatus(text, sub = "", mode = "") {
+  const el = document.getElementById("status");
+  document.getElementById("statusText").textContent = text;
+  document.getElementById("statusSub").textContent = sub;
+  el.className = "status" + (mode ? " " + mode : "");
+}
+
+function log(msg) {
+  const el = document.getElementById("log");
+  if (!el) return;
+  const t = new Date().toLocaleTimeString([], { hour12: false });
+  el.textContent = `${t}  ${msg}\n` + el.textContent.split("\n").slice(0, 5).join("\n");
+  console.log("[howdy-friend]", msg);
+}
+
+function syncOverlaySize(canvas, video) {
+  if (canvas.width !== video.videoWidth || canvas.height !== video.videoHeight) {
+    canvas.width = video.videoWidth; canvas.height = video.videoHeight;
+  }
+}
+
+function drawOverlay(canvas, video, face, hands) {
+  const g = canvas.getContext("2d");
+  g.clearRect(0, 0, canvas.width, canvas.height);
+  if (face) {
+    const b = face.boundingBox;
+    g.strokeStyle = "#f2c94c"; g.lineWidth = 3;
+    g.strokeRect(b.originX, b.originY, b.width, b.height);
+  }
+  for (const hand of hands) {
+    g.fillStyle = "#2bb673";
+    for (const p of hand) { g.beginPath(); g.arc(p.x * canvas.width, p.y * canvas.height, 4, 0, Math.PI * 2); g.fill(); }
+  }
+}
+
+const clamp = (v, lo, hi) => Math.max(lo, Math.min(hi, v));
+const lerp = (a, b, t) => a + (b - a) * t;

index.html

@@ -0,0 +1,124 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
+    <title>Howdy Friend</title>
+    <link rel="icon" href="/public/icon.svg" type="image/svg+xml" />
+
+    <link rel="modulepreload" href="https://cdn.jsdelivr.net/npm/@pollen-robotics/reachy-mini-sdk@1.8.0/+esm" />
+    <link rel="modulepreload" href="https://cdn.jsdelivr.net/npm/@pollen-robotics/reachy-mini-sdk@1.8.0/host/dist/entry/auto.js" />
+    <link rel="modulepreload" href="https://cdn.jsdelivr.net/npm/@pollen-robotics/reachy-mini-sdk@1.8.0/host/dist/entry/embed.js" />
+
+    <!-- Theme bootstrap (host passes ?theme=dark|light when iframing us) -->
+    <script>
+      (function () {
+        try {
+          var params = new URLSearchParams(window.location.search);
+          var raw = params.get("theme");
+          var mode;
+          if (raw === "dark" || raw === "light") mode = raw;
+          else if (window.matchMedia && window.matchMedia("(prefers-color-scheme: dark)").matches === false) mode = "light";
+          else mode = "dark";
+          document.documentElement.setAttribute("data-theme", mode);
+        } catch (_) {
+          document.documentElement.setAttribute("data-theme", "dark");
+        }
+      })();
+    </script>
+
+    <!-- HF Spaces OAuth placeholders. Substituted by HF at serve time because hf_oauth: true. -->
+    <script>
+      (function () {
+        var clientId = "__OAUTH_CLIENT_ID__";
+        var scopes = "__OAUTH_SCOPES__";
+        var spaceHost = "__SPACE_HOST__";
+        var spaceId = "__SPACE_ID__";
+        var looksSubstituted = clientId && clientId.indexOf("__") !== 0;
+        if (looksSubstituted) {
+          window.huggingface = window.huggingface || {};
+          window.huggingface.variables = {
+            OAUTH_CLIENT_ID: clientId,
+            OAUTH_SCOPES: scopes && scopes.indexOf("__") !== 0 ? scopes : "openid profile",
+            SPACE_HOST: spaceHost && spaceHost.indexOf("__") !== 0 ? spaceHost : "",
+            SPACE_ID: spaceId && spaceId.indexOf("__") !== 0 ? spaceId : "",
+          };
+        }
+      })();
+    </script>
+
+    <style>
+      :root {
+        --bg: #101214;
+        --fg: #f3f1ea;
+        --muted: #9aa0a6;
+        --accent: #f2c94c;
+        --ok: #2bb673;
+        --panel: #1a1d20;
+      }
+      [data-theme="light"] {
+        --bg: #f7f5ef;
+        --fg: #16181a;
+        --muted: #5d6368;
+        --panel: #ffffff;
+      }
+      * { box-sizing: border-box; }
+      html, body { margin: 0; height: 100%; background: var(--bg); color: var(--fg);
+        font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif; }
+      #root { display: none; }
+      #app { display: none; min-height: 100dvh; flex-direction: column; padding: 0.75rem;
+        padding-bottom: calc(0.75rem + env(safe-area-inset-bottom)); gap: 0.75rem; }
+      body.embed #app { display: flex; }
+      body.host #root { display: block; }
+
+      .stage { position: relative; width: 100%; aspect-ratio: 4 / 3; background: #000;
+        border-radius: 12px; overflow: hidden; }
+      .stage video, .stage canvas { position: absolute; inset: 0; width: 100%; height: 100%; object-fit: contain; }
+      .stage canvas { pointer-events: none; }
+
+      .status { display: flex; align-items: center; gap: 0.6rem; font-size: 1.05rem; min-height: 1.6rem; }
+      .status .dot { width: 0.7rem; height: 0.7rem; border-radius: 50%; background: var(--muted); flex: none;
+        transition: background 0.2s; }
+      .status.tracking .dot { background: var(--accent); }
+      .status.reacting .dot { background: var(--ok); }
+      .status small { color: var(--muted); }
+
+      .controls { display: grid; grid-template-columns: 1fr 1fr; gap: 0.5rem; }
+      button { min-height: 48px; border: 1px solid transparent; border-radius: 10px; background: var(--panel);
+        color: var(--fg); font: inherit; font-size: 1rem; padding: 0.6rem 0.9rem; cursor: pointer; }
+      button[aria-pressed="true"] { border-color: var(--accent); }
+      button:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
+      button.wide { grid-column: 1 / -1; }
+
+      .log { font-size: 0.85rem; color: var(--muted); white-space: pre-wrap; max-height: 6rem; overflow: auto; }
+      @media (prefers-reduced-motion: reduce) { .status .dot { transition: none; } }
+    </style>
+  </head>
+  <body>
+    <div id="root"></div>
+
+    <main id="app">
+      <div class="stage">
+        <video id="video" autoplay playsinline muted></video>
+        <canvas id="overlay"></canvas>
+      </div>
+
+      <div id="status" class="status">
+        <span class="dot"></span>
+        <span id="statusText">Loading vision models</span>
+        <small id="statusSub"></small>
+      </div>
+
+      <div class="controls">
+        <button id="btnTrack" aria-pressed="true">Tracking on</button>
+        <button id="btnBody" aria-pressed="true">Body turns</button>
+        <button id="btnMute" aria-pressed="false">Sound on</button>
+        <button id="btnWave">Test wave reaction</button>
+      </div>
+
+      <div id="log" class="log"></div>
+    </main>
+
+    <script type="module" src="/app.js"></script>
+  </body>
+</html>

plan.md

@@ -0,0 +1,82 @@
+# plan.md — Reachy Mini "Wave Hello" app
+
+## What you want
+
+A Reachy Mini app that:
+
+1. Watches the robot's camera for faces and keeps the head pointed at the nearest/largest face.
+2. Detects when a person waves at the robot.
+3. Responds to a wave by wagging its antennae and making happy noises.
+
+Robot: **Wireless** (onboard CM4). Reaction source: **prebuilt emotions library** (`pollen-robotics/reachy-mini-emotions-library`).
+
+## App flavor: JS / Hugging Face Space
+
+Chosen over Python because the Wireless CM4 is CPU- and memory-constrained. In the JS flavor all vision runs in the browser on your laptop or phone, the robot only streams video and receives motion commands. This matches the pattern used by the official `hand_tracker_v2` and `reachy_mini_telepresence` reference apps.
+
+Cost of this choice: WebRTC adds roughly 100-300 ms of latency between what the camera sees and the head command. Face tracking will feel slightly laggy compared to an on-robot loop. This is acceptable for social tracking; it would not be for fast target following. Graduating to Python later is easy if you decide you want tighter tracking.
+
+## Technical approach
+
+**Scaffold**
+- Clone `pollen-robotics/reachy_mini_telepresence` (React + Vite, already handles the video stream) and trim it. Alternative: `reachy_mini_minimal_conversation` (vanilla TS) if you prefer no React. I lean telepresence because the camera plumbing is already correct.
+- Pin `@pollen-robotics/reachy-mini-sdk@1.8.0`.
+- Standard host shell: `mountHost()` for OAuth, robot picker, top bar; `connectToHost()` in the embed.
+- `enableMicrophone: false` (no user-to-robot audio needed).
+- Mobile-first layout, per the guide.
+
+**Vision (in browser)**
+- `@mediapipe/tasks-vision` from npm: Face Detector (short-range, lightweight) and Hand Landmarker (1 hand). Both run in the browser via WebGL/WASM. Frames come from the `<video>` element attached with `reachy.attachVideo()`. Never `getUserMedia()`; that would grab your own laptop camera.
+- Run detection at ~15 fps in a `requestVideoFrameCallback` loop; skip frames if the previous inference hasn't finished.
+
+**Face tracking loop**
+- Pick the largest detected face. Compute its center offset from frame center, normalized to [-1, 1].
+- Proportional control on head yaw and pitch, with a small deadband (~4% of frame) and exponential smoothing so the head doesn't twitch.
+- Command via `reachy.setTarget()` at 10-20 Hz using a locally kept "last commanded pose" as the baseline (the guide warns against using telemetry as the baseline because it lags one RTT).
+- Clamp: pitch/roll within ±40°, keep yaw well inside limits. Body yaw stays fixed at 0 in v1; if the face drifts past ~35° head yaw, optionally rotate the body to follow (v2).
+- If no face is seen for ~3 s, ease back to a neutral pose.
+
+**Wave detection**
+- From Hand Landmarker: require an open hand (fingers extended, palm roughly facing camera) with the wrist above the hand's midpoint.
+- Track wrist x over a 1.2 s sliding window. A wave is: at least 2 direction reversals with horizontal travel above a threshold proportional to hand size.
+- Debounce: after a wave triggers, ignore new waves for ~4 s so one wave produces one reaction.
+
+**Happy reaction**
+- At startup, fetch a small set of "happy" moves from the emotions dataset (each is `<name>.json` + `<name>.ogg` at the dataset root) and cache them in memory.
+- On wave: pause the tracking loop, call `reachy.playMove(motionJson, { audioBlob })`, await `finished`, then resume tracking. `playMove` runs daemon-side with a single clock, so motion and sound stay in sync.
+- The recorded emotions already include antenna motion. If the library's "happy" clips are not antenna-heavy enough, I'll layer a short custom antenna wag via `setAntennasDeg` before or after the clip.
+
+**Cleanup**
+- `handle.onLeave()` → `safelyReturnToPose(reachy)`, cancel any in-flight `playMove`, stop the vision loop, close MediaPipe.
+
+**UI (minimal)**
+- Robot camera feed with face box and hand landmarks overlaid.
+- Status line: "Looking for faces" / "Tracking" / "Wave detected!"
+- Toggles: tracking on/off, reaction sensitivity, mute.
+
+## Files I will produce
+
+Per the guide's 3-file contract: `index.html`, `src/dispatch.ts`, `src/embed.tsx`, plus `src/vision/` helpers, `public/icon.svg`, `README.md` frontmatter with `sdk: static`, `hf_oauth: true`, and the `reachy_mini_js_app` tag.
+
+## Questions (answer inline)
+
+1. Which emotion clips should play on a wave? Options: (a) just `happy`, (b) a random pick from a short list I curate (happy, amazed, excited-type clips) so it doesn't feel identical every time.
+   Answer: random pick from a curated list (welcoming1, welcoming2, enthusiastic2, cheerful1, success2, loving1, helpful1)
+
+2. Should the robot also react when it first sees a face, before any wave (e.g. a small "noticed you" head tilt)? Or stay quiet until waved at?
+   Answer: yes, a short nod and a chirp when a face first appears
+
+3. Body rotation: keep the body fixed and only move the head (simpler, v1), or let the body turn to follow a person walking around it?
+   Answer: body turns to follow
+
+4. Your Hugging Face username and the Space name you want (e.g. `yourname/reachy_wave_hello`).
+   Answer: RickDakan/howdy_friend
+
+5. Do you have Node.js + npm on your machine? If not, I can switch to the bare-HTML + CDN path, but MediaPipe is easier via npm.
+   Answer: unknown, so use bare HTML + CDN path (no Node needed)
+
+## Not doing in v1
+
+- Multi-person tracking or identity recognition.
+- Custom recorded sounds.
+- Python fallback.