import AppKit import Foundation import WhisperKit // MARK: - Configuração (variáveis de ambiente sobrescrevem os padrões) let home = NSHomeDirectory() // Pasta de dados: modelos, glossary.json e (opcional) logs. Padrão: ~/Library/Application Support/LiveCaption let projectDir = ProcessInfo.processInfo.environment["LC_PROJECT"] ?? "\(home)/Library/Application Support/LiveCaption" let whisperDir = ProcessInfo.processInfo.environment["LC_WHISPER_DIR"] ?? "\(projectDir)/models/whisper/openai_whisper-small" let srcLang = ProcessInfo.processInfo.environment["LC_SRC"] ?? "pt" let dstLang = ProcessInfo.processInfo.environment["LC_DST"] ?? "Spanish" let mtURL = ProcessInfo.processInfo.environment["LC_MT_URL"] ?? "http://localhost:11434/api/chat" let mtModel = ProcessInfo.processInfo.environment["LC_MT_MODEL"] ?? "hymt2-1.8b" let sampleRate = 16000 let endSilenceSeconds: Float = 0.8 let decodeEverySeconds: Float = 0.6 let maxWindowSeconds: Float = 25 // MARK: - Utilidades func clean(_ s: String) -> String { let noTokens = s.replacingOccurrences(of: "<\\|[^|]*\\|>", with: "", options: .regularExpression) let noNoise = noTokens .replacingOccurrences(of: "\\[[^\\]]*\\]|\\([^)]*\\)|[♪♫]", with: " ", options: .regularExpression) .replacingOccurrences(of: "\\s+", with: " ", options: .regularExpression) return noNoise.trimmingCharacters(in: .whitespacesAndNewlines) } /// Divide em frases (. ! ? …) para traduzir aos poucos em vez de um bloco longo. func sentences(_ s: String) -> [String] { var out: [String] = [], cur = "" for ch in s { cur.append(ch) if ".!?…".contains(ch) { out.append(cur); cur = "" } } if !cur.trimmingCharacters(in: .whitespaces).isEmpty { out.append(cur) } return out.map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } } /// Frases que o Whisper alucina em silêncio/ruído. func isHallucination(_ s: String) -> Bool { let t = s.lowercased() let bad = ["legendas pela comunidade", "amara.org", "obrigado por assistir", "inscreva-se", "[música]", "(música)", "[ música ]"] let hasContent = t.contains { $0.isLetter || $0.isNumber } // eco do prompt de vocabulário em silêncio/ruído let vocabEcho = !glossary.whisperVocab.isEmpty && glossary.whisperVocab.contains { v in let rest = t.replacingOccurrences(of: v.lowercased(), with: "") return t.contains(v.lowercased()) && !rest.contains { $0.isLetter || $0.isNumber } } return t.isEmpty || !hasContent || vocabEcho || bad.contains { t.contains($0) } } /// Privacidade: por padrão NADA do que é dito é gravado em disco. LC_LOG=1 liga o histórico (logs/sessao-*.jsonl e logs/metricas-*.jsonl). let logEnabled = ProcessInfo.processInfo.environment["LC_LOG"] != nil let logURL: URL = { let df = DateFormatter() df.dateFormat = "yyyyMMdd-HHmmss" let dir = URL(fileURLWithPath: "\(projectDir)/logs") if logEnabled { try? FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) } return dir.appendingPathComponent("sessao-\(df.string(from: Date())).jsonl") }() func logLine(_ obj: [String: Any]) { guard logEnabled, let data = try? JSONSerialization.data(withJSONObject: obj), let line = String(data: data, encoding: .utf8) else { return } if let h = try? FileHandle(forWritingTo: logURL) { h.seekToEndOfFile(); h.write(Data((line + "\n").utf8)); try? h.close() } else { try? Data((line + "\n").utf8).write(to: logURL) } } // MARK: - Glossário (glossary.json no projeto; editável sem recompilar) struct Glossary { var whisperVocab: [String] = [] // desligado por padrão: custa ~2x no tempo do Whisper (medido) var keepTerms: [String] = [] var pre: [(NSRegularExpression, String)] = [] var post: [(NSRegularExpression, String)] = [] // correções do que o Whisper ouviu, antes de traduzir static func load() -> Glossary { var g = Glossary() var rules: [(String, String)] = [] if let data = FileManager.default.contents(atPath: "\(projectDir)/glossary.json"), let j = try? JSONSerialization.jsonObject(with: data) as? [String: Any] { if let v = j["whisper_vocab"] as? [String] { g.whisperVocab = v } if let k = j["keep_terms"] as? [String] { g.keepTerms = k } if let r = j["pre_substitutions"] as? [[String: String]] { rules = r.compactMap { d in d["pattern"].flatMap { p in d["replace"].map { (p, $0) } } } } } if ProcessInfo.processInfo.environment["LC_NO_VOCAB"] != nil { g.whisperVocab = [] } // para comparar com/sem vocabulário g.pre = rules.compactMap { (p, r) in (try? NSRegularExpression(pattern: p)).map { ($0, r) } } var fixes: [(String, String)] = [] if let data = FileManager.default.contents(atPath: "\(projectDir)/glossary.json"), let j = try? JSONSerialization.jsonObject(with: data) as? [String: Any], let r = j["post_corrections"] as? [[String: String]] { fixes = r.compactMap { d in d["pattern"].flatMap { p in d["replace"].map { (p, $0) } } } } g.post = fixes.compactMap { (p, r) in (try? NSRegularExpression(pattern: p)).map { ($0, r) } } return g } func applyPost(_ text: String) -> String { var t = text for (re, rep) in post { t = re.stringByReplacingMatches(in: t, range: NSRange(t.startIndex..., in: t), withTemplate: rep) } return t } func applyPre(_ text: String) -> String { var t = text for (re, rep) in pre { t = re.stringByReplacingMatches(in: t, range: NSRange(t.startIndex..., in: t), withTemplate: rep) } return t } /// Prompt de tradução com substituições prévias e referência de termos (formato de terminologia do Hunyuan-MT). func prompt(for text: String) -> String { let t = applyPre(text) let refs = keepTerms.filter { t.range(of: $0, options: .caseInsensitive) != nil } .map { "\($0) translated into \($0)\n" }.joined() return (refs.isEmpty ? "" : "Reference the following translations:\n\(refs)\n") + "Translate the following segment into \(dstLang), without additional explanation.\n\n\(t)" } } let glossary = Glossary.load() // MARK: - Tradução (backend plugável: API de chat compatível, aqui Ollama) /// Fila "mais recente ganha": o backlog é fundido num único pedido, então o atraso não acumula. actor TranslationQueue { private var pending = "" private var pendingSince: Date? private var running = false private let onResult: @Sendable (_ source: String, _ es: String, _ seconds: Double, _ queued: Double, _ ttft: Double) -> Void private let onDepth: @Sendable (Int) -> Void private let onPartial: @Sendable (String) -> Void init(onResult: @escaping @Sendable (String, String, Double, Double, Double) -> Void, onDepth: @escaping @Sendable (Int) -> Void, onPartial: @escaping @Sendable (String) -> Void) { self.onResult = onResult self.onDepth = onDepth self.onPartial = onPartial } func enqueue(_ text: String) { pending = pending.isEmpty ? text : pending + " " + text if pending.count > 500 { pending = String(pending.suffix(500)) } if pendingSince == nil { pendingSince = Date() } onDepth(running ? 1 : 0) if !running { running = true; Task { await self.drain() } } } private func drain() async { while !pending.isEmpty { let text = pending let since = pendingSince ?? Date() pending = ""; pendingSince = nil let t0 = Date() let (out, ttft) = await Translation.translateStream(text, onPartial: onPartial) let dt = Date().timeIntervalSince(t0) onResult(text, out, dt, t0.timeIntervalSince(since), ttft) } running = false onDepth(0) } } enum Translation { /// Tradução em streaming (NDJSON do Ollama): chama onPartial com o texto acumulado, só acrescentando. /// Devolve (texto final, segundos até a 1ª palavra). static func translateStream(_ text: String, onPartial: @Sendable (String) -> Void) async -> (String, Double) { let prompt = glossary.prompt(for: text) let body: [String: Any] = [ "model": mtModel, "stream": true, "keep_alive": "2h", "messages": [["role": "user", "content": prompt]], "options": ["num_predict": 240, "temperature": 0], ] guard let url = URL(string: mtURL), let data = try? JSONSerialization.data(withJSONObject: body) else { return ("⚠︎", 0) } var req = URLRequest(url: url, timeoutInterval: 20) req.httpMethod = "POST" req.setValue("application/json", forHTTPHeaderField: "Content-Type") req.httpBody = data let t0 = Date() var acc = "", ttft = 0.0 do { let (bytes, _) = try await URLSession.shared.bytes(for: req) for try await line in bytes.lines { guard let d = line.data(using: .utf8), let j = try? JSONSerialization.jsonObject(with: d) as? [String: Any] else { continue } if let m = j["message"] as? [String: Any], let c = m["content"] as? String, !c.isEmpty { if acc.isEmpty { ttft = Date().timeIntervalSince(t0) } acc += c onPartial(acc.trimmingCharacters(in: .whitespacesAndNewlines)) } if (j["done"] as? Bool) == true { break } } } catch { if acc.isEmpty { return ("⚠︎ tradução indisponível", 0) } } return (acc.trimmingCharacters(in: .whitespacesAndNewlines), ttft) } static func translate(_ text: String) async -> String { let prompt = "Translate the following segment into \(dstLang), without additional explanation.\n\n\(text)" let body: [String: Any] = [ "model": mtModel, "stream": false, "keep_alive": "2h", "messages": [["role": "user", "content": prompt]], "options": ["num_predict": 240, "temperature": 0], ] guard let url = URL(string: mtURL), let data = try? JSONSerialization.data(withJSONObject: body) else { return "⚠︎" } var req = URLRequest(url: url, timeoutInterval: 20) req.httpMethod = "POST" req.setValue("application/json", forHTTPHeaderField: "Content-Type") req.httpBody = data do { let (d, _) = try await URLSession.shared.data(for: req) if let j = try JSONSerialization.jsonObject(with: d) as? [String: Any], let m = j["message"] as? [String: Any], let c = m["content"] as? String { return c.trimmingCharacters(in: .whitespacesAndNewlines) } } catch {} return "⚠︎ tradução indisponível" } } // MARK: - Interface @MainActor final class CaptionView { let panel: NSPanel private let textView = NSTextView() private let scroll = NSScrollView() private let infoLabel = NSTextField(labelWithString: "") private let toggleButton = NSButton(title: "", target: nil, action: nil) private let quitButton = NSButton(title: "", target: nil, action: nil) var onToggle: (() -> Void)? private var confirmedES: [String] = [] private var live = "" // frase sendo traduzida agora (só cresce, nunca é reescrita) private let maxChars = 900 private let finalColor = NSColor(red: 1, green: 0.86, blue: 0.3, alpha: 1) // frase mais recente private let olderColor = NSColor(red: 0.62, green: 0.55, blue: 0.3, alpha: 1) // frases anteriores private let font = NSFont.systemFont(ofSize: 30, weight: .semibold) init() { let w: CGFloat = 1100, h: CGFloat = 460 let screen = NSScreen.main?.visibleFrame ?? NSRect(x: 0, y: 0, width: 1440, height: 900) panel = NSPanel(contentRect: NSRect(x: screen.midX - w / 2, y: screen.minY + 40, width: w, height: h), styleMask: [.borderless, .resizable, .nonactivatingPanel], backing: .buffered, defer: false) panel.level = .floating panel.isOpaque = false panel.backgroundColor = .clear panel.isMovableByWindowBackground = true panel.hidesOnDeactivate = false panel.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary] let bg = NSView(frame: NSRect(x: 0, y: 0, width: w, height: h)) bg.wantsLayer = true bg.layer?.backgroundColor = NSColor(white: 0, alpha: 0.80).cgColor bg.layer?.cornerRadius = 16 bg.autoresizingMask = [.width, .height] scroll.frame = NSRect(x: 22, y: 34, width: w - 44, height: h - 52) scroll.autoresizingMask = [.width, .height] scroll.drawsBackground = false scroll.hasVerticalScroller = false scroll.borderType = .noBorder textView.frame = NSRect(x: 0, y: 0, width: scroll.frame.width, height: scroll.frame.height) textView.autoresizingMask = [.width] textView.isEditable = false textView.isSelectable = false textView.drawsBackground = false textView.isVerticallyResizable = true textView.isHorizontallyResizable = false textView.textContainer?.widthTracksTextView = true textView.textContainerInset = .zero scroll.documentView = textView bg.addSubview(scroll) quitButton.frame = NSRect(x: w - 16 - 74, y: 6, width: 74, height: 24) toggleButton.frame = NSRect(x: w - 16 - 74 - 8 - 104, y: 6, width: 104, height: 24) for b in [toggleButton, quitButton] { b.autoresizingMask = [.minXMargin] b.isBordered = false b.wantsLayer = true b.layer?.cornerRadius = 7 b.target = self bg.addSubview(b) } toggleButton.action = #selector(toggleTapped) quitButton.action = #selector(quitTapped) styleButton(quitButton, "✕ Sair", fill: NSColor(red: 0.55, green: 0.18, blue: 0.18, alpha: 1)) styleButton(toggleButton, "⏸ Pausar", fill: NSColor(white: 0.32, alpha: 1)) infoLabel.frame = NSRect(x: 22, y: 9, width: w - 44 - 200, height: 16) infoLabel.autoresizingMask = [.width] infoLabel.font = .monospacedSystemFont(ofSize: 11, weight: .regular) infoLabel.textColor = NSColor(white: 0.65, alpha: 1) bg.addSubview(infoLabel) panel.contentView = bg panel.orderFrontRegardless() setInfo("iniciando…") } private func render() { // histórico em frases inteiras (descarta as mais antigas), a mais nova em destaque var older = confirmedES, newest = live if newest.isEmpty { newest = older.popLast() ?? "" } while older.joined(separator: " ").count + newest.count > maxChars && !older.isEmpty { older.removeFirst() } let para = NSMutableParagraphStyle() para.lineSpacing = 6 let attr = NSMutableAttributedString() if !older.isEmpty { attr.append(NSAttributedString(string: older.joined(separator: " ") + " ", attributes: [.foregroundColor: olderColor, .font: font, .paragraphStyle: para])) } attr.append(NSAttributedString(string: newest, attributes: [.foregroundColor: finalColor, .font: font, .paragraphStyle: para])) textView.textStorage?.setAttributedString(attr) textView.scrollToEndOfDocument(nil) } func addConfirmed(_ s: String) {} // o português não é exibido; mantido por compatibilidade func setPending(_ s: String) {} func setLive(_ s: String) { live = s; render() } func addTranslation(_ s: String) { confirmedES.append(s) if confirmedES.count > 40 { confirmedES.removeFirst() } live = "" render() } func setInfo(_ s: String) { infoLabel.stringValue = s } /// Depuração: LC_SNAPSHOT=/caminho.png desenha a janela num PNG (não precisa de permissão de gravação de tela). func snapshot(to path: String) { guard let v = panel.contentView, let rep = v.bitmapImageRepForCachingDisplay(in: v.bounds) else { return } v.cacheDisplay(in: v.bounds, to: rep) if let png = rep.representation(using: .png, properties: [:]) { try? png.write(to: URL(fileURLWithPath: path)) } } @objc private func toggleTapped() { onToggle?() } @objc private func quitTapped() { NSApp.terminate(nil) } private func styleButton(_ b: NSButton, _ title: String, fill: NSColor) { b.layer?.backgroundColor = fill.cgColor let para = NSMutableParagraphStyle(); para.alignment = .center b.attributedTitle = NSAttributedString(string: title, attributes: [ .foregroundColor: NSColor.white, .font: NSFont.systemFont(ofSize: 13, weight: .semibold), .paragraphStyle: para]) } func setPaused(_ paused: Bool) { if paused { styleButton(toggleButton, "▶ Retomar", fill: NSColor(red: 0.16, green: 0.45, blue: 0.22, alpha: 1)) } else { styleButton(toggleButton, "⏸ Pausar", fill: NSColor(white: 0.32, alpha: 1)) } textView.alphaValue = paused ? 0.35 : 1 if paused { infoLabel.stringValue = "⏸ PAUSADO — microfone desligado" } } } /// Buffer de áudio para o modo de teste (arquivo), com a mesma interface mínima do microfone. final class FeedBuffer: @unchecked Sendable { private var buf = [Float]() private let lock = NSLock() func append(_ x: [Float]) { lock.lock(); buf.append(contentsOf: x); lock.unlock() } func snapshot() -> [Float] { lock.lock(); defer { lock.unlock() }; return buf } var count: Int { lock.lock(); defer { lock.unlock() }; return buf.count } func purge(keepingLast k: Int) { lock.lock(); if buf.count > k { buf.removeFirst(buf.count - k) }; lock.unlock() } /// Há voz nos últimos N segundos? (pico de RMS em blocos de 100 ms) func voiceInLast(seconds: Double) -> Bool { lock.lock(); defer { lock.unlock() } let n = min(buf.count, Int(seconds * Double(sampleRate))), blk = sampleRate / 10 var i = buf.count - n while i < buf.count { let e = min(buf.count, i + blk); var sum: Float = 0 for j in i.. 0.01 { return true } i = e } return false } } // MARK: - Motor: mic → Whisper (janela limitada) → tradução final class Engine: @unchecked Sendable { let ui: CaptionView var whisper: WhisperKit! var translator: TranslationQueue! private var lastWhisperSec = 0.0, lastMTSec = 0.0, lastQueued = 0.0, lastEndToEs = 0.0, lastRTF = 0.0 private var depth = 0 private var lastTTFT = 0.0, mTTFTSum = 0.0 private(set) var paused = false func togglePause() { paused.toggle() let p = paused Task { @MainActor in self.ui.setPaused(p) } } /// Tradução provisória do trecho ainda não fechado; só roda se o tradutor estiver ocioso (não cria fila). // métricas por intervalo (gravadas em logs/metricas-*.jsonl) private var mDecodes = 0, mDecSum = 0.0, mDecMax = 0.0, mRtfSum = 0.0 private var mMT = 0, mMTSum = 0.0, mMTMax = 0.0, mSent = 0, winSecs = 0.0 private let metricsURL: URL = { let df = DateFormatter(); df.dateFormat = "yyyyMMdd-HHmmss" return URL(fileURLWithPath: "\(projectDir)/logs/metricas-\(df.string(from: Date())).jsonl") }() private func footprintMB() -> Double { var info = task_vm_info_data_t() var count = mach_msg_type_number_t(MemoryLayout.size / MemoryLayout.size) let r = withUnsafeMutablePointer(to: &info) { $0.withMemoryRebound(to: integer_t.self, capacity: Int(count)) { task_info(mach_task_self_, task_flavor_t(TASK_VM_INFO), $0, &count) } } return r == KERN_SUCCESS ? Double(info.phys_footprint) / 1_048_576 : 0 } private func cpuSeconds() -> Double { var ru = rusage(); getrusage(RUSAGE_SELF, &ru) return Double(ru.ru_utime.tv_sec) + Double(ru.ru_utime.tv_usec) / 1e6 + Double(ru.ru_stime.tv_sec) + Double(ru.ru_stime.tv_usec) / 1e6 } func startMetrics(every seconds: Double = 30) { let t0 = Date() Task.detached { [self] in var lastCPU = cpuSeconds(), lastWall = Date() while true { try? await Task.sleep(nanoseconds: UInt64(seconds * 1e9)) let now = Date(), cpu = cpuSeconds() let cpuPct = (cpu - lastCPU) / max(0.001, now.timeIntervalSince(lastWall)) * 100 lastCPU = cpu; lastWall = now let thermal = ["nominal", "fair", "serious", "critical"][min(3, ProcessInfo.processInfo.thermalState.rawValue)] let row: [String: Any] = [ "min": now.timeIntervalSince(t0) / 60, "rss_mb": footprintMB(), "cpu_pct": cpuPct, "thermal": thermal, "decodes": mDecodes, "whisper_avg_s": mDecodes > 0 ? mDecSum / Double(mDecodes) : 0, "whisper_max_s": mDecMax, "rtf_avg": mDecodes > 0 ? mRtfSum / Double(mDecodes) : 0, "mt_n": mMT, "mt_avg_s": mMT > 0 ? mMTSum / Double(mMT) : 0, "ttft_avg_s": mMT > 0 ? mTTFTSum / Double(mMT) : 0, "mt_max_s": mMTMax, "frases": mSent, "janela_s": winSecs, ] mDecodes = 0; mDecSum = 0; mDecMax = 0; mRtfSum = 0; mMT = 0; mMTSum = 0; mMTMax = 0; mSent = 0; mTTFTSum = 0 if let d = try? JSONSerialization.data(withJSONObject: row), let l = String(data: d, encoding: .utf8) { if let h = try? FileHandle(forWritingTo: metricsURL) { h.seekToEndOfFile(); h.write(Data((l + "\n").utf8)); try? h.close() } else { try? Data((l + "\n").utf8).write(to: metricsURL) } } } } } private var confirmWall: [Date] = [] // instantes de fechamento de frase, para medir fim→legenda es init(ui: CaptionView) { self.ui = ui } func status() { if paused { return } let s = String(format: "whisper %.1fs (RTF %.2f) · 1ª palavra %.1fs · tradução completa %.1fs · fim da frase→es %.1fs · fila %d", lastWhisperSec, lastRTF, lastTTFT, lastMTSec, lastEndToEs, depth) Task { @MainActor in self.ui.setInfo(s) } } func run() async { await MainActor.run { ui.setInfo("carregando Whisper (a 1ª vez compila para o Neural Engine, pode levar ~1 min)…") } do { let cfg = WhisperKitConfig(modelFolder: whisperDir, tokenizerFolder: URL(fileURLWithPath: "\(projectDir)/models/whisper/hf"), verbose: false, prewarm: true, load: true, download: false) whisper = try await WhisperKit(cfg) } catch { await MainActor.run { ui.setInfo("ERRO ao carregar Whisper: \(error.localizedDescription)") } return } await MainActor.run { ui.setInfo("aquecendo tradutor…") } _ = await Translation.translate("Olá.") translator = TranslationQueue(onResult: { [weak self] src, es, dt, queued, ttft in guard let self else { return } let endToEs = self.confirmWall.first.map { Date().timeIntervalSince($0) } ?? dt self.confirmWall.removeAll() self.lastMTSec = dt; self.lastQueued = queued; self.lastEndToEs = endToEs; self.lastTTFT = ttft; self.mTTFTSum += ttft self.mMT += 1; self.mMTSum += dt; self.mMTMax = max(self.mMTMax, dt); self.mSent += 1 logLine(["t": ISO8601DateFormatter().string(from: Date()), "pt": src, "es": es, "mt_s": dt, "ttft_s": ttft, "fila_s": queued, "fim_ate_es_s": endToEs]) Task { @MainActor in self.ui.addTranslation(es) } self.status() }, onDepth: { [weak self] d in self?.depth = d; self?.status() }, onPartial: { [weak self] p in guard let self else { return } Task { @MainActor in self.ui.setLive(p) } }) if logEnabled { startMetrics() } let getSamples: () -> [Float] let getCount: () -> Int let isVoice: () -> Bool let purge: (Int) -> Void var stopMic: () -> Void = {} var startMic: () -> Void = {} if let rawPath = ProcessInfo.processInfo.environment["LC_WAV"] { // modo teste: alimenta o pipeline com um arquivo f32le 16 kHz mono, em tempo real e em loop guard let data = FileManager.default.contents(atPath: rawPath), data.count > 4 else { await MainActor.run { ui.setInfo("ERRO: não consegui ler \(rawPath)") }; return } let file: [Float] = data.withUnsafeBytes { Array($0.bindMemory(to: Float.self)) } let feed = FeedBuffer() let start = Date() Task.detached { var fed = 0 while true { try? await Task.sleep(nanoseconds: 100_000_000) let want = Int(Date().timeIntervalSince(start) * Double(sampleRate)) if want > fed { var chunk = [Float](); chunk.reserveCapacity(want - fed) for i in fed.. Privacidade > Microfone") } return } let ap = whisper.audioProcessor // LC_SAVE_AUDIO=1: grava o microfone em logs/audio-*.f32 (mesmo formato do LC_WAV, para reproduzir/comparar modelos) var recorder: (([Float]) -> Void)? = nil if ProcessInfo.processInfo.environment["LC_SAVE_AUDIO"] != nil { let df = DateFormatter(); df.dateFormat = "yyyyMMdd-HHmmss" try? FileManager.default.createDirectory(atPath: "\(projectDir)/logs", withIntermediateDirectories: true) let path = "\(projectDir)/logs/audio-\(df.string(from: Date())).f32" FileManager.default.createFile(atPath: path, contents: nil) if let fh = FileHandle(forWritingAtPath: path) { recorder = { chunk in chunk.withUnsafeBufferPointer { fh.write(Data(buffer: $0)) } } await MainActor.run { ui.setInfo("gravando áudio em \(path)") } } } startMic = { try? ap.startRecordingLive(callback: recorder) } stopMic = { ap.stopRecording() } do { try ap.startRecordingLive(callback: recorder) } catch { await MainActor.run { ui.setInfo("ERRO no microfone: \(error.localizedDescription)") } return } getSamples = { Array(ap.audioSamples) } getCount = { ap.audioSamples.count } isVoice = { AudioProcessor.isVoiceDetected(in: ap.relativeEnergy, nextBufferInSeconds: endSilenceSeconds, silenceThreshold: 0.3) } purge = { ap.purgeAudioSamples(keepingLast: $0) } await MainActor.run { ui.setInfo("ouvindo… (fale em \(srcLang))") } } var opts = DecodingOptions() opts.verbose = false opts.task = .transcribe opts.language = srcLang opts.detectLanguage = false opts.temperature = 0 opts.temperatureFallbackCount = 0 // a re-tentativa com temperatura dobrava o tempo das travadas opts.sampleLength = 140 opts.skipSpecialTokens = true opts.withoutTimestamps = false opts.wordTimestamps = false opts.suppressBlank = true opts.noSpeechThreshold = 0.6 opts.compressionRatioThreshold = 2.4 opts.logProbThreshold = -1.0 if let tok = whisper.tokenizer, !glossary.whisperVocab.isEmpty { opts.promptTokens = tok.encode(text: " " + glossary.whisperVocab.joined(separator: ", ")) } var lastDecodedCount = 0 var hasPendingSpeech = false func decode(_ samples: [Float]) async -> [TranscriptionSegment] { let t0 = Date() let winS = Double(samples.count) / Double(sampleRate) let maxTokens = 30 + Int(winS * 10) // corta decodificação em loop: fala normal fica bem abaixo disso let res = try? await whisper.transcribe(audioArray: samples, decodeOptions: opts) { progress in progress.tokens.count > maxTokens ? false : nil } let dt = Date().timeIntervalSince(t0) lastWhisperSec = dt lastRTF = dt / max(0.1, Double(samples.count) / Double(sampleRate)) if dt > 3 { logLine(["evento": "decode_lento", "s": dt, "janela_s": Double(samples.count) / Double(sampleRate), "segs": res?.segments.count ?? 0]) } mDecodes += 1; mDecSum += dt; mDecMax = max(mDecMax, dt); mRtfSum += lastRTF let segs = res?.segments ?? [] // segmentos demais para a duração = timestamps em loop (degeneração): descarta if segs.count > 3 && Double(segs.count) / max(1, winS) > 0.8 { logLine(["evento": "decode_descartado", "segs": segs.count, "janela_s": winS]) return [] } return segs } func confirm(_ text: String) { let t = glossary.applyPost(clean(text)) guard !isHallucination(t) else { return } confirmWall.append(Date()) Task { @MainActor in self.ui.addConfirmed(t) } let parts = sentences(t).filter { $0.contains { $0.isLetter || $0.isNumber } } Task { for sent in parts { await self.translator.enqueue(sent) } } status() } var pausedApplied = false while true { try? await Task.sleep(nanoseconds: 150_000_000) if paused { if !pausedApplied { stopMic(); purge(0) // desliga o microfone e descarta o que estava em andamento lastDecodedCount = 0; hasPendingSpeech = false; pausedApplied = true } continue } else if pausedApplied { startMic(); pausedApplied = false await MainActor.run { ui.setInfo("ouvindo… (fale em \(srcLang))") } } let all = getSamples() let count = all.count let secs = Float(count) / Float(sampleRate) winSecs = Double(secs) let voiceNow = isVoice() if !voiceNow { if hasPendingSpeech { // fim de frase: decodificação final do que sobrou, confirma tudo e zera a janela let segs = await decode(all) let text = segs.map { clean($0.text) }.joined(separator: " ") confirm(text) purge(Int(0.3 * Float(sampleRate))) lastDecodedCount = 0; hasPendingSpeech = false await MainActor.run { ui.setPending("") } } else if secs > 2 { purge(Int(0.5 * Float(sampleRate))) // só silêncio: não deixa crescer lastDecodedCount = 0 } continue } hasPendingSpeech = true let newSecs = Float(count - lastDecodedCount) / Float(sampleRate) guard newSecs >= decodeEverySeconds else { continue } lastDecodedCount = count let segs = await decode(all) let texts = segs.map { clean($0.text) } if secs > maxWindowSeconds || segs.count >= 2 { // janela longa sem pausa: fecha tudo menos o último segmento (ou tudo se passou do limite) let keepLast = secs > maxWindowSeconds ? 0 : 1 let closed = texts.dropLast(keepLast) confirm(closed.joined(separator: " ")) let cutEnd = secs > maxWindowSeconds ? secs : (segs.dropLast(keepLast).last?.end ?? 0) guard cutEnd > 0 else { continue } let removeSamples = Int(cutEnd * Float(sampleRate)) purge(max(0, getCount() - removeSamples)) lastDecodedCount = 0 let rest = keepLast == 1 ? (texts.last ?? "") : "" await MainActor.run { ui.setPending(rest) } } status() } } } // MARK: - App @MainActor final class AppDelegate: NSObject, NSApplicationDelegate { var view: CaptionView! var engine: Engine! var usr1: DispatchSourceSignal? @objc func togglePause() { engine.togglePause() } func applicationDidFinishLaunching(_ n: Notification) { let menu = NSMenu(), item = NSMenuItem(), sub = NSMenu() let pause = NSMenuItem(title: "Pausar / Retomar", action: #selector(togglePause), keyEquivalent: "p") pause.target = self sub.addItem(pause) sub.addItem(withTitle: "Sair do LiveCaption", action: #selector(NSApplication.terminate(_:)), keyEquivalent: "q") item.submenu = sub; menu.addItem(item); NSApp.mainMenu = menu view = CaptionView() engine = Engine(ui: view) let e = engine! if let snap = ProcessInfo.processInfo.environment["LC_SNAPSHOT"] { let v = view! DispatchQueue.main.asyncAfter(deadline: .now() + 4) { v.setInfo("whisper 0.4s · 1ª palavra 0.2s · tradução completa 0.7s"); v.snapshot(to: snap) } } view.onToggle = { e.togglePause() } // `kill -USR1 ` (ou scripts/livecaption.sh pause) alterna pausa/retomada: dá para ligar a qualquer atalho externo signal(SIGUSR1, SIG_IGN) let src = DispatchSource.makeSignalSource(signal: SIGUSR1, queue: .main) src.setEventHandler { e.togglePause() } src.resume() usr1 = src Task.detached { await e.run() } } func applicationShouldTerminateAfterLastWindowClosed(_ app: NSApplication) -> Bool { true } } MainActor.assumeIsolated { let app = NSApplication.shared app.setActivationPolicy(.regular) let delegate = AppDelegate() app.delegate = delegate app.run() }