Spaces:
Running
Running
Commit History
-
feat: Implement remote audio management and diagnostics Running
- Introduced a new `createRemoteAudioHost` function to manage the remote audio element's container. - Added functions for creating and clearing remote audio elements, enhancing audio track handling. - Updated `attachRemoteAudioTrack` and `detachRemoteAudioTrack` functions to support new binding logic for remote audio tracks. - Enhanced diagnostics logging for remote audio track subscription and detachment, improving tracking of audio state changes. - Refactored existing audio management functions to utilize the new structure, ensuring better organization and clarity in audio handling.1877afd -
refactor: Remove observed total latency from metrics calculations
- Eliminated the `observed_total_latency` parameter from various metrics-related functions, including `ChannelPublisher`, `TurnMetrics`, and `MetricsCollector`. - Updated latency calculations to use strict component sums instead of observed values, enhancing clarity and consistency in latency reporting. - Adjusted related tests to reflect the changes in latency handling and ensure accurate validation of metrics.b0c775f
-
refactor: Remove LLM to TTS handoff latency from metrics and UI
- Eliminated the `llm_to_tts_handoff_latency` metric from various components, including `ChannelMetrics`, `TurnMetrics`, and UI elements. - Updated related calculations and tests to ensure the removal does not affect overall latency reporting. - Adjusted tool pipeline views and metrics collection to reflect the absence of the handoff metric, enhancing clarity in latency measurements.ee7cdab -
feat: Enhance metrics collection and turn tracing for assistant responses
- Added new attributes in `PendingUserUtterance` and `TraceTurn` to track assistant response states and recovery from false turns. - Updated `MetricsCollector` to handle reopening user utterances and managing superseded speech IDs, improving the accuracy of conversation tracking. - Enhanced `TurnTracer` to support recovery of pre-response turns and ignore stale events, ensuring more reliable turn management. - Expanded unit tests to validate the new features and ensure proper functionality of the updated metrics and tracing mechanisms.cd02c6a -
feat: Enhance metrics collection and tracing for assistant text
- Introduced new methods in `MetricsCollector` for handling streamed assistant text, including delta submissions and flush observations. - Updated `TurnTracer` to support new flags and methods for tracking streamed assistant text and its correlation with conversation items. - Added new data classes `PendingAssistantItemRecord` and `PendingUnscopedStreamRecord` to manage assistant items and unscoped streamed text. - Enhanced session handling to install tracing for text output when metrics collector attributes are present. - Expanded unit tests to validate the new features and ensure proper functionality of the updated metrics and tracing mechanisms.a524a82 -
feat: Enhance metrics collection and assistant item correlation
- Added support for correlating conversation items with speech items in the `MetricsCollector`, allowing for improved tracking of assistant responses. - Introduced new data classes `AssistantItemSpeechBinding` and `PendingAssistantItemEvent` to manage the relationship between chat items and speech IDs. - Updated methods in `MetricsCollector` to handle the new item correlation logic, including adjustments to item addition and speech handling. - Enhanced `TurnTracer` to support new flags for tracking speech completion and assistant text observations. - Added unit tests to validate the new correlation features and ensure proper functionality of the updated metrics collection mechanisms.6549fd6 -
feat: Enhance turn tracing and metrics collection
- Introduced `LANGFUSE_ASSISTANT_TEXT_GRACE_TIMEOUT_MS` in `.env.example` and `settings.py` to manage the grace period for waiting on assistant text before finalizing traces. - Refactored `MetricsCollector` and `TurnTracer` to support new metrics and improve handling of user utterances and assistant text sources. - Updated `PocketTTS` initialization to streamline parameter handling. - Enhanced tests to validate the new features and ensure proper functionality of the updated tracing and metrics collection mechanisms.04178a2
-
codex fixex
7fdafe4 -
feat: Introduce MCP connection timeout handling and enhance configuration
- Added `MCP_CONN_TIMEOUT_SEC` setting to manage timeout for MCP tool requests in `settings.py`. - Updated `.env.example` to include MCP timeout configuration. - Enhanced `LLMRuntimeConfig` and `ConfiguredMCPServerHTTP` to utilize the new MCP timeout setting. - Implemented error handling for MCP tool timeouts and unavailability, providing user guidance in case of failures. - Updated assistant instructions to reflect new behavior regarding tool retries after failures. - Enhanced tests to validate MCP timeout handling and ensure proper functionality of the new settings.8193c41 -
feat: Enhance voice activity detection and turn tracing features
- Updated VAD parameters in `.env.example` for improved responsiveness and accuracy, including adjustments to `VAD_MIN_SILENCE_DURATION`, `VAD_THRESHOLD`, and endpointing delays. - Introduced a new `PendingUserUtterance` class in `metrics_collector.py` to manage user utterances more effectively, allowing for better tracking of speech-to-text transitions. - Refactored `TurnTracer` to support coalescing of user transcripts, enabling the merging of immediate continuations into prior turns for a more seamless conversation flow. - Updated settings in `settings.py` to reflect new defaults for voice processing and turn detection. - Enhanced tests to validate the new features and ensure proper functionality of the updated VAD and turn tracing mechanisms.4a03ace
-
feat: Implement NVIDIA TTS stream patch for improved synthesis handling
- Added a patch to the NVIDIA TTS stream to enhance metrics and shutdown handling until LiveKit resolves the underlying issues. - Refactored the `_run` method of the NVIDIA TTS stream to support asynchronous processing and improved error handling during synthesis. - Updated the TTS factory to apply the patch when creating an NVIDIA TTS instance. - Enhanced tests to verify the application of the patch and ensure proper functionality of the NVIDIA TTS provider.9fbd33c -
feat: Add Deepgram STT provider support and enhance configuration options
- Introduced Deepgram as a new STT provider in `STTSettings`, including settings for model ID and language. - Updated `.env.example` and `README.md` to document the new Deepgram STT settings and their usage. - Enhanced the assistant's configuration summary to include details for the Deepgram STT provider. - Refactored the STT factory to support initialization of Deepgram STT engines, with appropriate logging for configuration. - Added tests to validate the integration of Deepgram STT and ensure proper handling of settings across different scenarios.c394e23 -
feat: Add NVIDIA TTS provider support and enhance configuration options
- Introduced NVIDIA as a new TTS provider in `VoiceSettings`, including settings for API key, voice, language code, server, function ID, and SSL usage. - Updated `.env.example` and `README.md` to document the new NVIDIA TTS settings and their usage. - Enhanced the assistant's configuration summary to include details for the NVIDIA TTS provider. - Refactored the TTS factory to support initialization of NVIDIA TTS engines, with appropriate logging for configuration. - Added tests to validate the integration of NVIDIA TTS and ensure proper handling of settings across different scenarios.6e34c93 -
feat: Implement dynamic TTS footer component and enhance footer HTML generation
- Added `_resolve_tts_footer_component` function to dynamically generate TTS provider links based on settings. - Updated `generate_footer_html` to utilize the new TTS footer component, improving flexibility for different TTS providers. - Enhanced tests to validate footer generation for both PocketTTS and Deepgram providers, ensuring correct display based on settings.419a1a4 -
feat: Add support for Deepgram TTS provider and related configurations
- Introduced `TTS_PROVIDER` and `DEEPGRAM_API_KEY` settings in `VoiceSettings` to support Deepgram as a TTS option. - Updated `.env.example` to document the new TTS provider settings and their usage. - Refactored the assistant's configuration summary to include details for the selected TTS provider. - Enhanced session management to utilize the new TTS factory for creating TTS engines. - Added tests to validate the integration of Deepgram and ensure proper handling of TTS settings.9b5c112 -
refactor: Revise assistant instructions and configuration summary formatting
- Updated the assistant instructions to prioritize concise responses and clarify tool usage, emphasizing the pipeline identity. - Refactored the configuration summary formatting to improve readability, changing from a single line to a structured multi-line format. - Enhanced tests to validate the updated instructions and ensure they reflect the new response style and setup summary accurately.5a8aabe -
feat: Introduce MCP startup greeting timeout and PocketTTS connection timeout settings
- Added `MCP_STARTUP_GREETING_TIMEOUT_SEC` to `LLMSettings` and `VoiceSettings` for controlling the startup greeting interruption behavior. - Introduced `POCKET_TTS_CONN_TIMEOUT_SEC` to define the timeout for PocketTTS synthesis attempts. - Updated `.env.example` and `README.md` to document the new timeout settings and their intended usage. - Refactored session management to utilize the new timeout settings for improved control over greeting and TTS behavior. - Enhanced tests to validate the new timeout configurations and ensure proper functionality.62404fa -
feat: Enhance LiveKit worker configuration and documentation
- Added `LIVEKIT_INITIALIZE_PROCESS_TIMEOUT_SEC` to `.env.example` and `LiveKitSettings` for configuring idle worker bootstrap timeout. - Updated `README.md` to document the new timeout setting and its intended usage. - Refactored server initialization in `session.py` to incorporate the new timeout setting. - Enhanced tests to validate the new LiveKit settings and ensure proper initialization behavior.60ef510 -
refactor: Consolidate LLM runtime configuration using LLMSettings
- Refactored the LLM runtime initialization to utilize a single LLMSettings object, simplifying parameter management. - Updated the assistant instructions to promote concise responses and clarify tool usage. - Enhanced test cases to validate the new LLMSettings integration and ensure proper functionality across different scenarios.1641cad -
feat: Update Ollama configuration for cloud and local usage
19bb8f7 -
refactor: Simplify MCP startup greeting for improved user interaction
7d4efce -
feat: Add support for extra MCP server URLs and enhance assistant instructions
- Introduced to allow configuration of additional MCP servers in and settings. - Updated to document the new configuration option and its usage. - Enhanced the assistant's instructions to include a summary of the current configuration, improving user awareness of the system setup. - Implemented a new function to resolve and deduplicate MCP server URLs. - Added tests to validate the new functionality and ensure proper handling of MCP server configurations.6febd52 -
feat: Enhance assistant instructions and footer display for LLM providers
- Updated assistant instructions to include the current date and clarify capabilities and limitations. - Modified footer display in the Streamlit app to show LLM provider names alongside their models for better user clarity. - Added tests to ensure the new assistant instructions and footer display functionality work as intended.a24d5f0 -
feat: Update LLM and PocketTTS connection options in session management
- Added a new method to build connection options for LLM and PocketTTS requests, consolidating timeout and retry settings. - Updated README.md to reflect changes in connection behavior for both LLM and PocketTTS. - Enhanced .env.example with a comment clarifying shared API timeout/retry tuning parameters.3344a84 -
feat: Update default Ollama model and enhance trace history management
- Changed the default Ollama model from 'kimi-k2.5' to 'qwen3-coder-next' in settings.py. - Modified trace history management in main.js to push new trace entries and sort them by creation time and trace ID, ensuring proper order and limiting history size.a92fd8b -
feat: Add Langfuse tracing support and UI enhancements
- Updated .env.example and README.md to include LANGFUSE_PROJECT_ID and LANGFUSE_PUBLIC_TRACES configurations. - Enhanced session bootstrap payload to include Langfuse settings for tracing. - Implemented UI components for displaying trace history and dropdown functionality. - Updated main.js to manage trace dropdown interactions and render trace history. - Added tests to validate Langfuse tracing behavior and ensure proper metadata handling.3f6837a
-
feat: Update LLM runtime configuration and tests for thinking behavior
5e3ba57 -
feat: Enhance turn pipeline summary with partial updates and tool activity tracking
- Introduced a method to publish partial turn pipeline summaries during tool step events. - Updated MetricsCollector and TurnTracer to handle partial summaries and tool activity checks. - Enhanced UI to reflect loading states for tool execution and total turn durations based on summary completeness. - Added tests to ensure proper publishing of partial summaries during tool execution phases.f801a21 -
feat: Introduce turn pipeline summary and enhance tool execution metrics
- Added a new method to publish a detailed turn pipeline summary, including phases and tool execution metrics.0a97994
-
feat: Update Ollama configuration and validation for cloud usage
- Enhanced .env.example and README.md to include examples for using Ollama Cloud with OpenAI-compatible endpoints. - Implemented validation in llm_runtime.py to prevent the use of ':cloud' aliases with the Ollama Cloud endpoint. - Updated default LLM provider to 'ollama' in settings.py and adjusted default model to 'qwen2.5:7b'. - Added tests to ensure proper rejection of invalid model configurations for Ollama Cloud.0167890 -
feat: Enhance metrics collection with event timestamps for assistant interactions
- Added event_created_at and item_created_at parameters to track timestamps for conversation items in MetricsCollector. - Updated _on_assistant_text and related methods to utilize these timestamps for improved tracing accuracy. - Refactored text extraction logic to return both the latest assistant text and its creation timestamp. - Introduced tests to validate the handling of delayed pre-tool assistant events and their timestamps.758164b -
feat: Enhance assistant behavior and tool feedback mechanisms
- Updated assistant instructions to prevent tool calls for self-description requests. - Introduced allowed_tool_names parameter to control tool usage in pre-tool feedback. - Enhanced MetricsCollector and TurnTracer to support tool step announcements and error handling. - Added tests to validate new assistant instructions and tool feedback behavior.c13fca7
-
feat: Add post-tool response timeout and enhance error handling in channel metrics
d0f8e17 -
feat: Enhance metrics collection and tracing with speech_id support
- Added speech_id attribute to MetricsCollector and TraceTurn for improved tracking of audio metrics. - Updated methods in MetricsCollector and TurnTracer to include speech_id in metrics and tracing. - Introduced new dataclasses for tool execution and LLM/TTS call tracing to capture detailed metrics. - Enhanced tests to validate the inclusion of speech_id and new metrics in tracing spans.bf5b959 -
refactor: Restructure agent components and remove unused files
- Updated directory structure in README to reflect new organization of agent components. - Refactored imports in to align with new module paths. - Removed obsolete files related to channel metrics, turn tracing, metrics collection, LLM runtime, STT factory, and tool feedback, streamlining the codebase. - Enhanced clarity and maintainability by consolidating related functionalities into dedicated directories.a1d8504 -
[200~feat: Enhance tool tracing and feedback mechanisms
- Added attributes to TraceTurn for tracking tool phases and timestamps. - Implemented attach_tool_step_started method to manage tool step announcements. - Updated MetricsCollector to trigger tool step started callback. - Enhanced tests to verify tool step tracking and post-response handling.~9bb60e3 -
feat: Add soundfile dependency and implement pre-tool feedback mechanism
c0055e3
-
feat: Implement LLM warm-up functionality and update connection settings
fa9dd6d -
feat: Implement startup greeting monitoring and enhance greeting handling
- Added asynchronous monitoring for startup greeting to handle timeouts and interruptions. - Refactored to support MCP runtime with improved error handling. - Introduced shutdown-safe task scheduling for startup greeting. - Updated tests to cover new greeting monitoring functionality and error scenarios.7d6d879 -
feat: Enhance telemetry with tool call tracing and metrics collection
- Introduced ToolCallTrace dataclass to capture details of function tool calls.c83e29a -
feat: Add LLM runtime and STT factory implementations
- Introduced to manage LLM configurations and MCP runtime decisions, supporting NVIDIA and Ollama providers. - Implemented to create STT instances based on the configured provider, supporting NVIDIA and Moonshine STT. - Added a test for generating footer HTML in the Streamlit app, ensuring correct behavior with the Ollama provider.8f7da58 -
Add MCP runtime support and update configuration: Introduce MCP_ENABLED setting in settings.py and .env.example, enhance README with MCP runtime details, and implement MCP runtime decision logic in agent.py. Update agent instructions and ensure fallback to legacy runtime when MCP is unavailable.
882a5d7
-
Update dependencies and enhance logging: Modify pyproject.toml to include additional features in livekit-agents and update transformers version. Add cryptography package to uv.lock and refactor metrics logging in agent.py to improve clarity and remove unused code. Adjust maximum endpointing delay in settings.py and update UI tooltip descriptions for better user understanding.
43dafc9
-
Update deployment environment configuration: Replace hardcoded default environment in agent.py
3a97ef8 -
Integrate Langchain usage patch and update UI metrics: Apply Langchain usage patch in agent.py, adjust pipeline stage layout in index.html, and enhance metric descriptions in main.js for improved clarity and functionality.
e9aa04f -
Enhance metrics handling and add EOU support: Update ChannelPublisher and TurnTracer to include end-of-utterance (EOU) metrics, refactor latency calculations, and improve metric attribute handling. Modify UI tooltips for clarity and adjust test cases to validate new metrics structure.
9ef4d80 -
Add LLM connection settings and enhance error handling: Introduce new parameters for LLM connection timeout, retry attempts, and stall timeout in .env.example and settings.py. Update agent.py to log errors and session closures, and implement a watchdog for LLM stall detection in MetricsCollector.
3d758c5
-
Refactor PocketTTS and test cases: Remove unused sample rate parameters in _tensor_to_pcm_bytes
3decc8a -
Update PocketTTS configuration and logging: Change sample rate handling to enforce native 24kHz, adjust idle process count in .env.example, and enhance audio diagnostics in main.js for better tracking of remote audio track events.
93ad4ad
-
Enhance agent metrics logging and configuration: Add detailed logging for various metrics (VAD, EOU, STT, LLM, TTS) in agent.py,
4730cee