Enhance episode control and jogging management in mujoco_server.py and nova_jogger.py
Georg
c1ce8c4
- Added episode duration tracking and watchdog functionality to automatically truncate episodes exceeding the maximum duration.
- Introduced timestamps for teleoperation commands to improve debugging and state management.
- Updated jogging state management to ensure proper handling of active jogging commands and WebSocket connection states in nova_jogger.py.
- Refactored episode control state initialization and reset logic for better clarity and functionality.
Enhance overlay camera state management in mujoco_server.py
Georg
8dfd85d
- Added logic to store a reference to the old renderer before updating the overlay camera state, preventing race conditions.
- Updated the state management to ensure the old renderer is closed only after the new state is set, mitigating use-after-free issues.
- Implemented error handling when closing the old renderer to improve robustness.
Enhance homing logic in mujoco_server.py to support timeout handling
Georg
2a49683
- Added timeout functionality to the homing process, allowing the system to return a timeout status if homing exceeds a specified duration.
- Implemented a polling mechanism to check the homing state, improving responsiveness during the homing operation.
- Refactored existing homing checks to streamline the logic and ensure proper handling of active homing states.
Implement blocking homing endpoint and refactor homing logic in mujoco_server.py
Georg
e9e2294
- Introduced a new blocking API endpoint (`/homing`) for UR5 that pauses jogging commands while executing homing, enhancing control during the homing process.
- Refactored the homing logic to remove the need for continuous home messages, improving efficiency and reducing potential drift.
- Updated the frontend to utilize the new blocking endpoint, simplifying the user interaction for homing.
- Removed deprecated home and stop_home message types from protocol_types.py to streamline the codebase.
- Enhanced tests to validate the new blocking homing functionality and ensure proper behavior during homing operations.
- Updated README.md to document the new homing endpoint and its usage.
Implement dynamic camera management and enhance overlay functionality in mujoco_server.py
Georg
2425670
- Introduced dynamic camera configuration capabilities, allowing users to register auxiliary cameras via a new API endpoint.
- Updated overlay camera state management to support dynamic camera states, improving the responsiveness of the UI to camera events.
- Enhanced the handling of camera events in the frontend, ensuring that the UI refreshes automatically when new cameras are added.
- Added comprehensive tests for the dynamic camera functionality, verifying state announcements and environment updates.
- Updated README.md to document the new dynamic camera API and its integration with the existing system.
Refactor depth snapshot processing in mujoco_server.py for metric depth representation
Georg
84b67c7
- Updated the capture_depth_snapshot function to convert depth values from meters to millimeters, ensuring precision in depth rendering.
- Introduced dynamic handling of maximum depth based on the MuJoCo visualization settings, improving adaptability to different environments.
- Enhanced depth normalization and clipping to handle invalid values more effectively, resulting in more accurate depth images.
Update .gitignore and enhance depth snapshot rendering in mujoco_server.py
Georg
bec2a82
- Added MUJOCO_LOG.TXT to .gitignore to prevent logging files from being tracked.
- Modified the capture_depth_snapshot function to dynamically set render height and width based on the camera name, improving flexibility in depth rendering.
multi-axes jogging
Georg
7e77839
tracking
Georg
25f4c68
Refactor Dockerfile to streamline build process and remove frontend build stage
Georg
8772f94
- Removed the multi-stage build for frontend assets, simplifying the Dockerfile.
- Consolidated application code copying to include the static frontend directory directly.
- Updated comments for clarity on the changes made to the build process.
Refactor homing control logic in mujoco_server.py for improved accuracy and responsiveness
Georg
3a6146c
- Adjusted joint velocity calculations to prevent overshoot and ensure smoother homing behavior.
- Introduced tighter tolerance checks for stopping individual joints, enhancing precision during the homing process.
- Removed redundant debug print statements to streamline code and improve clarity.
- Updated velocity damping mechanism to gradually reduce speed as joints approach their target positions, ensuring a more controlled movement.
Remove unused camera reset function in handle_ws_message of mujoco_server.py to streamline code and improve clarity.
Georg
55d7c05
Implement velocity-based jogging control in mujoco_server.py
Georg
36a37c0
- Added logic to stop active jogging when all velocity inputs are near zero, enhancing control responsiveness.
- Retained existing functionality to apply teleoperation movement when velocities are detected, ensuring smooth operation during active commands.
- This update improves the user experience by preventing unintended movements when no input is provided.
Enhance multi-axis homing and Nova API v2 integration
Georg
59625d9
- Updated the homing functionality in `mujoco_server.py` to support multi-axis movements using the Nova API v2, allowing all joints to move simultaneously for faster homing.
- Improved the `NovaJogger` class to utilize a unified v2 endpoint for joint and cartesian jogging, enhancing command efficiency and mode switching.
- Added comprehensive tests for multi-axis homing and jogging functionality, ensuring robust performance and validation of the new features.
- Updated `README.md` to reflect the new capabilities and integration details with the Nova API v2.
- Introduced new scripts for testing multi-axis homing and validating the v2 jogging commands, ensuring proper functionality before deployment.
Enhance homing functionality and client management in mujoco_server.py
Georg
0551bb3
- Updated the homing state management to include timeout checks, preventing drift when home messages cease.
- Improved the handling of identified and unidentified clients in the `broadcast_state()` function, ensuring accurate client status reporting.
- Introduced continuous command sending in the NovaJogger class for smoother joint movements during jogging.
- Enhanced the UI to display both identified and unidentified clients, improving user feedback on connection status.
- Added comprehensive tests for homing behavior, including drift prevention and overshoot detection, ensuring robust performance in various scenarios.
- Updated README.md to reflect changes in homing behavior and client management.
Add scene object extraction and update state broadcasting in mujoco_server.py
Georg
ec12dc0
- Introduced a new function `_get_scene_objects()` to extract position and orientation data for scene objects, specifically for UR5 scenes.
- Enhanced the `broadcast_state()` function to include scene object information in the state broadcast to connected clients.
- Updated `protocol_types.py` to define a new `SceneObject` type and included a list of scene objects in the `StateData` structure.
- Modified `README.md` to document the new `scene_objects` field, detailing its structure and usage.
Update UR5 home pose for Nova integration in environment and scene files
Georg
4f6eb67
- Modified the home position values in `ur5_env.py`, `scene_t_push.xml`, and `scene_with_gripper.xml` to reflect a custom configuration for improved integration with Nova.
- Adjusted joint angles to enhance the robot's initial state, ensuring better performance during simulations.
Update UR5 home pose in scene.xml for improved robot positioning
Georg
7ba9543
- Modified the home keyframe's qpos and ctrl values to reflect a more accurate home position for the UR5 robot.
- This change aims to enhance the robot's initial state configuration, ensuring better performance during simulations.
Enhance Nova API state management in frontend
Georg
18ba7d2
- Introduced a new state tracking mechanism for the Nova API to minimize unnecessary UI updates.
- Updated connection handling to only refresh UI elements when there are changes in the Nova API state.
- Improved error detection and notification for Nova connection issues, providing clearer feedback to users.
- Refactored badge visibility and controller indicators to reflect the current Nova API state more accurately.
- Enhanced mode text updates based on connection status, ensuring users receive timely information about the system's state.
Implement frontend integration and HTML template loading in mujoco_server.py
Georg
598c3a8
- Added a new HTML file for the frontend, establishing a structured layout for the Nova Sim interface.
- Updated the mujoco_server.py to load the HTML template dynamically, replacing inline HTML with a more maintainable approach.
- Enhanced the response handling to prevent browser caching, ensuring users receive the latest version of the frontend assets.
- This commit sets the foundation for future frontend enhancements and improves the overall project structure.
Refactor mujoco_server.py and protocol_types.py for external client support
Georg
223bbeb
- Replaced trainer-specific WebSocket handling with a more generic external client model, allowing for better integration of various client types.
- Updated message types and payload structures to reflect the new client identity and notification system, ensuring backward compatibility with legacy trainer messages.
- Enhanced the broadcast mechanisms to notify all connected clients about their connection status, improving the overall communication flow.
- Removed deprecated trainer-related code and tests, streamlining the server's functionality and focusing on external client interactions.
- Updated README.md to reflect changes in client messaging and connection handling, providing clearer guidance for developers.
Remove frontend and Makefile components from the project
Georg
11dd56c
- Deleted the entire frontend directory, including configuration files, source code, and styles, to streamline the project structure.
- Removed the Makefile, which contained development commands for building and running the frontend, as it is no longer needed.
- This cleanup aims to simplify the repository and focus on backend functionalities.
Update mujoco_server.py to replace jogging commands with homing commands and enhance action handling
Georg
51b2fc6
- Removed `start_jog` and `stop_jog` commands, replacing them with `home` and `stop_home` for improved homing functionality.
- Updated WebSocket message handling to support continuous homing commands and refined the logic for sending action messages.
- Adjusted the `index` function to streamline the homing process, ensuring smoother joint movements and better user experience.
- Enhanced the .gitignore file to exclude server logs, maintaining a cleaner repository.
Update Docker configuration and add frontend build process
Georg
eaf03f4
- Changed the exposed port for the nova-sim service from 3004 to 5001 in both docker-compose files to avoid conflicts.
- Added a multi-stage build process in the Dockerfile to compile frontend assets using Node.js and Vite, improving deployment efficiency.
- Introduced a Makefile for simplified development commands, including `make dev` for starting backend and frontend servers with hot reload.
- Updated .gitignore to exclude frontend build artifacts and development logs, ensuring a cleaner repository.
- Enhanced README.md to reflect new setup instructions and project structure, emphasizing the use of the Makefile for development.
Refactor teleoperation commands to actions in mujoco_server.py and update protocol documentation
Georg
9013e5d
- Renamed `teleop_command` to `teleop_action` and `command` to `action` for consistency across the codebase.
- Updated message handling to support new action-based velocity control for UR5 robots, including backward compatibility for legacy message types.
- Enhanced the README.md and PROTOCOL_CHANGES.md to reflect the new action structure and provide clear migration guidance for developers.
- Introduced a new `protocol_types.py` file with TypeScript-style type definitions for all protocol messages, improving type safety and IDE support.
- Updated tests to validate the new action message structure and ensure backward compatibility with existing command formats.
Add homing functionality to mujoco_server.py and implement test script
Georg
f6af091
- Introduced a state-based homing mechanism for UR5 robots, allowing for smoother joint movements towards home positions.
- Added a new `homing_state` dictionary to track the current state of the homing process, including joint errors and jogging status.
- Implemented logic to handle joint overshoot and stuck conditions during the homing sequence.
- Created a `test_home.py` script to simulate continuous home commands, testing the homing functionality and ensuring proper behavior during operation.
- Updated WebSocket message handling to support starting and stopping the homing process based on user input.
Enhance mujoco_server.py and README.md for improved teleoperation and environment management
Georg
9078db7
- Initialized last_teleop_command with default values to ensure consistent command structure.
- Updated switch_robot function to handle gripper state for UR5 robots, improving teleoperation control.
- Refactored the /env endpoint to return static environment information, including camera feeds and robot properties, enhancing API usability.
- Revised README.md to reflect changes in dependencies and clarify the usage of the new /env endpoint for environment information.
- Added detailed documentation on the new camera feed structure and updated examples for better clarity.
Update scene_t_push.xml for simulation stability improvements
Georg
044130b
- Added a timestep parameter to the simulation options for better time resolution.
- Adjusted solref and solimp values for the push stick and t_object geometries to enhance simulation stability and performance.
Add kill script for clean server restarts on port 3004
Georg
85d769f
- Introduced a new `kill.sh` script to terminate any processes bound to port 3004, ensuring the nova-sim server can be restarted without conflicts.
- The script checks for the presence of `lsof` and provides appropriate messages based on the server's state, enhancing usability and error handling.
Remove MUJOCO_LOG.TXT file containing simulation instability warnings to clean up project files.
Georg
03886be
Implement camera and environment reset functions in mujoco_server.py
Georg
cd29740
- Added _reset_camera_for_current_robot to configure camera settings based on the active robot.
- Introduced _reset_active_environment to reset the simulation environment and camera together.
- Updated _handle_gym_ws_message to call _reset_active_environment on specific message types.
- Refactored reset logic in handle_ws_message to utilize the new reset functions, improving code organization and maintainability.
Enhance mujoco_server.py and UR5 environment for improved teleoperation and episode control
Georg
c525614
- Introduced new mechanisms for handling teleoperation commands, including a snapshot of the last command for better state management.
- Updated the reward calculation in the UR5 environment to prioritize task rewards, with a fallback to distance-based rewards.
- Added gym-style control message handling over the unified WebSocket, allowing for richer interactions and state updates.
- Implemented a robust episode control flag system to manage episode termination and truncation without interfering with the gym client's lifecycle.
- Enhanced documentation in README.md to clarify server startup options and the unified WebSocket API for trainers and clients.
- Added unit tests to verify the functionality of the episode control mechanism, ensuring thread safety and correct behavior across multiple episodes.
Add overlay camera render resolution settings in mujoco_server.py
Georg
fc65a06
- Introduced new environment variables for overlay render width and height to enhance customization of camera outputs.
- Updated the renderer initialization to utilize these new settings, improving the flexibility of the rendering process.
- This change aims to streamline the rendering configuration for better performance and adaptability in various environments.
Remove UI notification for episode control requests in mujoco_server.py
Georg
4a81746
- Eliminated the broadcast notification to the UI when an episode control action is requested, streamlining the communication process.
- This change focuses on reducing unnecessary UI updates while maintaining the core functionality of episode management.
Enhance trainer client management and connection status in mujoco_server.py
Georg
d77b79f
- Introduced new functions to register and unregister trainer clients, improving connection tracking.
- Added functionality to build a payload summarizing connected trainer clients, enhancing real-time status updates.
- Updated the trainer websocket handler to manage client identities and connection states more effectively.
- Modified UI elements to reflect trainer connection status with improved visual indicators, enhancing user experience during teleoperation.
Add trainer connection status broadcasting in mujoco_server.py
Georg
90be8a0
- Implemented a new function to notify UI clients about the trainer's connection status, enhancing real-time feedback.
- Updated the trainer websocket handler to call the new broadcasting function upon client connection and disconnection.
- Modified the UI message handling to update the trainer connection status dynamically, improving user experience during teleoperation.
Refactor UR5 environment for enhanced Nova state management and UI adjustments
Georg
3305880
- Updated the UR5 environment to include new attributes for storing Nova joint snapshots and TCP positions, improving state management.
- Modified the shutdown process to optionally preserve the Nova state, enhancing the robustness of the environment during transitions.
- Implemented methods to store and apply the last Nova state, ensuring smoother operation and better integration with user commands.
- Adjusted the UI overlay positioning in mujoco_server.py for improved visual alignment.
Enhance mujoco_server.py and UR5 environment for Nova API integration and UI improvements
Georg
6aec869
- Added functionality to check for Nova credentials and determine their presence for default toggle state.
- Updated the UR5 environment to support Nova API configuration, including methods for enabling/disabling Nova integration and handling state synchronization.
- Enhanced the UI to reflect Nova connection status with improved visual indicators and connection state management.
- Implemented automatic toggling of Nova mode based on user interactions and connection status, improving user experience during teleoperation.
- Refined the handling of trainer connection status in the UI, providing real-time feedback on trainer connectivity.
Refactor mujoco_server.py to enhance arm teleoperation controls
Georg
eeed5a9
- Removed the "Go to Home Position" button from the UI for a cleaner interface.
- Introduced a key mapping system for arm jogging, allowing users to control the arm's movement using keyboard inputs.
- Implemented functions to handle key down and key up events for improved teleoperation responsiveness.
- Updated the key state change logic to accommodate the new arm control features, enhancing user interaction during teleoperation.
Add environment loading utility and integrate into Nova API and Jogger
Georg
07bb707
- Introduced a new utility module for loading environment variables from `.env.local` to streamline configuration management.
- Integrated the environment loading functionality into the Nova API and Jogger modules to ensure proper configuration during runtime.
- Updated the UR5 environment to utilize the loaded home pose for joint initialization, enhancing simulation consistency.
Refactor UR5 environment methods for object orientation and positioning
Georg
d428055
- Renamed method for generating T-object orientation to clarify its purpose.
- Removed unused target body quaternion setting method to streamline the code.
- Updated randomization logic to incorporate the new orientation method for improved object placement.
Add MUJOCO_LOG.TXT for simulation warnings and update UR5 environment for target positioning
Georg
a625bfc
- Created MUJOCO_LOG.TXT to log simulation warnings related to NaN and Inf values in QACC, indicating instability.
- Updated UR5 environment constants for table dimensions and object heights to improve target positioning and alignment with the end effector.
- Adjusted the spawn height for objects and targets in the UR5 environment to ensure proper placement on the table surface.
Update UR5 environment and mujoco_server.py for enhanced target positioning and randomization
Georg
3c5e479
- Adjusted camera overlay presets in mujoco_server.py for improved visualization.
- Introduced new methods in UR5 environment to randomize target object positions and orientations, enhancing simulation variability.
- Updated target height offset logic for better alignment with the end effector's position.
- Added constants for table dimensions to facilitate object placement within defined boundaries.
Reset UI environment on episode control requests in mujoco_server.py
Georg
0aa95e0
- Added functionality to reset the environment automatically when an episode control action is requested via the UI, allowing users to continue without manual resets.
- This enhancement improves user experience by streamlining the episode management process.
Implement episode control and enhance UR5 environment dynamics
Georg
c49a482
- Added episode control state management in mujoco_server.py to handle termination and truncation requests from the UI.
- Introduced functions to signal and consume episode control flags, allowing for better integration with the training process.
- Enhanced the UR5 environment's reward calculation by incorporating orientation penalties based on object alignment, improving target positioning accuracy.
- Updated the scene configuration for the UR5 model to include improved physical properties for better simulation behavior.