Spaces:
Runtime error
Runtime error
Commit History
-
merchant id missing
f7b5a05
-
fix: use plural entity type constants for queue names; fix inventory levels sync
0979938
-
catalogue invetory sync issue
8b188c4
-
refactor: replace hardcoded entity_type strings with constants from models.py
f09390e -
fix: merchant and employee queue name mismatch - use singular to match SCM-MS publisher
ca218f8 -
fix: catalogue queue name mismatch - use singular 'catalogue' to match SCM-MS publisher
51050a1 -
fix: limit SQLAlchemy pool size to prevent connection exhaustion
5db0a21 -
feat: sync catalogue_pricing and catalogue_inventory tables
After upserting catalogue_ref, also populate: - trans.catalogue_pricing: one row per pricing level (ncnf/cnf/distributor/retail) - trans.catalogue_inventory: reorder_level, lead_time_days, moq from inventory/procurement703d340 -
fix: correct MongoDB collection names for warehouse and uom
scm_warehouses -> scm_warehouse scm_uom_master -> scm_uom_group87bb00a -
fix: use plural entity_type to match publisher queue key names
Workers were subscribing to sync:queue:catalogue/merchant/employee/warehouse but publisher writes to sync:queue:catalogues/merchants/employees/warehouses. Queue items were never consumed due to this key name mismatch.2103664
-
fix(redis): cut workers to 7 total, remove per-iteration Redis calls to reduce connection pressure
dfbedd9 -
fix(redis): use correct redis.exceptions import to re-raise ConnectionError
77e12bc -
fix(redis): increase pool max_connections to 50, add exponential backoff on ConnectionError
0708ac4 -
feat(logging): implement production-grade structured logging with JSON and rotating files
- Add JSONFormatter for structured JSON output compatible with log aggregators (ELK, CloudWatch, Datadog) - Add ConsoleFormatter with color-coded output for local development - Implement rotating file handlers with configurable size and backup limits - Add separate error.log file for ERROR and CRITICAL level logs - Create LoggerAdapter for injecting worker/entity context into log records - Add structured startup/shutdown events with service metadata - Extend .env.example with LOG_FORMAT, LOG_DIR, LOG_MAX_BYTES, and LOG_BACKUP_COUNT configuration options - Update worker_manager.py to use new logging context for worker lifecycle events - Update base_service.py to emit structured logs for sync operations - Update main.py to initialize logging with startup event - Enables production-ready log aggregation while maintaining human-readable console output for local developmentf2bfacd -
refactor(redis-queue): reduce logging verbosity in enqueue and dequeue operations
- Remove debug log from enqueue method that logged sync operation details - Remove debug log from dequeue method timeout exception handler - Decrease noise in queue operations to improve log clarity and performancec021ed8 -
refactor(sync-worker-ms): reduce logging verbosity in worker loop and queue operations
- Change health check logging from every 10 iterations to every 180 iterations (~3 minutes) - Remove redundant dequeue attempt logging that occurred at iteration 1 and every 50 iterations - Remove duplicate null check for sync_op after dequeue operation - Remove debug and info level logs from Redis queue dequeue method - Reduces noise in logs while maintaining essential health check visibilityc3290dc
-
feat(sync-worker-ms): add delete_from_postgres method to merchant handler
- Add delete_from_postgres method to MerchantSyncHandler class - Implement DELETE operation for trans.merchants_ref table in PostgreSQL - Include comprehensive logging for successful deletions and not-found cases - Add error handling with detailed exception logging - Return True on successful deletion or when merchant not found - Maintain consistency with delete patterns in employee, warehouse, and uom handlersa2386f0 -
feat: add delete_from_postgres methods to employee, warehouse, and uom handlers
- Fixed employee handler: corrected table name to trans.employees_ref (plural) - Fixed employee handler: corrected column name to employee_id (not user_id) - Added delete_from_postgres() to WarehouseSyncHandler for trans.scm_warehouse_ref - Added delete_from_postgres() to UOMSyncHandler for trans.scm_uom_group_ref - All handlers now support DELETE operations from PostgreSQL - Consistent error handling and logging across all delete methods0d1ff6a -
Add DELETE operation support to sync worker
- Added 'delete' to valid operations in SyncOperation model - Added delete_from_postgres method to base handler - Updated sync method to handle DELETE operations separately - Updated base_service to pass operation type to handler - Implemented delete_from_postgres in CatalogueSyncHandler - DELETE operations now properly remove records from PostgreSQL Fixes issue where catalogue deletions in MongoDB were not synced to PostgreSQLd197a66 -
Fix catalogue sync: support status at root level
- Updated validate_required_fields to check status in both meta.status and root level - Updated extract_nested_fields to extract status, created_at, updated_at from root level as fallback - Fixes issue where catalogues with status at root level were passing validation but timestamps were None44a2b7c
-
feat(worker-manager): implement Redis connection pooling for sync workers
- Add Redis ConnectionPool with max_connections=15 for improved resource management - Enable socket keepalive to maintain persistent connections - Set socket_connect_timeout to 5 seconds for faster failure detection - Enable retry_on_timeout for automatic reconnection on timeout events - Add health_check_interval=30 for periodic connection validation - Update logging to reflect connection pool configuration - Improves connection stability and resource efficiency for sync worker operationse86dee7 -
first commit
6391dfd