- Add @pytest.mark.unit markers to all test functions
- Implement test_reporter_ran_tracking_still_works test
- Replace `if False: yield` with cleaner _empty_async_generator helper
- Add _create_task_prompt mock to test_internal_messages_are_filtered
- Use ORCH_MSG_KIND_* constants instead of string literals in _process_event
## Problem
After report synthesis, extra "JUDGING: ManagerAgent" and "PROGRESS: Step 11"
events appeared in the UI, confusing users. Root cause: we were treating
`ExecutorCompletedEvent` as a UI event when it's actually internal framework
bookkeeping (auto-emitted by MS Agent Framework for every executor).
## Solution
1. **Silence ExecutorCompletedEvent**: Remove UI event emission, keep only
internal state tracking (reporter_ran flag)
2. **Add metadata filtering**: Filter out `task_ledger` and `instruction`
messages from AgentRunUpdateEvent stream
3. **Remove dead code**: Delete unused `_handle_completion_event` and
`_get_event_type_for_agent` methods
## Changes
- src/orchestrators/advanced.py: Silence completion events, add metadata filter
- tests/unit/test_orchestrator_noise.py: New regression tests
- tests/unit/orchestrators/test_accumulator_pattern.py: Update expectations
- docs/bugs/P2_EXECUTOR_COMPLETED_EVENT_UI_NOISE.md: Full bug documentation
## Validation
- Senior …
* fix(deps): Upgrade agent-framework-core to 1.0.0b251204 (SPEC-18)
Breaking API changes in upstream require event class migration:
- MagenticAgentDeltaEvent → AgentRunUpdateEvent
- MagenticAgentMessageEvent → ExecutorCompletedEvent
- MagenticFinalResultEvent → (removed, use WorkflowOutputEvent)
Upstream fixed the repr bug we worked around in SPEC-17.
Keep Accumulator Pattern as safety net for now.
Closes P0: HuggingFace Spaces deployment blocked
* docs: Add SPEC-18 documentation and fix test docstring
- Add docs/specs/SPEC_18_AGENT_FRAMEWORK_UPGRADE.md with full migration guide
- Update test_accumulator_pattern.py docstring to reference new event classes
Part of SPEC-18: Agent Framework Core Upgrade
* refactor: Address CodeRabbit review feedback
- Use defensive getattr for event.data.text (consistency)
- Remove redundant isinstance check in _handle_final_event
- Update sync dates in requirements.txt and workflow-diagrams.md
- Add language specifier to SPEC-18 code block
Note:…
## Problem
The workflow terminates without ReportAgent producing a synthesis report.
Users see search/hypothesis/judge output but get "Research complete." with
no actual research report. This primarily affects Free Tier (HuggingFace 7B
manager model) which doesn't reliably delegate to ReportAgent.
## Solution
1. Track `reporter_ran` flag when ReportAgent produces output
2. On workflow termination, if ReportAgent never ran, force synthesis via
`_force_synthesis()` method (similar to `_handle_timeout()`)
3. Skip duplicate final events (both MagenticFinalResultEvent and
WorkflowOutputEvent were yielding "Research complete.")
## Testing
- 313 unit tests pass
- Linting and type checking pass
Fixes: P1 No Synthesis Free Tier
fix(p2): Resolve duplicate report content and first-turn timeout bugs (#123)
This commit completes BYOK (Bring Your Own Key) propagation through the
ENTIRE architecture - from UI to LLM clients AND embedding services.
## Changes
### LLM Layer (Chat/Agents)
- `get_model(api_key=None)` - PydanticAI model factory now accepts BYOK
- `get_chat_client(api_key=None)` - Already fixed in prior commit
- All `create_*_agent(api_key=None)` functions updated
- `JudgeHandler`, `LLMSubIterationJudge` accept api_key
### Embedding Layer (RAG/Memory)
- `LlamaIndexRAGService(api_key=None)` - Prioritizes BYOK over env
- `get_embedding_service(api_key=None)` - Activates premium tier on BYOK
- `get_embedding_service_if_available(api_key=None)` - Same
### Orchestrators (Plumbing)
- `AdvancedOrchestrator` - Captures _api_key, passes to both chat client AND embeddings
- `HierarchicalOrchestrator` - Same pattern
- `LangGraphOrchestrator` - Patched (deprecated but won't crash)
- `orchestrators/factory.py` - Passes api_key to all orchestrators
### Cleanup
- Removed dead Anthropic cod…
fix(P1): Replace stale 72B model fallbacks with 7B to avoid Novita 500 errors
Claude
Unverified signature
a4327d1
Bug: Three files had hardcoded fallbacks to Qwen2.5-72B-Instruct which
routes to Novita provider (unreliable, 500 errors). The config.py was
updated to use 7B, but these fallbacks were stale.
Files fixed:
- src/clients/huggingface.py:59 - HuggingFaceChatClient init
- src/agent_factory/judges.py:85 - get_model() HuggingFace branch
- src/orchestrators/langgraph_orchestrator.py:43 - LangGraph init (deprecated)
All fallbacks now use Qwen/Qwen2.5-7B-Instruct to stay on HuggingFace
native serverless infrastructure (models < 30B params).
See: CLAUDE.md "CRITICAL: HuggingFace Free Tier Architecture" section
All 310 unit tests pass.
fix(P0): Implement Accumulator Pattern to resolve Repr Bug (#117)
Changes based on CodeRabbit PR review:
1. docs/bugs/P0_HUGGINGFACE_TOOL_CALLING_BROKEN.md:
- Add 'text' language to fenced code block (MD040)
- Wrap bare GitHub URL in markdown link (MD034)
- Update workaround section to reflect implementation status
2. src/orchestrators/advanced.py:
- Add plain string handling to _extract_text() for WorkflowOutputEvent.data
- Filter repr-style noise from string payloads
3. src/clients/huggingface.py:
- JSON-encode structured tool results instead of str()
- Track call_id -> name mapping for tool result messages
- Add 'name' field to tool messages (required by some APIs)
fix(P0): Add function invoking marker and improve ChatMessage extraction
Changes:
1. Add __function_invoking_chat_client__ marker to HuggingFaceChatClient
- Without this marker, agent_framework ignores tools passed to the client
- The warning "does not support function invoking" is now eliminated
2. Improve _extract_text in AdvancedOrchestrator:
- Handle ChatMessage.contents (list of FunctionCallContent, TextContent)
- Extract tool call names when text content is empty
- Return empty string instead of object repr for graceful fallback
3. Fix WorkflowOutputEvent handling:
- Use _extract_text instead of str() to avoid object repr in output
- Provide meaningful fallback message when no text available
These fixes ensure the HuggingFace Free Tier can properly:
- Declare function calling capability to the framework
- Display tool calls and text content in events
- Show clean output instead of <ChatMessage object at ...>
fix(P1): Switch from Llama-3.1-70B to Qwen2.5-72B for HuggingFace free tier
- Extract magic number 45 to class constant _EST_SECONDS_PER_ROUND (advanced.py)
- Add @pytest.mark.unit to test_advanced_p2_dead_zones.py
- Remove debug print statements from test
- Add pytestmark = pytest.mark.unit to test_magentic_judge_termination.py
- Fix import ordering (stdlib before third-party)
- Add fallback p=0.15 for progress events without iteration (app.py)
- Refactor max_iters lookup with getattr chains to reduce statement count
- Fix SPEC_16 markdown: list spacing (1. vs 1. ) and code block languages
All 327 tests pass.
1. Add @pytest.mark.unit to test class for proper categorization
2. Handle user_task before text extraction (edge case fix)
3. Improve truncation tests to actually detect regression:
- Test sentence boundary truncation with proper position
- Test word boundary fallback separately
4. Remove unused imports
Co-authored-by: CodeRabbit AI <coderabbit@users.noreply.github.com>
Problem: Advanced orchestrator exposed raw internal framework events
from agent-framework-core to users:
- `Manager (task_ledger): We are working to address...` (truncated)
- `Manager (instruction): Conduct targeted searches...` (truncated)
- All mapped to type="judging" regardless of actual purpose
Solution:
1. Filter internal events: `task_ledger` and `instruction` now hidden
2. Transform: `user_task` → type="progress" with friendly message
3. Smart truncation: Cut at sentence/word boundaries, not mid-word
Tests: tests/unit/orchestrators/test_advanced_events.py (4 tests)
Closes #106
docs: Clarify config parameter behavior in factory.py
Address CodeRabbit recommendation to document that the config
parameter is only used by simple and hierarchical modes.
Advanced mode uses settings.advanced_max_rounds instead.
- Added a new synthesis method in the JudgeHandler to utilize free HuggingFace Inference for narrative generation in Free Tier mode.
- Updated the Orchestrator to check for the judge's synthesis method, ensuring consistent behavior across judging and synthesis.
- Enhanced integration and unit tests to verify the correct usage of the new synthesis method and fallback mechanisms.
- Documented the bug related to synthesis incorrectly using server-side API keys, outlining the root cause and potential fixes.
This change addresses user confusion and improves the Free Tier experience by ensuring expected functionality without requiring an API key.
fix: P0 provider mismatch and code quality audit fixes (#102)
* fix(SPEC_11): address CodeRabbit review feedback
- Update run_full.py docstring to reference sexual health pipeline (not drug repurposing)
- Update run_full.py help text to use sexual health query example
- Fix app.py domain display to show "Sexual Health" (not "Sexual_Health")
- Update test_nodes.py mock data to use sexual health terms (Testosterone, Androgen Receptor)
- Add @pytest.mark.unit markers to test_judges.py test classes
- Add specific assertion for next_search_queries in test_judges.py
- Add missing @pytest.mark.asyncio marker to test_mcp_tools.py
- Update test_mcp_tools.py and test_clinicaltrials.py to use sexual health queries
- Fix SPEC_12 markdown bare code fence (MD040)
* fix(SPEC_11): comprehensive domain alignment audit
Second pass of CodeRabbit review fixes - found additional domain mismatches:
- examples/search_demo/run_search.py: Update docstring from "drug repurposing" to "sexual health"
- examples/orchestrator_demo/run_agent.py: Update help text from "metf…
feat(SPEC_11): finalize transition to Sexual Health Research Specialist
This commit completes the transition of DeepBoner to a dedicated Sexual Health Research Agent by removing all references to "general" and "drug repurposing" domains. Key changes include:
- Updated domain defaults to exclusively use "sexual_health".
- Replaced all example queries and documentation to reflect a focus on sexual health topics.
- Removed any lingering references to non-sexual health concepts in code and tests.
- Ensured all tests are aligned with the new domain focus, passing successfully.
This change reinforces the project's identity and simplifies the codebase, enhancing clarity and usability for users focused on sexual health research.
Closes #89.
* feat: implement SPEC_10 domain-agnostic refactor
* fix: complete SPEC_10 audit - license + HierarchicalOrchestrator domain
License Fixes:
- LICENSE: Fix copyright "Antibody Training Pipeline ESM" → "DeepBoner Contributors"
- pyproject.toml: Add missing license = "Apache-2.0" field
- README.md: Fix frontmatter license: mit → apache-2.0
Domain Threading Fixes:
- hierarchical.py: Add domain param to __init__ and pass to ResearchTeam
- factory.py: Pass domain to HierarchicalOrchestrator
All 237 tests pass. Domain now properly threaded through all orchestrators.
* test: enhance domain handling in orchestrators and judges
- Updated unit tests for `configure_orchestrator` to include mock mode and free tier scenarios, ensuring the domain is correctly passed to handlers.
- Refactored tests for `JudgeHandler` to mock model retrieval, allowing for domain acceptance without API key requirements.
- Improved `AdvancedOrchestrator` tests to mock API key validation and ensure domain handling i…
feat: Wire LlamaIndex RAG into Simple Mode (Tiered Embedding) (#83)
- Updated graph nodes to accept an optional EmbeddingService for improved evidence handling and deduplication.
- Refactored search, judge, resolve, and synthesize nodes to utilize the embedding service for enhanced functionality.
- Modified the research graph creation to bind the embedding service to worker nodes, ensuring seamless integration.
- Added logging for better traceability during node execution.
- Expanded unit and integration tests to cover new embedding service interactions and ensure robust functionality.
feat: Integrate LangGraph for enhanced research orchestration
- Added LangGraph dependencies to pyproject.toml and uv.lock for improved state management and orchestration capabilities.
- Implemented LangGraph-based orchestrator in src/orchestrators/langgraph_orchestrator.py, enabling structured research workflows.
- Developed graph nodes and state management in src/agents/graph, including search, judge, and synthesis functionalities.
- Updated app.py to support new "god" mode for advanced orchestration.
- Enhanced documentation for active bugs and architectural gaps related to LangGraph integration.
- Added unit and integration tests to ensure functionality and reliability of the new features.
* feat: implement SPEC_06 fixes for simple mode synthesis
Implements research-backed architecture changes for Simple mode:
- Diverse evidence selection with lost-in-the-middle mitigation
- Debiased judge prompt (scoring only, no decision making)
- Code-enforced termination criteria (prevents infinite loops)
- Real partial synthesis with drug candidates when max iterations reached
- Unit and integration tests for all new logic
* fix: address CodeRabbit review feedback
- Fix type annotation: recommendation str -> Literal["continue", "synthesize"]
- Add @pytest.mark.unit markers to unit tests
- Add @pytest.mark.integration markers to integration tests
- Simplify early/late evidence assertions using set intersection
- Replace next() with explicit filtering for clearer test diagnostics
- Extract hardcoded thresholds (50, 30) to TERMINATION_CRITERIA
- Fix e2e mock to accept iteration/max_iterations parameters
All 177 tests pass, mypy clean, ruff clean.
* docs: add P3 architectural gap r…
feat: add service loader + SPEC_06 + P0 bug report
Service Loader:
- Create src/utils/service_loader.py for centralized lazy loading
- All orchestrators now use centralized service loading
- Enforces OrchestratorProtocol on Advanced and Hierarchical
SPEC_06 (Simple Mode Synthesis - not yet implemented):
- Research-backed spec for fixing Simple mode never synthesizing
- Identifies 6 root causes with November 2025 LLM-as-Judge research
- Separates scoring from decision-making (debiased architecture)
P0 Bug Report:
- Documents Simple mode producing only citations (no synthesis)
- Confidence dropping to 0% in late iterations (context overflow)
- Judge never recommending "synthesize" even with 455 sources
CodeRabbit concerns from PR #70 addressed.
fix: Address audit findings in orchestrators package
Claude
11888fc
CRITICAL fixes:
- HierarchicalOrchestrator now accepts config parameter and uses max_iterations
- Added timeout protection (DEFAULT_TIMEOUT_SECONDS = 300.0)
HIGH priority fixes:
- Added OrchestratorProtocol to base.py for type safety
- Factory returns OrchestratorProtocol instead of Any
MEDIUM priority fixes:
- Added deprecation warnings for MagenticOrchestrator class alias
- Added deprecation warning for get_magentic_orchestrator() function
- Improved error handling with separate ImportError/Exception blocks
- Added null check before calling analyzer.analyze() in simple.py
Type safety improvements:
- simple.py uses proper StatisticalAnalyzer/EmbeddingService types via TYPE_CHECKING
- All public API functions have proper return type annotations
- Sorted __all__ alphabetically per ruff RUF022
All 147 tests pass, linting and mypy clean.
refactor: Organize orchestrators into dedicated package
Claude
e82d9c9
- Create src/orchestrators/ package with proper module structure
- base.py: Shared protocols (SearchHandlerProtocol, JudgeHandlerProtocol)
- simple.py: Basic search-judge loop orchestrator (was orchestrator.py)
- advanced.py: Multi-agent MS Agent Framework orchestrator (was orchestrator_magentic.py)
- hierarchical.py: Sub-iteration middleware orchestrator
- factory.py: Factory pattern for orchestrator creation
- __init__.py: Clean public API with facade pattern
Design principles applied:
- Single Responsibility (SRP): Each file has one clear job
- Interface Segregation (ISP): Protocols in base.py
- Factory Pattern (GoF): Centralized creation logic
- Facade Pattern: Clean public imports via __init__.py
All imports updated across:
- src/app.py, src/agents/
- tests/unit/, tests/e2e/, tests/integration/
- examples/orchestrator_demo/
All 147 tests pass, linting and typecheck clean.
