Buckets:

tahamajs
/

Sysmem2_in_AI

Files

xet

tahamajs/Sysmem2_in_AI / ComputerAssignments /CA24_multi_agent_systems /README.md

tahamajs

5 months ago

preview code

download

raw

10.3 kB

# CA24: Advanced Multi-Agent Systems

## 🎯 Overview

A comprehensive implementation of advanced multi-agent systems featuring modern communication protocols, sophisticated coordination mechanisms, and multi-agent reinforcement learning. This project demonstrates state-of-the-art techniques in distributed AI systems.

## ✨ Key Features

### 1. **Advanced Communication Systems**

- 📨 Multiple communication protocols (Message Passing, Publish-Subscribe, Broadcast, Multicast)
- 🎯 Priority-based message handling (Low, Medium, High, Critical)
- 🔀 Intelligent message routing and filtering
- 📊 Real-time communication monitoring and statistics
- 🌐 WebSocket support for distributed agents
- ⏱️ TTL (Time-To-Live) and retry mechanisms

### 2. **Advanced Coordination Mechanisms**

- 🏗️ **Hierarchical Coordination**: Top-down task allocation
- 💰 **Market-Based Allocation**: Auction-based task distribution
- 🤝 **Coalition Formation**: Dynamic agent grouping for complex tasks
- 🎯 **Consensus Algorithms**: Weighted average and majority vote
- 📋 **Task Allocation Methods**:
  - Round Robin
  - Load Balancing
  - Capability Matching
  - Auction-Based
  - Genetic Algorithm

### 3. **Multi-Agent Reinforcement Learning (MARL)**

- 🧠 **Multi-Agent DQN**: Deep Q-Network for multiple agents
- 🎭 **Multi-Agent A2C**: Advantage Actor-Critic for cooperative learning
- 🔄 **Experience Replay**: Efficient learning from past experiences
- 🎮 **Custom Environments**: Flexible multi-agent simulation environments
- 📈 **Training Metrics**: Comprehensive performance tracking

### 4. **Agent Types**

- 👔 **Coordinator Agent**: Task allocation and system coordination
- 🔬 **Researcher Agent**: Information gathering and analysis
- 📊 **Analyst Agent**: Pattern recognition and prediction
- 🤖 **ReAct Agent**: Reasoning and acting with LLMs
- 🔮 **Reflexion Agent**: Self-reflective learning
- 🦛 **HippoRAG Agent**: Advanced RAG with knowledge graphs
- 🔗 **Nexus Agent**: Multi-modal reasoning

## 📁 Project Structure

```
CA24_multi_agent_systems/
├── 📓 notebooks/           # Jupyter notebooks with examples
│   ├── CA24.ipynb         # Main comprehensive notebook
│   ├── 01_Advanced_Communication_Systems.ipynb
│   ├── 02_Advanced_Coordination.ipynb
│   ├── 03_Multi_Agent_RL.ipynb
│   └── 04_Complete_Projects.ipynb
├── 🔧 src/                # Source code
│   ├── core/              # Core multi-agent systems
│   │   ├── multi_agent_system.py
│   │   ├── advanced_communication.py
│   │   ├── advanced_coordination.py
│   │   ├── multi_agent_rl.py
│   │   ├── environment.py
│   │   ├── rag_system.py
│   │   └── shared_memory.py
│   ├── agents/            # Specialized agents
│   │   ├── react_agent.py
│   │   ├── reflexion_agent.py
│   │   ├── hipporag_agent.py
│   │   └── nexus_agent.py
│   ├── evaluation/        # Evaluation metrics
│   └── utils/             # Utilities
├── 🚀 scripts/            # Execution scripts
│   ├── main.py
│   └── run.sh
├── 🧪 tests/              # Test files
├── ⚙️ config/             # Configuration
├── 📊 data/               # Data and results
├── 💾 models/             # Saved models
├── 🔗 integrations/       # External integrations
├── 🎮 demos/              # Interactive demos
└── 📋 docs/               # Documentation
```

## 🚀 Quick Start

### Installation

```bash
# Clone the repository
cd CAs/CA24_multi_agent_systems
# Install dependencies
pip install -r requirements.txt
# Set up environment variables (optional)
export GEMINI_API_KEY='your_api_key_here'
```

### Running Examples

#### 1. Basic Multi-Agent System

```bash
cd scripts/
python main.py
```

#### 2. Advanced Communication Demo

```python
from core.advanced_communication import AdvancedCommunicationManager
from core.multi_agent_system import BaseAgent
# Create communication manager
comm_manager = AdvancedCommunicationManager("my_system")
# Register agents
await comm_manager.register_agent(agent1, ["topic_A"])
await comm_manager.register_agent(agent2, ["topic_B"])
# Send messages
await comm_manager.broadcast_message(
    sender="coordinator",
    content={"message": "Hello all agents!"},
    priority=MessagePriority.HIGH
)
```

#### 3. Coordination with Coalition Formation

```python
from core.advanced_coordination import AdvancedCoordinationManager, CoordinationStrategy
# Create coordination manager
coord_manager = AdvancedCoordinationManager(
    comm_manager,
    strategy=CoordinationStrategy.COALITION
)
# Register agents with capabilities
await coord_manager.register_agent(agent, capabilities)
# Allocate tasks
assigned_agent = await coord_manager.allocate_task(task)
```

#### 4. Multi-Agent Reinforcement Learning

```python
from core.multi_agent_rl import MultiAgentRLSystem
# Create MARL system
marl = MultiAgentRLSystem(
    num_agents=3,
    state_dim=4,
    action_dim=2,
    algorithm="dqn"
)
# Train agents
marl.train(num_episodes=1000, batch_size=32)
# Evaluate performance
results = marl.evaluate(num_episodes=10)
```

## 📚 Comprehensive Examples

### Example 1: Distributed Task Allocation

```python
import asyncio
from core.multi_agent_system import MultiAgentSystem
from core.advanced_coordination import CoordinationTask, AgentCapability
async def distributed_task_allocation():
    # Initialize system
    mas = MultiAgentSystem(gemini_config)
    await mas.initialize()
    # Create task with requirements
    task = CoordinationTask(
        description="Analyze market data",
        resource_requirements={
            "capabilities": ["analysis", "visualization"],
            "max_cost": 100.0
        },
        deadline=datetime.now() + timedelta(hours=1)
    )
    # Allocate task
    assigned_agent = await coord_manager.allocate_task(task)
    print(f"Task assigned to: {assigned_agent}")
    # Monitor progress
    status = await mas.get_system_status()
    print(f"System status: {status}")
asyncio.run(distributed_task_allocation())
```

### Example 2: Consensus-Based Decision Making

```python
async def consensus_decision():
    # Initial values from different agents
    initial_values = {
        "agent_1": 0.8,
        "agent_2": 0.6,
        "agent_3": 0.7,
        "agent_4": 0.9
    }
    # Reach consensus
    consensus_result = await coord_manager.reach_consensus(
        topic="resource_allocation",
        initial_values=initial_values,
        max_iterations=10
    )
    print(f"Consensus reached: {consensus_result}")
asyncio.run(consensus_decision())
```

### Example 3: Multi-Agent RL Training

```python
# Create cooperative multi-agent environment
marl_system = MultiAgentRLSystem(
    num_agents=4,
    state_dim=8,
    action_dim=4,
    algorithm="a2c"
)
# Train with custom parameters
marl_system.train(
    num_episodes=2000,
    batch_size=64,
    update_frequency=10
)
# Get training statistics
stats = marl_system.get_training_stats()
print(f"Training completed:")
print(f"  Average reward: {stats['average_rewards'][-1]:.2f}")
print(f"  Total steps: {stats['total_steps']}")
# Evaluate trained agents
eval_results = marl_system.evaluate(num_episodes=20)
print(f"Evaluation results: {eval_results}")
```

## 🎓 Jupyter Notebooks

Explore comprehensive tutorials and examples:

1. **CA24.ipynb** - Main comprehensive guide
2. **01_Advanced_Communication_Systems.ipynb** - Communication protocols
3. **02_Advanced_Coordination.ipynb** - Coordination mechanisms
4. **03_Multi_Agent_RL.ipynb** - Reinforcement learning
5. **04_Complete_Projects.ipynb** - End-to-end projects

## 🧪 Testing

```bash
# Run all tests
cd tests/
python -m pytest
# Run specific test
python test_communication.py
python test_coordination.py
python test_marl.py
```

## 📊 Performance Metrics

The system tracks various performance metrics:

- **Communication**: Message throughput, latency, delivery rate
- **Coordination**: Task allocation efficiency, coalition formation time
- **Learning**: Episode rewards, convergence rate, success rate
- **System**: CPU/memory usage, agent utilization

## 🔧 Configuration

### Environment Variables

```bash
# API Keys
export GEMINI_API_KEY='your_gemini_key'
export OPENAI_API_KEY='your_openai_key'
# System Configuration
export MAX_AGENTS=10
export MESSAGE_TIMEOUT=5.0
export COORDINATION_STRATEGY='hierarchical'
```

### Configuration Files

Edit `config/system_config.yaml` for advanced settings:

```yaml
communication:
  protocol: message_passing
  max_retries: 3
  ttl: 60
coordination:
  strategy: hierarchical
  allocation_method: capability_matching
reinforcement_learning:
  algorithm: dqn
  learning_rate: 0.001
  gamma: 0.99
  epsilon: 1.0
```

## 📖 Documentation

Detailed documentation available in `docs/`:

- **Architecture Guide**: System design and components
- **API Reference**: Complete API documentation
- **Best Practices**: Guidelines for multi-agent systems
- **Troubleshooting**: Common issues and solutions

## 🎯 Key Concepts Implemented

### 1. Cooperative Distributed Problem Solving (CDPS)

- Decentralized control and data storage
- Efficient communication protocols
- Loose coupling between agents

### 2. Consensus Dynamics

- Weighted average consensus
- Majority vote mechanisms
- Convergence guarantees

### 3. Distributed Constraint Optimization (DCOP)

- Constraint satisfaction
- Resource allocation
- Optimization algorithms

### 4. Algorithmic Game Theory

- Mechanism design
- Equilibrium analysis
- Strategic behavior modeling

### 5. Multi-Agent Reinforcement Learning (MARL)

- Independent learners
- Centralized training, decentralized execution
- Cooperative and competitive scenarios

---

## 📄 License

This project is part of the System2_in_AI CA collection.

## 🙏 Acknowledgments

- **Research Papers**: Based on latest MAS research
- **Frameworks**: LangChain, Ray RLlib, PettingZoo
- **Community**: Open-source multi-agent systems community

---

**Built with ❤️ for advanced multi-agent systems research and applications**

Xet Storage Details

Size:: 10.3 kB
Xet hash:: c59a6a423b7576f484d817a95801b8fbbe6900cb50aeac773c7017838af09f19

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# CA24: Advanced Multi-Agent Systems

	## 🎯 Overview

	A comprehensive implementation of advanced multi-agent systems featuring modern communication protocols, sophisticated coordination mechanisms, and multi-agent reinforcement learning. This project demonstrates state-of-the-art techniques in distributed AI systems.

	## ✨ Key Features

	### 1. Advanced Communication Systems

	- 📨 Multiple communication protocols (Message Passing, Publish-Subscribe, Broadcast, Multicast)
	- 🎯 Priority-based message handling (Low, Medium, High, Critical)
	- 🔀 Intelligent message routing and filtering
	- 📊 Real-time communication monitoring and statistics
	- 🌐 WebSocket support for distributed agents
	- ⏱️ TTL (Time-To-Live) and retry mechanisms

	### 2. Advanced Coordination Mechanisms

	- 🏗️ Hierarchical Coordination: Top-down task allocation
	- 💰 Market-Based Allocation: Auction-based task distribution
	- 🤝 Coalition Formation: Dynamic agent grouping for complex tasks
	- 🎯 Consensus Algorithms: Weighted average and majority vote
	- 📋 Task Allocation Methods:
	- Round Robin
	- Load Balancing
	- Capability Matching
	- Auction-Based
	- Genetic Algorithm

	### 3. Multi-Agent Reinforcement Learning (MARL)

	- 🧠 Multi-Agent DQN: Deep Q-Network for multiple agents
	- 🎭 Multi-Agent A2C: Advantage Actor-Critic for cooperative learning
	- 🔄 Experience Replay: Efficient learning from past experiences
	- 🎮 Custom Environments: Flexible multi-agent simulation environments
	- 📈 Training Metrics: Comprehensive performance tracking

	### 4. Agent Types

	- 👔 Coordinator Agent: Task allocation and system coordination
	- 🔬 Researcher Agent: Information gathering and analysis
	- 📊 Analyst Agent: Pattern recognition and prediction
	- 🤖 ReAct Agent: Reasoning and acting with LLMs
	- 🔮 Reflexion Agent: Self-reflective learning
	- 🦛 HippoRAG Agent: Advanced RAG with knowledge graphs
	- 🔗 Nexus Agent: Multi-modal reasoning

	## 📁 Project Structure

	```
	CA24_multi_agent_systems/
	├── 📓 notebooks/ # Jupyter notebooks with examples
	│ ├── CA24.ipynb # Main comprehensive notebook
	│ ├── 01_Advanced_Communication_Systems.ipynb
	│ ├── 02_Advanced_Coordination.ipynb
	│ ├── 03_Multi_Agent_RL.ipynb
	│ └── 04_Complete_Projects.ipynb
	├── 🔧 src/ # Source code
	│ ├── core/ # Core multi-agent systems
	│ │ ├── multi_agent_system.py
	│ │ ├── advanced_communication.py
	│ │ ├── advanced_coordination.py
	│ │ ├── multi_agent_rl.py
	│ │ ├── environment.py
	│ │ ├── rag_system.py
	│ │ └── shared_memory.py
	│ ├── agents/ # Specialized agents
	│ │ ├── react_agent.py
	│ │ ├── reflexion_agent.py
	│ │ ├── hipporag_agent.py
	│ │ └── nexus_agent.py
	│ ├── evaluation/ # Evaluation metrics
	│ └── utils/ # Utilities
	├── 🚀 scripts/ # Execution scripts
	│ ├── main.py
	│ └── run.sh
	├── 🧪 tests/ # Test files
	├── ⚙️ config/ # Configuration
	├── 📊 data/ # Data and results
	├── 💾 models/ # Saved models
	├── 🔗 integrations/ # External integrations
	├── 🎮 demos/ # Interactive demos
	└── 📋 docs/ # Documentation
	```

	## 🚀 Quick Start

	### Installation

	```bash
	# Clone the repository
	cd CAs/CA24_multi_agent_systems

	# Install dependencies
	pip install -r requirements.txt

	# Set up environment variables (optional)
	export GEMINI_API_KEY='your_api_key_here'
	```

	### Running Examples

	#### 1. Basic Multi-Agent System

	```bash
	cd scripts/
	python main.py
	```

	#### 2. Advanced Communication Demo

	```python
	from core.advanced_communication import AdvancedCommunicationManager
	from core.multi_agent_system import BaseAgent

	# Create communication manager
	comm_manager = AdvancedCommunicationManager("my_system")

	# Register agents
	await comm_manager.register_agent(agent1, ["topic_A"])
	await comm_manager.register_agent(agent2, ["topic_B"])

	# Send messages
	await comm_manager.broadcast_message(
	sender="coordinator",
	content={"message": "Hello all agents!"},
	priority=MessagePriority.HIGH
	)
	```

	#### 3. Coordination with Coalition Formation

	```python
	from core.advanced_coordination import AdvancedCoordinationManager, CoordinationStrategy

	# Create coordination manager
	coord_manager = AdvancedCoordinationManager(
	comm_manager,
	strategy=CoordinationStrategy.COALITION
	)

	# Register agents with capabilities
	await coord_manager.register_agent(agent, capabilities)

	# Allocate tasks
	assigned_agent = await coord_manager.allocate_task(task)
	```

	#### 4. Multi-Agent Reinforcement Learning

	```python
	from core.multi_agent_rl import MultiAgentRLSystem

	# Create MARL system
	marl = MultiAgentRLSystem(
	num_agents=3,
	state_dim=4,
	action_dim=2,
	algorithm="dqn"
	)

	# Train agents
	marl.train(num_episodes=1000, batch_size=32)

	# Evaluate performance
	results = marl.evaluate(num_episodes=10)
	```

	## 📚 Comprehensive Examples

	### Example 1: Distributed Task Allocation

	```python
	import asyncio
	from core.multi_agent_system import MultiAgentSystem
	from core.advanced_coordination import CoordinationTask, AgentCapability

	async def distributed_task_allocation():
	# Initialize system
	mas = MultiAgentSystem(gemini_config)
	await mas.initialize()

	# Create task with requirements
	task = CoordinationTask(
	description="Analyze market data",
	resource_requirements={
	"capabilities": ["analysis", "visualization"],
	"max_cost": 100.0
	},
	deadline=datetime.now() + timedelta(hours=1)
	)

	# Allocate task
	assigned_agent = await coord_manager.allocate_task(task)
	print(f"Task assigned to: {assigned_agent}")

	# Monitor progress
	status = await mas.get_system_status()
	print(f"System status: {status}")

	asyncio.run(distributed_task_allocation())
	```

	### Example 2: Consensus-Based Decision Making

	```python
	async def consensus_decision():
	# Initial values from different agents
	initial_values = {
	"agent_1": 0.8,
	"agent_2": 0.6,
	"agent_3": 0.7,
	"agent_4": 0.9
	}

	# Reach consensus
	consensus_result = await coord_manager.reach_consensus(
	topic="resource_allocation",
	initial_values=initial_values,
	max_iterations=10
	)

	print(f"Consensus reached: {consensus_result}")

	asyncio.run(consensus_decision())
	```

	### Example 3: Multi-Agent RL Training

	```python
	# Create cooperative multi-agent environment
	marl_system = MultiAgentRLSystem(
	num_agents=4,
	state_dim=8,
	action_dim=4,
	algorithm="a2c"
	)

	# Train with custom parameters
	marl_system.train(
	num_episodes=2000,
	batch_size=64,
	update_frequency=10
	)

	# Get training statistics
	stats = marl_system.get_training_stats()
	print(f"Training completed:")
	print(f" Average reward: {stats['average_rewards'][-1]:.2f}")
	print(f" Total steps: {stats['total_steps']}")

	# Evaluate trained agents
	eval_results = marl_system.evaluate(num_episodes=20)
	print(f"Evaluation results: {eval_results}")
	```

	## 🎓 Jupyter Notebooks

	Explore comprehensive tutorials and examples:

	1. CA24.ipynb - Main comprehensive guide
	2. 01_Advanced_Communication_Systems.ipynb - Communication protocols
	3. 02_Advanced_Coordination.ipynb - Coordination mechanisms
	4. 03_Multi_Agent_RL.ipynb - Reinforcement learning
	5. 04_Complete_Projects.ipynb - End-to-end projects

	## 🧪 Testing

	```bash
	# Run all tests
	cd tests/
	python -m pytest

	# Run specific test
	python test_communication.py
	python test_coordination.py
	python test_marl.py
	```

	## 📊 Performance Metrics

	The system tracks various performance metrics:

	- Communication: Message throughput, latency, delivery rate
	- Coordination: Task allocation efficiency, coalition formation time
	- Learning: Episode rewards, convergence rate, success rate
	- System: CPU/memory usage, agent utilization

	## 🔧 Configuration

	### Environment Variables

	```bash
	# API Keys
	export GEMINI_API_KEY='your_gemini_key'
	export OPENAI_API_KEY='your_openai_key'

	# System Configuration
	export MAX_AGENTS=10
	export MESSAGE_TIMEOUT=5.0
	export COORDINATION_STRATEGY='hierarchical'
	```

	### Configuration Files

	Edit `config/system_config.yaml` for advanced settings:

	```yaml
	communication:
	protocol: message_passing
	max_retries: 3
	ttl: 60

	coordination:
	strategy: hierarchical
	allocation_method: capability_matching

	reinforcement_learning:
	algorithm: dqn
	learning_rate: 0.001
	gamma: 0.99
	epsilon: 1.0
	```

	## 📖 Documentation

	Detailed documentation available in `docs/`:

	- Architecture Guide: System design and components
	- API Reference: Complete API documentation
	- Best Practices: Guidelines for multi-agent systems
	- Troubleshooting: Common issues and solutions

	## 🎯 Key Concepts Implemented

	### 1. Cooperative Distributed Problem Solving (CDPS)

	- Decentralized control and data storage
	- Efficient communication protocols
	- Loose coupling between agents

	### 2. Consensus Dynamics

	- Weighted average consensus
	- Majority vote mechanisms
	- Convergence guarantees

	### 3. Distributed Constraint Optimization (DCOP)

	- Constraint satisfaction
	- Resource allocation
	- Optimization algorithms

	### 4. Algorithmic Game Theory

	- Mechanism design
	- Equilibrium analysis
	- Strategic behavior modeling

	### 5. Multi-Agent Reinforcement Learning (MARL)

	- Independent learners
	- Centralized training, decentralized execution
	- Cooperative and competitive scenarios

	---

	## 📄 License

	This project is part of the System2_in_AI CA collection.

	## 🙏 Acknowledgments

	- Research Papers: Based on latest MAS research
	- Frameworks: LangChain, Ray RLlib, PettingZoo
	- Community: Open-source multi-agent systems community

	---

	Built with ❤️ for advanced multi-agent systems research and applications