Open Source • Apache 2.0 Licensed • Community Driven
Test Multi-Agent Systems
Before Production
The first pytest-based framework for testing multi-agent AI systems. Simulate conversations, verify behaviors, and ensure reliability.
Quick Install
pip install maia-test-framework
Why Choose Maia?
Built for modern AI development workflows with enterprise-grade testing capabilities
👥
Multi-Agent Simulation
Simulate complex conversations and interactions between multiple AI agents with ease.
⚡
Extensible Providers
Integrate seamlessly with LiteLLM, LangChain, CrewAI, and other AI model providers.
🛡️
Built-in Assertions
Comprehensive testing suite with behavior verification and participation checks.
💻
Pytest Integration
Leverage familiar pytest patterns for writing and organizing your agent tests.
Simple. Powerful. Intuitive.
Write tests that feel natural with our pytest-based approach. No complex setup, just clean, readable test code.
- ✓Multi-agent conversation simulation
- ✓Behavior assertions and validation
- ✓Provider-agnostic integration
- ✓Real-time dashboard monitoring
test_conversation.py
async def test_conversation_direct_message(self): self.create_agent( name="Alice", provider=GenericLiteLLMProvider(config={ "model": "ollama/mistral", "api_base": "http://localhost:11434" }), system_message="You are a weather assistant." ) self.create_agent( name="Bob", provider=OllamaProvider(config={ "model": "mistral" }), system_message="You are an assistant who suggests clothing." ) session = self.create_session( ["Alice", "Bob"], assertions=[assert_professional_tone], ) await session.user_says("Please describe the usual weather in London in July, including temperature and conditions.") weather_response = await session.agent_responds("Alice") assert "warm" in weather_response.content await session.agent_says("Alice", "Bob", f"Given the weather: {weather_response.content}, what clothes should I wear?") await session.agent_responds("Bob") assert_agent_participated(session, "Bob")
Built-in Dashboard
Visualize test results, monitor agent interactions, and debug with our powerful dashboard
Customer Service Integration Test
test_20240802_143022|Duration: 2.30s
Passed
JUDGE: 8.5/10
6
Total Messages
0
Tool Calls
92.3%
Success Rate
245ms
Avg Response
8.5/10
Judge Score
Participating Agents
CU
CustomerServiceAgent
GenericLiteLLMProvider
3 msgs
IN
InventoryAgent
GenericLiteLLMProvider
1 msg
PR
PricingAgent
CrewAIProvider
1 msg
User Message
2:30:22 PM
user→CustomerServiceAgent
Hi, I'm looking for the iPhone 15 Pro. Is it in stock and what's the current price?
Agent Message
2:30:22 PM
CustomerServiceAgent→InventoryAgent
Check if iPhone 15 Pro is in the stock.
Agent Message
2:30:22 PM
InventoryAgent→CustomerServiceAgent
Inventory Check Result: iPhone 15 Pro - 128GB (12 units), 256GB (8 units), 512GB (3 units)
Agent Message
2:30:23 PM
CustomerServiceAgent→PricingAgent
Check the price for iPhone 15 Pro - 128GB (12 units), 256GB (8 units), 512GB (3 units)
Agent Message
2:30:23 PM
PricingAgent→CustomerServiceAgent
Current Prices: 128GB - $999, 256GB - $1099, 512GB - $1299
Agent Message
2:30:23 PM
CustomerServiceAgent→user
Great news! The iPhone 15 Pro is in stock in all storage variants: 128GB - $999 (12 units), 256GB - $1099 (8 units), 512GB - $1299 (3 units). All prices are current as of today. Would you like me to help you with placing an order or do you have any other questions?