Basic tests

This guide covers writing basic tests with Maia.

Scenario 1: Two agents talk to each other without a user trigger.

class TestConversationSessions(MaiaTest):
    def setup_agents(self):
        self.create_agent(
            name="Alice",
            provider=GenericLiteLLMProvider(config={
                "model": "ollama/mistral",
                "api_base": "http://localhost:11434"
            }),
            system_message="You are a weather assistant. Only describe the weather.",
        )

        self.create_agent(
            name="Bob",
            provider=GenericLiteLLMProvider(config={
                "model": "ollama/mistral",
                "api_base": "http://localhost:11434"
            }),
            system_message="You are an assistant who only suggests clothing.",
        )

@pytest.mark.asyncio
  async def test_agent_to_agent_conversation(self):
      session = self.create_session(["Alice", "Bob"])
      
      # Alice initiates conversation with Bob
      await session.agent_says("Alice", "Bob", "Given the weather: rainy and 20 degrees Celsius, what clothes should I wear?")
      response = await session.agent_responds("Bob")
      assert_agent_participated(session, "Bob")
      
      # Bob responds back to Alice
      await session.agent_says("Bob", "Alice", f"Based on my info: {response.content}")
      response = await session.agent_responds("Alice")
      assert_agent_participated(session, "Alice")

Explanation

This is a basic scenario where one agent talks to another. We are asserting participation to be sure that the agents are responding. We are using GenericLiteLLMProvider as a provider.

Scenario 2: Check the content of the message.

class TestContentAssertions(MaiaTest):
    def setup_agents(self):
        self.create_agent(
            name="Alice",
            provider=GenericLiteLLMProvider(config={
                "model": "ollama/mistral",
                "api_base": "http://localhost:11434"
            }),
            system_message="You are a helpful AI assistant. You will follow user instructions precisely."
        )

    def setup_session(self):
        self.create_session(
            session_id="common_session"
        )
    @pytest.mark.asyncio
    async def test_pattern_contains_assertion(self):
        session = self.extend_session(
            "common_session",
            agent_names=["Alice"],
            assertions=[partial(assert_contains_pattern, pattern="sunny")]
        )
        await session.user_says("What is the weather in a typically sunny place?")
        await session.agent_responds("Alice")

Explanation

We set up the agent and session in the setup phase; then, in the test, we extend the session by providing the agent's name. We are also adding an assertion that checks that the word sunny will appear in every message, including the agent's response.

Scenario 3: Trigger a multi-turn conversation between agents.


    @pytest.mark.asyncio
    async def test_multi_turn_agent_conversation(self):
        self.create_agent(
            name="Alice",
            provider=self.get_provider("ollama"),
            system_message="You are a weather assistant."
        )
        self.create_agent(
            name="Bob",
            provider=self.get_provider("ollama"),
            system_message="You are a helpful assistant who is also a pirate."
        )
    
        session = self.create_session(["Alice", "Bob"])
        
        conversation_log = await session.run_agent_conversation(
            initiator="Bob",
            responder="Alice", 
            initial_message="What's the weather?",
            max_turns=3
        )
        
        assert len(conversation_log) == 7  # 3 turns each + initial message

We can also trigger a multi-turn conversation between agents by executing run_agent_conversation on the session. It will trigger an automatic conversation limited by max_turns. Such a test can be useful to see how agents behave together without user interaction.

In the tests above, we are using a config file to get the provider. This concept is explained in Config File

Validator

Dashboard