MIRIX: Revolutionizing Personal AI with Advanced Multi-Agent Memory System

MIRIX is a groundbreaking multi-agent personal assistant that redefines the capabilities of AI through its advanced memory system. Unlike traditional AI assistants, MIRIX builds and refines its memory dynamically by observing screen activities and engaging in natural conversations. This innovative approach enables it to retain and recall information with remarkable accuracy, making it an indispensable tool for both personal and professional use.

Key Features of MIRIX

1. Multi-Agent Memory System: At the heart of MIRIX lies a sophisticated memory architecture comprising six specialized components:

• Core Memory: Stores fundamental information about the user and the assistant's persona.
• Episodic Memory: Captures specific events or experiences, allowing for detailed recall of past interactions.
• Semantic Memory: Organizes knowledge in a structured manner, enabling logical reasoning and inference.
• Procedural Memory: Retains procedural knowledge, such as routines or step-by-step tasks, making it easier to automate repetitive processes.
• Resource Memory: Manages resources like files, links, or other digital assets, ensuring they are readily accessible when needed.
• Knowledge Memory: Stores vast amounts of factual information, which can be retrieved and utilized for complex queries.

These memory components are managed by dedicated agents, each with specific roles and responsibilities. This distributed approach ensures that MIRIX can handle a wide range of tasks efficiently while maintaining coherence in its responses.

2. Screen Activity Tracking: One of the standout features of MIRIX is its ability to track screen activities continuously. By capturing visual data from the user's screen, MIRIX consolidates this information into structured memories. This feature allows the assistant to contextually understand and respond to user actions, making it a powerful tool for productivity and learning.

3. Privacy-First Design: In an era where data privacy is paramount, MIRIX prioritizes user control over their data. All long-term data is stored locally on the user's device, with robust privacy settings that allow users to manage what information is shared or retained. This ensures that sensitive data remains secure and protected.

4. Advanced Search Capabilities: MIRIX leverages PostgreSQL-native BM25 full-text search combined with vector similarity support. This dual approach enables users to retrieve relevant information quickly, whether it's a specific piece of data or a complex query requiring semantic understanding.

5. Multi-Modal Input Processing: MIRIX seamlessly processes multiple input types, including text, images, voice, and screen captures. This versatility allows users to interact with the assistant in the most natural way possible, whether through typing, speaking, or sharing visual content.

Getting Started with MIRIX

MIRIX offers flexibility in deployment, catering to both cloud and local setups, ensuring that users can choose the option that best fits their needs.

Option A: Cloud (Hosted API)

For users who prefer a cloud-based solution, MIRIX provides a hosted API. This option is ideal for those who want to leverage the power of MIRIX without the hassle of local setup. Users can install the MIRIX package via pip and obtain an API key to start interacting with the system.

pip install mirix==0.1.6

Users can then integrate MIRIX into their applications using a simple Python script:

from mirix import MirixClient
client = MirixClient(api_key="your_api_key_here")  # or set MIRIX_API_KEY in your environment
client.initialize_meta_agent(provider="openai")  # See configs in mirix/configs/examples/mirix_openai.yaml
# Simple add example
client.add(
    user_id="demo-user", 
    messages=[
        {"role": "user", "content": [{"type": "text", "text": "The moon now has a president."}]},
        {"role": "assistant", "content": [{"type": "text", "text": "Noted."}]},
    ],
)

For more detailed examples, users can refer to the README.md or the samples/run_client.py file provided in the repository.

Option B: Local (Backend + Dashboard, No Docker)

For users who prefer a local setup without Docker, MIRIX offers a backend and dashboard solution. This option provides full control over the data while eliminating the need for cloud dependency.

Step 1: Backend & Dashboard Setup Users can install the necessary dependencies using pip and npm:

pip install -r requirements.txt

In terminal 1, users can start the server with:

python scripts/start_server.py

In terminal 2, they can set up the dashboard:

cd dashboard
npm install
npm run dev

The dashboard will be accessible at http://localhost:5173, while the API will be available at http://localhost:8531. Users can create an API key in the dashboard and set it as an environmental variable (MIRIX_API_KEY).

Step 2: Local Development Considerations It is crucial to note that for local development, users should not use the PyPI package. Instead, they must run their code from the root folder of the project to ensure proper importing of the local mirix module. If the PyPI package was previously installed, it should be uninstalled first:

pip uninstall mirix

Option C: Local (Backend Only, No Dashboard)

For users who prefer a minimal setup without a dashboard, MIRIX offers a backend-only option. This is ideal for developers who want to integrate MIRIX into their applications without additional overhead.

Step 1: Backend Setup Users can install the dependencies and start the server with:

pip install -r requirements.txt
python scripts/start_server.py --allow-terminal-user-creation

The API will be available at http://localhost:8531. Users can then create or load a client from the API using their preferred client name or ID.

Step 2: Example Code Here’s an example of how to initialize MIRIX with a local setup:

from mirix import MirixClient
client = MirixClient(
    api_key="your-api-key",  # if you set MIRIX_API_KEY as the environmental variable, this is optional
    base_url="http://localhost:8531",
)
# If no API key is required (Option C):
from mirix import MirixTerminalClient
client = MirixClient(
    client_id="demo-client-id", 
    client_name="demo-client-name", 
    base_url="http://localhost:8531",
)
client.initialize_meta_agent(
    config={
        "llm_config": {
            "model": "gpt-4o-mini",
            "model_endpoint_type": "openai",
            "model_endpoint": "https://api.openai.com/v1",
            "context_window": 128000,
        },
        "build_embeddings_for_memory": True,
        "embedding_config": {
            "embedding_model": "text-embedding-3-small",
            "embedding_endpoint": "https://api.openai.com/v1",
            "embedding_endpoint_type": "openai",
            "embedding_dim": 1536,
        },
        "meta_agent_config": {
            "agents": [
                "core_memory_agent", 
                "resource_memory_agent",
                "semantic_memory_agent",
                "episodic_memory_agent",
                "procedural_memory_agent",
                "knowledge_memory_agent",
                "reflexion_agent",
                "background_agent",
            ],
            "memory": {
                "core": [{"label": "human", "value": ""}, {"label": "persona", "value": "I am a helpful assistant."}],
                "decay": {
                    "fade_after_days": 30,
                    "expire_after_days": 90,
                },
            },
        },
    }
)
client.add(
    user_id="demo-user", 
    messages=[
        {"role": "user", "content": [{"type": "text", "text": "The moon now has a president."}]},
        {"role": "assistant", "content": [{"type": "text", "text": "Noted."}]},
    ],
)
memories = client.retrieve_with_conversation(
    user_id="demo-user", 
    messages=[
        {"role": "user", "content": [{"type": "text", "text": "What did we discuss on MirixDB in the last 4 days?"}]},
    ],
    limit=5,
)
print(memories)

For further API examples, users can explore samples/run_client.py.

Updates and Branches

MIRIX has undergone significant updates with version 0.1.6, which introduces a brand-new release line focused solely on the memory system. This version is designed to be plugged into any existing agents, making it highly versatile and adaptable. The desktop personal assistant (frontend + backend) has been deprecated in favor of this modular approach.

For users who still require the earlier desktop application with a built-in agent, they can switch to the desktop-agent branch. This ensures that both new and existing users have access to the features they need while benefiting from ongoing improvements.

License and Community

MIRIX is released under the Apache License 2.0, which grants users the freedom to use, modify, and distribute the software while adhering to the terms of the license. For more details, users can refer to the LICENSE file in the repository.

The MIRIX community is vibrant and welcoming, with multiple channels for engagement:

1. Discord Community: Users can join the Discord server (https://discord.gg/FXtXJuRf) for real-time discussions, support, and updates. This platform fosters collaboration among users and developers alike.

2. Weekly Discussion Sessions: Hosted every Friday night from 8-9 PM PST, these sessions provide a space for users to discuss issues, share ideas, and connect with the development team. The Zoom link for these sessions is https://ucsd.zoom.us/j/96278791276.

3. WeChat Group: Users can add the WeChat account ari_asm to join the group chat, offering another avenue for community interaction and support.

Acknowledgments

MIRIX acknowledges the contributions of Letta (https://github.com/letta-ai/letta), an open-source framework that served as the foundation for its memory system. This collaboration has been instrumental in shaping MIRIX into the powerful tool it is today.

Conclusion

MIRIX represents a significant leap forward in personal AI technology, combining advanced memory systems with multi-modal input processing and privacy-first design. Whether deployed in the cloud or locally, MIRIX offers unparalleled flexibility and power, making it an essential tool for anyone looking to enhance their productivity or explore the potential of AI. With its active community and ongoing development, MIRIX is poised to redefine the landscape of personal assistants, setting a new standard for what AI can achieve.