Concurrency
How Mix achieves true parallel processing across sessions and tool calls while maintaining safety and isolation.
Mix enables multiple sessions to process simultaneously and parallel tool execution within messages, dramatically improving performance while maintaining safety and session isolation.
Architecture Overview
The concurrency system operates at two levels: Session-Level Concurrency (multiple sessions running simultaneously) and Tool-Level Concurrency (parallel tool execution within messages).
Key Features
- No Session Blocking: Multiple concurrent requests per session
- Parallel Tool Execution: All tool calls in a message execute simultaneously
- Maintained Safety: Session isolation and proper cancellation preserved
Implementation Details
1. Session-Level Concurrency
The agent maintains session cancellation tracking through a thread-safe map that associates each session ID with its context cancellation function. This enables multiple requests per session to run simultaneously without blocking.
2. Tool-Level Concurrency
The agent executes all tools in a message concurrently using a dependency-aware execution engine. This approach analyzes tool relationships and runs independent operations in parallel while respecting dependencies.
3. Tool Dependencies
Read-only and stateless tools execute in parallel. File operations automatically detect dependencies.
Performance
Parallel tool execution provides 2-5x performance improvements. Multiple file reads, searches, and independent operations run simultaneously while maintaining proper file operation dependencies.
Safety Mechanisms
- Session Isolation: Isolated storage, contexts, and providers per session
- Graceful Cancellation: Per-session context cancellation via
activeContexts - Error Handling: Individual tool failures don't block other operations
Benefits
- No session busy errors, 2-5x faster multi-tool operations
- Better resource utilization and reduced contention
Core Components
- agent.go: Session tracking with
activeContextsandexecuteToolsWithDependencies() - concurrency_tools.go: Parallel tool execution with dependency analysis
- HTTP handlers: Concurrent request support across sessions
Key Files
mix_agent/internal/llm/agent/agent.go: Core agent service implementing session-level concurrencymix_agent/internal/llm/agent/concurrency_tools.go: Parallel tool execution enginemix_agent/internal/http/sse.go: Concurrent SSE streaming support