Concurrency

Mix enables multiple sessions to process simultaneously and parallel tool execution within messages, dramatically improving performance while maintaining safety and session isolation.

Architecture Overview

The concurrency system operates at two levels: Session-Level Concurrency (multiple sessions running simultaneously) and Tool-Level Concurrency (parallel tool execution within messages).

Key Features

• No Session Blocking: Multiple concurrent requests per session
• Parallel Tool Execution: All tool calls in a message execute simultaneously
• Maintained Safety: Session isolation and proper cancellation preserved

Implementation Details

1. Session-Level Concurrency

Implementation:

// Session cancellation tracking
activeContexts sync.Map  // Maps session ID to context.CancelFunc

Result: Multiple requests per session run simultaneously without blocking.

2. Tool-Level Concurrency

Implementation:

// Execute all tools concurrently
toolResults, err := a.executeToolsWithDependencies(ctx, sessionID, toolCalls, assistantMsg)

3. Tool Dependencies

Read-only and stateless tools execute in parallel. File operations automatically detect dependencies.

Performance

Parallel tool execution provides 2-5x performance improvements. Multiple file reads, searches, and independent operations run simultaneously while maintaining proper file operation dependencies.

Safety Mechanisms

• Session Isolation: Isolated storage, contexts, and providers per session
• Graceful Cancellation: Per-session context cancellation via activeContexts
• Error Handling: Individual tool failures don't block other operations

Benefits

• No session busy errors, 2-5x faster multi-tool operations
• Better resource utilization and reduced contention

Core Components

• agent.go: Session tracking with activeContexts and executeToolsWithDependencies()
• concurrency_tools.go: Parallel tool execution with dependency analysis
• HTTP handlers: Concurrent request support across sessions

Key Files

• [mix_agent/internal/llm/agent/agent.go](Core agent service implementing session-level concurrency)
• [mix_agent/internal/llm/agent/concurrency_tools.go](Parallel tool execution engine)
• [mix_agent/internal/http/sse.go](Concurrent SSE streaming support)