A cache-aware harness for building adaptive LLM systems around evolving context.

Quickstart

1. Install

   pip install .

   Or with tracing support:

   pip install ".[tracing]"

2. Configure Create a .env file in your project root with your model provider keys:

   GEMINI_API_KEY=your_key_here
   OPENAI_API_KEY=your_key_here
   # MLFLOW_TRACKING_URI=http://localhost:5000  # Optional, for tracing

3. Run

   context-harness --model gemini/gemini-3-flash-preview

Architecture Overview

context-harness is built around an append-only event log that tracks every interaction, model call, and tool result.

• EventLog: An in-memory store of Event objects (USER_MESSAGE, TOOL_CALL, USAGE, etc.).
• Assembler & ContextPack: The Assembler transforms the EventLog and SessionState into a ContextPack—a model-ready bundle of messages and tool definitions.
• ModelAdapter: A protocol for LLM interaction. The default LiteLLMAdapter provides multi-provider support with usage and cost tracking.
• Tool: A simple protocol (name, description, parameters, execute) for extending agent capabilities.
• SessionState: Tracks transient session metadata including session_id, turn_count, and the current system_prompt.
• TraceLogger: Integrates with MLflow to provide session-grouped tracing of model calls and tool executions.

Data Flow

User Input ──▶ ChatLoop ───────────────────┐
                  │                        │
                  ▼                        ▼
              EventLog ◀─── Assembler ──▶ SessionState
                  │            │           (System Prompt)
                  ▼            ▼
              ModelAdapter ◀── ContextPack
                  │
                  ▼
              LLM Provider ──▶ Tool Execution ──▶ EventLog

Built-in Tools

• bash: Execute shell commands and capture stdout/stderr.
• web_search: Search the web via DuckDuckGo and return URLs with descriptions.
• web_fetch: Fetch a URL and return its readable text content (automatically cleans HTML).

CLI Commands

Observability

• MLflow Tracing: When the [tracing] extra is installed, context-harness automatically logs traces to MLflow. Traces are tagged with mlflow.trace.session to group all turns and tool calls within a single session.
• Usage Tracking: Every model response captures token counts, provider-specific costs, and latency. Usage is logged as events, allowing for detailed audit trails.

Configuration

• Environment: Managed via .env (API keys, MLFLOW_TRACKING_URI).
• Customization:
  • ChatLoop(max_iterations=...): Control the maximum number of tool-calling loops per turn.
  • LiteLLMAdapter(max_context_tokens=...): Set context limits for the Assembler.
  • tools=[...]: Pass any object implementing the Tool protocol to the ChatLoop.