Project Guide | AI Agent Production Lab

Live Demo

Open the public GitHub Pages demo
Scope: credential-free, synthetic-data demo for product teams and technical evaluators.

Self-contained lab for agent runtime reliability, evaluation, tracing, and cost accounting. The lab uses a deterministic planner so the full workflow can be tested without external APIs or credentials.

System Overview

A production-readiness lab for agents that makes planning, tracing, cost, and evaluation visible before real rollout.

Area	Details
Users	AI platform teams, backend teams, and product teams moving agents beyond demos.
Technical path	Validate the demo, README, architecture notes, and quality gate before deeper workflow review.
System scope	Deterministic planning fixtures, traces, cost accounting, eval assertions, and HTML reports.
Operating boundary	Lab fixtures are controlled; production rollouts need workflow-specific evals, rate limits, and approval paths.
Evaluation path	Run the lab scripts and inspect generated HTML reports, traces, and eval assertions.

Evaluation Path

Start here: Read the trace/cost/eval loop before looking at individual tools.
Local demo: Run python3 scripts/run_demo.py and open the generated HTML report under artifacts/.
Checks: Run python3 -m unittest discover -s tests and python3 -m agent_lab.evals examples/tasks.json.

Architecture Notes

Architecture guide summarizes the system scope, first files to inspect, runtime commands, and known boundaries.
Quality notes lists the local checks, CI surface, and release expectations for this repository.
Enterprise readiness notes outlines security, data, operations, integration, and handoff expectations.

What It Demonstrates

bounded tool execution
deterministic traces
eval cases with pass/fail assertions
estimated token and cost accounting
HTML and JSON report generation
CI-friendly tests with no network dependency

Architecture

flowchart LR
    Task["Task fixture"] --> Planner["Deterministic planner"]
    Planner --> Runtime["Agent runtime"]
    Runtime --> Tools["Allowed tools"]
    Tools --> Trace["Trace events"]
    Trace --> Eval["Eval runner"]
    Eval --> Report["JSON and HTML reports"]

Quick Start

python3 -m unittest discover -s tests
python3 scripts/run_demo.py

The demo writes:

artifacts/agent-report.json
artifacts/agent-report.html

Design Boundary

This repository does not try to simulate a full model provider. It isolates the production concerns around an agent loop: tool allowlisting, bounded execution, trace shape, evaluation scoring, and reproducible reports.

Consolidated Patterns

Provider-neutral agent patterns folds scattered cookbook notes into a single runtime, eval, tracing, retrieval, and fallback checklist.

Project Layout

agent_lab/
  runtime.py   # runtime, tools, planner, trace events
  evals.py     # eval case loading and scoring
  report.py    # JSON and HTML report writer
examples/
  tasks.json   # deterministic eval cases
docs/
  provider-neutral-agent-patterns.md
scripts/
  run_demo.py  # local report generator
tests/
  test_runtime.py

Verification

python3 -m unittest discover -s tests
python3 -m agent_lab.evals examples/tasks.json

All fixtures are synthetic.

Cloud + AI Architecture

Cloud + AI architecture blueprint
Machine-readable architecture manifest
Validation command: python3 scripts/validate_architecture_blueprint.py

Enterprise Productization

Product operating model defines the product scope, trust boundary, operating checks, and service path for this repository.

System Architecture

System architecture maps the runtime boundary, data/control flow, cloud or local deployment surface, and operating assumptions for this repository.

Service Architecture

Service architecture defines the cloud resources, account information, cost controls, and production guardrails needed to turn this repo into a scoped service without publishing public financial assumptions.

Search And Service Surface

Public entry: free production checklist and sample report
Paid boundary: Agent Reliability Audit fixed-scope private audit from USD 1,500
Canonical URL: https://ai-agent-production-lab.pages.dev/
Lead capture: https://kim3310-doeon-kim-portfolio.pages.dev/?offer=ai-agent-production-lab&inquiry=agent-reliability-audit#private-inquiry
Resource route: https://kim3310-doeon-kim-portfolio.pages.dev/resources/ai-agent-production-lab/
Commercial route: https://kim3310-doeon-kim-portfolio.pages.dev/?offer=ai-agent-production-lab#service-offers
Machine-readable offer: docs/service-offer.json
Search growth implementation: docs/search-growth-implementation.md
Revenue architecture: docs/revenue-architecture.md