{"harness": {"recipes": 3, "summary": "Kilo CLI (packages/opencode, a fork of OpenCode; core services in packages/core) is a Bun/TypeScript coding agent. Its entrypoint is packages/opencode/src/index.ts (yargs CLI, script name `kilo`); the non-interactive one-shot mode is `kilo run --format json --auto --model provider/model \"<prompt>\"`, which boots the in-process HTTP server, creates a session rooted at the cwd, streams one JSON event per line, auto-approves permission requests, and exits. Model calls go through the Vercel AI SDK; providers come from JSON config (KILO_CONFIG_CONTENT holds inline config), and a custom provider with npm \"@ai-sdk/openai-compatible\" plus options.baseURL sends OpenAI chat-completions requests (streaming + tool calls) to the proxy, exactly the way the repo's own CLI subprocess tests do (test/lib/test-provider.ts, test/lib/cli-process.ts). The overlay installs Bun 1.3.14 under /opt/harness/bun, copies the repo to /opt/harness/src, runs `bun install --frozen-lockfile`, and writes a /opt/harness/bin/kilo shim (symlinked into /usr/local/bin and added via profile.d so `bash -lc` keeps it on PATH) that launches src/index.ts with the same bun flags as the repo's dev/test tooling, with the @opentui/solid preload resolved to an absolute path at build time and offline-safe KILO_DISABLE_* defaults exported so the CLI never fetches models.dev, plugins, or updates.", "recipe": "kilo-org-kilocode@24167bd703f56b8adc39cf94263cb85736963777", "harness": "kilo-org-kilocode", "first_run": "2026-09-23T14:22:32", "domains": ["swe"], "runs": 8, "base_image": "debian:bookworm-slim", "finished": 8, "last_run": "2026-09-25T23:38:46", "repo": "https://github.com/kilo-org/kilocode", "last_run_id": "20260925T233225-kilo-org-kil-d996c3e6b2fa-polyglot_go_palindrome-products", "use_case": "Writes, edits, and debugs code across multiple files in any language using natural language instructions from VS Code, JetBrains, or the terminal.", "tasksets": ["aider_polyglot", "algotune", "bigcodebench_hard_complete", "bird-bench", "humanevalfix", "swebench-verified"], "results": {"bird-bench/card_games__372": {"best_reward": 1, "last": "2026-09-24T23:21:52", "scored": 1, "last_run": "20260924T231950-sw-kilocode-card_games__372", "last_tests": null, "last_outcome": "scored", "last_reward": 1, "runs": 1}, "algotune/algotune-vertex-cover": {"best_reward": 1, "last": "2026-09-25T22:57:19", "scored": 1, "last_run": "20260925T224236-kilo-org-kil-b0bb044ff780-algotune-vertex-cover", "last_tests": {"summary": "2 failed, 1 passed", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "scored", "last_reward": 1, "runs": 1}, "aider_polyglot/polyglot_go_palindrome-products": {"best_reward": 0, "last": "2026-09-25T23:38:46", "scored": 2, "last_run": "20260925T233225-kilo-org-kil-d996c3e6b2fa-polyglot_go_palindrome-products", "last_tests": null, "last_outcome": "scored", "last_reward": 0, "runs": 2}, "aider_polyglot/polyglot_python_bowling": {"best_reward": 1, "last": "2026-09-23T14:22:32", "scored": 1, "last_run": "20260923T134847-kilo-org-kil-polyglot_python_bowling", "last_tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "scored", "last_reward": 1, "runs": 1}, "bigcodebench_hard_complete/bigcodebench_1057": {"best_reward": 1, "last": "2026-09-25T23:27:30", "scored": 1, "last_run": "20260925T230452-kilo-org-kil-ac8dba9d8b5d-bigcodebench_1057", "last_tests": {"summary": "6 passed", "total": 6, "passed": 6, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "scored", "last_reward": 1, "runs": 1}, "swebench-verified/psf__requests-5414": {"best_reward": 1, "last": "2026-09-24T23:07:38", "scored": 1, "last_run": "20260924T230539-sw-kilocode-psf__requests-5414", "last_tests": {"summary": "131 passed, 1 xfailed, 158 errors", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "scored", "last_reward": 1, "runs": 1}, "humanevalfix/python-12": {"best_reward": 1, "last": "2026-09-24T23:19:21", "scored": 1, "last_run": "20260924T231500-sw-kilocode-python-12", "last_tests": {"summary": "1 passed", "total": 1, "passed": 1, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "scored", "last_reward": 1, "runs": 1}}, "models": ["bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0"], "tasks_tried": 7, "api_style": "openai", "commit": "24167bd703f56b8adc39cf94263cb85736963777", "scored": 8}, "profile": {"evidence": "README.md describes it as \"The open source coding agent for building with AI in VS Code, JetBrains, or the CLI.\" System prompt at packages/opencode/src/session/prompt/default.txt identifies it as \"an interactive CLI tool that helps users with software engineering tasks.\" Tool registry (packages/opencode/src/tool/registry.ts) provides Edit, Write, Read, Shell, Grep, Glob, Git operations, WebSearch, WebFetch, LSP diagnostics, Task delegation, and MCP server integration. Git service (packages/opencode/src/git/index.ts) supports diff, patch, status, branch operations. Supports multiple specialized agents (Code, Plan, Ask, Debug) and 500+ AI models.", "harness": "kilo-org-kilocode", "domains": ["swe"], "source": "claude -p", "cost_usd": 0.38582550000000004, "capabilities": ["edits-files", "runs-shell", "runs-tests", "browses-web", "uses-git", "multi-agent", "long-horizon", "reads-docs", "calls-apis"], "seconds": 87, "languages": ["go", "java", "javascript", "python", "rust"], "at": "2026-09-25T23:05:44", "use_case": "Writes, edits, and debugs code across multiple files in any language using natural language instructions from VS Code, JetBrains, or the terminal.", "not_for": ["Direct SQL database operations (no built-in SQL client)", "GUI automation or computer control beyond browser opening", "Specialized data science workflows"], "commit": "24167bd703f56b8adc39cf94263cb85736963777"}, "recommendations": {"model": "haiku", "at": "2026-09-25T23:40:43", "profile_source": "claude -p", "harness": "kilo-org-kilocode", "recs": [{"language": "rust", "task": "polyglot_rust_bowling", "score": null, "taskset": "aider_polyglot", "why": "Rust is in your language profile but untried; bowling scoring passed with 31 tests in Python, so this tests if you can generalize the same algorithmic solution across languages.", "domain": "swe"}, {"taskset": "aider_polyglot", "score": null, "language": "javascript", "task": "polyglot_javascript_triangle", "domain": "swe", "why": "JavaScript is in your language profile but untried; geometry classification is simpler than the Go palindrome task that failed, so this tests a cleaner baseline in a new language."}, {"task": "quixbugs-python-sieve", "language": "python", "taskset": "quixbugs", "score": null, "why": "New taskset focused on pure bug-fixing with one-line edits; tests your core capability on focused, minimal-blast-radius changes.", "domain": "swe"}, {"task": "python-21", "score": null, "language": null, "domain": "swe", "taskset": "humanevalfix", "why": "You passed python-12 in this taskset; this is a different single-function fix to test consistency and surface any task-specific brittleness."}, {"domain": "swe", "score": null, "task": "sphinx-doc__sphinx-8595", "language": null, "taskset": "swebench-verified", "why": "New swebench task marked as <15 min fix; tests whether you can handle real codebase understanding on a different project than the requests fix that had test errors."}], "source": "llm", "cost_usd": 0.03393625, "based_on_run": "20260925T233225-kilo-org-kil-d996c3e6b2fa-polyglot_go_palindrome-products"}, "runs": [{"run": "20260925T233225-kilo-org-kil-d996c3e6b2fa-polyglot_go_palindrome-products", "started": "2026-09-25T23:38:46", "finished": "2026-09-25T23:39:56", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "aider_polyglot", "name": "polyglot_go_palindrome-products"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": null, "calls": 12, "seconds": 50, "input_tokens": 201797, "output_tokens": 3071, "errors": 0, "last_action": "read: {\"filePath\": \"/app/palindrome_products.go\"}", "outcome": "scored", "verifier_says": "reward 0"}, {"run": "20260925T230452-kilo-org-kil-ac8dba9d8b5d-bigcodebench_1057", "started": "2026-09-25T23:27:30", "finished": "2026-09-25T23:28:25", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "bigcodebench_hard_complete", "name": "bigcodebench_1057"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "6 passed", "total": 6, "passed": 6, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 11, "seconds": 49, "input_tokens": 198996, "output_tokens": 4101, "errors": 0, "last_action": "read: {\"filePath\": \"/workspace/solution.py\"}", "outcome": "scored", "verifier_says": "reward 1 \u00b7 6 passed"}, {"run": "20260925T224236-kilo-org-kil-b0bb044ff780-algotune-vertex-cover", "started": "2026-09-25T22:57:19", "finished": "2026-09-25T23:03:43", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "algotune", "name": "algotune-vertex-cover"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 1, "tests": {"summary": "2 failed, 1 passed", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 55, "seconds": 378, "input_tokens": 2285858, "output_tokens": 33703, "errors": 0, "last_action": "bash: cd /app && python3 << 'EOF' import sys sys.path.insert(0, '/app') from solver import Solver import time def is_valid_cov\u2026", "outcome": "scored", "verifier_says": "reward 1 \u00b7 2 failed, 1 passed \u00b7 verifier exited 1"}, {"run": "20260924T231950-sw-kilocode-card_games__372", "started": "2026-09-24T23:21:52", "finished": "2026-09-24T23:22:13", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "bird-bench", "name": "card_games__372"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": null, "calls": 4, "seconds": 16, "input_tokens": 64869, "output_tokens": 390, "errors": 0, "last_action": "write: {\"filePath\": \"/app/answer.sql\", \"content\": \"SELECT COUNT(*) FROM cards WHERE toughness = '99';\"}", "outcome": "scored", "verifier_says": "reward 1"}, {"run": "20260924T231500-sw-kilocode-python-12", "started": "2026-09-24T23:19:21", "finished": "2026-09-24T23:19:49", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "humanevalfix", "name": "python-12"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "1 passed", "total": 1, "passed": 1, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 7, "seconds": 23, "input_tokens": 110066, "output_tokens": 1078, "errors": 0, "last_action": "read: {\"filePath\": \"/workspace/Python__12.py\"}", "outcome": "scored", "verifier_says": "reward 1 \u00b7 1 passed"}, {"run": "20260924T230539-sw-kilocode-psf__requests-5414", "started": "2026-09-24T23:07:38", "finished": "2026-09-24T23:14:57", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "swebench-verified", "name": "psf__requests-5414"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "131 passed, 1 xfailed, 158 errors", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 91, "seconds": 426, "input_tokens": 4327113, "output_tokens": 19843, "errors": 0, "last_action": "bash: cd /testbed && python -m pytest tests/test_requests.py::TestPreparingURLs::test_preparing_url tests/test_requests.py::Te\u2026", "outcome": "scored", "verifier_says": "reward 1 \u00b7 131 passed, 1 xfailed, 158 errors"}, {"run": "20260924T225619-sw-kilocode-polyglot_go_palindrome-products", "started": "2026-09-24T23:02:49", "finished": "2026-09-24T23:05:36", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "aider_polyglot", "name": "polyglot_go_palindrome-products"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": null, "calls": 18, "seconds": 114, "input_tokens": 316849, "output_tokens": 4788, "errors": 0, "last_action": "read: {\"filePath\": \"/app/palindrome_products.go\"}", "outcome": "scored", "verifier_says": "reward 0"}, {"run": "20260923T134847-kilo-org-kil-polyglot_python_bowling", "started": "2026-09-23T14:22:32", "finished": "2026-09-23T14:27:53", "status": "done", "kind": "harbor", "harness": "kilo-org-kilocode", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 54, "seconds": 317, "input_tokens": 2306099, "output_tokens": 43338, "errors": 0, "last_action": "bash: cd /app && python3 << 'EOF' from bowling import BowlingGame print(\"=== FINAL IMPLEMENTATION CHECK ===\\n\") # Check that m\u2026", "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}]}