{"harness": {"recipes": 1, "summary": "Aider is a terminal AI pair-programming harness written in Python (package `aider-chat`, console entrypoint `aider = aider.main:main`). It edits files in the current working directory (using git for its repo map and change tracking) and makes every LLM call through litellm: `aider/llm.py` lazy-loads litellm and `Model.send_completion` in `aider/models.py` calls `litellm.completion` with the model name, streaming flag and optional tool spec. Using an `openai/<name>` model makes litellm's OpenAI chat-completions client honour `OPENAI_API_BASE`/`OPENAI_BASE_URL`, so all requests (main, weak and editor models) go to `$PROXY_URL/v1/chat/completions`. A single non-interactive task is run with `--message \"$TASK\"` plus `--yes-always`, which answers every confirmation, applies the edits and exits after one turn; chat and full LLM request/response history files are written under /out.", "recipe": "aider-ai-aider@5dc9490bb35f9729ef2c95d00a19ccd30c26339c", "tasks_passed": 3, "harness": "aider-ai-aider", "first_run": "2026-09-23T11:32:24", "domains": ["swe"], "runs": 19, "base_image": "python:3.12-slim-bookworm", "finished": 19, "last_run": "2026-09-25T16:09:45", "repo": "https://github.com/Aider-AI/aider", "passes": 8, "last_run_id": "20260925T090301-aider-ai-aid-psf__requests-5414", "use_case": "A terminal pair-programming assistant: you add files from your git repo to the chat, describe a change or bug fix, and the LLM edits the code with SEARCH/REPLACE diffs, auto-commits the change, and can re-run your linter/tests to fix failures.", "tasksets": ["aider_polyglot", "bird-bench", "humanevalfix", "swebench-verified"], "results": {"bird-bench/card_games__372": {"passes": 1, "last": "2026-09-24T22:52:19", "last_run": "20260924T225115-sw-aider-card_games__372", "last_tests": null, "last_outcome": "pass", "last_reward": 1, "runs": 1}, "aider_polyglot/polyglot_python_bowling": {"passes": 6, "last": "2026-09-24T18:46:14", "last_run": "20260924T184611-aider-ai-aid-polyglot_python_bowling", "last_tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "pass", "last_reward": 1, "runs": 12}, "aider_polyglot/polyglot_go_palindrome-products": {"passes": 0, "last": "2026-09-25T16:06:29", "last_run": "20260925T090255-aider-ai-aid-polyglot_go_palindrome-products", "last_tests": null, "last_outcome": "fail", "last_reward": 0, "runs": 2}, "swebench-verified/psf__requests-5414": {"passes": 0, "last": "2026-09-25T16:09:45", "last_run": "20260925T090301-aider-ai-aid-psf__requests-5414", "last_tests": {"summary": "1 failed, 130 passed, 1 xfailed, 158 errors", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "fail", "last_reward": 0, "runs": 3}, "humanevalfix/python-12": {"passes": 1, "last": "2026-09-24T22:50:50", "last_run": "20260924T224953-sw-aider-python-12", "last_tests": {"summary": "1 passed", "total": 1, "passed": 1, "failed": 0, "agent_written": 0, "failed_names": []}, "last_outcome": "pass", "last_reward": 1, "runs": 1}}, "models": ["bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0"], "tasks_tried": 5, "api_style": "openai", "commit": "5dc9490bb35f9729ef2c95d00a19ccd30c26339c"}, "profile": {"evidence": "README.md calls it \"AI Pair Programming in Your Terminal\" and lists repo map, git auto-commit, lint and test, web pages/images, and 100+ languages. aider/coders/editblock_prompts.py:8-29 sets the SEARCH/REPLACE edit protocol and says to ask the user to add files. aider/commands.py has cmd_web (219), cmd_commit (337), cmd_lint (356), cmd_undo (553), cmd_git (967), cmd_test (993) and cmd_run (1013).", "harness": "aider-ai-aider", "domains": ["swe"], "source": "claude -p", "cost_usd": 0.0870634, "capabilities": ["edits-files", "runs-shell", "runs-tests", "uses-git", "browses-web", "reads-docs"], "seconds": 14, "languages": ["cpp", "css", "go", "html", "java", "javascript", "php", "python", "ruby", "rust"], "at": "2026-09-24T23:30:49", "use_case": "A terminal pair-programming assistant: you add files from your git repo to the chat, describe a change or bug fix, and the LLM edits the code with SEARCH/REPLACE diffs, auto-commits the change, and can re-run your linter/tests to fix failures.", "not_for": ["The model does not call tools on its own. Edits come back as text blocks, and shell commands are only suggested; the user confirms before they run", "Not a long-horizon agent that plans and explores by itself: the user has to add the relevant files to the chat", "Not built for data/SQL, ops/infra, or non-coding work"], "commit": "5dc9490bb35f9729ef2c95d00a19ccd30c26339c"}, "recommendations": {"model": "haiku", "at": "2026-09-25T16:11:47", "profile_source": "claude -p", "harness": "aider-ai-aider", "recs": [{"language": "cpp", "domain": "swe", "taskset": "aider_polyglot", "task": "polyglot_cpp_allergies", "why": "Establish C++ support as a covered compiled language in your core polyglot test suite.", "score": null}, {"score": null, "language": "javascript", "taskset": "aider_polyglot", "task": "polyglot_javascript_triangle", "why": "Extend to JavaScript by solving a similar geometric problem as your passing bowling task to verify dynamic-language support.", "domain": "swe"}, {"domain": "swe", "why": "Confirm Rust support by tackling the same bowling-scoring logic in a systems language, testing cross-language consistency.", "taskset": "aider_polyglot", "score": null, "language": "rust", "task": "polyglot_rust_bowling"}, {"why": "Test single-line bug-fixing in a new taskset to validate that focused editing is a core pair-programming capability.", "taskset": "quixbugs", "task": "quixbugs-python-sieve", "language": "python", "domain": "swe", "score": null}, {"why": "Assess function-fixing consistency on python-11 now that python-12 passed, confirming this pattern is replicable beyond single trials.", "domain": "swe", "task": "python-11", "taskset": "humanevalfix", "score": null, "language": null}], "source": "llm", "cost_usd": 0.054720500000000005, "based_on_run": "20260925T090301-aider-ai-aid-psf__requests-5414"}, "runs": [{"run": "20260925T090301-aider-ai-aid-psf__requests-5414", "started": "2026-09-25T16:09:45", "finished": "2026-09-25T16:10:26", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "swebench-verified", "name": "psf__requests-5414"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 1, "tests": {"summary": "1 failed, 130 passed, 1 xfailed, 158 errors", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 3, "seconds": 23, "input_tokens": 67486, "output_tokens": 1477, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 1 failed, 130 passed, 1 xfailed, 158 errors \u00b7 verifier exited 1"}, {"run": "20260925T090255-aider-ai-aid-polyglot_go_palindrome-products", "started": "2026-09-25T16:06:29", "finished": "2026-09-25T16:07:02", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_go_palindrome-products"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": null, "calls": 1, "seconds": 12, "input_tokens": 2629, "output_tokens": 904, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0"}, {"run": "20260924T225830-sw-aider-psf__requests-5414", "started": "2026-09-24T22:58:34", "finished": "2026-09-24T22:59:15", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "swebench-verified", "name": "psf__requests-5414"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 1, "tests": {"summary": "8 failed, 123 passed, 1 xfailed, 158 errors", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 2, "seconds": 23, "input_tokens": 39125, "output_tokens": 1604, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 8 failed, 123 passed, 1 xfailed, 158 errors \u00b7 verifier exited 1"}, {"run": "20260924T225115-sw-aider-card_games__372", "started": "2026-09-24T22:52:19", "finished": "2026-09-24T22:52:49", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "bird-bench", "name": "card_games__372"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": null, "calls": 1, "seconds": 21, "input_tokens": 3470, "output_tokens": 241, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1"}, {"run": "20260924T224953-sw-aider-python-12", "started": "2026-09-24T22:50:50", "finished": "2026-09-24T22:51:13", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "humanevalfix", "name": "python-12"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "1 passed", "total": 1, "passed": 1, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 16, "input_tokens": 2421, "output_tokens": 1574, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 1 passed"}, {"run": "20260924T224501-sw-aider-psf__requests-5414", "started": "2026-09-24T22:46:09", "finished": "2026-09-24T22:49:51", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "swebench-verified", "name": "psf__requests-5414"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 1, "tests": {"summary": "8 failed, 123 passed, 1 xfailed, 158 errors", "total": 0, "passed": 0, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 3, "seconds": 30, "input_tokens": 67493, "output_tokens": 1426, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 8 failed, 123 passed, 1 xfailed, 158 errors \u00b7 verifier exited 1"}, {"run": "20260924T224300-sw-aider-polyglot_go_palindrome-products", "started": "2026-09-24T22:43:51", "finished": "2026-09-24T22:45:00", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_go_palindrome-products"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": null, "calls": 1, "seconds": 12, "input_tokens": 2628, "output_tokens": 936, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0"}, {"run": "20260924T184611-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T18:46:14", "finished": "2026-09-24T18:46:38", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 19, "input_tokens": 3064, "output_tokens": 2532, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}, {"run": "20260924T183432-polyglot_python_bowling", "started": "2026-09-24T18:34:34", "finished": "2026-09-24T18:35:00", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 21, "input_tokens": 3064, "output_tokens": 2529, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}, {"run": "20260924T180336-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T18:03:40", "finished": "2026-09-24T18:04:04", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 19, "input_tokens": 3064, "output_tokens": 2529, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}, {"run": "refactorcheck-polyglot_python_bowling", "started": "2026-09-24T17:37:48", "finished": "2026-09-24T17:38:14", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 21, "input_tokens": 3064, "output_tokens": 2529, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}, {"run": "20260924T170312-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T17:03:15", "finished": "2026-09-24T17:03:39", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 20, "input_tokens": 3064, "output_tokens": 2532, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}, {"run": "20260924T162919-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T16:29:22", "finished": "2026-09-24T16:29:45", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": {"summary": "25 failed, 6 passed", "total": 31, "passed": 6, "failed": 25, "agent_written": 0, "failed_names": ["test_a_spare_followed_by_zeros_is_worth_ten_points", "test_a_spare_in_the_last_frame_gets_a_one_roll_bonus_that_is_counted_once", "test_a_strike_earns_ten_points_in_a_frame_with_a_single_roll", "test_a_strike_in_the_last_frame_gets_a_two_roll_bonus_that_is_counted_once", "test_a_strike_with_the_one_roll_bonus_after_a_spare_in_the_last_frame_does_not_get_a_bonus", "test_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_bonus_roll_for_a_spare_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_both_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_cannot_roll_after_bonus_roll_for_spare", "test_cannot_roll_after_bonus_rolls_for_strike", "test_cannot_roll_if_game_already_has_ten_frames", "test_consecutive_spares_each_get_a_one_roll_bonus", "test_consecutive_strikes_each_get_the_two_roll_bonus", "test_points_scored_in_the_roll_after_a_spare_are_counted_twice", "test_points_scored_in_the_two_rolls_after_a_strike_are_counted_twice_as_a_bonus", "test_rolling_a_spare_with_the_two_roll_bonus_does_not_get_a_bonus_roll", "test_second_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_should_be_able_to_score_a_game_with_all_zeros", "test_should_be_able_to_score_a_game_with_no_strikes_or_spares", "test_strikes_with_the_two_roll_bonus_do_not_get_bonus_rolls", "test_the_second_bonus_rolls_after_a_strike_in_the_last_frame_cannot_be_a_strike_if_the_first_one_is_not_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_can_score_more_than_10_points_if_one_is_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_two_rolls_in_a_frame_cannot_score_more_than_10_points"]}, "calls": 1, "seconds": 18, "input_tokens": 3064, "output_tokens": 2466, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 25 failed, 6 passed"}, {"run": "20260924T162753-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T16:27:55", "finished": "2026-09-24T16:28:19", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": {"summary": "25 failed, 6 passed", "total": 31, "passed": 6, "failed": 25, "agent_written": 0, "failed_names": ["test_a_spare_followed_by_zeros_is_worth_ten_points", "test_a_spare_in_the_last_frame_gets_a_one_roll_bonus_that_is_counted_once", "test_a_strike_earns_ten_points_in_a_frame_with_a_single_roll", "test_a_strike_in_the_last_frame_gets_a_two_roll_bonus_that_is_counted_once", "test_a_strike_with_the_one_roll_bonus_after_a_spare_in_the_last_frame_does_not_get_a_bonus", "test_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_bonus_roll_for_a_spare_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_both_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_cannot_roll_after_bonus_roll_for_spare", "test_cannot_roll_after_bonus_rolls_for_strike", "test_cannot_roll_if_game_already_has_ten_frames", "test_consecutive_spares_each_get_a_one_roll_bonus", "test_consecutive_strikes_each_get_the_two_roll_bonus", "test_points_scored_in_the_roll_after_a_spare_are_counted_twice", "test_points_scored_in_the_two_rolls_after_a_strike_are_counted_twice_as_a_bonus", "test_rolling_a_spare_with_the_two_roll_bonus_does_not_get_a_bonus_roll", "test_second_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_should_be_able_to_score_a_game_with_all_zeros", "test_should_be_able_to_score_a_game_with_no_strikes_or_spares", "test_strikes_with_the_two_roll_bonus_do_not_get_bonus_rolls", "test_the_second_bonus_rolls_after_a_strike_in_the_last_frame_cannot_be_a_strike_if_the_first_one_is_not_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_can_score_more_than_10_points_if_one_is_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_two_rolls_in_a_frame_cannot_score_more_than_10_points"]}, "calls": 1, "seconds": 19, "input_tokens": 3064, "output_tokens": 2466, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 25 failed, 6 passed"}, {"run": "20260924T162614-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T16:26:17", "finished": "2026-09-24T16:26:40", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": {"summary": "25 failed, 6 passed", "total": 31, "passed": 6, "failed": 25, "agent_written": 0, "failed_names": ["test_a_spare_followed_by_zeros_is_worth_ten_points", "test_a_spare_in_the_last_frame_gets_a_one_roll_bonus_that_is_counted_once", "test_a_strike_earns_ten_points_in_a_frame_with_a_single_roll", "test_a_strike_in_the_last_frame_gets_a_two_roll_bonus_that_is_counted_once", "test_a_strike_with_the_one_roll_bonus_after_a_spare_in_the_last_frame_does_not_get_a_bonus", "test_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_bonus_roll_for_a_spare_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_both_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_cannot_roll_after_bonus_roll_for_spare", "test_cannot_roll_after_bonus_rolls_for_strike", "test_cannot_roll_if_game_already_has_ten_frames", "test_consecutive_spares_each_get_a_one_roll_bonus", "test_consecutive_strikes_each_get_the_two_roll_bonus", "test_points_scored_in_the_roll_after_a_spare_are_counted_twice", "test_points_scored_in_the_two_rolls_after_a_strike_are_counted_twice_as_a_bonus", "test_rolling_a_spare_with_the_two_roll_bonus_does_not_get_a_bonus_roll", "test_second_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_should_be_able_to_score_a_game_with_all_zeros", "test_should_be_able_to_score_a_game_with_no_strikes_or_spares", "test_strikes_with_the_two_roll_bonus_do_not_get_bonus_rolls", "test_the_second_bonus_rolls_after_a_strike_in_the_last_frame_cannot_be_a_strike_if_the_first_one_is_not_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_can_score_more_than_10_points_if_one_is_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_two_rolls_in_a_frame_cannot_score_more_than_10_points"]}, "calls": 1, "seconds": 18, "input_tokens": 3064, "output_tokens": 2466, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 25 failed, 6 passed"}, {"run": "20260924T162204-aider-ai-aid-polyglot_python_bowling", "started": "2026-09-24T16:22:08", "finished": "2026-09-24T16:22:33", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": {"summary": "25 failed, 6 passed", "total": 31, "passed": 6, "failed": 25, "agent_written": 0, "failed_names": ["test_a_spare_followed_by_zeros_is_worth_ten_points", "test_a_spare_in_the_last_frame_gets_a_one_roll_bonus_that_is_counted_once", "test_a_strike_earns_ten_points_in_a_frame_with_a_single_roll", "test_a_strike_in_the_last_frame_gets_a_two_roll_bonus_that_is_counted_once", "test_a_strike_with_the_one_roll_bonus_after_a_spare_in_the_last_frame_does_not_get_a_bonus", "test_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_bonus_roll_for_a_spare_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_both_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_cannot_roll_after_bonus_roll_for_spare", "test_cannot_roll_after_bonus_rolls_for_strike", "test_cannot_roll_if_game_already_has_ten_frames", "test_consecutive_spares_each_get_a_one_roll_bonus", "test_consecutive_strikes_each_get_the_two_roll_bonus", "test_points_scored_in_the_roll_after_a_spare_are_counted_twice", "test_points_scored_in_the_two_rolls_after_a_strike_are_counted_twice_as_a_bonus", "test_rolling_a_spare_with_the_two_roll_bonus_does_not_get_a_bonus_roll", "test_second_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_should_be_able_to_score_a_game_with_all_zeros", "test_should_be_able_to_score_a_game_with_no_strikes_or_spares", "test_strikes_with_the_two_roll_bonus_do_not_get_bonus_rolls", "test_the_second_bonus_rolls_after_a_strike_in_the_last_frame_cannot_be_a_strike_if_the_first_one_is_not_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_can_score_more_than_10_points_if_one_is_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_two_rolls_in_a_frame_cannot_score_more_than_10_points"]}, "calls": 1, "seconds": 20, "input_tokens": 3064, "output_tokens": 2466, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 25 failed, 6 passed"}, {"run": "20260924T161445-amd64-aider-polyglot_python_bowling", "started": "2026-09-24T16:14:48", "finished": "2026-09-24T16:15:11", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": {"summary": "26 failed, 5 passed", "total": 31, "passed": 5, "failed": 26, "agent_written": 0, "failed_names": ["test_a_spare_followed_by_zeros_is_worth_ten_points", "test_a_spare_in_the_last_frame_gets_a_one_roll_bonus_that_is_counted_once", "test_a_strike_earns_ten_points_in_a_frame_with_a_single_roll", "test_a_strike_in_the_last_frame_gets_a_two_roll_bonus_that_is_counted_once", "test_a_strike_with_the_one_roll_bonus_after_a_spare_in_the_last_frame_does_not_get_a_bonus", "test_an_incomplete_game_cannot_be_scored", "test_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_bonus_roll_for_a_spare_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_both_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_cannot_roll_after_bonus_roll_for_spare", "test_cannot_roll_after_bonus_rolls_for_strike", "test_cannot_roll_if_game_already_has_ten_frames", "test_consecutive_spares_each_get_a_one_roll_bonus", "test_consecutive_strikes_each_get_the_two_roll_bonus", "test_last_two_strikes_followed_by_only_last_bonus_with_non_strike_points", "test_points_scored_in_the_roll_after_a_spare_are_counted_twice", "test_points_scored_in_the_two_rolls_after_a_strike_are_counted_twice_as_a_bonus", "test_rolling_a_spare_with_the_two_roll_bonus_does_not_get_a_bonus_roll", "test_second_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_should_be_able_to_score_a_game_with_all_zeros", "test_should_be_able_to_score_a_game_with_no_strikes_or_spares", "test_strikes_with_the_two_roll_bonus_do_not_get_bonus_rolls", "test_the_second_bonus_rolls_after_a_strike_in_the_last_frame_cannot_be_a_strike_if_the_first_one_is_not_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_can_score_more_than_10_points_if_one_is_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points"]}, "calls": 1, "seconds": 18, "input_tokens": 3064, "output_tokens": 2365, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 26 failed, 5 passed"}, {"run": "20260924T160732-amd64-aider-polyglot_python_bowling", "started": "2026-09-24T16:09:56", "finished": "2026-09-24T16:10:20", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 0, "verifier_rc": 0, "tests": {"summary": "25 failed, 6 passed", "total": 31, "passed": 6, "failed": 25, "agent_written": 0, "failed_names": ["test_a_spare_followed_by_zeros_is_worth_ten_points", "test_a_spare_in_the_last_frame_gets_a_one_roll_bonus_that_is_counted_once", "test_a_strike_earns_ten_points_in_a_frame_with_a_single_roll", "test_a_strike_in_the_last_frame_gets_a_two_roll_bonus_that_is_counted_once", "test_a_strike_with_the_one_roll_bonus_after_a_spare_in_the_last_frame_does_not_get_a_bonus", "test_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_bonus_roll_for_a_spare_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_both_bonus_rolls_for_a_strike_in_the_last_frame_must_be_rolled_before_score_can_be_calculated", "test_cannot_roll_after_bonus_roll_for_spare", "test_cannot_roll_after_bonus_rolls_for_strike", "test_cannot_roll_if_game_already_has_ten_frames", "test_consecutive_spares_each_get_a_one_roll_bonus", "test_consecutive_strikes_each_get_the_two_roll_bonus", "test_points_scored_in_the_roll_after_a_spare_are_counted_twice", "test_points_scored_in_the_two_rolls_after_a_strike_are_counted_twice_as_a_bonus", "test_rolling_a_spare_with_the_two_roll_bonus_does_not_get_a_bonus_roll", "test_second_bonus_roll_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_should_be_able_to_score_a_game_with_all_zeros", "test_should_be_able_to_score_a_game_with_no_strikes_or_spares", "test_strikes_with_the_two_roll_bonus_do_not_get_bonus_rolls", "test_the_second_bonus_rolls_after_a_strike_in_the_last_frame_cannot_be_a_strike_if_the_first_one_is_not_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_can_score_more_than_10_points_if_one_is_a_strike", "test_two_bonus_rolls_after_a_strike_in_the_last_frame_cannot_score_more_than_10_points", "test_two_rolls_in_a_frame_cannot_score_more_than_10_points"]}, "calls": 1, "seconds": 19, "input_tokens": 3064, "output_tokens": 2476, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 0 \u00b7 25 failed, 6 passed"}, {"run": "20260923T112810-aider-ai-aider-polyglot_python_bowling", "started": "2026-09-23T11:32:24", "finished": "2026-09-23T11:32:43", "status": "done", "kind": "harbor", "harness": "aider-ai-aider", "task": {"taskset": "aider_polyglot", "name": "polyglot_python_bowling"}, "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", "reward": 1, "verifier_rc": 0, "tests": {"summary": "31 passed", "total": 31, "passed": 31, "failed": 0, "agent_written": 0, "failed_names": []}, "calls": 1, "seconds": 17, "input_tokens": 3063, "output_tokens": 2463, "errors": 0, "last_action": null, "outcome": "scored", "verifier_says": "reward 1 \u00b7 31 passed"}]}