{"taskset": {"n_runnable": 0, "name": "ACE", "url_data": "https://huggingface.co/datasets/mercor/ACE", "catalog_id": "ace-mercor", "owner_type": "company", "cited_by": [], "taskset": "acebench-special", "domain": "enterprise-ops", "task_kind": "evaluation criteria for DIY/food/shopping/gaming tasks", "environment": "live-web", "grading": "rubric", "path": "datasets/acebench-special", "n_single_container": 150, "n_tasks": 150, "domain_raw": "consumer", "sample_instruction": "# Tool Analysis Task\n\nThe user's request **may contain incorrect parameter values**. Check whether any parameter value is wrong or invalid.\n\n## Question\nuser: Can you evaluate my rehab plan ID: 1234WrongID, on 2023-10-21 with an effectiveness of 8?\n\n## Current Time\nThe current time is October 21, 2023, Saturday\u3002\n\n## Available Tools\n\n```json\n[\n  {\n    \"name\": \"RehabPlanEvaluator\",\n    \"description\": \"Evaluate the effectiveness of the current rehabilitation plan and suggest improvements.\",\n    \"parameters\": {\n      \"type\": \"dict\",\n      \"properties\": {\n        \"plan_id\": {\n          \"description\": \"The unique identifier for the rehabilitation plan.\",\n          \"type\": \"string\",\n          \"pattern\": \"^[a-zA-Z0-9-]+$\"\n        },\n        \"evaluation_date\": {\n          \"description\": \"The date of the plan evaluation in YYYY-MM-DD format.\",\n          \"type\": \"string\",\n          \"pattern\": \"^\\\\d{4}-\\\\d{2}-\\\\d{2}$\"\n        },\n        \"effectiveness_score\": {\n          \"description\": \"A score representing the effectiveness of the plan, from 1 to 10.\",\n          \"type\": \"integer\",\n          \"minimum\": 1,\n          \"maximum\": 10\n        },\n        \"comments\": {\n          \"description\": \"Comments or suggestions for improving the rehabilitation plan.\",\n          \"type\": \"string\"\n        }\n      },\n      \"required\": [\n        \"plan_id\",\n        \"evaluation_date\",\n        \"effectiveness_score\"\n      ]\n    }\n  },\n  {\n    \"name\": \"ComprehensiveRecoveryPlan\",\n    \"description\": \"Generate a comp", "owner_org": "Mercor", "url_repo": null, "url_paper": null, "license": "cc-by-4.0", "languages": [], "difficulties": {"medium": 150}, "categories": ["tool-use"]}, "tasks": [{"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_0", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_1", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_10", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_11", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_12", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_13", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_14", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_15", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_16", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_17", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_18", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_19", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_2", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_20", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_21", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_22", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_23", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_24", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_25", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_26", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_27", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_28", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_29", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_3", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_30", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_31", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_32", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_33", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_34", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_35", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_36", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_37", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_38", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_39", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_4", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_40", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_41", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_42", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_43", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_44", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_45", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_46", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_47", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_48", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_49", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_5", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_6", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_7", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_8", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_error_param_9", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_0", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_1", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_10", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_11", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_12", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_13", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_14", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_15", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_16", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_17", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_18", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_19", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_2", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_20", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_21", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_22", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_23", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_24", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_25", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_26", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_27", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_28", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_29", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_3", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_30", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_31", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_32", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_33", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_34", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_35", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_36", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_37", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_38", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_39", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_4", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_40", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_41", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_42", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_43", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_44", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_45", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_46", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_47", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_48", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_49", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_5", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_6", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_7", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_8", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_incomplete_9", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_0", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_1", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_10", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_11", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_12", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_13", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_14", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_15", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_16", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_17", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_18", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_19", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_2", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_20", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_21", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_22", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_23", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_24", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_25", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_26", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_27", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_28", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_29", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_3", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_30", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_31", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_32", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_33", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_34", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_35", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_36", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_37", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_38", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_39", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_4", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_40", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_41", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_42", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_43", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_44", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_45", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_46", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_47", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_48", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_49", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_5", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_6", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_7", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_8", "taskset": "acebench-special"}, {"agent_timeout": 300, "category": "tool-use", "compose": false, "difficulty": "medium", "language": "", "oracle": null, "runnable": false, "tags": [], "task": "ace-bench_special_irrelevant_9", "taskset": "acebench-special"}], "next": null}