{"taskset": {"n_runnable": 0, "name": "GPQA Diamond", "url_data": "https://huggingface.co/datasets/Idavidrein/gpqa", "catalog_id": "gpqa-diamond", "owner_type": "academic", "cited_by": ["DeepSeek-V4.1-Flash", "GLM-5", "GLM-5.1", "GLM-5.2", "GPT-5.6 Sol", "GPT-6 Astra", "Kimi K2.6", "Kimi K3", "Nemotron 3 Ultra", "Nemotron 3.5 Lightning", "Qwen3.8-27B", "Qwen3.8-Flash-Next", "Qwen3.8-Max"], "taskset": "gpqa-diamond", "domain": "science-math", "task_kind": "graduate-level science multiple choice", "environment": "none", "grading": "exact-match", "path": "datasets/gpqa-diamond", "n_single_container": 198, "n_tasks": 198, "domain_raw": "science", "sample_instruction": "What is the correct answer to this question: \nTwo quantum states with energies E1 and E2 have a lifetime of 10^-9 sec and 10^-8 sec, respectively. We want to clearly distinguish these two energy levels. Which one of the following options could be their energy difference so that they can be clearly resolved?\n\nChoices:\n  (A) 10^-8 eV\n\n  (B) 10^-11 eV\n  (C) 10^-9 eV\n  (D) 10^-4 eV\n\nLet's think step by step:\n\n[First, reason through the problem carefully. Consider all relevant concepts, formulas, and principles. Break down complex problems into manageable steps. Evaluate each answer choice critically.]\n\nBased on your reasoning above, determine the single, most likely answer choice. Then write ONLY the letter of your final answer (A, B, C, or D) to `/app/answer.txt`.\n\n**Important:**\n- This is a challenging graduate-level question in biology, physics, or chemistry\n- Do not try to solve the question in a single step. Break it down into smaller steps.\n- You should ONLY interact with the environment provided to you AND NEVER ASK FOR HUMAN HELP.\n- The answer file must contain only a single uppercase letter (A, B, C, or D) with no additional text or explanation\n- You can show your work and reasoning before writing the final answer, but the file `/app/answer.txt` should contain only the letter", "owner_org": "NYU / Cohere / Anthropic (Rein et al.)", "url_repo": "https://github.com/idavidrein/gpqa", "url_paper": "https://arxiv.org/abs/2311.12022", "license": "MIT", "languages": [], "difficulties": {"difficult": 198}, "categories": ["reasoning"]}, "tasks": [{"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "0", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "1", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "10", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "100", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "101", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "102", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "103", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "104", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "105", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "106", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "107", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "108", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "109", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "11", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "110", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "111", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "112", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "113", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "114", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "115", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "116", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "117", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "118", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "119", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "12", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "120", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "121", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "122", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "123", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "124", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "125", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "126", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "127", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "128", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "129", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "13", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "130", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "131", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "132", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "133", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "134", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "electromagnetism and photonics"], "task": "135", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "136", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "137", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "138", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "139", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "14", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "140", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "141", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "142", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "143", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "144", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "145", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "146", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "147", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "148", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "149", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "15", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "150", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "151", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "152", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "153", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "154", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "155", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "156", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "genetics"], "task": "157", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "158", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "optics and acoustics"], "task": "159", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "16", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "160", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "161", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "162", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "163", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "164", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "165", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "166", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "genetics"], "task": "167", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "168", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "169", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "17", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "170", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "171", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "172", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "173", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "electromagnetism and photonics"], "task": "174", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "175", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "176", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "177", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "178", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "179", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "18", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "180", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "condensed matter physics"], "task": "181", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "182", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "183", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "184", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "185", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "186", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "187", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "188", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "189", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "19", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "190", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "electromagnetism and photonics"], "task": "191", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "192", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "193", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "194", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "195", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "196", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "197", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "2", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "20", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "21", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "22", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "23", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "24", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "25", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "26", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "27", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "genetics"], "task": "28", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "29", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "electromagnetism and photonics"], "task": "3", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "30", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "31", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "32", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "33", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "34", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "35", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "36", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "37", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "38", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "39", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "4", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "40", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "41", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "42", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "43", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "44", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "45", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "46", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "47", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "48", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "49", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "5", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "electromagnetism and photonics"], "task": "50", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "51", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "52", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "53", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "54", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "55", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "56", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "57", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "58", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "59", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "6", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "60", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "61", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "62", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "inorganic chemistry"], "task": "63", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "64", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "65", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "66", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "quantum mechanics"], "task": "67", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "high-energy particle physics"], "task": "68", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "69", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "genetics"], "task": "7", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "70", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "71", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "relativistic mechanics"], "task": "72", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "73", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "74", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "75", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "76", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "electromagnetism and photonics"], "task": "77", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "78", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "79", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "8", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "80", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "81", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "82", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "83", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "84", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "85", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "86", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "87", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "88", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "89", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "astrophysics"], "task": "9", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "90", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "91", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "92", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "93", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "94", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "physics", "physics (general)"], "task": "95", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "biology", "molecular biology"], "task": "96", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "97", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "chemistry (general)"], "task": "98", "taskset": "gpqa-diamond"}, {"agent_timeout": 3600, "category": "reasoning", "compose": false, "difficulty": "difficult", "language": "", "oracle": null, "runnable": false, "tags": ["gpqa", "multiple-choice", "chemistry", "organic chemistry"], "task": "99", "taskset": "gpqa-diamond"}], "next": null}