# gpqa-diamond - domain: science-math - n_tasks: 198 - n_runnable: 0 - owner_org: NYU / Cohere / Anthropic (Rein et al.) - grading: exact-match - task_kind: graduate-level science multiple choice - url_repo: https://github.com/idavidrein/gpqa - url_paper: https://arxiv.org/abs/2311.12022 ## Tasks | task | difficulty | language | runnable | harnesses tried | |---|---|---|---|---| | [0](https://harnessreport.com/tasks/gpqa-diamond/0.md) | difficult | | | 0 | | [1](https://harnessreport.com/tasks/gpqa-diamond/1.md) | difficult | | | 0 | | [10](https://harnessreport.com/tasks/gpqa-diamond/10.md) | difficult | | | 0 | | [100](https://harnessreport.com/tasks/gpqa-diamond/100.md) | difficult | | | 0 | | [101](https://harnessreport.com/tasks/gpqa-diamond/101.md) | difficult | | | 0 | | [102](https://harnessreport.com/tasks/gpqa-diamond/102.md) | difficult | | | 0 | | [103](https://harnessreport.com/tasks/gpqa-diamond/103.md) | difficult | | | 0 | | [104](https://harnessreport.com/tasks/gpqa-diamond/104.md) | difficult | | | 0 | | [105](https://harnessreport.com/tasks/gpqa-diamond/105.md) | difficult | | | 0 | | [106](https://harnessreport.com/tasks/gpqa-diamond/106.md) | difficult | | | 0 | | [107](https://harnessreport.com/tasks/gpqa-diamond/107.md) | difficult | | | 0 | | [108](https://harnessreport.com/tasks/gpqa-diamond/108.md) | difficult | | | 0 | | [109](https://harnessreport.com/tasks/gpqa-diamond/109.md) | difficult | | | 0 | | [11](https://harnessreport.com/tasks/gpqa-diamond/11.md) | difficult | | | 0 | | [110](https://harnessreport.com/tasks/gpqa-diamond/110.md) | difficult | | | 0 | | [111](https://harnessreport.com/tasks/gpqa-diamond/111.md) | difficult | | | 0 | | [112](https://harnessreport.com/tasks/gpqa-diamond/112.md) | difficult | | | 0 | | [113](https://harnessreport.com/tasks/gpqa-diamond/113.md) | difficult | | | 0 | | [114](https://harnessreport.com/tasks/gpqa-diamond/114.md) | difficult | | | 0 | | [115](https://harnessreport.com/tasks/gpqa-diamond/115.md) | difficult | | | 0 | | [116](https://harnessreport.com/tasks/gpqa-diamond/116.md) | difficult | | | 0 | | [117](https://harnessreport.com/tasks/gpqa-diamond/117.md) | difficult | | | 0 | | [118](https://harnessreport.com/tasks/gpqa-diamond/118.md) | difficult | | | 0 | | [119](https://harnessreport.com/tasks/gpqa-diamond/119.md) | difficult | | | 0 | | [12](https://harnessreport.com/tasks/gpqa-diamond/12.md) | difficult | | | 0 | | [120](https://harnessreport.com/tasks/gpqa-diamond/120.md) | difficult | | | 0 | | [121](https://harnessreport.com/tasks/gpqa-diamond/121.md) | difficult | | | 0 | | [122](https://harnessreport.com/tasks/gpqa-diamond/122.md) | difficult | | | 0 | | [123](https://harnessreport.com/tasks/gpqa-diamond/123.md) | difficult | | | 0 | | [124](https://harnessreport.com/tasks/gpqa-diamond/124.md) | difficult | | | 0 | | [125](https://harnessreport.com/tasks/gpqa-diamond/125.md) | difficult | | | 0 | | [126](https://harnessreport.com/tasks/gpqa-diamond/126.md) | difficult | | | 0 | | [127](https://harnessreport.com/tasks/gpqa-diamond/127.md) | difficult | | | 0 | | [128](https://harnessreport.com/tasks/gpqa-diamond/128.md) | difficult | | | 0 | | [129](https://harnessreport.com/tasks/gpqa-diamond/129.md) | difficult | | | 0 | | [13](https://harnessreport.com/tasks/gpqa-diamond/13.md) | difficult | | | 0 | | [130](https://harnessreport.com/tasks/gpqa-diamond/130.md) | difficult | | | 0 | | [131](https://harnessreport.com/tasks/gpqa-diamond/131.md) | difficult | | | 0 | | [132](https://harnessreport.com/tasks/gpqa-diamond/132.md) | difficult | | | 0 | | [133](https://harnessreport.com/tasks/gpqa-diamond/133.md) | difficult | | | 0 | | [134](https://harnessreport.com/tasks/gpqa-diamond/134.md) | difficult | | | 0 | | [135](https://harnessreport.com/tasks/gpqa-diamond/135.md) | difficult | | | 0 | | [136](https://harnessreport.com/tasks/gpqa-diamond/136.md) | difficult | | | 0 | | [137](https://harnessreport.com/tasks/gpqa-diamond/137.md) | difficult | | | 0 | | [138](https://harnessreport.com/tasks/gpqa-diamond/138.md) | difficult | | | 0 | | [139](https://harnessreport.com/tasks/gpqa-diamond/139.md) | difficult | | | 0 | | [14](https://harnessreport.com/tasks/gpqa-diamond/14.md) | difficult | | | 0 | | [140](https://harnessreport.com/tasks/gpqa-diamond/140.md) | difficult | | | 0 | | [141](https://harnessreport.com/tasks/gpqa-diamond/141.md) | difficult | | | 0 | | [142](https://harnessreport.com/tasks/gpqa-diamond/142.md) | difficult | | | 0 | | [143](https://harnessreport.com/tasks/gpqa-diamond/143.md) | difficult | | | 0 | | [144](https://harnessreport.com/tasks/gpqa-diamond/144.md) | difficult | | | 0 | | [145](https://harnessreport.com/tasks/gpqa-diamond/145.md) | difficult | | | 0 | | [146](https://harnessreport.com/tasks/gpqa-diamond/146.md) | difficult | | | 0 | | [147](https://harnessreport.com/tasks/gpqa-diamond/147.md) | difficult | | | 0 | | [148](https://harnessreport.com/tasks/gpqa-diamond/148.md) | difficult | | | 0 | | [149](https://harnessreport.com/tasks/gpqa-diamond/149.md) | difficult | | | 0 | | [15](https://harnessreport.com/tasks/gpqa-diamond/15.md) | difficult | | | 0 | | [150](https://harnessreport.com/tasks/gpqa-diamond/150.md) | difficult | | | 0 | | [151](https://harnessreport.com/tasks/gpqa-diamond/151.md) | difficult | | | 0 | | [152](https://harnessreport.com/tasks/gpqa-diamond/152.md) | difficult | | | 0 | | [153](https://harnessreport.com/tasks/gpqa-diamond/153.md) | difficult | | | 0 | | [154](https://harnessreport.com/tasks/gpqa-diamond/154.md) | difficult | | | 0 | | [155](https://harnessreport.com/tasks/gpqa-diamond/155.md) | difficult | | | 0 | | [156](https://harnessreport.com/tasks/gpqa-diamond/156.md) | difficult | | | 0 | | [157](https://harnessreport.com/tasks/gpqa-diamond/157.md) | difficult | | | 0 | | [158](https://harnessreport.com/tasks/gpqa-diamond/158.md) | difficult | | | 0 | | [159](https://harnessreport.com/tasks/gpqa-diamond/159.md) | difficult | | | 0 | | [16](https://harnessreport.com/tasks/gpqa-diamond/16.md) | difficult | | | 0 | | [160](https://harnessreport.com/tasks/gpqa-diamond/160.md) | difficult | | | 0 | | [161](https://harnessreport.com/tasks/gpqa-diamond/161.md) | difficult | | | 0 | | [162](https://harnessreport.com/tasks/gpqa-diamond/162.md) | difficult | | | 0 | | [163](https://harnessreport.com/tasks/gpqa-diamond/163.md) | difficult | | | 0 | | [164](https://harnessreport.com/tasks/gpqa-diamond/164.md) | difficult | | | 0 | | [165](https://harnessreport.com/tasks/gpqa-diamond/165.md) | difficult | | | 0 | | [166](https://harnessreport.com/tasks/gpqa-diamond/166.md) | difficult | | | 0 | | [167](https://harnessreport.com/tasks/gpqa-diamond/167.md) | difficult | | | 0 | | [168](https://harnessreport.com/tasks/gpqa-diamond/168.md) | difficult | | | 0 | | [169](https://harnessreport.com/tasks/gpqa-diamond/169.md) | difficult | | | 0 | | [17](https://harnessreport.com/tasks/gpqa-diamond/17.md) | difficult | | | 0 | | [170](https://harnessreport.com/tasks/gpqa-diamond/170.md) | difficult | | | 0 | | [171](https://harnessreport.com/tasks/gpqa-diamond/171.md) | difficult | | | 0 | | [172](https://harnessreport.com/tasks/gpqa-diamond/172.md) | difficult | | | 0 | | [173](https://harnessreport.com/tasks/gpqa-diamond/173.md) | difficult | | | 0 | | [174](https://harnessreport.com/tasks/gpqa-diamond/174.md) | difficult | | | 0 | | [175](https://harnessreport.com/tasks/gpqa-diamond/175.md) | difficult | | | 0 | | [176](https://harnessreport.com/tasks/gpqa-diamond/176.md) | difficult | | | 0 | | [177](https://harnessreport.com/tasks/gpqa-diamond/177.md) | difficult | | | 0 | | [178](https://harnessreport.com/tasks/gpqa-diamond/178.md) | difficult | | | 0 | | [179](https://harnessreport.com/tasks/gpqa-diamond/179.md) | difficult | | | 0 | | [18](https://harnessreport.com/tasks/gpqa-diamond/18.md) | difficult | | | 0 | | [180](https://harnessreport.com/tasks/gpqa-diamond/180.md) | difficult | | | 0 | | [181](https://harnessreport.com/tasks/gpqa-diamond/181.md) | difficult | | | 0 | | [182](https://harnessreport.com/tasks/gpqa-diamond/182.md) | difficult | | | 0 | | [183](https://harnessreport.com/tasks/gpqa-diamond/183.md) | difficult | | | 0 | | [184](https://harnessreport.com/tasks/gpqa-diamond/184.md) | difficult | | | 0 | | [185](https://harnessreport.com/tasks/gpqa-diamond/185.md) | difficult | | | 0 | | [186](https://harnessreport.com/tasks/gpqa-diamond/186.md) | difficult | | | 0 | | [187](https://harnessreport.com/tasks/gpqa-diamond/187.md) | difficult | | | 0 | | [188](https://harnessreport.com/tasks/gpqa-diamond/188.md) | difficult | | | 0 | | [189](https://harnessreport.com/tasks/gpqa-diamond/189.md) | difficult | | | 0 | | [19](https://harnessreport.com/tasks/gpqa-diamond/19.md) | difficult | | | 0 | | [190](https://harnessreport.com/tasks/gpqa-diamond/190.md) | difficult | | | 0 | | [191](https://harnessreport.com/tasks/gpqa-diamond/191.md) | difficult | | | 0 | | [192](https://harnessreport.com/tasks/gpqa-diamond/192.md) | difficult | | | 0 | | [193](https://harnessreport.com/tasks/gpqa-diamond/193.md) | difficult | | | 0 | | [194](https://harnessreport.com/tasks/gpqa-diamond/194.md) | difficult | | | 0 | | [195](https://harnessreport.com/tasks/gpqa-diamond/195.md) | difficult | | | 0 | | [196](https://harnessreport.com/tasks/gpqa-diamond/196.md) | difficult | | | 0 | | [197](https://harnessreport.com/tasks/gpqa-diamond/197.md) | difficult | | | 0 | | [2](https://harnessreport.com/tasks/gpqa-diamond/2.md) | difficult | | | 0 | | [20](https://harnessreport.com/tasks/gpqa-diamond/20.md) | difficult | | | 0 | | [21](https://harnessreport.com/tasks/gpqa-diamond/21.md) | difficult | | | 0 | | [22](https://harnessreport.com/tasks/gpqa-diamond/22.md) | difficult | | | 0 | | [23](https://harnessreport.com/tasks/gpqa-diamond/23.md) | difficult | | | 0 | | [24](https://harnessreport.com/tasks/gpqa-diamond/24.md) | difficult | | | 0 | | [25](https://harnessreport.com/tasks/gpqa-diamond/25.md) | difficult | | | 0 | | [26](https://harnessreport.com/tasks/gpqa-diamond/26.md) | difficult | | | 0 | | [27](https://harnessreport.com/tasks/gpqa-diamond/27.md) | difficult | | | 0 | | [28](https://harnessreport.com/tasks/gpqa-diamond/28.md) | difficult | | | 0 | | [29](https://harnessreport.com/tasks/gpqa-diamond/29.md) | difficult | | | 0 | | [3](https://harnessreport.com/tasks/gpqa-diamond/3.md) | difficult | | | 0 | | [30](https://harnessreport.com/tasks/gpqa-diamond/30.md) | difficult | | | 0 | | [31](https://harnessreport.com/tasks/gpqa-diamond/31.md) | difficult | | | 0 | | [32](https://harnessreport.com/tasks/gpqa-diamond/32.md) | difficult | | | 0 | | [33](https://harnessreport.com/tasks/gpqa-diamond/33.md) | difficult | | | 0 | | [34](https://harnessreport.com/tasks/gpqa-diamond/34.md) | difficult | | | 0 | | [35](https://harnessreport.com/tasks/gpqa-diamond/35.md) | difficult | | | 0 | | [36](https://harnessreport.com/tasks/gpqa-diamond/36.md) | difficult | | | 0 | | [37](https://harnessreport.com/tasks/gpqa-diamond/37.md) | difficult | | | 0 | | [38](https://harnessreport.com/tasks/gpqa-diamond/38.md) | difficult | | | 0 | | [39](https://harnessreport.com/tasks/gpqa-diamond/39.md) | difficult | | | 0 | | [4](https://harnessreport.com/tasks/gpqa-diamond/4.md) | difficult | | | 0 | | [40](https://harnessreport.com/tasks/gpqa-diamond/40.md) | difficult | | | 0 | | [41](https://harnessreport.com/tasks/gpqa-diamond/41.md) | difficult | | | 0 | | [42](https://harnessreport.com/tasks/gpqa-diamond/42.md) | difficult | | | 0 | | [43](https://harnessreport.com/tasks/gpqa-diamond/43.md) | difficult | | | 0 | | [44](https://harnessreport.com/tasks/gpqa-diamond/44.md) | difficult | | | 0 | | [45](https://harnessreport.com/tasks/gpqa-diamond/45.md) | difficult | | | 0 | | [46](https://harnessreport.com/tasks/gpqa-diamond/46.md) | difficult | | | 0 | | [47](https://harnessreport.com/tasks/gpqa-diamond/47.md) | difficult | | | 0 | | [48](https://harnessreport.com/tasks/gpqa-diamond/48.md) | difficult | | | 0 | | [49](https://harnessreport.com/tasks/gpqa-diamond/49.md) | difficult | | | 0 | | [5](https://harnessreport.com/tasks/gpqa-diamond/5.md) | difficult | | | 0 | | [50](https://harnessreport.com/tasks/gpqa-diamond/50.md) | difficult | | | 0 | | [51](https://harnessreport.com/tasks/gpqa-diamond/51.md) | difficult | | | 0 | | [52](https://harnessreport.com/tasks/gpqa-diamond/52.md) | difficult | | | 0 | | [53](https://harnessreport.com/tasks/gpqa-diamond/53.md) | difficult | | | 0 | | [54](https://harnessreport.com/tasks/gpqa-diamond/54.md) | difficult | | | 0 | | [55](https://harnessreport.com/tasks/gpqa-diamond/55.md) | difficult | | | 0 | | [56](https://harnessreport.com/tasks/gpqa-diamond/56.md) | difficult | | | 0 | | [57](https://harnessreport.com/tasks/gpqa-diamond/57.md) | difficult | | | 0 | | [58](https://harnessreport.com/tasks/gpqa-diamond/58.md) | difficult | | | 0 | | [59](https://harnessreport.com/tasks/gpqa-diamond/59.md) | difficult | | | 0 | | [6](https://harnessreport.com/tasks/gpqa-diamond/6.md) | difficult | | | 0 | | [60](https://harnessreport.com/tasks/gpqa-diamond/60.md) | difficult | | | 0 | | [61](https://harnessreport.com/tasks/gpqa-diamond/61.md) | difficult | | | 0 | | [62](https://harnessreport.com/tasks/gpqa-diamond/62.md) | difficult | | | 0 | | [63](https://harnessreport.com/tasks/gpqa-diamond/63.md) | difficult | | | 0 | | [64](https://harnessreport.com/tasks/gpqa-diamond/64.md) | difficult | | | 0 | | [65](https://harnessreport.com/tasks/gpqa-diamond/65.md) | difficult | | | 0 | | [66](https://harnessreport.com/tasks/gpqa-diamond/66.md) | difficult | | | 0 | | [67](https://harnessreport.com/tasks/gpqa-diamond/67.md) | difficult | | | 0 | | [68](https://harnessreport.com/tasks/gpqa-diamond/68.md) | difficult | | | 0 | | [69](https://harnessreport.com/tasks/gpqa-diamond/69.md) | difficult | | | 0 | | [7](https://harnessreport.com/tasks/gpqa-diamond/7.md) | difficult | | | 0 | | [70](https://harnessreport.com/tasks/gpqa-diamond/70.md) | difficult | | | 0 | | [71](https://harnessreport.com/tasks/gpqa-diamond/71.md) | difficult | | | 0 | | [72](https://harnessreport.com/tasks/gpqa-diamond/72.md) | difficult | | | 0 | | [73](https://harnessreport.com/tasks/gpqa-diamond/73.md) | difficult | | | 0 | | [74](https://harnessreport.com/tasks/gpqa-diamond/74.md) | difficult | | | 0 | | [75](https://harnessreport.com/tasks/gpqa-diamond/75.md) | difficult | | | 0 | | [76](https://harnessreport.com/tasks/gpqa-diamond/76.md) | difficult | | | 0 | | [77](https://harnessreport.com/tasks/gpqa-diamond/77.md) | difficult | | | 0 | | [78](https://harnessreport.com/tasks/gpqa-diamond/78.md) | difficult | | | 0 | | [79](https://harnessreport.com/tasks/gpqa-diamond/79.md) | difficult | | | 0 | | [8](https://harnessreport.com/tasks/gpqa-diamond/8.md) | difficult | | | 0 | | [80](https://harnessreport.com/tasks/gpqa-diamond/80.md) | difficult | | | 0 | | [81](https://harnessreport.com/tasks/gpqa-diamond/81.md) | difficult | | | 0 | | [82](https://harnessreport.com/tasks/gpqa-diamond/82.md) | difficult | | | 0 | | [83](https://harnessreport.com/tasks/gpqa-diamond/83.md) | difficult | | | 0 | | [84](https://harnessreport.com/tasks/gpqa-diamond/84.md) | difficult | | | 0 | | [85](https://harnessreport.com/tasks/gpqa-diamond/85.md) | difficult | | | 0 | | [86](https://harnessreport.com/tasks/gpqa-diamond/86.md) | difficult | | | 0 | | [87](https://harnessreport.com/tasks/gpqa-diamond/87.md) | difficult | | | 0 | | [88](https://harnessreport.com/tasks/gpqa-diamond/88.md) | difficult | | | 0 | | [89](https://harnessreport.com/tasks/gpqa-diamond/89.md) | difficult | | | 0 | | [9](https://harnessreport.com/tasks/gpqa-diamond/9.md) | difficult | | | 0 | | [90](https://harnessreport.com/tasks/gpqa-diamond/90.md) | difficult | | | 0 | | [91](https://harnessreport.com/tasks/gpqa-diamond/91.md) | difficult | | | 0 | | [92](https://harnessreport.com/tasks/gpqa-diamond/92.md) | difficult | | | 0 | | [93](https://harnessreport.com/tasks/gpqa-diamond/93.md) | difficult | | | 0 | | [94](https://harnessreport.com/tasks/gpqa-diamond/94.md) | difficult | | | 0 | | [95](https://harnessreport.com/tasks/gpqa-diamond/95.md) | difficult | | | 0 | | [96](https://harnessreport.com/tasks/gpqa-diamond/96.md) | difficult | | | 0 | | [97](https://harnessreport.com/tasks/gpqa-diamond/97.md) | difficult | | | 0 | | [98](https://harnessreport.com/tasks/gpqa-diamond/98.md) | difficult | | | 0 | | [99](https://harnessreport.com/tasks/gpqa-diamond/99.md) | difficult | | | 0 | --- Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt ยท MCP: https://harnessreport.com/mcp