# bigcodebench_hard_complete

- domain: swe
- n_tasks: 145
- n_runnable: 3

## Tasks

| task | difficulty | language | runnable | harnesses tried |
|---|---|---|---|---|
| [bigcodebench_100](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_100.md) | medium |  |  | 0 |
| [bigcodebench_1003](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1003.md) | medium |  |  | 0 |
| [bigcodebench_1004](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1004.md) | medium |  |  | 0 |
| [bigcodebench_1006](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1006.md) | medium |  |  | 0 |
| [bigcodebench_1008](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1008.md) | medium |  |  | 0 |
| [bigcodebench_1012](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1012.md) | medium |  |  | 0 |
| [bigcodebench_1013](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1013.md) | medium |  |  | 0 |
| [bigcodebench_1015](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1015.md) | medium |  |  | 0 |
| [bigcodebench_1019](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1019.md) | medium |  |  | 0 |
| [bigcodebench_1020](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1020.md) | medium |  |  | 0 |
| [bigcodebench_1022](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1022.md) | medium |  |  | 0 |
| [bigcodebench_1039](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1039.md) | medium |  |  | 0 |
| [bigcodebench_1040](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1040.md) | medium |  |  | 0 |
| [bigcodebench_1042](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1042.md) | medium |  |  | 0 |
| [bigcodebench_1053](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1053.md) | medium |  |  | 0 |
| [bigcodebench_1057](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1057.md) | medium |  | yes | 1 |
| [bigcodebench_1077](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1077.md) | medium |  |  | 0 |
| [bigcodebench_108](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_108.md) | medium |  |  | 0 |
| [bigcodebench_1085](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1085.md) | medium |  |  | 0 |
| [bigcodebench_1124](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1124.md) | medium |  |  | 0 |
| [bigcodebench_1129](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1129.md) | medium |  |  | 0 |
| [bigcodebench_1137](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_1137.md) | medium |  |  | 0 |
| [bigcodebench_120](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_120.md) | medium |  |  | 0 |
| [bigcodebench_123](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_123.md) | medium |  |  | 0 |
| [bigcodebench_124](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_124.md) | medium |  |  | 0 |
| [bigcodebench_129](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_129.md) | medium |  |  | 0 |
| [bigcodebench_13](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_13.md) | medium |  |  | 0 |
| [bigcodebench_139](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_139.md) | medium |  |  | 0 |
| [bigcodebench_147](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_147.md) | medium |  |  | 0 |
| [bigcodebench_15](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_15.md) | medium |  |  | 0 |
| [bigcodebench_161](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_161.md) | medium |  |  | 0 |
| [bigcodebench_162](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_162.md) | medium |  | yes | 0 |
| [bigcodebench_17](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_17.md) | medium |  |  | 0 |
| [bigcodebench_177](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_177.md) | medium |  |  | 0 |
| [bigcodebench_184](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_184.md) | medium |  |  | 0 |
| [bigcodebench_187](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_187.md) | medium |  |  | 0 |
| [bigcodebench_19](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_19.md) | medium |  |  | 0 |
| [bigcodebench_199](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_199.md) | medium |  |  | 0 |
| [bigcodebench_208](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_208.md) | medium |  |  | 0 |
| [bigcodebench_211](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_211.md) | medium |  |  | 0 |
| [bigcodebench_214](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_214.md) | medium |  |  | 0 |
| [bigcodebench_227](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_227.md) | medium |  |  | 0 |
| [bigcodebench_239](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_239.md) | medium |  |  | 0 |
| [bigcodebench_241](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_241.md) | medium |  |  | 0 |
| [bigcodebench_267](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_267.md) | medium |  |  | 0 |
| [bigcodebench_273](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_273.md) | medium |  |  | 0 |
| [bigcodebench_274](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_274.md) | medium |  |  | 0 |
| [bigcodebench_287](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_287.md) | medium |  |  | 0 |
| [bigcodebench_302](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_302.md) | medium |  |  | 0 |
| [bigcodebench_308](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_308.md) | medium |  |  | 0 |
| [bigcodebench_310](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_310.md) | medium |  |  | 0 |
| [bigcodebench_313](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_313.md) | medium |  |  | 0 |
| [bigcodebench_324](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_324.md) | medium |  |  | 0 |
| [bigcodebench_326](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_326.md) | medium |  |  | 0 |
| [bigcodebench_34](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_34.md) | medium |  |  | 0 |
| [bigcodebench_341](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_341.md) | medium |  |  | 0 |
| [bigcodebench_346](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_346.md) | medium |  |  | 0 |
| [bigcodebench_360](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_360.md) | medium |  | yes | 0 |
| [bigcodebench_367](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_367.md) | medium |  |  | 0 |
| [bigcodebench_368](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_368.md) | medium |  |  | 0 |
| [bigcodebench_37](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_37.md) | medium |  |  | 0 |
| [bigcodebench_374](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_374.md) | medium |  |  | 0 |
| [bigcodebench_399](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_399.md) | medium |  |  | 0 |
| [bigcodebench_401](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_401.md) | medium |  |  | 0 |
| [bigcodebench_409](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_409.md) | medium |  |  | 0 |
| [bigcodebench_417](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_417.md) | medium |  |  | 0 |
| [bigcodebench_418](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_418.md) | medium |  |  | 0 |
| [bigcodebench_424](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_424.md) | medium |  |  | 0 |
| [bigcodebench_443](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_443.md) | medium |  |  | 0 |
| [bigcodebench_445](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_445.md) | medium |  |  | 0 |
| [bigcodebench_454](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_454.md) | medium |  |  | 0 |
| [bigcodebench_458](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_458.md) | medium |  |  | 0 |
| [bigcodebench_461](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_461.md) | medium |  |  | 0 |
| [bigcodebench_477](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_477.md) | medium |  |  | 0 |
| [bigcodebench_486](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_486.md) | medium |  |  | 0 |
| [bigcodebench_492](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_492.md) | medium |  |  | 0 |
| [bigcodebench_501](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_501.md) | medium |  |  | 0 |
| [bigcodebench_502](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_502.md) | medium |  |  | 0 |
| [bigcodebench_503](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_503.md) | medium |  |  | 0 |
| [bigcodebench_509](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_509.md) | medium |  |  | 0 |
| [bigcodebench_511](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_511.md) | medium |  |  | 0 |
| [bigcodebench_513](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_513.md) | medium |  |  | 0 |
| [bigcodebench_526](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_526.md) | medium |  |  | 0 |
| [bigcodebench_528](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_528.md) | medium |  |  | 0 |
| [bigcodebench_530](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_530.md) | medium |  |  | 0 |
| [bigcodebench_532](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_532.md) | medium |  |  | 0 |
| [bigcodebench_553](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_553.md) | medium |  |  | 0 |
| [bigcodebench_560](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_560.md) | medium |  |  | 0 |
| [bigcodebench_567](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_567.md) | medium |  |  | 0 |
| [bigcodebench_574](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_574.md) | medium |  |  | 0 |
| [bigcodebench_579](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_579.md) | medium |  |  | 0 |
| [bigcodebench_582](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_582.md) | medium |  |  | 0 |
| [bigcodebench_583](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_583.md) | medium |  |  | 0 |
| [bigcodebench_587](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_587.md) | medium |  |  | 0 |
| [bigcodebench_592](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_592.md) | medium |  |  | 0 |
| [bigcodebench_594](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_594.md) | medium |  |  | 0 |
| [bigcodebench_618](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_618.md) | medium |  |  | 0 |
| [bigcodebench_636](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_636.md) | medium |  |  | 0 |
| [bigcodebench_637](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_637.md) | medium |  |  | 0 |
| [bigcodebench_654](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_654.md) | medium |  |  | 0 |
| [bigcodebench_655](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_655.md) | medium |  |  | 0 |
| [bigcodebench_657](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_657.md) | medium |  |  | 0 |
| [bigcodebench_678](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_678.md) | medium |  |  | 0 |
| [bigcodebench_720](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_720.md) | medium |  |  | 0 |
| [bigcodebench_723](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_723.md) | medium |  |  | 0 |
| [bigcodebench_752](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_752.md) | medium |  |  | 0 |
| [bigcodebench_760](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_760.md) | medium |  |  | 0 |
| [bigcodebench_763](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_763.md) | medium |  |  | 0 |
| [bigcodebench_765](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_765.md) | medium |  |  | 0 |
| [bigcodebench_771](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_771.md) | medium |  |  | 0 |
| [bigcodebench_777](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_777.md) | medium |  |  | 0 |
| [bigcodebench_785](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_785.md) | medium |  |  | 0 |
| [bigcodebench_800](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_800.md) | medium |  |  | 0 |
| [bigcodebench_82](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_82.md) | medium |  |  | 0 |
| [bigcodebench_826](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_826.md) | medium |  |  | 0 |
| [bigcodebench_845](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_845.md) | medium |  |  | 0 |
| [bigcodebench_854](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_854.md) | medium |  |  | 0 |
| [bigcodebench_857](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_857.md) | medium |  |  | 0 |
| [bigcodebench_865](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_865.md) | medium |  |  | 0 |
| [bigcodebench_870](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_870.md) | medium |  |  | 0 |
| [bigcodebench_879](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_879.md) | medium |  |  | 0 |
| [bigcodebench_89](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_89.md) | medium |  |  | 0 |
| [bigcodebench_897](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_897.md) | medium |  |  | 0 |
| [bigcodebench_906](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_906.md) | medium |  |  | 0 |
| [bigcodebench_914](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_914.md) | medium |  |  | 0 |
| [bigcodebench_915](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_915.md) | medium |  |  | 0 |
| [bigcodebench_916](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_916.md) | medium |  |  | 0 |
| [bigcodebench_917](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_917.md) | medium |  |  | 0 |
| [bigcodebench_92](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_92.md) | medium |  |  | 0 |
| [bigcodebench_928](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_928.md) | medium |  |  | 0 |
| [bigcodebench_93](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_93.md) | medium |  |  | 0 |
| [bigcodebench_942](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_942.md) | medium |  |  | 0 |
| [bigcodebench_945](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_945.md) | medium |  |  | 0 |
| [bigcodebench_952](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_952.md) | medium |  |  | 0 |
| [bigcodebench_955](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_955.md) | medium |  |  | 0 |
| [bigcodebench_963](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_963.md) | medium |  |  | 0 |
| [bigcodebench_964](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_964.md) | medium |  |  | 0 |
| [bigcodebench_969](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_969.md) | medium |  |  | 0 |
| [bigcodebench_971](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_971.md) | medium |  |  | 0 |
| [bigcodebench_985](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_985.md) | medium |  |  | 0 |
| [bigcodebench_988](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_988.md) | medium |  |  | 0 |
| [bigcodebench_99](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_99.md) | medium |  |  | 0 |
| [bigcodebench_990](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_990.md) | medium |  |  | 0 |
| [bigcodebench_998](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_998.md) | medium |  |  | 0 |
| [bigcodebench_999](https://harnessreport.com/tasks/bigcodebench_hard_complete/bigcodebench_999.md) | medium |  |  | 0 |

---
Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt · MCP: https://harnessreport.com/mcp