# swe-lancer - domain: swe - n_tasks: 463 - n_runnable: 0 - owner_org: OpenAI - grading: tests - task_kind: IC SWE patches + SWE-manager proposal selection, priced in USD - url_repo: https://github.com/openai/SWELancer-Benchmark - url_paper: https://arxiv.org/abs/2502.12115 ## Tasks | task | difficulty | language | runnable | harnesses tried | |---|---|---|---|---| | [12155-manager-0](https://harnessreport.com/tasks/swe-lancer/12155-manager-0.md) | hard | | | 0 | | [12155_1](https://harnessreport.com/tasks/swe-lancer/12155_1.md) | hard | | | 0 | | [13006-manager-0](https://harnessreport.com/tasks/swe-lancer/13006-manager-0.md) | hard | | | 0 | | [13066_1](https://harnessreport.com/tasks/swe-lancer/13066_1.md) | hard | | | 0 | | [14127](https://harnessreport.com/tasks/swe-lancer/14127.md) | hard | | | 0 | | [14223_1](https://harnessreport.com/tasks/swe-lancer/14223_1.md) | hard | | | 0 | | [14268](https://harnessreport.com/tasks/swe-lancer/14268.md) | hard | | | 0 | | [14268-manager-0](https://harnessreport.com/tasks/swe-lancer/14268-manager-0.md) | hard | | | 0 | | [14294](https://harnessreport.com/tasks/swe-lancer/14294.md) | hard | | | 0 | | [14294-manager-0](https://harnessreport.com/tasks/swe-lancer/14294-manager-0.md) | hard | | | 0 | | [14340](https://harnessreport.com/tasks/swe-lancer/14340.md) | hard | | | 0 | | [14857](https://harnessreport.com/tasks/swe-lancer/14857.md) | hard | | | 0 | | [14958](https://harnessreport.com/tasks/swe-lancer/14958.md) | hard | | | 0 | | [14958-manager-0](https://harnessreport.com/tasks/swe-lancer/14958-manager-0.md) | hard | | | 0 | | [15048-manager-0](https://harnessreport.com/tasks/swe-lancer/15048-manager-0.md) | hard | | | 0 | | [15108](https://harnessreport.com/tasks/swe-lancer/15108.md) | hard | | | 0 | | [15193](https://harnessreport.com/tasks/swe-lancer/15193.md) | hard | | | 0 | | [15193-manager-0](https://harnessreport.com/tasks/swe-lancer/15193-manager-0.md) | hard | | | 0 | | [15289-manager-0](https://harnessreport.com/tasks/swe-lancer/15289-manager-0.md) | hard | | | 0 | | [15289_1](https://harnessreport.com/tasks/swe-lancer/15289_1.md) | hard | | | 0 | | [15321](https://harnessreport.com/tasks/swe-lancer/15321.md) | hard | | | 0 | | [15784-manager-0](https://harnessreport.com/tasks/swe-lancer/15784-manager-0.md) | hard | | | 0 | | [15815-manager-0](https://harnessreport.com/tasks/swe-lancer/15815-manager-0.md) | hard | | | 0 | | [15815_1](https://harnessreport.com/tasks/swe-lancer/15815_1.md) | hard | | | 0 | | [15925](https://harnessreport.com/tasks/swe-lancer/15925.md) | hard | | | 0 | | [16912-manager-0](https://harnessreport.com/tasks/swe-lancer/16912-manager-0.md) | hard | | | 0 | | [16912_4](https://harnessreport.com/tasks/swe-lancer/16912_4.md) | hard | | | 0 | | [16921-manager-0](https://harnessreport.com/tasks/swe-lancer/16921-manager-0.md) | hard | | | 0 | | [16921_1](https://harnessreport.com/tasks/swe-lancer/16921_1.md) | hard | | | 0 | | [16935-manager-0](https://harnessreport.com/tasks/swe-lancer/16935-manager-0.md) | hard | | | 0 | | [16946-manager-0](https://harnessreport.com/tasks/swe-lancer/16946-manager-0.md) | hard | | | 0 | | [16977-manager-0](https://harnessreport.com/tasks/swe-lancer/16977-manager-0.md) | hard | | | 0 | | [16985-manager-0](https://harnessreport.com/tasks/swe-lancer/16985-manager-0.md) | hard | | | 0 | | [17073-manager-0](https://harnessreport.com/tasks/swe-lancer/17073-manager-0.md) | hard | | | 0 | | [17092-manager-0](https://harnessreport.com/tasks/swe-lancer/17092-manager-0.md) | hard | | | 0 | | [17155-manager-0](https://harnessreport.com/tasks/swe-lancer/17155-manager-0.md) | hard | | | 0 | | [17163-manager-0](https://harnessreport.com/tasks/swe-lancer/17163-manager-0.md) | hard | | | 0 | | [17166-manager-0](https://harnessreport.com/tasks/swe-lancer/17166-manager-0.md) | hard | | | 0 | | [17187-manager-0](https://harnessreport.com/tasks/swe-lancer/17187-manager-0.md) | hard | | | 0 | | [17387-manager-0](https://harnessreport.com/tasks/swe-lancer/17387-manager-0.md) | hard | | | 0 | | [17387_1](https://harnessreport.com/tasks/swe-lancer/17387_1.md) | hard | | | 0 | | [18157-manager-0](https://harnessreport.com/tasks/swe-lancer/18157-manager-0.md) | hard | | | 0 | | [18157_893](https://harnessreport.com/tasks/swe-lancer/18157_893.md) | hard | | | 0 | | [18207-manager-0](https://harnessreport.com/tasks/swe-lancer/18207-manager-0.md) | hard | | | 0 | | [18207_923](https://harnessreport.com/tasks/swe-lancer/18207_923.md) | hard | | | 0 | | [18218-manager-0](https://harnessreport.com/tasks/swe-lancer/18218-manager-0.md) | hard | | | 0 | | [18230_905](https://harnessreport.com/tasks/swe-lancer/18230_905.md) | hard | | | 0 | | [18244-manager-0](https://harnessreport.com/tasks/swe-lancer/18244-manager-0.md) | hard | | | 0 | | [18300-manager-0](https://harnessreport.com/tasks/swe-lancer/18300-manager-0.md) | hard | | | 0 | | [18311-manager-0](https://harnessreport.com/tasks/swe-lancer/18311-manager-0.md) | hard | | | 0 | | [18434-manager-0](https://harnessreport.com/tasks/swe-lancer/18434-manager-0.md) | hard | | | 0 | | [18447-manager-0](https://harnessreport.com/tasks/swe-lancer/18447-manager-0.md) | hard | | | 0 | | [18535-manager-0](https://harnessreport.com/tasks/swe-lancer/18535-manager-0.md) | hard | | | 0 | | [18569-manager-0](https://harnessreport.com/tasks/swe-lancer/18569-manager-0.md) | hard | | | 0 | | [18641_837](https://harnessreport.com/tasks/swe-lancer/18641_837.md) | hard | | | 0 | | [18647-manager-0](https://harnessreport.com/tasks/swe-lancer/18647-manager-0.md) | hard | | | 0 | | [18647_838](https://harnessreport.com/tasks/swe-lancer/18647_838.md) | hard | | | 0 | | [18658-manager-0](https://harnessreport.com/tasks/swe-lancer/18658-manager-0.md) | hard | | | 0 | | [18659_831](https://harnessreport.com/tasks/swe-lancer/18659_831.md) | hard | | | 0 | | [18660_869](https://harnessreport.com/tasks/swe-lancer/18660_869.md) | hard | | | 0 | | [18667-manager-0](https://harnessreport.com/tasks/swe-lancer/18667-manager-0.md) | hard | | | 0 | | [18681-manager-0](https://harnessreport.com/tasks/swe-lancer/18681-manager-0.md) | hard | | | 0 | | [18688-manager-0](https://harnessreport.com/tasks/swe-lancer/18688-manager-0.md) | hard | | | 0 | | [18736-manager-0](https://harnessreport.com/tasks/swe-lancer/18736-manager-0.md) | hard | | | 0 | | [18742-manager-0](https://harnessreport.com/tasks/swe-lancer/18742-manager-0.md) | hard | | | 0 | | [18746_833](https://harnessreport.com/tasks/swe-lancer/18746_833.md) | hard | | | 0 | | [18796-manager-0](https://harnessreport.com/tasks/swe-lancer/18796-manager-0.md) | hard | | | 0 | | [18813-manager-0](https://harnessreport.com/tasks/swe-lancer/18813-manager-0.md) | hard | | | 0 | | [18827-manager-0](https://harnessreport.com/tasks/swe-lancer/18827-manager-0.md) | hard | | | 0 | | [18827_741](https://harnessreport.com/tasks/swe-lancer/18827_741.md) | hard | | | 0 | | [18836-manager-0](https://harnessreport.com/tasks/swe-lancer/18836-manager-0.md) | hard | | | 0 | | [18891_881](https://harnessreport.com/tasks/swe-lancer/18891_881.md) | hard | | | 0 | | [18909-manager-0](https://harnessreport.com/tasks/swe-lancer/18909-manager-0.md) | hard | | | 0 | | [18909_734](https://harnessreport.com/tasks/swe-lancer/18909_734.md) | hard | | | 0 | | [18910-manager-0](https://harnessreport.com/tasks/swe-lancer/18910-manager-0.md) | hard | | | 0 | | [18910_733](https://harnessreport.com/tasks/swe-lancer/18910_733.md) | hard | | | 0 | | [18911-manager-0](https://harnessreport.com/tasks/swe-lancer/18911-manager-0.md) | hard | | | 0 | | [18983-manager-0](https://harnessreport.com/tasks/swe-lancer/18983-manager-0.md) | hard | | | 0 | | [19022-manager-0](https://harnessreport.com/tasks/swe-lancer/19022-manager-0.md) | hard | | | 0 | | [19022_888](https://harnessreport.com/tasks/swe-lancer/19022_888.md) | hard | | | 0 | | [19023_868](https://harnessreport.com/tasks/swe-lancer/19023_868.md) | hard | | | 0 | | [19048-manager-0](https://harnessreport.com/tasks/swe-lancer/19048-manager-0.md) | hard | | | 0 | | [19073-manager-0](https://harnessreport.com/tasks/swe-lancer/19073-manager-0.md) | hard | | | 0 | | [19132-manager-0](https://harnessreport.com/tasks/swe-lancer/19132-manager-0.md) | hard | | | 0 | | [19132_872](https://harnessreport.com/tasks/swe-lancer/19132_872.md) | hard | | | 0 | | [19135-manager-0](https://harnessreport.com/tasks/swe-lancer/19135-manager-0.md) | hard | | | 0 | | [19135_873](https://harnessreport.com/tasks/swe-lancer/19135_873.md) | hard | | | 0 | | [19167-manager-0](https://harnessreport.com/tasks/swe-lancer/19167-manager-0.md) | hard | | | 0 | | [19232-manager-0](https://harnessreport.com/tasks/swe-lancer/19232-manager-0.md) | hard | | | 0 | | [19289-manager-0](https://harnessreport.com/tasks/swe-lancer/19289-manager-0.md) | hard | | | 0 | | [19301-manager-0](https://harnessreport.com/tasks/swe-lancer/19301-manager-0.md) | hard | | | 0 | | [19359-manager-0](https://harnessreport.com/tasks/swe-lancer/19359-manager-0.md) | hard | | | 0 | | [19362_850](https://harnessreport.com/tasks/swe-lancer/19362_850.md) | hard | | | 0 | | [19364-manager-0](https://harnessreport.com/tasks/swe-lancer/19364-manager-0.md) | hard | | | 0 | | [19366-manager-0](https://harnessreport.com/tasks/swe-lancer/19366-manager-0.md) | hard | | | 0 | | [19398-manager-0](https://harnessreport.com/tasks/swe-lancer/19398-manager-0.md) | hard | | | 0 | | [19398_889](https://harnessreport.com/tasks/swe-lancer/19398_889.md) | hard | | | 0 | | [20079](https://harnessreport.com/tasks/swe-lancer/20079.md) | hard | | | 0 | | [20569-manager-0](https://harnessreport.com/tasks/swe-lancer/20569-manager-0.md) | hard | | | 0 | | [20569_1122](https://harnessreport.com/tasks/swe-lancer/20569_1122.md) | hard | | | 0 | | [20589-manager-0](https://harnessreport.com/tasks/swe-lancer/20589-manager-0.md) | hard | | | 0 | | [20608-manager-0](https://harnessreport.com/tasks/swe-lancer/20608-manager-0.md) | hard | | | 0 | | [20672-manager-0](https://harnessreport.com/tasks/swe-lancer/20672-manager-0.md) | hard | | | 0 | | [20672_1019](https://harnessreport.com/tasks/swe-lancer/20672_1019.md) | hard | | | 0 | | [20675-manager-0](https://harnessreport.com/tasks/swe-lancer/20675-manager-0.md) | hard | | | 0 | | [20703_962](https://harnessreport.com/tasks/swe-lancer/20703_962.md) | hard | | | 0 | | [22624](https://harnessreport.com/tasks/swe-lancer/22624.md) | hard | | | 0 | | [24145-manager-0](https://harnessreport.com/tasks/swe-lancer/24145-manager-0.md) | hard | | | 0 | | [24145_783](https://harnessreport.com/tasks/swe-lancer/24145_783.md) | hard | | | 0 | | [24155-manager-0](https://harnessreport.com/tasks/swe-lancer/24155-manager-0.md) | hard | | | 0 | | [24156-manager-0](https://harnessreport.com/tasks/swe-lancer/24156-manager-0.md) | hard | | | 0 | | [24157-manager-0](https://harnessreport.com/tasks/swe-lancer/24157-manager-0.md) | hard | | | 0 | | [24237-manager-0](https://harnessreport.com/tasks/swe-lancer/24237-manager-0.md) | hard | | | 0 | | [24261-manager-0](https://harnessreport.com/tasks/swe-lancer/24261-manager-0.md) | hard | | | 0 | | [24263-manager-0](https://harnessreport.com/tasks/swe-lancer/24263-manager-0.md) | hard | | | 0 | | [24366-manager-0](https://harnessreport.com/tasks/swe-lancer/24366-manager-0.md) | hard | | | 0 | | [24368-manager-0](https://harnessreport.com/tasks/swe-lancer/24368-manager-0.md) | hard | | | 0 | | [24393-manager-0](https://harnessreport.com/tasks/swe-lancer/24393-manager-0.md) | hard | | | 0 | | [24476-manager-0](https://harnessreport.com/tasks/swe-lancer/24476-manager-0.md) | hard | | | 0 | | [24508-manager-0](https://harnessreport.com/tasks/swe-lancer/24508-manager-0.md) | hard | | | 0 | | [24508_791](https://harnessreport.com/tasks/swe-lancer/24508_791.md) | hard | | | 0 | | [24522-manager-0](https://harnessreport.com/tasks/swe-lancer/24522-manager-0.md) | hard | | | 0 | | [24642-manager-0](https://harnessreport.com/tasks/swe-lancer/24642-manager-0.md) | hard | | | 0 | | [24654-manager-0](https://harnessreport.com/tasks/swe-lancer/24654-manager-0.md) | hard | | | 0 | | [25405-manager-0](https://harnessreport.com/tasks/swe-lancer/25405-manager-0.md) | hard | | | 0 | | [25448-manager-0](https://harnessreport.com/tasks/swe-lancer/25448-manager-0.md) | hard | | | 0 | | [25491-manager-0](https://harnessreport.com/tasks/swe-lancer/25491-manager-0.md) | hard | | | 0 | | [25495-manager-0](https://harnessreport.com/tasks/swe-lancer/25495-manager-0.md) | hard | | | 0 | | [25499-manager-0](https://harnessreport.com/tasks/swe-lancer/25499-manager-0.md) | hard | | | 0 | | [25527-manager-0](https://harnessreport.com/tasks/swe-lancer/25527-manager-0.md) | hard | | | 0 | | [25550-manager-0](https://harnessreport.com/tasks/swe-lancer/25550-manager-0.md) | hard | | | 0 | | [25562-manager-0](https://harnessreport.com/tasks/swe-lancer/25562-manager-0.md) | hard | | | 0 | | [25570-manager-0](https://harnessreport.com/tasks/swe-lancer/25570-manager-0.md) | hard | | | 0 | | [25570_780](https://harnessreport.com/tasks/swe-lancer/25570_780.md) | hard | | | 0 | | [25577-manager-0](https://harnessreport.com/tasks/swe-lancer/25577-manager-0.md) | hard | | | 0 | | [25582-manager-0](https://harnessreport.com/tasks/swe-lancer/25582-manager-0.md) | hard | | | 0 | | [25589-manager-0](https://harnessreport.com/tasks/swe-lancer/25589-manager-0.md) | hard | | | 0 | | [25595-manager-0](https://harnessreport.com/tasks/swe-lancer/25595-manager-0.md) | hard | | | 0 | | [25640-manager-0](https://harnessreport.com/tasks/swe-lancer/25640-manager-0.md) | hard | | | 0 | | [25640_655](https://harnessreport.com/tasks/swe-lancer/25640_655.md) | hard | | | 0 | | [25646-manager-0](https://harnessreport.com/tasks/swe-lancer/25646-manager-0.md) | hard | | | 0 | | [25693-manager-0](https://harnessreport.com/tasks/swe-lancer/25693-manager-0.md) | hard | | | 0 | | [25859-manager-0](https://harnessreport.com/tasks/swe-lancer/25859-manager-0.md) | hard | | | 0 | | [25901-manager-0](https://harnessreport.com/tasks/swe-lancer/25901-manager-0.md) | hard | | | 0 | | [25901_945](https://harnessreport.com/tasks/swe-lancer/25901_945.md) | hard | | | 0 | | [25929-manager-0](https://harnessreport.com/tasks/swe-lancer/25929-manager-0.md) | hard | | | 0 | | [25932-manager-0](https://harnessreport.com/tasks/swe-lancer/25932-manager-0.md) | hard | | | 0 | | [25975-manager-0](https://harnessreport.com/tasks/swe-lancer/25975-manager-0.md) | hard | | | 0 | | [26009-manager-0](https://harnessreport.com/tasks/swe-lancer/26009-manager-0.md) | hard | | | 0 | | [26015-manager-0](https://harnessreport.com/tasks/swe-lancer/26015-manager-0.md) | hard | | | 0 | | [26048-manager-0](https://harnessreport.com/tasks/swe-lancer/26048-manager-0.md) | hard | | | 0 | | [26120-manager-0](https://harnessreport.com/tasks/swe-lancer/26120-manager-0.md) | hard | | | 0 | | [26161_924](https://harnessreport.com/tasks/swe-lancer/26161_924.md) | hard | | | 0 | | [26167-manager-0](https://harnessreport.com/tasks/swe-lancer/26167-manager-0.md) | hard | | | 0 | | [26209-manager-0](https://harnessreport.com/tasks/swe-lancer/26209-manager-0.md) | hard | | | 0 | | [26210-manager-0](https://harnessreport.com/tasks/swe-lancer/26210-manager-0.md) | hard | | | 0 | | [26228-manager-0](https://harnessreport.com/tasks/swe-lancer/26228-manager-0.md) | hard | | | 0 | | [26228_719](https://harnessreport.com/tasks/swe-lancer/26228_719.md) | hard | | | 0 | | [26258-manager-0](https://harnessreport.com/tasks/swe-lancer/26258-manager-0.md) | hard | | | 0 | | [26258_492](https://harnessreport.com/tasks/swe-lancer/26258_492.md) | hard | | | 0 | | [26648-manager-0](https://harnessreport.com/tasks/swe-lancer/26648-manager-0.md) | hard | | | 0 | | [26648_972](https://harnessreport.com/tasks/swe-lancer/26648_972.md) | hard | | | 0 | | [27029-manager-0](https://harnessreport.com/tasks/swe-lancer/27029-manager-0.md) | hard | | | 0 | | [27029_876](https://harnessreport.com/tasks/swe-lancer/27029_876.md) | hard | | | 0 | | [27092-manager-0](https://harnessreport.com/tasks/swe-lancer/27092-manager-0.md) | hard | | | 0 | | [27106-manager-0](https://harnessreport.com/tasks/swe-lancer/27106-manager-0.md) | hard | | | 0 | | [27107-manager-0](https://harnessreport.com/tasks/swe-lancer/27107-manager-0.md) | hard | | | 0 | | [27114-manager-0](https://harnessreport.com/tasks/swe-lancer/27114-manager-0.md) | hard | | | 0 | | [27153-manager-0](https://harnessreport.com/tasks/swe-lancer/27153-manager-0.md) | hard | | | 0 | | [27156-manager-0](https://harnessreport.com/tasks/swe-lancer/27156-manager-0.md) | hard | | | 0 | | [27156_665](https://harnessreport.com/tasks/swe-lancer/27156_665.md) | hard | | | 0 | | [27226_729](https://harnessreport.com/tasks/swe-lancer/27226_729.md) | hard | | | 0 | | [27243-manager-0](https://harnessreport.com/tasks/swe-lancer/27243-manager-0.md) | hard | | | 0 | | [27243_705](https://harnessreport.com/tasks/swe-lancer/27243_705.md) | hard | | | 0 | | [27353_776](https://harnessreport.com/tasks/swe-lancer/27353_776.md) | hard | | | 0 | | [27363_760](https://harnessreport.com/tasks/swe-lancer/27363_760.md) | hard | | | 0 | | [27538-manager-0](https://harnessreport.com/tasks/swe-lancer/27538-manager-0.md) | hard | | | 0 | | [27538_794](https://harnessreport.com/tasks/swe-lancer/27538_794.md) | hard | | | 0 | | [28019-manager-0](https://harnessreport.com/tasks/swe-lancer/28019-manager-0.md) | hard | | | 0 | | [28030-manager-0](https://harnessreport.com/tasks/swe-lancer/28030-manager-0.md) | hard | | | 0 | | [28030_832](https://harnessreport.com/tasks/swe-lancer/28030_832.md) | hard | | | 0 | | [28059_803](https://harnessreport.com/tasks/swe-lancer/28059_803.md) | hard | | | 0 | | [28077-manager-0](https://harnessreport.com/tasks/swe-lancer/28077-manager-0.md) | hard | | | 0 | | [28077_782](https://harnessreport.com/tasks/swe-lancer/28077_782.md) | hard | | | 0 | | [28087-manager-0](https://harnessreport.com/tasks/swe-lancer/28087-manager-0.md) | hard | | | 0 | | [28087_800](https://harnessreport.com/tasks/swe-lancer/28087_800.md) | hard | | | 0 | | [28093-manager-0](https://harnessreport.com/tasks/swe-lancer/28093-manager-0.md) | hard | | | 0 | | [28093_799](https://harnessreport.com/tasks/swe-lancer/28093_799.md) | hard | | | 0 | | [28096_836](https://harnessreport.com/tasks/swe-lancer/28096_836.md) | hard | | | 0 | | [28132-manager-0](https://harnessreport.com/tasks/swe-lancer/28132-manager-0.md) | hard | | | 0 | | [28148_414](https://harnessreport.com/tasks/swe-lancer/28148_414.md) | hard | | | 0 | | [28153-manager-0](https://harnessreport.com/tasks/swe-lancer/28153-manager-0.md) | hard | | | 0 | | [28153_817](https://harnessreport.com/tasks/swe-lancer/28153_817.md) | hard | | | 0 | | [28170-manager-0](https://harnessreport.com/tasks/swe-lancer/28170-manager-0.md) | hard | | | 0 | | [28250-manager-0](https://harnessreport.com/tasks/swe-lancer/28250-manager-0.md) | hard | | | 0 | | [28250_990](https://harnessreport.com/tasks/swe-lancer/28250_990.md) | hard | | | 0 | | [28265-manager-0](https://harnessreport.com/tasks/swe-lancer/28265-manager-0.md) | hard | | | 0 | | [28265_616](https://harnessreport.com/tasks/swe-lancer/28265_616.md) | hard | | | 0 | | [28324-manager-0](https://harnessreport.com/tasks/swe-lancer/28324-manager-0.md) | hard | | | 0 | | [28324_808](https://harnessreport.com/tasks/swe-lancer/28324_808.md) | hard | | | 0 | More: https://harnessreport.com/tasks/swe-lancer.md?after=28324_808 --- Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt · MCP: https://harnessreport.com/mcp