{"task": {"agent_timeout": 3000, "task": "codegen__prometheus__prometheus-5759", "verifier_timeout": 3000, "instruction": "This is a code generation task. You are expected to write working code that solves the described problem.\n<issue>\n      ## Bug Report\n**What did you do?**\nWe use recording rules to create metrics with information about system. Something like location and status to be able to pass this information to alertmanager. This information is updated through an automated process and it reloads prometheus.\nAfter the update to version 2.10 we see the metric with old and new labels.\n\ne.g.\n```\nasset_info{instance=\"test1\",status=\"a\",location=\"a\"}\nasset_info{instance=\"test1\",status=\"b\",location=\"a\"}\n```\n\nIn version 2.8.0 there was only one metric after the prometheus reload.\n\nThe query we use to trigger this looks like this.\n```\nup{exported_instance=\"i\"} * on(exported_instance) group_left(status, location) asset_info\n```\n\n**What did you expect to see?**\nAfter reload the asset information is updated and not reported twice.\n\n**What did you see instead? Under which circumstances?**\nThe asset metric is shown twice instead of one after an update of a value.\n\n**Environment**\n\n* Prometheus version:\n```\n# prometheus --version\nprometheus, version 2.10.0 (branch: HEAD, revision: d20e84d0fb64aff2f62a977adc8cfb656da4e286)\n  build user:       root@a49185acd9b0\n  build date:       20190525-12:28:13\n  go version:       go1.12.5\n```\n\n* Testcase\n```\n#!/bin/bash\n\nfunction cleanup() {\n    kill $WEB\n    kill $PROMETHEUS\n}\n\ntrap cleanup SIGINT SIGTERM\n\nTMP=$(mktemp -d)\n\necho \"changing to $TMP\"\n\ncd $TMP\necho 'up{instance=\"i\"} 1' > metrics\n\n\ncat > prometheus.yml <<EOF\nglobal:\n  scrape_interval: 10s\n  scrape_timeout: 5s\n  evaluation_interval: 10s\n  external_labels:\n    prometheus_scraper: test\n\nrule_files:\n  - rules.yml\n\nscrape_configs:\n  - job_name: 'example1'\n    metrics_path: '/metrics'\n    static_configs:\n      - targets: [127.0.0.1:8000]\nEOF\n\ncat > rules.yml <<EOF\ngroups:\n  - name: asset_info\n    interval: 10s\n    rules:\n      - record: asset_info\n        expr: 1\n        labels:\n          exported_instance: \"i\"\n          asset_status: \"test\"\n          asset_location: \"test\"\n          testlabel: \"a\"\n\n  - name: linux\n    interval: 10s\n    rules:\n      - alert: TestAlert\n        expr: up{exported_instance=\"i\"} * on(exported_instance) group_left(asset_status, asset_location) asset_info\n        for: 2m\nEOF\n\npython -m SimpleHTTPServer &\nWEB=$!\nprometheus &\nPROMETHEUS=$!\n\nsleep 15\n\nsed -e 's/testlabel: \".*\"/testlabel: \"'$RANDOM'\"/' -i rules.yml\nkill -SIGHUP $PROMETHEUS\n\nwait\n```\n\nIf you run this, you will see evaluation errors after some seconds.\n\n**Affected version**\nI have bisected it and found that change e15d8c5802f98b6a508022500acf44e91adc7d87 is the culprit here. It is also the case in master branch.\n\n</issue>\nFocus on implementing the required functionality correctly and efficiently. Treat this as a programming challenge.\nYou are not allowed to read git history.\n", "memory": "8192m", "runnable": false, "difficulty": "hard", "language": "", "cpus": "", "instruction_truncated": false, "category": "code-generation", "compose": false, "has_solution": true, "oracle": null, "docker_image": "", "taskset": "devopsgym", "tags": ["code-generation", "devops-bench"]}, "runs": []}