# devopsgym / codegen__prometheus__prometheus-5759 - taskset: [devopsgym](https://harnessreport.com/tasks/devopsgym.md) - difficulty: hard - category: code-generation - language: - runnable from the site: no - agent timeout: 3000s ## Results by harness _none yet_ ## Instruction ``` This is a code generation task. You are expected to write working code that solves the described problem. <issue> ## Bug Report **What did you do?** We use recording rules to create metrics with information about system. Something like location and status to be able to pass this information to alertmanager. This information is updated through an automated process and it reloads prometheus. After the update to version 2.10 we see the metric with old and new labels. e.g. ``` asset_info{instance="test1",status="a",location="a"} asset_info{instance="test1",status="b",location="a"} ``` In version 2.8.0 there was only one metric after the prometheus reload. The query we use to trigger this looks like this. ``` up{exported_instance="i"} * on(exported_instance) group_left(status, location) asset_info ``` **What did you expect to see?** After reload the asset information is updated and not reported twice. **What did you see instead? Under which circumstances?** The asset metric is shown twice instead of one after an update of a value. **Environment** * Prometheus version: ``` # prometheus --version prometheus, version 2.10.0 (branch: HEAD, revision: d20e84d0fb64aff2f62a977adc8cfb656da4e286) build user: root@a49185acd9b0 build date: 20190525-12:28:13 go version: go1.12.5 ``` * Testcase ``` #!/bin/bash function cleanup() { kill $WEB kill $PROMETHEUS } trap cleanup SIGINT SIGTERM TMP=$(mktemp -d) echo "changing to $TMP" cd $TMP echo 'up{instance="i"} 1' > metrics cat > prometheus.yml <<EOF global: scrape_interval: 10s scrape_timeout: 5s evaluation_interval: 10s external_labels: prometheus_scraper: test rule_files: - rules.yml scrape_configs: - job_name: 'example1' metrics_path: '/metrics' static_configs: - targets: [127.0.0.1:8000] EOF cat > rules.yml <<EOF groups: - name: asset_info interval: 10s rules: - record: asset_info expr: 1 labels: exported_instance: "i" asset_status: "test" asset_location: "test" testlabel: "a" - name: linux interval: 10s rules: - alert: TestAlert expr: up{exported_instance="i"} * on(exported_instance) group_left(asset_status, asset_location) asset_info for: 2m EOF python -m SimpleHTTPServer & WEB=$! prometheus & PROMETHEUS=$! sleep 15 sed -e 's/testlabel: ".*"/testlabel: "'$RANDOM'"/' -i rules.yml kill -SIGHUP $PROMETHEUS wait ``` If you run this, you will see evaluation errors after some seconds. **Affected version** I have bisected it and found that change e15d8c5802f98b6a508022500acf44e91adc7d87 is the culprit here. It is also the case in master branch. </issue> Focus on implementing the required functionality correctly and efficiently. Treat this as a programming challenge. You are not allowed to read git history. ``` --- Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt · MCP: https://harnessreport.com/mcp