# swegym / pandas-dev__pandas-56523

- taskset: [swegym](https://harnessreport.com/tasks/swegym.md)
- difficulty: hard
- category: debugging
- language: 
- runnable from the site: no
- agent timeout: 3000s

## Results by harness

_none yet_

## Instruction

```
PERF: merge_ordered(how="inner") without prior filtering is slow and uses much more memory.
### Pandas version checks

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this issue exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [ ] I have confirmed this issue exists on the main branch of pandas.


### Reproducible Example

```python
import pandas as pd
import numpy as np

start, stop, step = 0, 100_000_000, 2
df1 = pd.DataFrame({"key": np.arange(start, stop, step), "val1": np.arange(start, stop, step), "val2": np.arange(start, stop, step)}, copy=False)
df1.info()

start, stop, step = 50_000_000, 70_000_000, 1
df2 = pd.DataFrame({"key": np.arange(start, stop, step), "val3": np.arange(start, stop, step), "val4": np.arange(start, stop, step)}, copy=False)
df2.info()

# This filtering should not be necessary for inner join as it only drops unused data
df1 = df1.query("key >= @df2.key.min() and key <= @df2.key.max()")
df2 = df2.query("key in @df1.key")

df = pd.merge_ordered(df1, df2, on="key", how="inner")
df.info()
```

### Installed Versions

<details>

```
INSTALLED VERSIONS
------------------
commit              : 2a953cf80b77e4348bf50ed724f8abc0d814d9dd
python              : 3.10.9.final.0
python-bits         : 64
OS                  : Linux
OS-release          : 6.2.0-36-generic
Version             : #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2
machine             : x86_64
processor           : x86_64
byteorder           : little
LC_ALL              : None
LANG                : pl_PL.UTF-8
LOCALE              : pl_PL.UTF-8

pandas              : 2.1.3
numpy               : 1.26.2
pytz                : 2023.3.post1
dateutil            : 2.8.2
setuptools          : 65.5.0
pip                 : 22.3.1
Cython              : None
pytest              : None
hypothesis          : None
sphinx              : None
blosc               : None
feather             : None
xlsxwriter          : None
lxml.etree          : None
html5lib            : None
pymysql             : None
psycopg2            : None
jinja2              : None
IPython             : None
pandas_datareader   : None
bs4                 : None
bottleneck          : None
dataframe-api-compat: None
fastparquet         : None
fsspec              : None
gcsfs               : None
matplotlib          : 3.8.2
numba               : None
numexpr             : None
odfpy               : None
openpyxl            : None
pandas_gbq          : None
pyarrow             : None
pyreadstat          : None
pyxlsb              : None
s3fs                : None
scipy               : None
sqlalchemy          : None
tables              : None
tabulate            : None
xarray              : None
xlrd                : None
zstandard           : None
tzdata              : 2023.3
qtpy                : None
pyqt5               : None
```

</details>


### Prior Performance

![diff](https://github.com/pandas-dev/pandas/assets/8440636/530b6169-3892-44ea-bc79-5a25e067ccc0)
Plot generated by `memory_profiler`:
* black line: with 2 queries before `merge_ordered`
* blue line: without 2 queries before `merge_ordered`
```
---
Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt · MCP: https://harnessreport.com/mcp