# swegym / pandas-dev__pandas-51854 - taskset: [swegym](https://harnessreport.com/tasks/swegym.md) - difficulty: hard - category: debugging - language: - runnable from the site: no - agent timeout: 3000s ## Results by harness _none yet_ ## Instruction ``` BUG: Pandas 2.0rc0 can't handle string column with PyArrow (large_string error) ### Pandas version checks - [X] I have checked that this issue has not already been reported. - [X] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas. - [ ] I have confirmed this bug exists on the [main branch](https://pandas.pydata.org/docs/dev/getting_started/install.html#installing-the-development-version-of-pandas) of pandas. ### Reproducible Example ```python import polars as pl from datetime import datetime dataframe = pl.DataFrame({"integer": [1, 2, 3], "date": [ (datetime(2022, 1, 1)), (datetime(2022, 1, 2)), (datetime(2022, 1, 3)) ], "float":[4.0, 5.0, 6.0], "strings": ["first", "second", "third"] }) df_pd_pyarrow = dataframe.to_pandas(use_pyarrow_extension_array=True) #ERROR *** NotImplementedError: large_string *** ``` ### Issue Description Hi guys! Polars is another tool to work with dataframes. When converting to Pandas, one dataframe in PyArrow (generated by Polars), which has one column with string datatype, I got this error: `NotImplementedError: large_string` This only occurs in Pandas 2.0rc0. With Pandas 1.5.3 it works without error. [This not appears to be a Polars' bug, as whats was tested here](https://github.com/pola-rs/polars/issues/7431) ### Expected Behavior It's supposed to work, as its works with v 1.5.3 ### Installed Versions <details> ``` pandas.show_versions() INSTALLED VERSIONS #------------------ commit : 1a2e300170efc08cb509a0b4ff6248f8d55ae777 python : 3.10.6.final.0 python-bits : 64 OS : Linux OS-release : 5.10.16.3-microsoft-standard-WSL2 Version : #1 SMP Fri Apr 2 22:23:49 UTC 2021 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : C.UTF-8 LOCALE : en_US.UTF-8 pandas : 2.0.0rc0 numpy : 1.24.2 pytz : 2022.7.1 dateutil : 2.8.2 setuptools : 59.6.0 pip : 22.0.2 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : 8.11.0 pandas_datareader: None bs4 : 4.11.2 bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : 11.0.0 pyreadstat : None pyxlsb : None s3fs : None scipy : None snappy : None sqlalchemy : 2.0.5.post1 tables : None tabulate : None xarray : None xlrd : None zstandard : None tzdata : None qtpy : None pyqt5 : None ``` </details> ``` --- Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt · MCP: https://harnessreport.com/mcp