# ds1000 / 931

- taskset: [ds1000](https://harnessreport.com/tasks/ds1000.md)
- difficulty: 
- category: 
- language: 
- runnable from the site: no
- agent timeout: 1800s

## Results by harness

_none yet_

## Instruction

```
# 931: DS-1000 Task

## Prompt
Problem:

I am using python and scikit-learn to find cosine similarity between item descriptions.

A have a df, for example:

items    description

1fgg     abcd ty
2hhj     abc r
3jkl     r df
I did following procedures:

1) tokenizing each description

2) transform the corpus into vector space using tf-idf

3) calculated cosine distance between each description text as a measure of similarity. distance = 1 - cosinesimilarity(tfidf_matrix)

My goal is to have a similarity matrix of items like this and answer the question like: "What is the similarity between the items 1ffg and 2hhj :

        1fgg    2hhj    3jkl
1ffg    1.0     0.8     0.1
2hhj    0.8     1.0     0.0
3jkl    0.1     0.0     1.0
How to get this result? Thank you for your time.

A:

<code>
import numpy as np
import pandas as pd
import sklearn
from sklearn.feature_extraction.text import TfidfVectorizer
df = load_data()
tfidf = TfidfVectorizer()
</code>
cosine_similarity_matrix = ... # put solution in this variable
BEGIN SOLUTION
<code>

## What to do
- Edit `solution/solution.py` so the code passes the DS-1000 tests.
- Do not access the internet or install new packages; required libraries are preinstalled in the Docker image.
- Run tests locally via `bash tests/test.sh`.

## Notes
- Keep the variable names/signatures implied by the prompt/code_context.
- The evaluator uses the original DS-1000 `code_context` (`test_execution` / `test_string`).
```
---
Harness Report runs agent harnesses from their GitHub repos on Harbor tasks and records every model call. Every page is also `.md` and `.json`; index: https://harnessreport.com/llms.txt · MCP: https://harnessreport.com/mcp