wehub-resource-sync
32 lines
1.7 KiB
Markdown
32 lines
1.7 KiB
Markdown
# Case Study
|
|
|
|
This directory includes some case analysis. We compare the both method(grep + Claude Context semantic search) and the traditional grep only method.
|
|
|
|
These cases are selected from the Princeton NLP's [SWE-bench_Verified](https://openai.com/index/introducing-swe-bench-verified/) dataset. The results and the logs are generated by the [run_evaluation.py](../run_evaluation.py) script. For more details, please refer to the [evaluation README.md](../README.md) file.
|
|
|
|
- 📁 [django_14170](./django_14170/): Query optimization in YearLookup breaks filtering by "__iso_year"
|
|
- 📁 [pydata_xarray_6938](./pydata_xarray_6938/): `.swap_dims()` can modify original object
|
|
|
|
Each case study includes:
|
|
- **Original Issue**: The GitHub issue description and requirements
|
|
- **Problem Analysis**: Technical breakdown of the bug and expected solution
|
|
- **Method Comparison**: Detailed comparison of both approaches
|
|
- **Conversation Logs**: The interaction records showing how the LLM agent call the ols and generate the final answer.
|
|
- **Results**: Performance metrics and outcome analysis
|
|
|
|
## Key Results
|
|
Compared with traditional grep only, the both method(grep + Claude Context semantic search) is more efficient and accurate.
|
|
|
|
## Why Grep Fails
|
|
|
|
1. **Information Overload** - Generates hundreds of irrelevant matches
|
|
2. **No Semantic Understanding** - Only literal text matching
|
|
3. **Context Loss** - Can't understand code relationships
|
|
4. **Inefficient Navigation** - Produces many irrelevant results
|
|
|
|
## How Grep + Semantic Search Wins
|
|
|
|
1. **Intelligent Filtering** - Automatically ranks by relevance
|
|
2. **Conceptual Understanding** - Grasps code meaning and relationships
|
|
3. **Efficient Navigation** - Direct targeting of relevant sections
|