license:cc-by-4.0 region:us
---
license: cc-by-4.0
dataset_info:
  features:
  - name: code
    dtype: string
  - name: signature
    dtype: string
  - name: docstring
    dtype: string
  - name: loss_without_docstring
    dtype: float64
  - name: loss_with_docstring
    dtype: float64
  - name: factor
    dtype: float64
  - name: rendered
    dtype: string
  - name: quality_prob
    dtype: float64
  - name: learning_prob
    dtype: float64
  splits:
  - name: train
    num_bytes: 59688305.816243604
    num_examples: 34488
  download_size: 30704027
  dataset_size: 59688305.816243604
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---

Filtered version of code search net python subset, with filtering based on perplexity with/without docstring, learning value/quality classifiers, and manual filtering.

Original data with perplexity filtering is from [here](https://huggingface.co/datasets/bjoernp/code_search_net_python_processed_400k), with credit to bjoernp.