language:en license:apache-2.0 PyPi package dataset named entity recognition text classification NLP Large doi:10.57967/hf/0456 region:us
---
license: apache-2.0
language:
- en
tags:
- PyPi
- package
- dataset
- named entity recognition
- text classification
- NLP
- Large
pretty_name: kfj pypi
---

# kfj-pypi Dataset

This dataset contains a collection of PyPI packages scraped from PyPI. The dataset includes metadata about each package, including its name, version, description, author, license, and more. The dataset is intended to be used for research and development in various natural language processing (NLP) applications such as named entity recognition and text classification.

## Usage

To use this dataset, you can download it from [Hugging Face Datasets](https://huggingface.co/datasets/KingfernJohn/kfj-pypi-packages-metadata) using the `datasets` library in Python:

```python
from datasets import load_dataset

dataset = load_dataset("kfj-pypi-packages-metadata")
```

This will load the kfj-pypi dataset into a Python variable, which you can then use to access the metadata for each package.


## Info

The dataset contains metadata of 161,346 packages, with a total size of 743MB (.zip 304MB).
We skipped packages that returned no metadata to avoid empty files.

Please note that the dataset is currently being updated and more packages will be added soon.
version 0.1

## Structure

```json
{
  "name": "",
  "version": "",
  "description": "",
  "author": "",
  "author_email": "",
  "maintainer": "",
  "maintainer_email": "",
  "license": "",
  "keywords": "",
  "classifiers": "",
  "download_url": "",
  "platform": "",
  "homepage": "",
  "project_urls": "",
  "requires_python": "",
  "requires_dist": "",
  "provides_dist": "",
  "obsoletes_dist": "",
  "summary": ""
}
```