task_categories:text-generation task_ids:language-modeling size_categories:unknown language:code license:apache-2.0 region:us
---
annotations_creators: []
language:
- code
license:
- apache-2.0
pretty_name: github-code
size_categories:
- unknown
source_datasets: []
task_categories:
- text-generation
task_ids:
- language-modeling
---
# Dataset Card for AI-HDLCoder

## Dataset Description
The GitHub Code dataset consists of 100M code files from GitHub in VHDL programming language with extensions totaling in 1.69 GB of data. The dataset was created from the public GitHub dataset on Google BiqQuery at Anhalt University of Applied Sciences.

## Considerations for Using the Data

The dataset is created for research purposes and consists of source code from a wide range of repositories. As such they can potentially include harmful or biased code as well as sensitive information like passwords or usernames.

### Languages

```python
{
    "VHDL": [".vhdl", "vhd"],
}
```

## Dataset Structure

### Data Instances

```python
{
  "repo_name": "sebgod/linguist",
  "path": "samples/VHDL/foo.vhd",
  "copies": "91",
  "size": "217",
  "content": "-- VHDL example file\n\nlibrary ieee;\nuse ieee.std_logic_1164.all;\n\nentity inverter is\n\tport(a : in std_logic;\n\t     b : out std_logic);\nend entity;\n\narchitecture rtl of inverter is\nbegin\n\tb \u003c\u003d not a;\nend architecture;\n",
  "license": "mit"
}, {
  "repo_name": "cybernet14/linguist",
  "path": "samples/VHDL/foo.vhd",
  "copies": "91",
  "size": "217",
  "content": "-- VHDL example file\n\nlibrary ieee;\nuse ieee.std_logic_1164.all;\n\nentity inverter is\n\tport(a : in std_logic;\n\t     b : out std_logic);\nend entity;\n\narchitecture rtl of inverter is\nbegin\n\tb \u003c\u003d not a;\nend architecture;\n",
  "license": "mit"
}
```

### Data Fields

|Field|Type|Description|
|---|---|---|
|code|string|content of source file|
|repo_name|string|name of the GitHub repository|
|path|string|path of file in GitHub repository|
|language|string|programming language as inferred by extension|
|license|string|license of GitHub repository|
|size|int|size of source file in bytes|

### Data Splits

The dataset contains a train split only

### Licensing Information

```python
[
  'agpl-3.0',
  'artistic-2.0',
  'mpl-2.0',
  'cc0-1.0',
  'mit',
  'gpl-2.0',
  'gpl-3.0',
  'lgpl-3.0',
  'apache-2.0',
  'bsd-3-clause'
 ]
```

### v1.0
- Initial release of dataset
- The query was executed on 21.07.2023, 00:10:30 UTC+2
