task_categories:text-generation size_categories:100K<n<1M language:code license:other region:us
---
license: other
task_categories:
- text-generation
language:
- code
pretty_name: github-r-repos
size_categories:
- 100K<n<1M
---

## GitHub R repositories dataset

R source files from GitHub.

This dataset has been created using the public GitHub datasets from Google BigQuery.
This is the actual query that has been used to export the data:

```
EXPORT DATA
  OPTIONS (
    uri = 'gs://your-bucket/gh-r/*.parquet',
    format = 'PARQUET') as
(
select
  f.id, f.repo_name, f.path,
  c.content, c.size
from (
  SELECT distinct 
    id, repo_name, path
  FROM `bigquery-public-data.github_repos.files`
  where ends_with(path, ".R")
) as f
left join `bigquery-public-data.github_repos.contents` as c on f.id = c.id
)

EXPORT_DATA
 OPTIONS (
    uri = 'gs://your-bucket/licenses.parquet',
    format = 'PARQUET') as
(select * from `bigquery-public-data.github_repos.licenses`)
```

Files were then exported and processed locally with files in the root of this repository.
Datasets in this repository contain data from reositories with different licenses.

The data schema is:

```
id: string
repo_name: string
path: string
content: string
size: int32
license: string
```

Last updated: Jun 6th 2023
