region:us
This is a sample of GitHub commits and files reconstructed from the Software Heritage dataset.
It contains the latest 1024 commits in all of `pytorch/*` and `huggingface/*` repos.

The tables are split to avoid an explosion of rows (lots of repeated files between commits), so you will need to pre-filter the commits before adding the file contents.

Table descriptions:
## 1. `commits`
The commit message table. Join it with `commit_filepath` on `commits.directory_id == commit_filepath.directory_id` and `commits.parent_directory_id == commit_filepath.directory_id`.

* origin - str	(repo url, example: `https://github.com/huggingface/datasets`)
* full_name - str	(repo name, example: `huggingface/datasets`)
* commit_id - str	(example: `56b114ebfd5399252dc23f9df207f87c5397b50a`)
* parent_commit_id - str	(previous commit id, example: `8c826fb80f7f8135f6e632d34c8f59134f5983c8`)
* snapshot_id - str	(Software Heritage snapshot id, example: `d76232879a5912b1eaca91e8889863117bca66a4`)
* visit_date - datetime[ns]	(Software Heritage crawler visit date, example: `2022-11-28 13:03:09.100114`)
* branch_name - str	(example: `refs/heads/main`)
* revision_date - datetime[ns] (commit date, example: `2022-04-27 17:30:41`)	
* committer_date - datetime[ns]	(commit date, example: `2022-04-27 17:30:41`)		
* author - binary	(Software Heritage anonymized commit's author name)
* message - str	(commit message, example: `update auth when mirroring datasets on the hub (#4242)`)
* directory_id - str (root directory id to join this table with the files, example: `84c6cc5b2c156ed3251674c43dd411d731183bb3`)
* parent_directory_id - str	(parent commit's root directory id, example: `e927ce1cdecf6286f7e23204ed656373c9921f89`)

## 2. `commit_filepath`
The file paths associated with each commit. Join it with `file_conents` on `blob_id`

* directory_id - str	(root directory id of the corresponding commit, example: `001331910958befd665d94c85c23471a8fc1ab19`)
* blob_id - str	(Software Heritage file blob id, example: `47953673b7b51c2585402a91d434f5fe4d9dc105`)
* content_id - str	(Software Heritage content id, example: `d6b3dab547a59efe5246edf06a42e8e85776acb1`)
* path - str	(file path inside the repository, example: `	/core/src/components/tab-bar/usage/javascript.md`)
* length - i64 (file length in bytes, example: `529`)

## 3. `file_contents`
The contents of all non-binary files (i.e. not images/media/data).

* blob_id - str	(Software Heritage file blob id, example: `47953673b7b51c2585402a91d434f5fe4d9dc105`)
* content - str	(file contents, always a UTF-8 string)
* src_encoding - str	(original file's encoding, example: `UTF-8`)
language - str	(programming language label, example: `Python`)
is_vendor - bool	(True if it's a vendor file, e.g. 3rd party library)
is_generated - bool  (True if the file is auto-generated)