task_categories:text-generation size_categories:1M<n<10M code haskell region:us
---
dataset_info:
  features:
  - name: repo_name
    dtype: string
  - name: path
    dtype: string
  - name: license
    dtype: string
  - name: full_code
    dtype: string
  - name: full_size
    dtype: int64
  - name: uncommented_code
    dtype: string
  - name: uncommented_size
    dtype: int64
  - name: function_only_code
    dtype: string
  - name: function_only_size
    dtype: int64
  - name: is_commented
    dtype: bool
  - name: is_signatured
    dtype: bool
  - name: n_ast_errors
    dtype: int64
  - name: ast_max_depth
    dtype: int64
  - name: n_whitespaces
    dtype: int64
  - name: n_ast_nodes
    dtype: int64
  - name: n_ast_terminals
    dtype: int64
  - name: n_ast_nonterminals
    dtype: int64
  - name: loc
    dtype: int64
  - name: cycloplexity
    dtype: int64
  splits:
  - name: train
    num_bytes: 3094608763
    num_examples: 3263408
  download_size: 1168831903
  dataset_size: 3094608763
task_categories:
- text-generation
tags:
- code
- haskell
size_categories:
- 1M<n<10M
---
# Dataset Card for "github-code-haskell-function"

Rows: 3.26M
Download Size: 1.17GB

This dataset is extracted from [github-code-haskell-file](https://huggingface.co/datasets/blastwind/github-code-haskell-file).

Each row has 3 flavors of the same function: 
`uncommented_code`: Includes the function and its closest signature. 
`function_only_code`: Includes the function only.
`full_code`: Includes the function and its closest [signature](https://wiki.haskell.org/Type_signature) and comment. 
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the function 
is neither a signature nor comment, `full_code` is just the function. If the previous neighbor is one though, include 
them appropriately, then search the previous neighbor for the other node with the same logic.

Further, each row also contains attribute values for my personal analysis project. The attributes are calculated from the code in column `uncommented_code`. 
7% (225k) of the rows have cyclomatic complexity and LOC valued at `-1` because [`homplexity`](https://github.com/BlastWind/homplexity) failed in parsing the row's `uncommented_code`.

