task_categories:text-classification task_categories:text2text-generation task_categories:summarization size_categories:10K<n<100K language:en typescript instruct code region:us
---
task_categories:
- text-classification
- text2text-generation
- summarization
language:
- en
tags:
- typescript
- instruct
- code
size_categories:
- 10K<n<100K
---

# typescript-instruct
A dataset of TypeScript snippets, processed from the typescript subset of [the-stack-smol](https://huggingface.co/datasets/bigcode/the-stack-smol).


# Processing
- Each source file is parsed with the TypeScript AST and queried for 'semantic chunks' of the following types. 
```
ClassDeclaration - 2401                                                                                                                                                         
ArrowFunction - 16443                                                                                                                                                           
MethodDeclaration - 12096                                                                                                                                                       
FunctionDeclaration - 3226                                                                                                                                                      
TypeAliasDeclaration - 1489                                                                                                                                                     
InterfaceDeclaration - 5240                                                                                                                                                     
EnumDeclaration - 214 
```
- Leading comments are added to the front of `content`
- Removed all chunks over max sequence length (2048)
- Deduplicated / cleaned up
- Generated instructions w/ `gpt-3.5-turbo`
  - Ran into of OpenAI API for the month, will finish other half next month



# Dataset Structure
```python
from datasets import load_dataset
load_dataset("bleugreen/typescript-instruct")
DatasetDict({
    train: Dataset({
      features: ['type', 'content', 'repo', 'path', 'language', 'instruction'],
      num_rows: 41109
  })
})
```