iva-codeint-swift-small GPT-2 is (small version - 239.4M parameters) trained from scratch to obtain results in the text-to-code task tailored for Swift language used
in native mobile development (iOS).

## Usage

```Python
from transformers import pipeline

pipe = pipeline("text-generation", model="mvasiliniuc/iva-codeint-swift-small")
outputs = pipe("func triggerNSNotification")

```

### Inference
```Python
API_URL = "https://api-inference.huggingface.co/models/mvasiliniuc/iva-codeint-swift-small"
headers = {"Authorization": "Bearer <key>"}
def query(payload):
	response = requests.post(API_URL, headers=headers, json=payload)
	return response.json()

output = query({
"inputs": """
/* 
A function that gets the current device operating system.
*/
"""
})
pprint.pprint(output, compact=True)
```

## Training

| Config | Value |
|------|------------------|
| seq length | 1024 |
| weight decay | 0.1 |
| learning rate | 0.0005 |
| max eval steps | -1 |
| shuffle buffer | 10000 |
| max train steps | 150000 |
| mixed precision | fp16 |
| num warmup steps | 2000 |
| train batch size | 5 |
| valid batch size | 5 |
| lr scheduler type | cosine |
| save checkpoint steps | 15000 |
| gradient checkpointing | false |
| gradient accumulation steps | 1 |

## Resources

Resources used for research: 
* [Training a causal language model from scratch](https://huggingface.co/learn/nlp-course/chapter7/6)
* [CodeParrot a GPT-2 model (1.5B parameters) trained to generate Python code](https://huggingface.co/codeparrot/codeparrot)

{"language": ["code"], "tags": ["gpt2", "code", "swift", "mobile", "generation"], "datasets": ["mvasiliniuc/iva-swift-codeint-clean-train", "mvasiliniuc/iva-swift-codeint-clean-valid"], "widget": [{"text": "/*\n  A function that returns the time zone currently configured on the device.\n*/\n", "example_title": "Get the current time zone of the device"}, {"text": "/*\n  A function that returns the current version of the operating system.\n*/\n", "example_title": "Get current device operating system"}, {"text": "/* \nA function that fires an NSNotification named 'MyUpdate'. \n*/\npublic func post", "example_title": "Post NSNotification"}, {"text": "/* \nA public function that saves a given String value in UserPreference at a given String key.\n*/\n", "example_title": "Save to UserPreference"}]}
transformers pytorch gpt2 text-generation code swift mobile generation dataset:mvasiliniuc/iva-swift-codeint-clean-train dataset:mvasiliniuc/iva-swift-codeint-clean-valid autotrain_compatible endpoints_compatible has_space text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-generation #code #swift #mobile #generation #dataset-mvasiliniuc/iva-swift-codeint-clean-train #dataset-mvasiliniuc/iva-swift-codeint-clean-valid #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 

iva-codeint-swift-small GPT-2 is (small version - 239.4M parameters) trained from scratch to obtain results in the text-to-code task tailored for Swift language used
in native mobile development (iOS).


Usage
-----


### Inference


Training
--------



Resources
---------


Resources used for research:


* Training a causal language model from scratch
* CodeParrot a GPT-2 model (1.5B parameters) trained to generate Python code



### Inference


Training
--------



Resources
---------


Resources used for research:


* Training a causal language model from scratch
* CodeParrot a GPT-2 model (1.5B parameters) trained to generate Python code
TAGS
#transformers #pytorch #gpt2 #text-generation #code #swift #mobile #generation #dataset-mvasiliniuc/iva-swift-codeint-clean-train #dataset-mvasiliniuc/iva-swift-codeint-clean-valid #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
 ### Inference


Training
--------



Resources
---------


Resources used for research:


* Training a causal language model from scratch
* CodeParrot a GPT-2 model (1.5B parameters) trained to generate Python code