|
--- |
|
license: llama3 |
|
license_link: LICENSE |
|
base_model: meta-llama/Meta-Llama-3-8B |
|
datasets: |
|
- ajibawa-2023/Python-Code-23k-ShareGPT |
|
language: |
|
- en |
|
tags: |
|
- code |
|
library_name: transformers |
|
--- |
|
|
|
|
|
|
|
Large Language Models (LLMs) are good with code generations. Sometimes LLMs do make mistakes in code generation. How about if they can give detailed explanation along with the code. |
|
This is what I have tried over here. The base Llama-3-8B model was used for training purpose. It is trained on around 23000+ set of codes. Each set having 2 conversations. |
|
This data was generated using GPT-3.5, GPT-4 etc. This conversation is in Vicuna/ShareGPT format. Each set, along with code, has detailed explanation. |
|
|
|
|
|
Training framework: [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) |
|
|
|
|
|
Training details: |
|
- epochs: 3 |
|
- learning rate: 0.0002 |
|
- learning rate scheduler type: cosine |
|
- warmup_steps: 10 |
|
- cutoff len (i.e. context length): 4096 |
|
- gradient_accumulation_steps: 4 |
|
- micro_batch_size: 4 |
|
- fine-tuning type: full parameters |
|
- optimizer: adamw_bnb_8bit |
|
|
|
|
|
Examples |
|
|
|
The following are some examples generated by our CodeLlama3-8B-python model: |
|
|
|
|
|
User:Based on the principle of the quicksort algorithm, implement a quicksort algorithm in Python without using any built-in library functions. The input should be a list where all elements are of float type, and it should return a sorted list. Remember to add comments to the code. |
|
|
|
|
|
Assistant: |
|
quicksort algorithm: |
|
""" |
|
A sorting algorithm that works by selecting a pivot element and partitioning the array around this pivot element. |
|
Then, recursively applying the same process to the two partitions until the entire array is sorted. |
|
""" |
|
|
|
```python |
|
def quicksort(arr): |
|
if len(arr) <= 1: |
|
return arr |
|
|
|
# Selecting a pivot element |
|
pivot = arr[len(arr)//2] |
|
|
|
# Partitioning the array around the pivot element |
|
left = [x for x in arr if x > pivot] |
|
middle = [x for x in arr if x == pivot] |
|
right = [x for x in arr if x < pivot] |
|
|
|
# Recursively applying the quicksort algorithm to the left and right partitions |
|
return quicksort(left) + middle + quicksort(right) |
|
``` |
|
|
|
The `quicksort` function takes an input list `arr` and returns a sorted list. It uses the quicksort algorithm to achieve this. |
|
|
|
|
|
|
|
|
|
|
|
User:Based on the principle of the KMP algorithm, implement a KMP algorithm in Python and add comments to the code. |
|
|
|
|
|
Assistant: |
|
```python |
|
def kmp_algorithm(pattern, text): |
|
n = len(pattern) |
|
m = len(text) |
|
|
|
# Build the table |
|
table = [0] * m |
|
length = 1 |
|
i = 1 |
|
while i < n: |
|
if pattern[i] == text[length]: |
|
table[length] = i |
|
length += 1 |
|
i += 1 |
|
else: |
|
if length > 1: |
|
length = min(length, table[i - 1]) |
|
table[i] = length |
|
i += 1 |
|
|
|
# Find all matches |
|
matches = [] |
|
i = 0 |
|
j = 0 |
|
while i < n and j < m: |
|
if pattern[i] == text[j]: |
|
i += 1 |
|
j += 1 |
|
if i == n: |
|
matches.append(text[j - i: j]) |
|
|
|
return matches |
|
``` |
|
|