metadata

license: mit
datasets:
  - HelloImSteven/applescript-lines-annotated
language:
  - en
tags:
  - applescript
  - code
widget:
  - text: on openFile(filePath, permission)
    example_title: Handler Definition
  - text: set myVar to 3
    example_title: Set Variable
  - text: open location "https://huggingface.co"
    example_title: open location
  - text: >-
      do shell script "echo 'Here are some numbers: 1, 5, 89, 156, 1053' | egrep
      -o '\\d*'"
    example_title: do shell script
  - text: use framework "CoreLocation"
    example_title: use framework
  - text: tell application "Notes" to make new note
    example_title: Make New Note
pipeline_tag: summarization

Model Description

This model summarizes individual lines of AppleScript code and provides insights into their purpose and functionality. The model is trained on the applescript-lines-annotated dataset.

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, AutoConfig, pipeline

model_selector = "HelloImSteven/AppleScript-Summarizer"
tokenizer = AutoTokenizer.from_pretrained(model_selector)

model = AutoModelForSeq2SeqLM.from_pretrained(model_selector)
config = AutoConfig.from_pretrained(model_selector)

pipe = pipeline("summarization", model=model, config=config, tokenizer=tokenizer)

raw_code = [
    "if myThing is 0 then set success to true",
    "set numInputs to 5",
    "tell application \"System Events\" to click button \"OK\" of window 1 of application process \"Finder\""
]

res = pipe(raw_code)
for i in range(len(res)):
    print("\n\nInput: '" + raw_code[i] + "'")
    print("\nSummary: " + res[i]["summary_text"] + '\n')

The model performs best when given adequate context, such as meaningful variable and handler names. For example, it will perform better when provided the input "on makeDirectory(dirPath)" than it will for "on makeDir(x)".

Example Outputs

Input	Output
on makeDirectory(dirPath)	This line of AppleScript code begins the definition of a handler named "makeDirectory". Based on its name, the handler is likely used to create a directory path from the path stored in the variable "dirPath".
open location "https://huggingface.co"	This is a line of AppleScript code that calls the built-in "open location" handler, providing the URL "https://huggingface.co" as a parameter. This is likely used to open a web browser.
do shell script "open -a 'Google Chrome'"	This is a line of AppleScript code that calls the "open -a" handler of the "Google Chrome" command, providing the output as a shell command. When executed, this line will open the Chrome web browser.

Limitations

This model is a work in progress. It is trained on a small dataset of annotated AppleScript lines, but that dataset will grow in time. For now, however, this leads to several limitations of the model:

There are concepts and line structures that are simply absent from the dataset, so the model will struggle with them. If you identify such a concept, please consider providing feedback (e.g. expected output).
While the model is fine-tuned from Bart and thus has adequate knowledge of outside concepts, it is not able to give explanations of all possible concepts. For example, for inputs involving the do shell script command, it will be able to provide explanations of some shell scripts better than others. This is not an area of focus for this model, so do not expect significant improvements in future versions.
This is not a conversational model.

Validation Metrics

Loss: 0.20628
Rouge1: 0.60299
Rouge2: 0.41118
rougeL: 0.52186
RougeLsum: 0.52207