<commit_before># Operations

Operation support is implemented by the `op` module:

    >>> import guild.op

For our tests, we'll use a helper function that returns an instance of
`guild.modelfile.OpDef`:

    >>> import guild.modelfile

    >>> def OpDef(cmd, name="op"):
    ...     data = [
    ...       {
    ...         "name": "model",
    ...         "operations": {
    ...           name: {
    ...             "cmd": cmd
    ...           }
    ...         }
    ...       }
    ...     ]
    ...     models = guild.modelfile.Modelfile(data, "./MODEL")
    ...     return models["model"].get_operation(name)

We'll also create a helper function that returns and instance of
`guild.op.Operation` given arguments to `OpDef` above:

    >>> def Operation(*args, **kw):
    ...     model = None # not used
    ...     return guild.op.Operation(model, OpDef(*args, **kw))

Note that the `"test"` argument is an operation reference, which is
not used in our tests.

## Command specs

Command specs are used to generate Python commands. The first part of
the spec is used as the Python script or module. It can be a module
name with or without a py extension.

Here's an operation with a simple "train" cmd:

    >>> op = Operation(cmd="train")
    >>> op.cmd_args
    ['python', '-um', 'guild.op_main', 'train']

Command specs may contain additional arguments, which will be included
in the Python command.

    >>> op = Operation(cmd="train epoch=10 tags='tag1 tag2'")
    >>> op.cmd_args
    ['python', '-um', 'guild.op_main', 'train', 'epoch=10', 'tags=tag1 tag2']

Command specs cannot be empty:

    >>> Operation(cmd="")
    Traceback (most recent call last):
    InvalidCmd

## Flag args

Flags are defined in MODEL files and provided as command line
arguments to the run command. `_flag_args` returns a list of command
line arg for a map of flag values.

Empty flags:

    >>> guild.op._flag_args({}, [])
    []

Single flag:

    >>> guild.op._flag_args({"epochs": 100}, [])
    ['--epochs', '100']

Multiple flags are returned in sorted order:

    >>> guild.op._flag_args({"epochs": 100, "data": "my-data"}, [])<commit_msg>Run op_main as a run script rather than as a module<commit_after># Operations

Operation support is implemented by the `op` module:

    >>> import guild.op

For our tests, we'll use a helper function that returns an instance of
`guild.modelfile.OpDef`:

    >>> import guild.modelfile

    >>> def OpDef(cmd, name="op"):
    ...     data = [
    ...       {
    ...         "name": "model",
    ...         "operations": {
    ...           name: {
    ...             "cmd": cmd
    ...           }
    ...         }
    ...       }
    ...     ]
    ...     models = guild.modelfile.Modelfile(data, "./MODEL")
    ...     return models["model"].get_operation(name)

We'll also create a helper function that returns and instance of
`guild.op.Operation` given arguments to `OpDef` above:

    >>> def Operation(*args, **kw):
    ...     model = None # not used
    ...     return guild.op.Operation(model, OpDef(*args, **kw))

Note that the `"test"` argument is an operation reference, which is
not used in our tests.

## Command specs

Command specs are used to generate Python commands. The first part of
the spec is used as the Python script or module. It can be a module
name with or without a py extension.

Here's an operation with a simple "train" cmd:

    >>> op = Operation(cmd="train")
    >>> op.cmd_args
    ['.../python...', '-u', 'guild/scripts/run', 'train']

Command specs may contain additional arguments, which will be included
in the Python command.

    >>> op = Operation(cmd="train epoch=10 tags='tag1 tag2'")
    >>> op.cmd_args
    ['.../python...', '-u', 'guild/scripts/run', 'train', 'epoch=10',
     'tags=tag1 tag2']

Command specs cannot be empty:

    >>> Operation(cmd="")
    Traceback (most recent call last):
    InvalidCmd

## Flag args

Flags are defined in MODEL files and provided as command line
arguments to the run command. `_flag_args` returns a list of command
line arg for a map of flag values.

Empty flags:

    >>> guild.op._flag_args({}, [])
    []

Single flag:

    >>> guild.op._flag_args({"epochs": 100}, [])
    ['--epochs', '100']

Multiple flags are returned in sorted order:

    >>> guild.op._flag_args({"epochs": 100, "data": "my-data"}, [])
{'fraction_non_alphanumeric': 0.1465775524314358, 'fraction_numerical': 0.007835906890988707, 'mean_word_length': 2.551554828150573, 'pattern_counts': {'":': 12, '<': 3, '<?xml version=': 0, '>': 75, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5678540', 'n_tokens_mistral': 1351, 'n_tokens_neox': 1288, 'n_words': 470}