<commit_before>IOpt documentation = #[Command line processing the Ioke way.

  Introduction.
 
  IOpt is a tool for command-line option analysis. It tries to take
  advantage from Ioke's homoiconic nature to make you write less.
  IOpt is inspired by similar tools like ruby's optparse, and tries 
  to make command line parsing as easy as possible for ioke programs,
  while still allowing a great degree of control.

  Features

  - Doen't impose an option style, by default IOpt provides support 
    for unix style options (short and long), but you can easily define
    how an option looks like for your application.

  - Interface for easily creating options that will store a cell on a given
    object, activate a value(methods, blocks, etc).

  - Option arguments can be any of those supported by Ioke, thus your
    options can take required arguments, optionals, keywords, and
    +rest, +:krest arguments just like any other ioke method.

  - IOpt support option priority, so that you can choose which flags
    should be handled before others (like --help or --version).
    Default priority is 0, more negative values are higher priority.
 
  Basic Usage.
     
     opt = IOpt mimic

     ;; print the opt object as text, showing the available options
     ;; By default an option's action is executed having the opt object
     ;; as receiver. (more on how to change this later)
     opt["--help"\] = method("Show help", @println. System exit)
     ;; set --help a negative priority so that its executed before
     ;; options with more a positive priority.
     opt["--help"\] priority = -10
       
     ;; Creating an option that takes one required argument:
     ;; this action would store the value on opt cell(:output)
     ;;   --output here       => opt output = "here"
     ;;   -o=/there           => opt output = "/there"
     ;;   -o21                => opt output = "21"
     opt["-o", "--output"\] = method("Set output file", file, @output = file)

     ;; Optional arguments are just as easy
     ;; will handle the following: 
     ;;    -s fancy    =>  opt style = "fancy"
     ;;    -sa         =>  opt style = "a"
     ;;    -s=simple   =>  opt style = "simple"
     opt["-s"\] = method("With style", style "neat", @style = style)

     ;; parse the arguments given to the ioke runtime.
     ;; IOpt will not modify the array given to the parse method,
     ;; this original input is later available at opt cell(:argv)
     opt parse(System programArguments)

     ;; Arguments that were not recognized as options nor 
     ;; were part of an option arguments can be accessed by
     opt programArguments

     ;; The original input given to the parse method is available at
     opt argv

  Action arguments.

  Actions (the code that handles an option) can be any Ioke activable
  value, methods, macros, etc. Because of this, your options can have
  any type of arguments supported by ioke: required, optional, keyword
  arguments, +rest and +:krest.

  Arguments are captured from the command line just before the option
  taking them and before any other option is found.

  The following is a complex example of an action taking all types of
  arguments:

      ;; This action can take any argument up until the following option.
      ;; e.g. 
      ;;
      ;;    --ioke bKey:10 hello say:me world aKey: 5 again
      ;;
      opt["--ioke"\] = method("Support ioke argument style",
            required, optional "defValue", +rest,
            aKey:, bKey: 22, +:krest,
         ;; illustrate how arguments were handled.
         required should == "hello"
         optional should == "world"
         rest should == ["again"\]
         aKey should == "5"
         bKey should == "10"
         krest should == { say: "me" })


  Specifing options in a DSL way.
      
  IOpt tries to make option definition as natural as possible, to
  this end, it provides the on macro.
  
  Suppose you have an object where you want to store a cell
  using an option
  
     ;; this will create a --file option taking one required 
     ;; argument. when called will store value on myObject cell(:file)
     opt on(myObject, "--file", "Which file should I use", :@file)

     ;; Or if you already have a method on myObject and would like
     ;; to expose it as an option, use something like the follwing.
     ;; This will generate a --my-method option taking arguments
     ;; as mandated by the method object.
     opt on(myObject, "--my-method", :myMethod)

     ;; Instead of giving myObject each time, you can use
     opt = IOpt on(myObject)
     opt on(\"--foo\", \"Set the foo\", :@foo)
     opt on(\"-b\", \"--activate-bar\", :bar)

     ;; If the last argument is not a symbol, then the on macro
     ;; creates a LexicalBlock to handle the option:
     opt on("--something", "-s", "The documentation", arg,
       thisLexicalScopeVar + arg)

  The Action object.

  When you define an option in IOpt, it will create a mimic of
  IOpt Action, these kind of objects know how to consume the
  arguments they need, the documentation and the receiver to 
  execute the action on.

  After creating an option, its action can be accessed like
  
     opt["--help"\] ;; => The action handling --help
     
  There are some things you can do with an action object:

     ;; ask for the options it handles
     opt["--help"\] flags ;; => ["--help", "-h"\]

     ;; define it's processing priority
     ;; (more negative values are higher priority)
     opt["--help"\] priority = -1

     ;; set the documentation displayed for --help
     ;; most of the times it is borrowed from the activable
     ;; value's documentation cell.
     opt["--help"\] documentation = "Display this help"

     ;; set the receiver object where to activate the action
     opt["--help"\] receiver = someOtherObject
       
     ;; change the arity for this action.
     ;; Most of the times this is borrowed from the activable
     ;; value, but if that is a macro, you will need to 
     ;; specify if your option can consume arguments from cli.
     ;; e.g. Suppose you have a macro on the app object.
     app weird = dmacro("Weird destructuring macro for CLI",
        [>one\]
        blabla ;; received one arg

        [>one, two\]
        something) ;; expected at most two args

     ;; Now, so that you can be abe to consume at most two arguments
     ;; from the command line, you need to specify the argumentsCode
     ;; that your macro would expect.
     opt on(app, :weird) argumentsCode = "a,b nil"

]; documentation
  <commit_msg>Improved documentation with an example app.<commit_after>IOpt documentation = #[Command line processing the Ioke way.

Introduction.

  IOpt is a tool for command line option analysis. It tries to
  take advantage of Ioke's homoiconic nature, to provide a DSL
  interface and make you write less.
  
  IOpt has been influenced by similar tools like ruby's optparse,
  trying to make command line parsing as easy as possible for ioke
  programs, but still providing great degree of control over
  how options look like, and how they should be processed.

  Features:

  - Doesn't impose an option style, by default IOpt provides support 
    for unix style options (short and long), but you can easily define
    how an option looks like for your application.

  - Interface for easily creating options that will store a cell on a given
    object, or activate a value, call methods, blocks, etc.

  - Option execution priority, so that you can choose which actions
    should be handled before others (like --help or --version).
    Default priority is 0, more negative values are higher priority.

  - Support for clustered short options, eg if -x -v -f are short options
    you can feed -xvf to your program.

  - Options can take arguments directly from command line.  
    These can be any of those supported by Ioke, thus your
    options can take required arguments, optionals, keywords, and
    +rest, +:krest arguments just like any other ioke method.

  - Option Arguments can be coerced to Ioke objects.
    By default IOpt recognizes option arguments that are literals
       nil, true, false, symbols, numbers.
    It is also pretty easy to create your own coercing strategies 
    so you could for example transform "yes" or "no" to true/false,
    or something more advanced. Coercion can be customized per
    option, so that you can adapt how arguments are send to the
    action.
    

Basic Usage

  The following is an interesting example application to show some IOpt features:

     app = Origin with(name: "app", version: "1.0", style: "none", verbosity: 1)
     app stdout = method("Default output strategy", block, block(System out))
     app outStrategy = app cell(:stdout)

     app loadConfig = method("Load configuration from io", io, "Reading" println)

     app usingOutput = method("Use new output strategy for file", file, append: false,
       if(file == "-", return(@outStrategy = @cell(:stdout)))
       @outStrategy = fn("New output strategy that writes to  #{file}", block,
         ensure(
           os = if(file kind?("java:net:Socket"), os getOutputStream,
             java:io:FileOutputStream new(file, append))
           ps = java:io:PrintStream new(os, true)
           block(ps),
           ps close,
           os close)))

     app run = method("Execute the application", args,
       outStrategy(fn(out,
           verbosity times(i,
             out println("#{i}: #{name} is running #{args inspect}, my style is #{style}")))
     ))

     ;; Done with app implementation, now let's define it's command line options
     opt = IOpt on(app) ; app will be the receiver object to execute actions on
     opt banner = "Usage: #{app name} [options\]"

     ; Print the opt object and exit. Highest priority option.
     opt on("-h", "--help", "HALP!", opt println. System exit) priority = -10

     ; Print the application version before any option with more positive prio.
     opt on("--version", "Print version and exit", version println) priority = -5
     
     ; An option taking one required argument, and calling app loadConfig
     opt on("-c", "--config", "Use config file", path,
       FileSystem withOpenFile(path, fn(f, self loadConfig(f))))
     
     ; Store a cell in app
     opt on("-s", "--style", "Set output style", :@style)
     
     ; increase app verbosity. takes an optional argument
     opt on("-v", "Increase verbosity", value = 1,
       @verbosity += value)

     ; An option that will activate the usingOutput cell on app
     ;
     ; help string will be taken from that cell's documentation
     ; This action will take aditional arguments according to usingOutput's arity
     ;
     ; Also this action uses a custom coercing strategy to convert
     ; english yes/no and japanese hai/iie to true/false ioke objects.
     ; if output is host:port then it will be converted to a java socket.
     ;
     ; This action priority is greater than the default (0), that means
     ; it will be executed after higher (more negative) actions, thus 
     ; if the user specified --style, this action will be executing having
     ; the user specified style already set.
     opt on("-o", "--output", :usingOutput) coercing (
       english: #/^yes|no$/ => method(t, t == "yes"),
       japanese: #/^hai|iie$/ => method(t, t == "hai"),
       net: #/:\\d+$/ => method(t,
         addr = t.split(":")
         java:net:Socket new(addr first, addr second toDecimal))
     ) priority = 1

     ; will parse the command line arguments before the first --
     ; and execute the actions by order of priority.
     opt parse!(System programArguments, stopAt: "--")
     ; give app the arguments that arent options
     app run(opt programArguments)

     ;; this are example command lines for app:
     ;; app --help
     ;; app -vvvv --version -c ~/.appConfig -o -
     ;; app -v4 -o there append:yes -s xml
     ;; app -o nihon.jp:2200 --style origami
 
More Advanced Usage

  If you need more control over how options are handled, IOpt provides
  great deal of flexibility. Please read the IOpt API to get familiar with it.


  IOpt parse

    Some times you would like to process the command line actions yourself,
  maybe you want to be sure no option is given twice, or that some option
  is always required, or have mutually exclusive actions, etc. You can 
  implement all of this by obtaining an IOpt CommandLine object, that
  represents the parsed command line, including all options found, etc.
  
      cl = opt parse(argv, stopAt: "--", includeUnknownOption: false)
      unless(cl unknownOptions emtpy, 
        error!("Unknown options: %[%s %]" format(cl unknownOptions)))
      unless(cl include?("--required"),
        error!("The --required option must be present!"))
      once = cl options select(o, o option == "--once")
      if(once length > 1, error!("You specified --once more than once"))
      if(cl programArguments empty?,
        error!("Please provide at least one argument"))
      sendToSubProgram(cl rest) ; elements found after --
      fileOption = cl options find(o, o option == "--file")
      fileOption args positional[0] = "IDontMindUserInput.txt"
      fileOption action priority = -99 ; set higher priority
      cl execute
  
]; documentation
  
{'fraction_non_alphanumeric': 0.08735165990686496, 'fraction_numerical': 0.0026288117770767614, 'mean_word_length': 2.726560313462077, 'pattern_counts': {'":': 1, '<': 3, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2444458', 'n_tokens_mistral': 3643, 'n_tokens_neox': 3439, 'n_words': 1735}