<commit_before>#`chidley`



##`chidley` converts *any* XML to Go structs (and therefor to JSON)
* By *any*, any XML that can be read by the Go [xml package](http://golang.org/pkg/encoding/xml/) decoder. 
* Where *convert* means, generates Go code that when compiled, will convert the XML to JSON
* or will just generate the Go structs that represent the input XML
* or converts XML to XML (useful for validation) 

Author: <NAME>
Language: Go

##New

#2015.07.24
`chidley` now supports the user naming of the resulting JAXB Java class package.
Previously the package name could only be `ca/gnewton/chidley/jaxb`.
Now, using the `-P name`, the `jaxb` default can be altered.

So `chidley -J -P "foobar" sample.xml` will result in Java classes with package name: `ca/gnewton/chidley/foobar`.


###Previous

`chidley` now has support for Java/JAXB. It generates appropriate Java/JAXB classes and associated maven pom.

See [Java/JAXB section below](#user-content-java) for usage.


##How does it work (with such a small memory footprint)
`chidley` uses the input XML to build a model of each XML element (aka tag).
It examines each instance of a tag, and builds a (single) prototypical representation; that is, the union of all the attributes and all of the child elements of all instances of the tag.
So even if there are million instances of a specific tag, there is only one model tag representation.
Note that a tag is unique by its namespace+tagname combination (in the Go xml package parlance, [`space + local`](http://golang.org/pkg/encoding/xml/#Name).
###Types
`chidley` by default makes all values (attributes, tag content) in the generated Go structs a string (which is always valid for XML attributes and content), but it has a flag (`-t`) where it will detect and use the most appropriate type. 
`chidley` tries to fit the **smallest** Go type. 
For example, if all instances of a tag contain a number, and all instances are -128 to 127, then it will use an `int8` in the Go struct.

##`chidley` binary
Compiled for 64bit Linux Fedora18, go version go1.3 linux/amd64

##Usage
```
Usage of ./chidley:
  -B=false: Add database metadata to created Go structs
  -D="java": Base directory for generated Java code (root of maven project)
  -G=false: Only write generated Go structs to stdout
  -J=false: Generated Java code for Java/JAXB
  -P="": Java package name (rightmost in full package name
  -W=false: Generate Go code to convert XML to JSON or XML (latter useful for validation) and write it to stdout
  -a="Attr": Prefix to attribute names
  -c=false: Read XML from standard input
  -d=false: Debug; prints out much information
  -e="Chi": Prefix to struct (element) names; must start with a capital
  -k="jaxb": App name for Java code (appended to ca.gnewton.chidley Java package name))
  -n=false: Use the XML namespace prefix as prefix to JSON name; prefix followed by 2 underscores (__)
  -p=false: Pretty-print json in generated code (if applicable)
  -r=false: Progress: every 50000 input tags (elements)
  -t=false: Use type info obtained from XML (int, bool, etc); default is to assume everything is a string; better chance at working if XMl sample is not complete
  -u=false: Filename interpreted as an URL
  -x=false: Add XMLName (Space, Local) for each XML element, to JSON
$
```

<commit_msg>Removed -B in documentation: implementation not done yet...<commit_after>#`chidley`



##`chidley` converts *any* XML to Go structs (and therefor to JSON)
* By *any*, any XML that can be read by the Go [xml package](http://golang.org/pkg/encoding/xml/) decoder. 
* Where *convert* means, generates Go code that when compiled, will convert the XML to JSON
* or will just generate the Go structs that represent the input XML
* or converts XML to XML (useful for validation) 

Author: <NAME>
Language: Go

##New

#2015.07.24
`chidley` now supports the user naming of the resulting JAXB Java class package.
Previously the package name could only be `ca/gnewton/chidley/jaxb`.
Now, using the `-P name`, the `jaxb` default can be altered.

So `chidley -J -P "foobar" sample.xml` will result in Java classes with package name: `ca/gnewton/chidley/foobar`.


###Previous

`chidley` now has support for Java/JAXB. It generates appropriate Java/JAXB classes and associated maven pom.

See [Java/JAXB section below](#user-content-java) for usage.


##How does it work (with such a small memory footprint)
`chidley` uses the input XML to build a model of each XML element (aka tag).
It examines each instance of a tag, and builds a (single) prototypical representation; that is, the union of all the attributes and all of the child elements of all instances of the tag.
So even if there are million instances of a specific tag, there is only one model tag representation.
Note that a tag is unique by its namespace+tagname combination (in the Go xml package parlance, [`space + local`](http://golang.org/pkg/encoding/xml/#Name).
###Types
`chidley` by default makes all values (attributes, tag content) in the generated Go structs a string (which is always valid for XML attributes and content), but it has a flag (`-t`) where it will detect and use the most appropriate type. 
`chidley` tries to fit the **smallest** Go type. 
For example, if all instances of a tag contain a number, and all instances are -128 to 127, then it will use an `int8` in the Go struct.

##`chidley` binary
Compiled for 64bit Linux Fedora18, go version go1.3 linux/amd64

##Usage
```
Usage of ./chidley:
  -D="java": Base directory for generated Java code (root of maven project)
  -G=false: Only write generated Go structs to stdout
  -J=false: Generated Java code for Java/JAXB
  -P="": Java package name (rightmost in full package name
  -W=false: Generate Go code to convert XML to JSON or XML (latter useful for validation) and write it to stdout
  -a="Attr": Prefix to attribute names
  -c=false: Read XML from standard input
  -d=false: Debug; prints out much information
  -e="Chi": Prefix to struct (element) names; must start with a capital
  -k="jaxb": App name for Java code (appended to ca.gnewton.chidley Java package name))
  -n=false: Use the XML namespace prefix as prefix to JSON name; prefix followed by 2 underscores (__)
  -p=false: Pretty-print json in generated code (if applicable)
  -r=false: Progress: every 50000 input tags (elements)
  -t=false: Use type info obtained from XML (int, bool, etc); default is to assume everything is a string; better chance at working if XMl sample is not complete
  -u=false: Filename interpreted as an URL
  -x=false: Add XMLName (Space, Local) for each XML element, to JSON
$
```


{'fraction_non_alphanumeric': 0.0848173860549351, 'fraction_numerical': 0.008753395713854513, 'mean_word_length': 4.197647058823529, 'pattern_counts': {'":': 10, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 10}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '551051', 'n_tokens_mistral': 1993, 'n_tokens_neox': 1873, 'n_words': 1001}