<commit_before>=================
python-anyconfig
=================

Generic access to configuration files in any formats (to be in the future).

* Author: <NAME> <<EMAIL>>
* License: MIT

Usage
======

see also: output of `python -c "import anyconfig; help(anyconfig)"`

anyconfig module
-------------------

To load single config file::

  import anyconfig

  # Config type (format) is automatically detected by filename (file
  # extension).
  data1 = anyconfig.load("/path/to/foo/conf.d/a.yml")

  # Loaded config data is a dict-like object.
  # examples:
  # data1["a"] => 1
  # data1["b"]["b1"] => "xyz"
  # data1["c"]["c1"]["c13"] => [1, 2, 3]

  # Same as above
  data2 = anyconfig.single_load("/path/to/foo/conf.d/a.yml")

  # Or you can specify config type explicitly.
  data3 = anyconfg.load("/path/to/foo/conf.d/b.conf", "yaml")

  # Same as above
  data4 = anyconfig.single_load("/path/to/foo/conf.d/b.conf", "yaml")


To load multiple config files::

  import anyconfig

  # Specify config files by list of paths:
  data1 = anyconfig.load(["/etc/foo.d/a.json", "/etc/foo.d/b.json"])

  # Specify config files by glob path pattern:
  data2 = anyconfig.load("/etc/foo.d/*.json")

  # Similar to above, but parameters in the former config file will be simply
  # overwritten by the later ones:
  data3 = anyconfig.load("/etc/foo.d/*.json", merge=anyconfig.MS_REPLACE)

CUI frontend
-------------

There is a CUI frontend 'anyconfig_cui' for demonstration purpose.

It can process various config files and output a summarized config file::

  $ anyconfig_cui -h
  Usage: anyconfg_cui [Options...] CONF_PATH_OR_PATTERN_0 [CONF_PATH_OR_PATTERN_1 ..]

  Examples:
    anyconfg_cui --list
    anyconfg_cui -I yaml /etc/xyz/conf.d/a.conf
    anyconfg_cui -I yaml '/etc/xyz/conf.d/*.conf' -o xyz.conf --otype json
    anyconfg_cui '/etc/xyz/conf.d/*.json' -o xyz.yml \
      --atype json -A '{"obsoletes": "sysdata", "conflicts": "sysdata-old"}'
    anyconfg_cui '/etc/xyz/conf.d/*.json' -o xyz.yml \
      -A obsoletes:sysdata;conflicts:sysdata-old
    anyconfg_cui /etc/foo.json /etc/foo/conf.d/x.json /etc/foo/conf.d/y.json<commit_msg>added explanation of merge storategy when loading multiple configs<commit_after>=================
python-anyconfig
=================

Generic access to configuration files in any formats (to be in the future).

* Author: <NAME> <<EMAIL>>
* License: MIT

Usage
======

see also: output of `python -c "import anyconfig; help(anyconfig)"`

anyconfig module
-------------------

To load single config file::

  import anyconfig

  # Config type (format) is automatically detected by filename (file
  # extension).
  data1 = anyconfig.load("/path/to/foo/conf.d/a.yml")

  # Loaded config data is a dict-like object.
  # examples:
  # data1["a"] => 1
  # data1["b"]["b1"] => "xyz"
  # data1["c"]["c1"]["c13"] => [1, 2, 3]

  # Same as above
  data2 = anyconfig.single_load("/path/to/foo/conf.d/a.yml")

  # Or you can specify config type explicitly.
  data3 = anyconfg.load("/path/to/foo/conf.d/b.conf", "yaml")

  # Same as above
  data4 = anyconfig.single_load("/path/to/foo/conf.d/b.conf", "yaml")


To load multiple config files::

  import anyconfig

  # Specify config files by list of paths:
  data1 = anyconfig.load(["/etc/foo.d/a.json", "/etc/foo.d/b.json"])

  # Specify config files by glob path pattern:
  data2 = anyconfig.load("/etc/foo.d/*.json")

  # Similar to above, but parameters in the former config file will be simply
  # overwritten by the later ones:
  data3 = anyconfig.load("/etc/foo.d/*.json", merge=anyconfig.MS_REPLACE)

On loading multiple config files, you can choose strategy to merge configs from
the followings:

* anyconfig.MS_REPLACE: Replace all configuration parameters provided in former
  config files are simply replaced w/ the ones in later config files.

  For example, if a.yml and b.yml are like followings:

  a.yml::

    a: 1
    b:
       - c: 0
       - c: 2
    d:
       e: "aaa"
       f: 3

  b.yml::

    b:
       - c: 3
    d:
       e: "bbb"

  then::

    load(["a.yml", "b.yml"], merge=anyconfig.MS_REPLACE)

  will give object such like::

    {'a': 1, 'b': [{'c': 3}], 'd': {'e': "bbb"}}

* anyconfig.MS_DICTS: Merge dicts recursively. That is, the following::

    load(["a.yml", "b.yml"], merge=anyconfig.MS_DICTS)

  will give object such like::

    {'a': 1, 'b': [{'c': 3}], 'd': {'e': "bbb", 'f': 3}}

* anyconfig.MS_DICTS_AND_LISTS: Merge dicts and lists recursively. That is, the
  following::

    load(["a.yml", "b.yml"], merge=anyconfig.MS_DICTS_AND_LISTS)

  will give object such like::

    {'a': 1, 'b': [{'c': 0}, {'c': 2}, {'c': 3}], 'd': {'e': "bbb", 'f': 3}}


CUI frontend
-------------

There is a CUI frontend 'anyconfig_cui' for demonstration purpose.

It can process various config files and output a summarized config file::

  $ anyconfig_cui -h
  Usage: anyconfg_cui [Options...] CONF_PATH_OR_PATTERN_0 [CONF_PATH_OR_PATTERN_1 ..]

  Examples:
    anyconfg_cui --list
    anyconfg_cui -I yaml /etc/xyz/conf.d/a.conf
    anyconfg_cui -I yaml '/etc/xyz/conf.d/*.conf' -o xyz.conf --otype json
    anyconfg_cui '/etc/xyz/conf.d/*.json' -o xyz.yml \
      --atype json -A '{"obsoletes": "sysdata", "conflicts": "sysdata-old"}'
    anyconfg_cui '/etc/xyz/conf.d/*.json' -o xyz.yml \
      -A obsoletes:sysdata;conflicts:sysdata-old
    anyconfg_cui /etc/foo.json /etc/foo/conf.d/x.json /etc/foo/conf.d/y.json
{'fraction_non_alphanumeric': 0.17498618530116045, 'fraction_numerical': 0.010130779149014552, 'mean_word_length': 3.1355674028941354, 'pattern_counts': {'":': 4, '<': 9, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3976663', 'n_tokens_mistral': 2069, 'n_tokens_neox': 1943, 'n_words': 548}