<commit_before>Quick start
===========

Installation
------------
The easiest is just to install it using PyPI::

      pip install bandicoot


Alternatively, you can download it from `here <https://github.com/yvesalexandre/bandicoot/archive/master.zip>`_. Once unzipped you can either import it ``import bandicoot as bc`` or install it::

        python setup.py install


Loading data
------------
bandicoot takes one files per user as standard input and assume that all the user records are in the same directory `records/`.

>>> B = bc.read_csv('my_user', 'records/', 'antennas.csv')

This :meth:`~bandicoot.io.read_csv` will load the records of the user `my_user` from `records/my_user.csv` and the antenna file from `antennas.csv`.


.. image:: _static/mini-mockups-01.png


bandicoot's records files are per user and structured as follow.

=========== ========= ================ =================== ============= ===========
interaction direction correspondent_id datetime            call_duration antenna_id
=========== ========= ================ =================== ============= ===========
call        in        8f8ad28de134     2012-05-20 20:30:37 137           13084
call        out       fe01d67aeccd     2012-05-20 20:31:42 542           13084
text        in        c8f538f1ccb2     2012-05-20 21:10:31               13087
=========== ========= ================ =================== ============= ===========


records.csv::

  interaction,direction,correspondent_id,datetime,call_duration,antenna_id
  call,in,8f8ad28de134,2012-05-20 20:30:37,137,13084
  call,out,fe01d67aeccd,2012-05-20 20:31:42,542,13084
  text,in,c8f538f1ccb2,2012-05-20 21:10:31,,13087

while the antennas file contains the latitude and longitude coordinates of the antennas::

  antenna_id,latitude,longitude
  13084,42.360888,-71.0877297
  13087,42.367709,-71.107692

Computing indicators
--------------------

The behavioral indicators can be computed using individual function such as :meth:`~bandicoot.individual.response_rate_text` and :meth:`~bandicoot.spatial.radius_of_gyration` or by using :meth:`~bandicoot.utils.all` which return all the indicators.


  >>> B = bc.tests.sample_user()  # Load a sample user
  >>> bc.utils.all(B)
  OrderedDict({'name': 'sample_user',
    'reporting': OrderedDict({'antennas_path': None,
    'groupby': 'week',
    'start_time': '2012-01-01 00:55:56',
    'end_time': '2014-11-27 00:31:44',
    ...,
    'call_duration': {'call': {'std': {'std': 85.48627089424896, 'mean': 238.69736346741757}, 'mean': {'std': 133.02496554053093, 'mean': 509.85158868177155}}},
    'percent_nocturnal': {'text': {'std': 0.11496930069267557, 'mean': 0.9108055090449453}, 'call': {'std': 0.11006480488825417, 'mean': 0.9244309521895726}},
    ...
    'percent_initiated_interactions': {'call': {'std': 0.22322545460001883, 'mean': 0.4735613383686402}},
    ...
    'radius_of_gyration': {'std': 0.26544350516080456, 'mean': 1.4274315700646447},
    'frequent_antennas': {'std': 0.8516613127743775, 'mean': 3.586206896551724}})

:meth:`~bandicoot.utils.all` returns a nested dictionary with all the indicators (:mod:`bandicoot.individual`, :mod:`bandicoot.spatial`, and :mod:`bandicoot.network`) and some reporting metrics (the name of the user, ``groupby``, the ``version`` of bandicoot used, the number of ``records_missing_locations``, etc)


By default, bandicoot computes the indicators on a **weekly basis** over all the weeks available. It then returns their mean and standard deviation in a nested dictionary. bandicoot consider weeks starting on Monday and ending on Sunday.

.. image:: _static/mini-mockups-02.png


The parameter ``groupby=None`` can be used to compute the indicators on the entire timeframe instead.

  >>> bc.utils.all(B, groupby=None)
  OrderedDict({'name': 'sample_user',
    'reporting': OrderedDict({'antennas_path': None,
    'groupby': None,
    'start_time': '2012-01-01 00:55:56',
    'end_time': '2014-11-27 00:31:44',
    ...,
    'call_duration': {'call': {'std': 288.20204388583556, 'mean': 509.09016393442624}},
    'percent_nocturnal': {'text': 0.9065040650406504, 'call': 0.9180327868852459},
    ...
    'percent_initiated_interactions': {'call': 0.4887295081967213},
    ...
    'radius_of_gyration': 1.5368293314872674,
    'frequent_antennas': 6})



Note that, while some indicators return a mean and a std per time period (week or the entire period), other return only one value. For example, :meth:`~bandicoot.individual.percent_initiated_interactions` and :meth:`~bandicoot.individual.active_days` return only one value per time period, the percentage of interactions initiated by the user (48.8%) and the number of days he has been active. Other, such as :meth:`~bandicoot.individual.call_duration` will return the mean and std of the value over the time period (509 seconds on average with a standard deviation of 288 seconds). If passed ``summary=extended``, bandicoot will also return the weekly median, min, max, kurtosis, and skewness::

  >>> bc.individual.call_duration(B, groupby=None)
  {'call': {'mean': 509.09016393442624, 'std': 288.20204388583556}}
  >>> bc.individual.call_duration(B, summary='extended', groupby=None)
  {'call': {'kurtosis': 1.7893082779332345,
  'max': 997.0,
  'mean': 509.09016393442624,
  'median': 512.5,
  'min': 7.0,
  'skewness': -0.025855693793360647,
  'std': 288.20204388583556}}

``summary=extended`` can also be passed to :meth:`~bandicoot.utils.all`::
<commit_msg>Updated quickstart to reflect new all() output, addition of split_week split_day.<commit_after>Quick start
===========

Installation
------------
The easiest is just to install it using PyPI::

      pip install bandicoot


Alternatively, you can download it from `here <https://github.com/yvesalexandre/bandicoot/archive/master.zip>`_. Once unzipped you can either import it ``import bandicoot as bc`` or install it::

        python setup.py install


Loading data
------------
bandicoot takes one files per user as standard input and assume that all the user records are in the same directory `records/`.

>>> B = bc.read_csv('my_user', 'records/', 'antennas.csv')

This :meth:`~bandicoot.io.read_csv` will load the records of the user `my_user` from `records/my_user.csv` and the antenna file from `antennas.csv`.


.. image:: _static/mini-mockups-01.png


bandicoot's records files are per user and structured as follow.

=========== ========= ================ =================== ============= ===========
interaction direction correspondent_id datetime            call_duration antenna_id
=========== ========= ================ =================== ============= ===========
call        in        8f8ad28de134     2012-05-20 20:30:37 137           13084
call        out       fe01d67aeccd     2012-05-20 20:31:42 542           13084
text        in        c8f538f1ccb2     2012-05-20 21:10:31               13087
=========== ========= ================ =================== ============= ===========


records.csv::

  interaction,direction,correspondent_id,datetime,call_duration,antenna_id
  call,in,8f8ad28de134,2012-05-20 20:30:37,137,13084
  call,out,fe01d67aeccd,2012-05-20 20:31:42,542,13084
  text,in,c8f538f1ccb2,2012-05-20 21:10:31,,13087

while the antennas file contains the latitude and longitude coordinates of the antennas::

  antenna_id,latitude,longitude
  13084,42.360888,-71.0877297
  13087,42.367709,-71.107692

Computing indicators
--------------------

The behavioral indicators can be computed using individual function such as :meth:`~bandicoot.individual.response_rate_text` and :meth:`~bandicoot.spatial.radius_of_gyration` or by using :meth:`~bandicoot.utils.all` which return all the indicators.


  >>> B = bc.tests.sample_user()  # Load a sample user
  >>> bc.utils.all(B)
  OrderedDict({'name': 'sample_user',
    'reporting': OrderedDict({'antennas_path': None,
    'groupby': 'week',
    'split_week': false, 
    'split_day': false, 
    'start_time': '2012-01-01 00:55:56',
    'end_time': '2014-11-27 00:31:44',
    ...,
    'call_duration': {
        'allweek': {
            'allday': {
                'call': {
                    'std': {
                        'std': 0.0, 
                        'mean': 202.5
                    }, 
                    'mean: {
                        'std': 0.0, 
                        'mean': 339.5
                    }
                }
            }
        }
    }, ,
    'percent_nocturnal': {
        'allweek': {
            'allday': {
                'text': {
                    'std': 0.0, 
                    'mean': 1.0
                }, 
                'call': {
                    'std': 0.0, 
                    'mean': 1.0
                }
            }
        }
    },
    ...
    'percent_initiated_interactions': {
        'allweek': {
            'allday': {
                'call': {
                    'std': 0.0, 
                    'mean': 0.5
                }
            }
        }
    },
    ...
    'radius_of_gyration': {
        'allweek': {
            'allday': {
                'std': 0.0, 
                'mean': 1.2777217936866738
            }
        }
    },
    'frequent_antennas': {
        'allweek': {
            'allday': {
                'std': 0.0, 
                'mean': 1.0
            }
        }
    }})

:meth:`~bandicoot.utils.all` returns a nested dictionary with all the indicators (:mod:`bandicoot.individual`, :mod:`bandicoot.spatial`, and :mod:`bandicoot.network`) and some reporting metrics (the name of the user, ``groupby``, the ``version`` of bandicoot used, the number of ``records_missing_locations``, etc)


By default, bandicoot computes the indicators on a **weekly basis** over all the weeks available. It then returns their mean and standard deviation in a nested dictionary. bandicoot consider weeks starting on Monday and ending on Sunday.  The parameter ``groupby=None`` can be used to compute the indicators on the entire timeframe instead.  (See below).

The keys ``allweek`` and ``allday`` become meaningful if we specify that data should be also be generated about the weekend and weeknights separately (``split_week=True``) or days and nights separately (``split_day=True``): 

.. image:: _static/mini-mockups-02.png

  >>> bc.utils.all(B, groupby=None, split_week=True, split_day=True)
  OrderedDict({'name': 'sample_user',
    'reporting': OrderedDict({'antennas_path': None,
    'groupby': None,
    'start_time': '2012-01-01 00:55:56',
    'end_time': '2014-11-27 00:31:44',
    'split_week': true, 
    'split_day': true, 
    ...,
    'call_duration': {
        'weekend': {
            'allday': {
                'call': {
                    'std': 202.5, 
                    'mean': 339.5
                }
            }, 
            'day': {
                'call': {
                    'std': 202.5, 
                    'mean': 339.5
                }
            }, 
            'night': {
                'call': {
                    'std': null, 
                    'mean': null
                }
            }
        }, 
        'weekday': {
            'allday': {
                'call': {
                    'std': null, 
                    'mean': null
                }
            }, 
            'day': {
                'call': {
                    'std': null, 
                    'mean': null
                }
            }, 
            'night': {
                'call': {
                    'std': null, 
                    'mean': null
                }
            }
        }, 
        'allweek': {
            'allday': {
                'call': {
                    'std': 202.5, 
                    'mean': 339.5
                }
            }, 
            'day': {
                'call': {
                    'std': 202.5, 
                    'mean': 339.5
                }
            }, 
            'night': {
                'call': {
                    'std': null, 
                    'mean': null
                }
            }
        }
    }, 
    ...
    })



Note that, while some indicators return a mean and a std per time period (week or the entire period), other return only one value. For example, :meth:`~bandicoot.individual.percent_initiated_interactions` and :meth:`~bandicoot.individual.active_days` return only one value per time period, the percentage of interactions initiated by the user (48.8%) and the number of days he has been active. Other, such as :meth:`~bandicoot.individual.call_duration` will return the mean and std of the value over the time period (509 seconds on average with a standard deviation of 288 seconds). If passed ``summary=extended``, bandicoot will also return the weekly median, min, max, kurtosis, and skewness::

  >>> bc.individual.call_duration(B, groupby=None)
  {'call': {'mean': 509.09016393442624, 'std': 288.20204388583556}}
  >>> bc.individual.call_duration(B, summary='extended', groupby=None)
  {'call': {'kurtosis': 1.7893082779332345,
  'max': 997.0,
  'mean': 509.09016393442624,
  'median': 512.5,
  'min': 7.0,
  'skewness': -0.025855693793360647,
  'std': 288.20204388583556}}

``summary=extended`` can also be passed to :meth:`~bandicoot.utils.all`::

{'fraction_non_alphanumeric': 0.16404833836858007, 'fraction_numerical': 0.09018126888217523, 'mean_word_length': 2.4427977119084763, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 41, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3581267', 'n_tokens_mistral': 4985, 'n_tokens_neox': 3953, 'n_words': 1064}