<commit_before>.. _analysis:

Analysis
========

The following provides a brief overview for the most common analysis required for each
type of Monte Carlo calculation.  The guides in :ref:`tutorials` provide a step-by-step
guide to analysing HANDE calculations and explain the reasoning behind the required
analysis parameters.

HANDE includes a variety of scripts and utilities in the ``tools`` subdirectory.  However,
these only provide a simple, command-line interface.  A comprehensive python module,
:ref:`pyhande`, drives all the analysis.  :ref:`pyhande` is extremely powerful for dealing
with complex analysis, data-driven investigation or bulk data analysis.

FCIQMC and CCMC
---------------
Basic Usage
^^^^^^^^^^^^^^^

QMC calculations print out data from a block of iterations (a 'report loop'), the length
of which is controlled by the **mc_cycles** input option.  Care should be taken analysing
this data and, in particular, producing accurate estimates of the errors in the means of
the energy estimators.  Almost all data is averaged over the report loop (see output for
further details).

Note that no data is lost when quantities are summed over report loops, as the
correlation length in the data is substantially longer than the length of the
report loop (typically 10-20 iterations).

As the particle distribution at one iteration is not independent from the distribution at
the previous iteration, estimators at each iteration are not independent.  This
correlation in the data needs to be taken into account when estimating standard errors.
A simple and effective way of doing this is to use a blocking analysis
[Flyvbjerg89]_.

The ``reblock_hande.py`` script (in the ``tools`` subdirectory) does this.  Run

.. code-block:: bash

    $ reblock_hande.py --help

to see the available options.  Estimates for the shift and projected energy are
typically obtained using

.. code-block:: bash

    $ reblock_hande.py --start N out

respectively, where ``N`` is the iteration from which data should be blocked (i.e.
after the calculation has equilibrated) and ``out`` is the file to which the
calculation output was saved. Without --start option, this script automatically 
estimates the appropriate ``N``, so you don't have to its value basically.

Note that reblock_hande.py can accept multiple output files for the case when
a calculation is restarted as follows:

.. code-block:: bash

    $ reblock_hande.py -m out1 out2 out3

More complicated analysis can be performed in python by
using the ``pyhande`` library --- ``reblock_hande.py`` simply provides a convenient
interface for the most common analysis tasks.

New Features
^^^^^^^^^^^^^^^

Canonical Total Energy MC
---------------------------

The configurations and resulting estimates in a canonical total energy
calculation are statistically independent and therefore no blocking analysis is
required. The ``analyse_canonical.py`` script is available in ``tools/canonical_energy/`` which
performs the appropriate averaging and standard error analysis on the output file
using the pyhande suite.

DMQMC
-----

No blocking analysis is required for the error analysis of DMQMC calculations
as estimates are averaged over statistically independent runs. The
``finite_temp_analysis.py`` script in ``tools/dmqmc`` can be used to perform a
standard error analysis of the Monte Carlo data for a number of different observables.<commit_msg>Update analysis.rst<commit_after>.. _analysis:

Analysis
========

The following provides a brief overview for the most common analysis required for each
type of Monte Carlo calculation.  The guides in :ref:`tutorials` provide a step-by-step
guide to analysing HANDE calculations and explain the reasoning behind the required
analysis parameters.

HANDE includes a variety of scripts and utilities in the ``tools`` subdirectory.  However,
these only provide a simple, command-line interface.  A comprehensive python module,
:ref:`pyhande`, drives all the analysis.  :ref:`pyhande` is extremely powerful for dealing
with complex analysis, data-driven investigation or bulk data analysis.

FCIQMC and CCMC
---------------
Fundamental Usage
^^^^^^^^^^^^^^^

QMC calculations print out data from a block of iterations (a 'report loop'), the length
of which is controlled by the **mc_cycles** input option.  Care should be taken analysing
this data and, in particular, producing accurate estimates of the errors in the means of
the energy estimators.  Almost all data is averaged over the report loop (see output for
further details).

Note that no data is lost when quantities are summed over report loops, as the
correlation length in the data is substantially longer than the length of the
report loop (typically 10-20 iterations).

As the particle distribution at one iteration is not independent from the distribution at
the previous iteration, estimators at each iteration are not independent.  This
correlation in the data needs to be taken into account when estimating standard errors.
A simple and effective way of doing this is to use a blocking analysis
[Flyvbjerg89]_.

The ``reblock_hande.py`` script (in the ``tools`` subdirectory) does this.  Run

.. code-block:: bash

    $ reblock_hande.py --help

to see the available options.  Estimates for the shift and projected energy are
typically obtained using

.. code-block:: bash

    $ reblock_hande.py --start N out

respectively, where ``N`` is the iteration from which data should be blocked (i.e.
after the calculation has equilibrated) and ``out`` is the file to which the
calculation output was saved. Without --start option, this script automatically 
estimates the appropriate ``N``, so you don't have to its value basically.

Note that reblock_hande.py can accept multiple output files for the case when
a calculation is restarted as follows:

.. code-block:: bash

    $ reblock_hande.py -m out1 out2 out3

More complicated analysis can be performed in python by
using the ``pyhande`` library --- ``reblock_hande.py`` simply provides a convenient
interface for the most common analysis tasks.

New Feature on Error Estimation
^^^^^^^^^^^^^^^
'N_0','Shift','# H psips',]
新たに実装した解析手法では、次のような改良点が見られる。
新たに実装した手法では、<\sum H_0j N_j>/<N_0>ではなく、
<\sum H_0j N_j / N_0>として、射影エネルギーを算定する。



New Feature on Warm-up Steps Detection
^^^^^^^^^^^^^^^


Canonical Total Energy MC
---------------------------

The configurations and resulting estimates in a canonical total energy
calculation are statistically independent and therefore no blocking analysis is
required. The ``analyse_canonical.py`` script is available in ``tools/canonical_energy/`` which
performs the appropriate averaging and standard error analysis on the output file
using the pyhande suite.

DMQMC
-----

No blocking analysis is required for the error analysis of DMQMC calculations
as estimates are averaged over statistically independent runs. The
``finite_temp_analysis.py`` script in ``tools/dmqmc`` can be used to perform a
standard error analysis of the Monte Carlo data for a number of different observables.
{'fraction_non_alphanumeric': 0.07769743371614915, 'fraction_numerical': 0.003261023677867574, 'mean_word_length': 4.401225114854517, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '106189', 'n_tokens_mistral': 1841, 'n_tokens_neox': 1738, 'n_words': 962}