<commit_before>.. _whatsnew_0101:

v0.10.1 (January 22, 2013)
---------------------------

This is a minor release from 0.10.0 and includes new features, enhancements,
and bug fixes. In particular, there is substantial new HDFStore functionality
contributed by <NAME>.

An undesired API breakage with functions taking the ``inplace`` option has been
reverted and deprecation warnings added.

API changes
~~~~~~~~~~~

- Functions taking an ``inplace`` option return the calling object as before. A
  deprecation message has been added
- Groupby aggregations Max/Min no longer exclude non-numeric data (:issue:`2700`)
- Resampling an empty DataFrame now returns an empty DataFrame instead of
  raising an exception (:issue:`2640`)
- The file reader will now raise an exception when NA values are found in an
  explicitly specified integer column instead of converting the column to float
  (:issue:`2631`)
- DatetimeIndex.unique now returns a DatetimeIndex with the same name and
- timezone instead of an array (:issue:`2563`)

New features
~~~~~~~~~~~~

- MySQL support for database (contribution from <NAME>)

HDFStore
~~~~~~~~

You may need to upgrade your existing data files. Please visit the
**compatibility** section in the main docs.


.. ipython:: python
   :suppress:
   :okexcept:

   os.remove('store.h5')

You can designate (and index) certain columns that you want to be able to
perform queries on a table, by passing a list to ``data_columns``

.. ipython:: python

   store = HDFStore('store.h5')
   df = DataFrame(randn(8, 3), index=date_range('1/1/2000', periods=8),
              columns=['A', 'B', 'C'])
   df['string'] = 'foo'
   df.ix[4:6,'string'] = np.nan
   df.ix[7:9,'string'] = 'bar'
   df['string2'] = 'cool'
   df

   # on-disk operations
   store.append('df', df, data_columns = ['B','C','string','string2'])
   store.select('df',[ 'B > 0', 'string == foo' ])

   # this is in-memory version of this type of selection
   df[(df.B > 0) & (df.string == 'foo')]

Retrieving unique values in an indexable or data column.

.. code-block:: python

   # not that these are deprecated as of 0.14.0
   # can be replicated by: store.select_column('df','index').unique()
   store.unique('df','index')
   store.unique('df','string')

You can now store ``datetime64`` in data columns

.. ipython:: python

    df_mixed               = df.copy()
    df_mixed['datetime64'] = Timestamp('20010102')
    df_mixed.ix[3:4,['A','B']] = np.nan

    store.append('df_mixed', df_mixed)
    df_mixed1 = store.select('df_mixed')
    df_mixed1
    df_mixed1.get_dtype_counts()

You can pass ``columns`` keyword to select to filter a list of the return
columns, this is equivalent to passing a
``Term('columns',list_of_columns_to_filter)``

.. ipython:: python

   store.select('df',columns = ['A','B'])
<commit_msg>Update v0.10.1.txt<commit_after>.. _whatsnew_0101:

v0.10.1 (January 22, 2013)
---------------------------

This is a minor release from 0.10.0 and includes new features, enhancements,
and bug fixes. In particular, there is substantial new HDFStore functionality
contributed by <NAME>.

An undesired API breakage with functions taking the ``inplace`` option has been
reverted and deprecation warnings added.

API changes
~~~~~~~~~~~

- Functions taking an ``inplace`` option return the calling object as before. A
  deprecation message has been added
- Groupby aggregations Max/Min no longer exclude non-numeric data (:issue:`2700`)
- Resampling an empty DataFrame now returns an empty DataFrame instead of
  raising an exception (:issue:`2640`)
- The file reader will now raise an exception when NA values are found in an
  explicitly specified integer column instead of converting the column to float
  (:issue:`2631`)
- DatetimeIndex.unique now returns a DatetimeIndex with the same name and
- timezone instead of an array (:issue:`2563`)

New features
~~~~~~~~~~~~

- MySQL support for database (contribution from <NAME>)

HDFStore
~~~~~~~~

You may need to upgrade your existing data files. Please visit the
**compatibility** section in the main docs.


.. ipython:: python
   :suppress:
   :okexcept:

   os.remove('store.h5')

You can designate (and index) certain columns that you want to be able to
perform queries on a table, by passing a list to ``data_columns``

.. ipython:: python

   store = HDFStore('store.h5')
   df = DataFrame(randn(8, 3), index=date_range('1/1/2000', periods=8),
              columns=['A', 'B', 'C'])
   df['string'] = 'foo'
   df.ix[4:6,'string'] = np.nan
   df.ix[7:9,'string'] = 'bar'
   df['string2'] = 'cool'
   df

   # on-disk operations
   store.append('df', df, data_columns = ['B','C','string','string2'])
   store.select('df',[ 'B > 0', 'string == foo' ])

   # this is in-memory version of this type of selection
   df[(df.B > 0) & (df.string == 'foo')]

Retrieving unique values in an indexable or data column.

.. code-block:: python

   # note that this is deprecated as of 0.14.0
   # can be replicated by: store.select_column('df','index').unique()
   store.unique('df','index')
   store.unique('df','string')

You can now store ``datetime64`` in data columns

.. ipython:: python

    df_mixed               = df.copy()
    df_mixed['datetime64'] = Timestamp('20010102')
    df_mixed.ix[3:4,['A','B']] = np.nan

    store.append('df_mixed', df_mixed)
    df_mixed1 = store.select('df_mixed')
    df_mixed1
    df_mixed1.get_dtype_counts()

You can pass ``columns`` keyword to select to filter a list of the return
columns, this is equivalent to passing a
``Term('columns',list_of_columns_to_filter)``

.. ipython:: python

   store.select('df',columns = ['A','B'])

{'fraction_non_alphanumeric': 0.14207165661582122, 'fraction_numerical': 0.026959914863426748, 'mean_word_length': 3.7666948436179206, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5734151', 'n_tokens_mistral': 1858, 'n_tokens_neox': 1716, 'n_words': 625}