<commit_before>.. _whatsnew_0700:

v.0.7.0 (Not Yet Released)
--------------------------

API Changes
~~~~~~~~~~~

New features
~~~~~~~~~~~~

- New unified :ref:`merge function <merging.join>` for efficiently performing
  full gamut of database / relational-algebra operations. Refactored existing
  join methods to use the new infrastructure, resulting in substantial
  performance gains (GH220_, GH249_, GH267_)

- New :ref:`unified concatenation function <merging.concat>` for concatenating
  Series, DataFrame or Panel objects along an axis. Can form union or
  intersection of the other axes. Improves performance of ``Series.append`` and
  ``DataFrame.append`` (GH468_, GH479_, GH273_)

- :ref:`Can <merging.concatenation>` pass multiple DataFrames to
  `DataFrame.append` to concatenate (stack) and multiple Series to
  ``Series.append`` too

- :ref:`Can<basics.dataframe.from_list_of_dicts>` pass list of dicts (e.g., a
  list of JSON objects) to DataFrame constructor (GH526_)

- You can now :ref:`set multiple columns <indexing.columns.multiple>` in a
  DataFrame via ``__getitem__``, useful for transformation (GH342_)

- Handle differently-indexed output values in ``DataFrame.apply`` (GH498_)

.. ipython:: python

   df = DataFrame(randn(10, 4))
   df.apply(lambda x: x.describe())

- :ref:`Add<indexing.reorderlevels>` ``reorder_levels`` method to Series and
  DataFrame (PR534_)

- :ref:`Add<indexing.dictionarylike>` dict-like ``get`` function to DataFrame
  and Panel (PR521_)

- :ref:`Add<basics.iterrows>` ``DataFrame.iterrows`` method for efficiently
  iterating through the rows of a DataFrame

- :ref:`Add<dsintro.to_panel>` ``DataFrame.to_panel`` with code adapted from
  ``LongPanel.to_long``

- :ref:`Add <basics.reindexing>` ``reindex_axis`` method added to DataFrame

- :ref:`Add <basics.stats>` ``level`` option to binary arithmetic functions on
  ``DataFrame`` and ``Series``

- :ref:`Add <indexing.advanced_reindex>` ``level`` option to the ``reindex``
  and ``align`` methods on Series and DataFrame for broadcasting values across
  a level (GH542_, PR552_, others)

- :ref:`Add <dsintro.panel_item_selection>` attribute-based item access to
  ``Panel`` and add IPython completion (PR563_)

- :ref:`Add <visualization.basic>` ``logy`` option to ``Series.plot`` for
  log-scaling on the Y axis

- :ref:`Add <io.formatting>` ``index`` and ``header`` options to
  ``DataFrame.to_string``

- :ref:`Can <merging.multiple_join>` pass multiple DataFrames to
  ``DataFrame.join`` to join on index (GH166_)

- :ref:`Can <merging.multiple_join>` pass multiple Panels to ``Panel.join``
  (GH115_)

- :ref:`Added <io.formatting>` ``justify`` argument to ``DataFrame.to_string``
  to allow different alignment of column headers

- :ref:`Add <groupby.atributes>` ``sort`` option to GroupBy to allow disabling
  sorting of the group keys for potential speedups (GH595_)

- :ref:`Can <basics.dataframe.from_series>` pass MaskedArray to Series
  constructor (PR563_)

- :ref:`Add <dsintro.panel_item_selection>` Panel item access via attributes
  and IPython completion (GH554_)

- Implement ``DataFrame.lookup``, fancy-indexing analogue for retrieving values
  given a sequence of row and column labels (GH338_)

Performance improvements
~~~~~~~~~~~~~~~~~~~~~~~~

:ref:`Cythonized GroupBy aggregations <groupby.aggregate.cython>` no longer
presort the data, thus achieving a significant speedup (GH93_). Here's a
graph of the performance of this operation over time on a dataset with 100,000
rows and 10,000 unique groups:
<commit_msg>DOC: release notes, what's new in 0.7.0<commit_after>.. _whatsnew_0700:

v.0.7.0 (Not Yet Released)
--------------------------

API Changes
~~~~~~~~~~~

New features
~~~~~~~~~~~~

- New unified :ref:`merge function <merging.join>` for efficiently performing
  full gamut of database / relational-algebra operations. Refactored existing
  join methods to use the new infrastructure, resulting in substantial
  performance gains (GH220_, GH249_, GH267_)

- New :ref:`unified concatenation function <merging.concat>` for concatenating
  Series, DataFrame or Panel objects along an axis. Can form union or
  intersection of the other axes. Improves performance of ``Series.append`` and
  ``DataFrame.append`` (GH468_, GH479_, GH273_)

- :ref:`Can <merging.concatenation>` pass multiple DataFrames to
  `DataFrame.append` to concatenate (stack) and multiple Series to
  ``Series.append`` too

- :ref:`Can<basics.dataframe.from_list_of_dicts>` pass list of dicts (e.g., a
  list of JSON objects) to DataFrame constructor (GH526_)

- You can now :ref:`set multiple columns <indexing.columns.multiple>` in a
  DataFrame via ``__getitem__``, useful for transformation (GH342_)

- Handle differently-indexed output values in ``DataFrame.apply`` (GH498_)

.. ipython:: python

   df = DataFrame(randn(10, 4))
   df.apply(lambda x: x.describe())

- :ref:`Add<indexing.reorderlevels>` ``reorder_levels`` method to Series and
  DataFrame (PR534_)

- :ref:`Add<indexing.dictionarylike>` dict-like ``get`` function to DataFrame
  and Panel (PR521_)

- :ref:`Add<basics.iterrows>` ``DataFrame.iterrows`` method for efficiently
  iterating through the rows of a DataFrame

- :ref:`Add<dsintro.to_panel>` ``DataFrame.to_panel`` with code adapted from
  ``LongPanel.to_long``

- :ref:`Add <basics.reindexing>` ``reindex_axis`` method added to DataFrame

- :ref:`Add <basics.stats>` ``level`` option to binary arithmetic functions on
  ``DataFrame`` and ``Series``

- :ref:`Add <indexing.advanced_reindex>` ``level`` option to the ``reindex``
  and ``align`` methods on Series and DataFrame for broadcasting values across
  a level (GH542_, PR552_, others)

- :ref:`Add <dsintro.panel_item_selection>` attribute-based item access to
  ``Panel`` and add IPython completion (PR563_)

- :ref:`Add <visualization.basic>` ``logy`` option to ``Series.plot`` for
  log-scaling on the Y axis

- :ref:`Add <io.formatting>` ``index`` and ``header`` options to
  ``DataFrame.to_string``

- :ref:`Can <merging.multiple_join>` pass multiple DataFrames to
  ``DataFrame.join`` to join on index (GH115_)

- :ref:`Can <merging.multiple_join>` pass multiple Panels to ``Panel.join``
  (GH115_)

- :ref:`Added <io.formatting>` ``justify`` argument to ``DataFrame.to_string``
  to allow different alignment of column headers

- :ref:`Add <groupby.atributes>` ``sort`` option to GroupBy to allow disabling
  sorting of the group keys for potential speedups (GH595_)

- :ref:`Can <basics.dataframe.from_series>` pass MaskedArray to Series
  constructor (PR563_)

- :ref:`Add <dsintro.panel_item_selection>` Panel item access via attributes
  and IPython completion (GH554_)

- Implement ``DataFrame.lookup``, fancy-indexing analogue for retrieving values
  given a sequence of row and column labels (GH338_)

- Can pass a :ref:`list of functions <groupby.aggregate.multifunc>` to
  aggregate with groupby on a DataFrame, yielding an aggregated result with
  hierarchical columns (GH166_)

Performance improvements
~~~~~~~~~~~~~~~~~~~~~~~~

:ref:`Cythonized GroupBy aggregations <groupby.aggregate.cython>` no longer
presort the data, thus achieving a significant speedup (GH93_). Here's a
graph of the performance of this operation over time on a dataset with 100,000
rows and 10,000 unique groups:

{'fraction_non_alphanumeric': 0.13341543513957307, 'fraction_numerical': 0.02353585112205802, 'mean_word_length': 4.884863123993559, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 48, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5370808', 'n_tokens_mistral': 2420, 'n_tokens_neox': 2169, 'n_words': 768}