<commit_before>* Riak 1.0.2 Release Notes
** Bugs Fixed
-[[https://issues.basho.com/show_bug.cgi?id=1227][bz1227 - badstate crash in handoff]]
-[[https://issues.basho.com/show_bug.cgi?id=1261][bz1261 - pipe mapreduce jobs fail in mixed 1.0.0/1.0.1 clusters]]
-[[https://issues.basho.com/show_bug.cgi?id=1263][bz1263 - bitcask-backed vnodes handle requests while dropping]]
-[[https://issues.basho.com/show_bug.cgi?id=1265][bz1265 - eleveldb not opened with configuration options from environment]]

** Other Additions
*** New stats
Two new stats categories have been added to riak stats.

-=node_get_fsm_siblings= reports the statstics on the number of siblings retrieved with get over the last minute.
-=node_get_fsm_objsize= reports an estimate of the object sizes retrieved with get over the last minute.

For both categories mean, median, max, 95th & 99th percentiles are given.  
The new stats are available using riak-admin status or from the /stats endpoint.
** Known Issues
*** Pipe based MapReduce during rolling upgrades
[[https://issues.basho.com/show_bug.cgi?id=1261][BZ1261]] 
A bug was found that caused pipe-based MapReduce to be
unusable during rolling upgrades from 1.0.0 to 1.0.1 (they work again
once the upgrade is complete).  The problem has been fixed such that
MapReduce will stay available when upgrading from 1.0.1 to 1.0.2, and
also from 1.0.2 onward.

If you are still using Riak 1.0.0, pipe-based MapReduce will still be
unusable during the rolling upgrade to 1.0.2.  We apologize for the
inconvenience.  If your MapReduce queries work on the legacy
subsystem, you may want to switch back to it for the duration of your
upgrade to 1.0.2.  To switch MapReduce subsystems, either edit the
'mapred_system' setting in your app.config and restart each node, or
connect to each nodes console and evaluate
'application:set_env(riak_kv, mapred_system, Choice).' where 'Choice'
is either 'legacy' or 'pipe'.

*** Large Ring Sizes

The new cluster protocol in 1.0 leads to a practical limit on the size
of the ring in a Riak cluster. While there is no hard coded limit, large
ring sizes lead to increasingly slower performance and potentially unstable
clusters. This limitation will be addressed in a future major release of
Riak. For the time being, it is recommended that users limit clusters to
no more than 256, 512, 1024 partitions depending on the performance of the
underlying hardware.

*** Ownership Handoff Stall

It has been observed that 1.0 clusters can end up in a state with ownership
handoff failing to complete. This state should not happen under normal
circumstances, but has been observed in cases where Riak nodes crashed due
to unrelated issues (eg. running out of disk space/memory) during cluster
membership changes. The behavior can be identified by looking at the output
of =riak-admin ring_status= and looking for transfers that are waiting on an
empty set. As an example:

#+BEGIN_SRC
============================== Ownership Handoff ==============================
Owner:      riak@host1
Next Owner: riak@host2

Index: 123456789123456789123456789123456789123456789123
  Waiting on: []
  Complete:   [riak_kv_vnode,riak_pipe_vnode]
#+END_SRC

To fix this issue, copy/paste the following code sequence into an Erlang
shell for each =Owner= node in =riak-admin ring_status= for which
this case has been identified. The Erlang shell can be accessed with =riak attach=

#+BEGIN_SRC
fun() ->
  Node = node(),
  {_Claimant, _RingReady, _Down, _MarkedDown, Changes} =
      riak_core_status:ring_status(),
  Stuck =
      [{Idx, Complete} || {{Owner, NextOwner}, Transfers} <- Changes,
                          {Idx, Waiting, Complete, Status} <- Transfers,
                          Owner =:= Node,
                          Waiting =:= [],
                          Status =:= awaiting],

  RT = fun(Ring, _) ->
           NewRing = lists:foldl(fun({Idx, Mods}, Ring1) ->
                         lists:foldl(fun(Mod, Ring2) ->
                             riak_core_ring:handoff_complete(Ring2, Idx, Mod)
                         end, Ring1, Mods)
                     end, Ring, Stuck),
           {new_ring, NewRing}
       end,

  case Stuck of
      [] ->
          ok;
      _ ->
          riak_core_ring_manager:ring_trans(RT, []),
          ok
  end
end().
#+END_SRC

*** Innostore on 32-bit systems

Depending on the partition count and the default stack size of the
operating system, using Innostore as the backend on 32-bit systems can
cause problems. A symptom of this problem would be a message similar
to this on the console or in the Riak error log:

#+BEGIN_SRC
10:22:38.719 [error] Failed to start riak_kv_innostore_backend for index 1415829711164312202009819681693899175291684651008. Reason: {eagain,[{erlang,open_port,[{spawn,innostore_drv},[binary]]},{innostore,connect,0},{riak_kv_innostore_backend,start,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}
10:22:38.871 [notice] "backend module failed to start."
#+END_SRC

The workaround for this problem is to reduce the stack size for the
user the Riak is run as (=riak= for most distributions). For example
on Linux, the current stack size can be viewed using =ulimit -s= and<commit_msg>Fix grammar typo<commit_after>* Riak 1.0.2 Release Notes
** Bugs Fixed
-[[https://issues.basho.com/show_bug.cgi?id=1227][bz1227 - badstate crash in handoff]]
-[[https://issues.basho.com/show_bug.cgi?id=1261][bz1261 - pipe mapreduce jobs fail in mixed 1.0.0/1.0.1 clusters]]
-[[https://issues.basho.com/show_bug.cgi?id=1263][bz1263 - bitcask-backed vnodes handle requests while dropping]]
-[[https://issues.basho.com/show_bug.cgi?id=1265][bz1265 - eleveldb not opened with configuration options from environment]]

** Other Additions
*** New stats
Two new stats categories have been added to riak stats.

-=node_get_fsm_siblings= reports the statstics on the number of siblings retrieved with get over the last minute.
-=node_get_fsm_objsize= reports an estimate of the object sizes retrieved with get over the last minute.

For both categories mean, median, max, 95th & 99th percentiles are given.  
The new stats are available using riak-admin status or from the /stats endpoint.
** Known Issues
*** Pipe based MapReduce during rolling upgrades
[[https://issues.basho.com/show_bug.cgi?id=1261][BZ1261]] 
A bug was found that caused pipe-based MapReduce to be
unusable during rolling upgrades from 1.0.0 to 1.0.1 (they work again
once the upgrade is complete).  The problem has been fixed such that
MapReduce will stay available when upgrading from 1.0.1 to 1.0.2, and
also from 1.0.2 onward.

If you are still using Riak 1.0.0, pipe-based MapReduce will still be
unusable during the rolling upgrade to 1.0.2.  We apologize for the
inconvenience.  If your MapReduce queries work on the legacy
subsystem, you may want to switch back to it for the duration of your
upgrade to 1.0.2.  To switch MapReduce subsystems, either edit the
'mapred_system' setting in your app.config and restart each node, or
connect to each nodes console and evaluate
'application:set_env(riak_kv, mapred_system, Choice).' where 'Choice'
is either 'legacy' or 'pipe'.

*** Large Ring Sizes

The new cluster protocol in 1.0 leads to a practical limit on the size
of the ring in a Riak cluster. While there is no hard coded limit, large
ring sizes lead to increasingly slower performance and potentially unstable
clusters. This limitation will be addressed in a future major release of
Riak. For the time being, it is recommended that users limit clusters to
no more than 256, 512, 1024 partitions depending on the performance of the
underlying hardware.

*** Ownership Handoff Stall

It has been observed that 1.0 clusters can end up in a state with ownership
handoff failing to complete. This state should not happen under normal
circumstances, but has been observed in cases where Riak nodes crashed due
to unrelated issues (eg. running out of disk space/memory) during cluster
membership changes. The behavior can be identified by looking at the output
of =riak-admin ring_status= and looking for transfers that are waiting on an
empty set. As an example:

#+BEGIN_SRC
============================== Ownership Handoff ==============================
Owner:      riak@host1
Next Owner: riak@host2

Index: 123456789123456789123456789123456789123456789123
  Waiting on: []
  Complete:   [riak_kv_vnode,riak_pipe_vnode]
#+END_SRC

To fix this issue, copy/paste the following code sequence into an Erlang
shell for each =Owner= node in =riak-admin ring_status= for which
this case has been identified. The Erlang shell can be accessed with =riak attach=

#+BEGIN_SRC
fun() ->
  Node = node(),
  {_Claimant, _RingReady, _Down, _MarkedDown, Changes} =
      riak_core_status:ring_status(),
  Stuck =
      [{Idx, Complete} || {{Owner, NextOwner}, Transfers} <- Changes,
                          {Idx, Waiting, Complete, Status} <- Transfers,
                          Owner =:= Node,
                          Waiting =:= [],
                          Status =:= awaiting],

  RT = fun(Ring, _) ->
           NewRing = lists:foldl(fun({Idx, Mods}, Ring1) ->
                         lists:foldl(fun(Mod, Ring2) ->
                             riak_core_ring:handoff_complete(Ring2, Idx, Mod)
                         end, Ring1, Mods)
                     end, Ring, Stuck),
           {new_ring, NewRing}
       end,

  case Stuck of
      [] ->
          ok;
      _ ->
          riak_core_ring_manager:ring_trans(RT, []),
          ok
  end
end().
#+END_SRC

*** Innostore on 32-bit systems

Depending on the partition count and the default stack size of the
operating system, using Innostore as the backend on 32-bit systems can
cause problems. A symptom of this problem would be a message similar
to this on the console or in the Riak error log:

#+BEGIN_SRC
10:22:38.719 [error] Failed to start riak_kv_innostore_backend for index 1415829711164312202009819681693899175291684651008. Reason: {eagain,[{erlang,open_port,[{spawn,innostore_drv},[binary]]},{innostore,connect,0},{riak_kv_innostore_backend,start,2},{riak_kv_vnode,init,1},{riak_core_vnode,init,1},{gen_fsm,init_it,6},{proc_lib,init_p_do_apply,3}]}
10:22:38.871 [notice] "backend module failed to start."
#+END_SRC

The workaround for this problem is to reduce the stack size for the
user Riak is run as (=riak= for most distributions). For example
on Linux, the current stack size can be viewed using =ulimit -s= and
{'fraction_non_alphanumeric': 0.0929258307150338, 'fraction_numerical': 0.042273636103970294, 'mean_word_length': 3.727272727272727, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 15, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5584889', 'n_tokens_mistral': 3490, 'n_tokens_neox': 3084, 'n_words': 1248}