<commit_before>/**
 * This package contains all public Data Access APIs.
 *
 * All data access is performed through the creation and execution of
 * {@link com.continuuity.api.data.Operation}s, of which there are two primary
 * types, {@link com.continuuity.api.data.ReadOperation} and
 * {@link com.continuuity.api.data.WriteOperation}.
 * 
 * Operations are executed through two primary pathways: a synchronous read
 * path and an asynchronous write path.
 * 
 * Read operations are performed one-at-a-time and inline within a flowlet
 * via the {@link com.continuuity.api.data.DataFabric}. The result of the
 * operation is returned from the executor.
 * 
 * For example, a flowlet might receive a tuple containing the user id of a
 * given user and we want to read the e-mail address for this user from the
 * data fabric. The below example, JoinUserEmailFlowlet, performs this join
 * inline using the read operation executor and adds the email address of
 * the user to a tuple and emits it.
 * 
 * <pre>
 * public class JoinUserEmailFlowlet extends AbstractComputeFlowlet {
 *   public void configure(StreamsConfigurator configurator) {
 *     // input = {(userid, byte[])}
 *     configurator.getDefaultTupleInputStream().setSchema(
 *         new TupleSchemaBuilder()
 *         .add("userid", byte[].class)
 *         .create());
 *     // output = {(userid, byte[]), (email, byte[])}
 *     configurator.getDefaultTupleOutputStream().setSchema(
 *         new TupleSchemaBuilder()
 *         .add("userid", byte[].class)
 *         .add("email", byte[].class)
 *         .create());
 *   }
 *
 *   public void process(Tuple tuple, TupleContext context,
 *       OutputCollector collector) {
 *     byte [] userid = tuple.get("userid");
 *     Read read = new Read(userid, "email".getBytes());
 *     ReadOperationExecutor executor = getFlowletLaunchContext().getReadExecutor();
 *     byte [] email = executor.execute(read).get("email".getBytes());
 *     collector.add(
 *         new TupleBuilder()
 *         .set("userid", userid)
 *         .set("email", email)
 *         .create());
 *   }
 * }
 * </pre>
 *
 * Write operations are performed within a transaction and as an asynchronous
 * batch at the end of every flowlet process call.  Writes are added to this
 * batch through OutputCollector.add(WriteOperation).
 * 
 * This batch will either completely succeed or completely fail, the database
 * will never be left in an inconsistent state.  If the batch is successful,
 * all the changes are committed permanently and the flowlet will receive a
 * Callback.onSuccess() callback.  If
 * the batch is not successfully transacted (a conditional operation could fail
 * or some other kind of non-retryable error may occur), all operations in the
 * batch are aborted, the database remains unchanged, and the flowlet will
 * receive an {@link Callback.onFailure()} callback.
 * 
 * As an example of performing a write operation, we will create a flowlet
 * which does the opposite of the previous example.  It will receive a tuple
 * containing a userid and email, store the email into the data fabric,
 * and then pass along a tuple containing only the userid.
 * 
 * <pre><commit_msg>ENG-749 Data Fabric java doc warnings<commit_after>/**
 * This package contains all public Data Access APIs.
 *
 * All data access is performed through the creation and execution of
 * {@link com.continuuity.api.data.Operation}s, of which there are two primary
 * types, {@link com.continuuity.api.data.ReadOperation} and
 * {@link com.continuuity.api.data.WriteOperation}.
 * 
 * Operations are executed through two primary pathways: a synchronous read
 * path and an asynchronous write path.
 * 
 * Read operations are performed one-at-a-time and inline within a flowlet
 * via the {@link com.continuuity.api.data.DataFabric}. The result of the
 * operation is returned from the executor.
 * 
 * For example, a flowlet might receive a tuple containing the user id of a
 * given user and we want to read the e-mail address for this user from the
 * data fabric. The below example, JoinUserEmailFlowlet, performs this join
 * inline using the read operation executor and adds the email address of
 * the user to a tuple and emits it.
 * 
 * <pre>
 * public class JoinUserEmailFlowlet extends AbstractComputeFlowlet {
 *   public void configure(StreamsConfigurator configurator) {
 *     // input = {(userid, byte[])}
 *     configurator.getDefaultTupleInputStream().setSchema(
 *         new TupleSchemaBuilder()
 *         .add("userid", byte[].class)
 *         .create());
 *     // output = {(userid, byte[]), (email, byte[])}
 *     configurator.getDefaultTupleOutputStream().setSchema(
 *         new TupleSchemaBuilder()
 *         .add("userid", byte[].class)
 *         .add("email", byte[].class)
 *         .create());
 *   }
 *
 *   public void process(Tuple tuple, TupleContext context,
 *       OutputCollector collector) {
 *     byte [] userid = tuple.get("userid");
 *     Read read = new Read(userid, "email".getBytes());
 *     ReadOperationExecutor executor = getFlowletLaunchContext().getReadExecutor();
 *     byte [] email = executor.execute(read).get("email".getBytes());
 *     collector.add(
 *         new TupleBuilder()
 *         .set("userid", userid)
 *         .set("email", email)
 *         .create());
 *   }
 * }
 * </pre>
 *
 * Write operations are performed within a transaction and as an asynchronous
 * batch at the end of every flowlet process call.  Writes are added to this
 * batch through OutputCollector.add(WriteOperation).
 * 
 * This batch will either completely succeed or completely fail, the database
 * will never be left in an inconsistent state.  If the batch is successful,
 * all the changes are committed permanently and the flowlet will receive an
 * onSuccess() callback.  If
 * the batch is not successfully transacted (a conditional operation could fail
 * or some other kind of non-retryable error may occur), all operations in the
 * batch are aborted, the database remains unchanged, and the flowlet will
 * receive an onFailure() callback.
 * 
 * As an example of performing a write operation, we will create a flowlet
 * which does the opposite of the previous example.  It will receive a tuple
 * containing a userid and email, store the email into the data fabric,
 * and then pass along a tuple containing only the userid.
 * 
 * <pre>
{'fraction_non_alphanumeric': 0.09577289034472002, 'fraction_numerical': 0.0004679457182966776, 'mean_word_length': 3.4343015214384507, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7388050', 'n_tokens_mistral': 1719, 'n_tokens_neox': 1681, 'n_words': 705}