<commit_before>// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements.  See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License.  You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

== Writing Accumulo Clients

=== Running Client Code

There are multiple ways to run Java code that uses Accumulo. Below is a list
of the different ways to execute client code.

* using java executable
* using the accumulo script
* using the tool script

In order to run client code written to run against Accumulo, you will need to
include the jars that Accumulo depends on in your classpath. Accumulo client
code depends on Hadoop and Zookeeper. For Hadoop add the hadoop client jar, all
of the jars in the Hadoop lib directory, and the conf directory to the
classpath. For Zookeeper 3.3 you only need to add the Zookeeper jar, and not
what is in the Zookeeper lib directory. You can run the following command on a
configured Accumulo system to see what its using for its classpath.

  $ACCUMULO_HOME/bin/accumulo classpath

Another option for running your code is to put a jar file in
+$ACCUMULO_HOME/lib/ext+. After doing this you can use the accumulo
script to execute your code. For example if you create a jar containing the
class +com.foo.Client+ and placed that in +lib/ext+, then you could use the command
+$ACCUMULO_HOME/bin/accumulo com.foo.Client+ to execute your code.

If you are writing map reduce job that access Accumulo, then you can use the
bin/tool.sh script to run those jobs. See the map reduce example.

=== Connecting

All clients must first identify the Accumulo instance to which they will be
communicating. Code to do this is as follows:

[source,java]
----
String instanceName = "myinstance";
String zooServers = "zooserver-one,zooserver-two"
Instance inst = new ZooKeeperInstance(instanceName, zooServers);

Connector conn = inst.getConnector("user", new PasswordToken("<PASSWORD>"));
----

=== Writing Data

Data are written to Accumulo by creating Mutation objects that represent all the
changes to the columns of a single row. The changes are made atomically in the
TabletServer. Clients then add Mutations to a BatchWriter which submits them to
the appropriate TabletServers.

Mutations can be created thus:

[source,java]
----
Text rowID = new Text("row1");
Text colFam = new Text("myColFam");
Text colQual = new Text("myColQual");
ColumnVisibility colVis = new ColumnVisibility("public");
long timestamp = System.currentTimeMillis();

Value value = new Value("myValue".getBytes());

Mutation mutation = new Mutation(rowID);
mutation.put(colFam, colQual, colVis, timestamp, value);
----

==== BatchWriter
The BatchWriter is highly optimized to send Mutations to multiple TabletServers
and automatically batches Mutations destined for the same TabletServer to
amortize network overhead. Care must be taken to avoid changing the contents of
any Object passed to the BatchWriter since it keeps objects in memory while
batching.

Mutations are added to a BatchWriter thus:

[source,java]
----
// BatchWriterConfig has reasonable defaults
BatchWriterConfig config = new BatchWriterConfig();
config.setMaxMemory(10000000L); // bytes available to batchwriter for buffering mutations

BatchWriter writer = conn.createBatchWriter("table", config)

writer.add(mutation);

writer.close();
----

An example of using the batch writer can be found at
+accumulo/docs/examples/README.batch+.

==== ConditionalWriter
The ConditionalWriter enables efficient, atomic read-modify-write operations on
rows.  The ConditionalWriter writes special Mutations which have a list of per
column conditions that must all be met before the mutation is applied.  The
conditions are checked in the tablet server while a row lock is
held\footnote{Mutations written by the BatchWriter will not obtain a row
lock.}.  The conditions that can be checked for a column are equality and
absence.  For example a conditional mutation can require that column A is
absent inorder to be applied.  Iterators can be applied when checking
conditions.  Using iterators, many other operations besides equality and
absence can be checked.  For example, using an iterator that converts values
less than 5 to 0 and everything else to 1, its possible to only apply a
mutation when a column is less than 5.<commit_msg>ACCUMULO-2884 Update source to remove TeX structure<commit_after>// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements.  See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License.  You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

== Writing Accumulo Clients

=== Running Client Code

There are multiple ways to run Java code that uses Accumulo. Below is a list
of the different ways to execute client code.

* using java executable
* using the accumulo script
* using the tool script

In order to run client code written to run against Accumulo, you will need to
include the jars that Accumulo depends on in your classpath. Accumulo client
code depends on Hadoop and Zookeeper. For Hadoop add the hadoop client jar, all
of the jars in the Hadoop lib directory, and the conf directory to the
classpath. For Zookeeper 3.3 you only need to add the Zookeeper jar, and not
what is in the Zookeeper lib directory. You can run the following command on a
configured Accumulo system to see what its using for its classpath.

  $ACCUMULO_HOME/bin/accumulo classpath

Another option for running your code is to put a jar file in
+$ACCUMULO_HOME/lib/ext+. After doing this you can use the accumulo
script to execute your code. For example if you create a jar containing the
class +com.foo.Client+ and placed that in +lib/ext+, then you could use the command
+$ACCUMULO_HOME/bin/accumulo com.foo.Client+ to execute your code.

If you are writing map reduce job that access Accumulo, then you can use the
bin/tool.sh script to run those jobs. See the map reduce example.

=== Connecting

All clients must first identify the Accumulo instance to which they will be
communicating. Code to do this is as follows:

[source,java]
----
String instanceName = "myinstance";
String zooServers = "zooserver-one,zooserver-two"
Instance inst = new ZooKeeperInstance(instanceName, zooServers);

Connector conn = inst.getConnector("user", new PasswordToken("<PASSWORD>"));
----

=== Writing Data

Data are written to Accumulo by creating Mutation objects that represent all the
changes to the columns of a single row. The changes are made atomically in the
TabletServer. Clients then add Mutations to a BatchWriter which submits them to
the appropriate TabletServers.

Mutations can be created thus:

[source,java]
----
Text rowID = new Text("row1");
Text colFam = new Text("myColFam");
Text colQual = new Text("myColQual");
ColumnVisibility colVis = new ColumnVisibility("public");
long timestamp = System.currentTimeMillis();

Value value = new Value("myValue".getBytes());

Mutation mutation = new Mutation(rowID);
mutation.put(colFam, colQual, colVis, timestamp, value);
----

==== BatchWriter
The BatchWriter is highly optimized to send Mutations to multiple TabletServers
and automatically batches Mutations destined for the same TabletServer to
amortize network overhead. Care must be taken to avoid changing the contents of
any Object passed to the BatchWriter since it keeps objects in memory while
batching.

Mutations are added to a BatchWriter thus:

[source,java]
----
// BatchWriterConfig has reasonable defaults
BatchWriterConfig config = new BatchWriterConfig();
config.setMaxMemory(10000000L); // bytes available to batchwriter for buffering mutations

BatchWriter writer = conn.createBatchWriter("table", config)

writer.add(mutation);

writer.close();
----

An example of using the batch writer can be found at
+accumulo/docs/examples/README.batch+.

==== ConditionalWriter
The ConditionalWriter enables efficient, atomic read-modify-write operations on
rows.  The ConditionalWriter writes special Mutations which have a list of per
column conditions that must all be met before the mutation is applied.  The
conditions are checked in the tablet server while a row lock is
held (Mutations written by the BatchWriter will not obtain a row
lock).  The conditions that can be checked for a column are equality and
absence.  For example a conditional mutation can require that column A is
absent inorder to be applied.  Iterators can be applied when checking
conditions.  Using iterators, many other operations besides equality and
absence can be checked.  For example, using an iterator that converts values
less than 5 to 0 and everything else to 1, its possible to only apply a
mutation when a column is less than 5.
{'fraction_non_alphanumeric': 0.06023362112747588, 'fraction_numerical': 0.004266124936515998, 'mean_word_length': 4.491355270496375, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2639663', 'n_tokens_mistral': 2596, 'n_tokens_neox': 2504, 'n_words': 1368}