<commit_before>---
title:  Function Execution
---

<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements.  See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

A client can invoke a server-resident function without the need to send the function code itself to the server.
The client can pass paramaters to the function and can collect and operate on the returned results.

## Server-side Requirements

To be callable from your client, a function must be resident on the server and registered as available for client access.
See [Executing a Function in <%=vars.product_name_long%>](/serverman/developing/function_exec/function_execution.html) 
in the <%=vars.product_name%> User Guide for details on how to write and register server-resident functions.

## Client-side Requirements

The client must connect to the server through a connection pool in order to invoke a server-side function.

## How Functions Execute

1.  The calling client application runs the `execute` method on the `Execution` object. The object must already be registered on the servers.
2.  The function is invoked on all servers where it needs to run. The locations are determined by the `FunctionService on*` 
method calls, region configuration, and any filters.
3.  If the function has results, the result is returned in a `ResultCollector` object.
4.  The client collects results using the result collector `getResult`.

In every client where you want to execute the function and process the results:

- Use one of the `FunctionService on*` methods to create an `Execution` object. The `on*` methods,
`onRegion`, `onServer` and `onServers`, define the highest level where the function is run. If
you use `onRegion` you can further narrow your run scope by setting key filters. The function run
using `onRegion` is a data dependent function – the others are data-independent functions.

You can run a data dependent function against custom partitioned and colocated partitioned regions. From the client, provide the appropriate key
 sets to the function call.

-  Use the `Execution` object as needed for additional function configuration. You can:
    -   Provide a set of data keys to `withFilter` to narrow the execution scope. This works only for `onRegion` Execution objects.
    -   Provide function arguments to `withArgs`.
    -   Define a custom `ResultCollector` to `withCollector`. See [Programming to Get Function Results](handling-function-results.html#handling-function-results).

-  Call the `Execution` object execute method to run the function.

## Processing Function Results

The client may use the default result collector. If the client needs special results handling, code
a custom `ResultsCollector` implementation to replace the default. Use the
`Execution::withCollector` method to define the custom collector.

For example, to program your client to get the results from a function, use the result collector returned from the function execution, like this:

``` pre
ResultCollectorPtr rc = FunctionService::onRegion(region)
                    ->withArgs(args)
                    ->withFilter(keySet)
                    ->withCollector(new MyCustomResultCollector())
                    .execute(Function);
CacheableVectorPtr functionResult = rc.getResult();
```

The `getResult` methods of the default result collector block until all results are received, then return the full result set.

To handle the results in a custom manner:

1.  Write a class that implements the `ResultCollector` interface to handle the results in a custom manner. The methods are of two types: one handles data and information from <%=vars.product_name%> and populates the results set, while the other returns the compiled results to the calling application:
    -  `addResult` is called when results arrive from the `Function` methods. Use `addResult` to add a single result to the ResultCollector.
    -  `endResults` is called to signal the end of all results from the function execution.
    -  `getResult` is available to your executing application (the one that calls `Execution.execute`) to retrieve the results. This may block until all results are available.
    -  `clearResults` is called to clear partial results from the results collector. This is used only for highly available `onRegion` functions where the calling application waits for the results. If the call fails, before <%=vars.product_name%> retries the execution, it calls `clearResults` to ready the instance for a clean set of results.
2.  Use the `Execution` object in your executing member to call `withCollector`, passing your custom collector, as shown in the example above.

# Examples

The native client source release contains examples of function execution written for .NET and
C++. The examples are located in `../examples/dotnet/FunctionExecutionCs` and
`../examples/cpp/function-execution`, respectively.

Both examples begin with a server-side script that runs `gfsh` commands to create
a region, simply called "partition_region", which is preloaded with a JAR file containing the
server-side Java function code. The function, called "ExampleMultiGetFunction", is defined in the
`examples/utilities` directory of your distribution. As its input parameter, the function takes an array of keys,
then performs a `get` on each key and returns an array containing the results.
The function does not load values into the data store. That is a separate operation, performed in these examples by
the client, and does not involve the server-side function.

As prerequisites, the client code must be aware of the connection to the server, the name of the function, and the expected type/format 
of the input parameter and return value.

The client:

- creates an execution object
- populates the execution object with input parameters
- invokes the object's execute method to invoke the server-side function.

If the client expects results, it must create a result
object. Optionally, the client can used a provided result collector which offers some predefined
methods for iterating over and processing return values.

## .NET Example
This section contains code snippets showing highlights of the .NET function execution example. They are not intended for cut-and-paste execution.
For the complete source, see the example source directory.

The .NET example creates a cache, then uses it to create a connection pool.

```csharp
   var cacheFactory = new CacheFactory()
       .Set("log-level", "none");
   var cache = cacheFactory.Create();

   var poolFactory = cache.GetPoolFactory()
       .AddLocator("localhost", 10334);
   var pool = poolFactory.Create("pool");
```

It then uses the connection pool to create a region, with the same characteristics and name as the server-side region (`partition_region`).

```csharp
   var regionFactory = cache.CreateRegionFactory(RegionShortcut.PROXY)
       .SetPoolName("pool");
   var region = regionFactory.Create<object, object>("partition_region");
```

It then populates the server's datastore with values, using the API and some sample key-value pairs.

```csharp
   string rtimmonsKey = "rtimmons";
   string rtimmonsValue = "<NAME>";
   string scharlesKey = "scharles";
   string scharlesValue = "<NAME>";
   region.Put(rtimmonsKey, rtimmonsValue, null);
   region.Put(scharlesKey, scharlesValue, null);
```

It then uses the API to retrieve the values and write them to the console as confirmation that the data has been stored. Again,
this is done without reference to the server-side example function.

```csharp
   var user1 = region.Get(rtimmonsKey, null);
   var user2 = region.Get(scharlesKey, null);

   Console.WriteLine(rtimmonsKey + " = " + user1);
   Console.WriteLine(scharlesKey + " = " + user2);
```

Next, the client retrieves those same values using the server-side example function.
The client code creates the input parameter, an array of keys whose values are to be retrieved.

```csharp
   ArrayList keyArgs = new ArrayList();
   keyArgs.Add(rtimmonsKey);
   keyArgs.Add(scharlesKey);
```

The client creates an execution object using `Client.FunctionService.OnRegion` and specifying the region.

```csharp
   var exc = Client.FunctionService<object>.OnRegion<object, object>(region);
```

The client then calls the server side function with its input arguments and stores the results in a Client.IResultCollector.

```csharp
   Client.IResultCollector<object> rc = exc.WithArgs<object>(keyArgs).Execute("ExampleMultiGetFunction");
```

It then loops through the results and prints the retrieved values.

```csharp
   ICollection<object> res = rc.GetResult();

   Console.WriteLine("Function Execution Results:");
   Console.WriteLine("   Count = {0}", res.Count);

   foreach (List<object> item in res)
   {
     foreach (object item2 in item)
     {
       Console.WriteLine("   value = {0}", item2.ToString());
     }
   }
```

## C++ Example
This section contains code snippets showing highlights of the C++ function execution example. They are not intended for cut-and-paste execution.
For the complete source, see the example source directory.

The C++ example creates a cache.

```cpp
Cache setupCache() {
  return CacheFactory()
      .set("log-level", "none")
      .create();
}
```

It then uses the cache to create a connection pool, 

```cpp
void createPool(const Cache& cache) {
  auto pool = cache.getPoolManager()
      .createFactory()
      .addServer("localhost", EXAMPLE_SERVER_PORT)
      .create("pool");
}
```

Then,using that pool, it creates a region with the same characteristics and name as the server-side region (`partition_region`).

```cpp
std::shared_ptr<Region> createRegion(Cache& cache) {
  auto regionFactory = cache.createRegionFactory(RegionShortcut::PROXY);
  auto region = regionFactory.setPoolName("pool").create("partition_region");

  return region;
}
```

It then populates the server's datastore with values, using the API and some sample key-value pairs.

```cpp
void populateRegion(const std::shared_ptr<Region>& region) {
  for (int i = 0; i < keys.size(); i++) {
    region->put(keys[i], values[i]);
  }
}
```

It then uses the API to retrieve the values and write them to the console as confirmation that the data has been stored. Again,
this is done without reference to the server-side example function.

```cpp
std::shared_ptr<CacheableVector> populateArguments() {
  auto arguments = CacheableVector::create();
  for (int i = 0; i < keys.size(); i++) {
    arguments->push_back(CacheableKey::create(keys[i]));
  }
  return arguments;
}
```

Next, the client retrieves those same values using the server-side example function.
The client code creates the input parameter, an array of keys whose values are to be retrieved.

```cpp
std::vector<std::string> executeFunctionOnServer(const std::shared_ptr<Region> region,
    const std::shared_ptr<CacheableVector> arguments) {
  std::vector<std::string> resultList;
```

The client creates an execution object using `Client.FunctionService.OnRegion` and specifying the region.

```cpp
  auto functionService = FunctionService::onServer(region->getRegionService());
```

The client then calls the server side function with its input arguments and stores the results in a Client.IResultCollector.

```cpp
  if(auto executeFunctionResult = functionService.withArgs(arguments).execute(getFuncIName)->getResult()) {
    for (auto &arrayList: *executeFunctionResult) {
      for (auto &cachedString: *std::dynamic_pointer_cast<CacheableArrayList>(arrayList)) {
        resultList.push_back(std::dynamic_pointer_cast<CacheableString>(cachedString)->value());
      }
    }
  } else {
    std::cout << "get executeFunctionResult is NULL\n";
  }

  return resultList;
}
```

It then loops through the results and prints the retrieved values.

```cpp
void printResults(const std::vector<std::string>& resultList) {
  std::cout << "Result count = " << resultList.size() << std::endl << std::endl;
  int i = 0;
  for (auto &cachedString: resultList) {
    std::cout << "\tResult[" << i << "]=" << cachedString << std::endl;
    ++i;
  }
```
<commit_msg>GEODE-4337, GEODE-4346 improve formatting<commit_after>---
title:  Function Execution
---

<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements.  See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

A client can invoke a server-resident function without the need to send the function code itself to the server.
The client can pass paramaters to the function and can collect and operate on the returned results.

## Server-side Requirements

To be callable from your client, a function must be resident on the server and registered as available for client access.
See [Executing a Function in <%=vars.product_name_long%>](/serverman/developing/function_exec/function_execution.html) 
in the <%=vars.product_name%> User Guide for details on how to write and register server-resident functions.

## Client-side Requirements

The client must connect to the server through a connection pool in order to invoke a server-side function.

## How Functions Execute

1.  The calling client application runs the `execute` method on the `Execution` object. The object must already be registered on the servers.
2.  The function is invoked on all servers where it needs to run. The locations are determined by the `FunctionService on*` 
method calls, region configuration, and any filters.
3.  If the function has results, the result is returned in a `ResultCollector` object.
4.  The client collects results using the result collector `getResult`.

In every client where you want to execute the function and process the results:

- Use one of the `FunctionService on*` methods to create an `Execution` object. The `on*` methods,
`onRegion`, `onServer` and `onServers`, define the highest level where the function is run. If
you use `onRegion` you can further narrow your run scope by setting key filters. The function run
using `onRegion` is a data dependent function – the others are data-independent functions.

You can run a data dependent function against custom partitioned and colocated partitioned regions. From the client, provide the appropriate key
 sets to the function call.

-  Use the `Execution` object as needed for additional function configuration. You can:
    -   Provide a set of data keys to `withFilter` to narrow the execution scope. This works only for `onRegion` Execution objects.
    -   Provide function arguments to `withArgs`.
    -   Define a custom `ResultCollector` to `withCollector`. See [Programming to Get Function Results](handling-function-results.html#handling-function-results).

-  Call the `Execution` object execute method to run the function.

## Processing Function Results

The client may use the default result collector. If the client needs special results handling, code
a custom `ResultsCollector` implementation to replace the default. Use the
`Execution::withCollector` method to define the custom collector.

For example, to program your client to get the results from a function, use the result collector returned from the function execution, like this:

```cpp
ResultCollectorPtr rc = FunctionService::onRegion(region)
                    ->withArgs(args)
                    ->withFilter(keySet)
                    ->withCollector(new MyCustomResultCollector())
                    .execute(Function);
CacheableVectorPtr functionResult = rc.getResult();
```

The `getResult` methods of the default result collector block until all results are received, then return the full result set.

To handle the results in a custom manner:

1.  Write a class that implements the `ResultCollector` interface to handle the results in a custom manner. The methods are of two types: one handles data and information from <%=vars.product_name%> and populates the results set, while the other returns the compiled results to the calling application:
    -  `addResult` is called when results arrive from the `Function` methods. Use `addResult` to add a single result to the ResultCollector.
    -  `endResults` is called to signal the end of all results from the function execution.
    -  `getResult` is available to your executing application (the one that calls `Execution.execute`) to retrieve the results. This may block until all results are available.
    -  `clearResults` is called to clear partial results from the results collector. This is used only for highly available `onRegion` functions where the calling application waits for the results. If the call fails, before <%=vars.product_name%> retries the execution, it calls `clearResults` to ready the instance for a clean set of results.
2.  Use the `Execution` object in your executing member to call `withCollector`, passing your custom collector, as shown in the example above.

# Examples

The native client source release contains examples of function execution written for .NET and
C++. The examples are located in `../examples/dotnet/FunctionExecutionCs` and
`../examples/cpp/function-execution`, respectively.

Both examples begin with a server-side script that runs `gfsh` commands to create
a region, simply called "partition_region", which is preloaded with a JAR file containing the
server-side Java function code. The function, called "ExampleMultiGetFunction", is defined in the
`examples/utilities` directory of your distribution. As its input parameter, the function takes an array of keys,
then performs a `get` on each key and returns an array containing the results.
The function does not load values into the data store. That is a separate operation, performed in these examples by
the client, and does not involve the server-side function.

As prerequisites, the client code must be aware of the connection to the server, the name of the function, and the expected type/format 
of the input parameter and return value.

The client:

- creates an execution object
- populates the execution object with input parameters
- invokes the object's execute method to invoke the server-side function.

If the client expects results, it must create a result
object. Optionally, the client can used a provided result collector which offers some predefined
methods for iterating over and processing return values.

## .NET Example
This section contains code snippets showing highlights of the .NET function execution example. They are not intended for cut-and-paste execution.
For the complete source, see the example source directory.

The .NET example creates a cache, then uses it to create a connection pool.

```csharp
   var cacheFactory = new CacheFactory()
       .Set("log-level", "none");
   var cache = cacheFactory.Create();

   var poolFactory = cache.GetPoolFactory()
       .AddLocator("localhost", 10334);
   var pool = poolFactory.Create("pool");
```

It then uses the connection pool to create a region, with the same characteristics and name as the server-side region (`partition_region`).

```csharp
   var regionFactory = cache.CreateRegionFactory(RegionShortcut.PROXY)
       .SetPoolName("pool");
   var region = regionFactory.Create<object, object>("partition_region");
```

It then populates the server's datastore with values, using the API and some sample key-value pairs.

```csharp
   string rtimmonsKey = "rtimmons";
   string rtimmonsValue = "<NAME>";
   string scharlesKey = "scharles";
   string scharlesValue = "<NAME>";
   region.Put(rtimmonsKey, rtimmonsValue, null);
   region.Put(scharlesKey, scharlesValue, null);
```

It then uses the API to retrieve the values and write them to the console as confirmation that the data has been stored. Again,
this is done without reference to the server-side example function.

```csharp
   var user1 = region.Get(rtimmonsKey, null);
   var user2 = region.Get(scharlesKey, null);

   Console.WriteLine(rtimmonsKey + " = " + user1);
   Console.WriteLine(scharlesKey + " = " + user2);
```

Next, the client retrieves those same values using the server-side example function.
The client code creates the input parameter, an array of keys whose values are to be retrieved.

```csharp
   ArrayList keyArgs = new ArrayList();
   keyArgs.Add(rtimmonsKey);
   keyArgs.Add(scharlesKey);
```

The client creates an execution object using `Client.FunctionService.OnRegion` and specifying the region.

```csharp
   var exc = Client.FunctionService<object>.OnRegion<object, object>(region);
```

The client then calls the server side function with its input arguments and stores the results in a Client.IResultCollector.

```csharp
   Client.IResultCollector<object> rc = exc.WithArgs<object>(keyArgs).Execute("ExampleMultiGetFunction");
```

It then loops through the results and prints the retrieved values.

```csharp
   ICollection<object> res = rc.GetResult();

   Console.WriteLine("Function Execution Results:");
   Console.WriteLine("   Count = {0}", res.Count);

   foreach (List<object> item in res)
   {
     foreach (object item2 in item)
     {
       Console.WriteLine("   value = {0}", item2.ToString());
     }
   }
```

## C++ Example
This section contains code snippets showing highlights of the C++ function execution example. They are not intended for cut-and-paste execution.
For the complete source, see the example source directory.

The C++ example creates a cache.

```cpp
Cache setupCache() {
  return CacheFactory()
      .set("log-level", "none")
      .create();
}
```

It then uses the cache to create a connection pool, 

```cpp
void createPool(const Cache& cache) {
  auto pool = cache.getPoolManager()
      .createFactory()
      .addServer("localhost", EXAMPLE_SERVER_PORT)
      .create("pool");
}
```

Then,using that pool, it creates a region with the same characteristics and name as the server-side region (`partition_region`).

```cpp
std::shared_ptr<Region> createRegion(Cache& cache) {
  auto regionFactory = cache.createRegionFactory(RegionShortcut::PROXY);
  auto region = regionFactory.setPoolName("pool").create("partition_region");

  return region;
}
```

It then populates the server's datastore with values, using the API and some sample key-value pairs.

```cpp
void populateRegion(const std::shared_ptr<Region>& region) {
  for (int i = 0; i < keys.size(); i++) {
    region->put(keys[i], values[i]);
  }
}
```

It then uses the API to retrieve the values and write them to the console as confirmation that the data has been stored. Again,
this is done without reference to the server-side example function.

```cpp
std::shared_ptr<CacheableVector> populateArguments() {
  auto arguments = CacheableVector::create();
  for (int i = 0; i < keys.size(); i++) {
    arguments->push_back(CacheableKey::create(keys[i]));
  }
  return arguments;
}
```

Next, the client retrieves those same values using the server-side example function.
The client code creates the input parameter, an array of keys whose values are to be retrieved.

```cpp
std::vector<std::string> executeFunctionOnServer(const std::shared_ptr<Region> region,
    const std::shared_ptr<CacheableVector> arguments) {
  std::vector<std::string> resultList;
```

The client creates an execution object using `Client.FunctionService.OnRegion` and specifying the region.

```cpp
  auto functionService = FunctionService::onServer(region->getRegionService());
```

The client then calls the server side function with its input arguments and stores the results in a Client.IResultCollector.

```cpp
  if(auto executeFunctionResult = functionService.withArgs(arguments).execute(getFuncIName)->getResult()) {
    for (auto &arrayList: *executeFunctionResult) {
      for (auto &cachedString: *std::dynamic_pointer_cast<CacheableArrayList>(arrayList)) {
        resultList.push_back(std::dynamic_pointer_cast<CacheableString>(cachedString)->value());
      }
    }
  } else {
    std::cout << "get executeFunctionResult is NULL\n";
  }

  return resultList;
}
```

It then loops through the results and prints the retrieved values.

```cpp
void printResults(const std::vector<std::string>& resultList) {
  std::cout << "Result count = " << resultList.size() << std::endl << std::endl;
  int i = 0;
  for (auto &cachedString: resultList) {
    std::cout << "\tResult[" << i << "]=" << cachedString << std::endl;
    ++i;
  }
```

{'fraction_non_alphanumeric': 0.0828920010984269, 'fraction_numerical': 0.0023537719195010002, 'mean_word_length': 4.41116535767353, 'pattern_counts': {'":': 0, '<': 95, '<?xml version=': 0, '>': 67, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3874230', 'n_tokens_mistral': 6698, 'n_tokens_neox': 6444, 'n_words': 3061}