<jupyter_start><jupyter_text>G2Engine<jupyter_code>import com.senzing.g2.engine.G2Engine;
import com.senzing.g2.engine.G2JNI;
import com.senzing.g2.engine.Result;<jupyter_output><empty_output><jupyter_text>Initialize variablesCreate variables used for G2Engine.<jupyter_code>String moduleName = "G2JNI";
String iniFilename = "{\"PIPELINE\": {\"SUPPORTPATH\": \"/opt/senzing/g2/data\"},\"SQL\": {\"CONNECTION\": \"sqlite3://na:na@/opt/senzing/g2/sqldb/G2C.db\",\"G2CONFIGFILE\": \"/opt/senzing/g2/python/g2config.json\"}}";
boolean verboseLogging = true;
Result<Long> configID = new Result<Long>();<jupyter_output><empty_output><jupyter_text>InitializationTo start using Senzing G2Engine, create and initialize an instance.This should be done once per process.The `init()` method accepts the following parameters:- **module_name:** A short name given to this instance of the G2 engine (i.e. your G2Module object)- **g2module_ini_pathname:** A fully qualified path to the G2 engine INI file (often /opt/senzing/g2/python/G2Module.ini)- **verbose_logging:** A boolean which enables diagnostic logging - this will print a massive amount of information to stdout (default = False)- **config_id:** (optional) The identifier value for the engine configuration can be returned here.Calling this function will return "0" upon success.<jupyter_code>G2Engine engine = new G2JNI();
int initResult = engine.initV2(moduleName, iniFilename, verboseLogging);
System.out.println(initResult);<jupyter_output><empty_output><jupyter_text>Prime EngineThe `primeEngine()` method may optionally be called to pre-initialize some of the heavier weight internal resources of the G2 engine.<jupyter_code>int ret = engine.primeEngine();
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.print(ret)
String stats = engine.stats();
System.out.println("STATS: " + stats);<jupyter_output><empty_output><jupyter_text>addRecord()Once the Senzing engine is initialized, use addRecord() to load a record into the Senzing repository -- addRecord() can be called as many times as desired and from multiple threads at the same time. The addRecord() function returns "0" upon success, and accepts four parameters as input:- **datasource_code:** The name of the data source the record is associated with. This value is configurable to the system- **record_id:** The record ID, used to identify distinct records- **data_string:** A JSON document with the attribute data for the record- **load_id:** The observation load ID for the record; value can be null and will default to data_source<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"SMITH\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"653 STATE ROUTE 7\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;

int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);

StringBuffer recordID2 = new StringBuffer();
int ret2 = engine.addRecordWithReturnedRecordID(dataSourceCode, recordID2, jsonData, loadID);

System.out.println("New Record ID: " + recordID2.toString());
if(ret!=0)
    System.out.println(engine.getLastException());
if(ret2!=0)
    System.out.println(engine.getLastException());
System.out.println(ret)<jupyter_output><empty_output><jupyter_text>getRecord()Use `getRecord()` to retrieve a single record from the data repository; the record is assigned in JSON form to a user-designated buffer, and the function itself returns "0" upon success. Once the Senzing engine is initialized, `getRecord()` can be called as many times as desired and from multiple threads at the same time. The `getRecord()` function accepts the following parameters as input:- **data_source:** The name of the data source the record is associated with. This value is configurable to the system- **record_id:** The record ID, used to identify the record for retrieval- **response:** A memory buffer for returning the response document; if an error occurred, an error response is stored here<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";
StringBuffer response = new StringBuffer();

/** getRecord */
int ret = engine.getRecord(dataSourceCode, recordID, response);

System.out.println("GET RECORD: " + response.toString());
System.out.println("");
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret)<jupyter_output><empty_output><jupyter_text>getEntityByRecordID()Use `getEntityByRecordID()` to retrieve entity data based on the record ID of a particular data record. This function accepts the following parameters as input:- **datasource_code:** The name of the data source the record is associated with. This value is configurable to the system- **record_id:** The record ID for a particular data record- **response:** A memory buffer for returning the response document; if an error occurred, an error response is stored here.<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";

long entityID = 1;
StringBuffer response = new StringBuffer();

int ret2 = engine.getEntityByEntityID(entityID, response);
System.out.println("GET Entity: " + response.toString());

response = new StringBuffer();

int ret3 = engine.getEntityByRecordID(dataSourceCode,recordID,response);
System.out.println("GET Entity: " + response.toString());
if(ret2!=0)
    System.out.println(engine.getLastException());
if(ret3!=0)
    System.out.println(engine.getLastException());
System.out.println(ret2)<jupyter_output><empty_output><jupyter_text>searchByAttributes()Use `searchByAttributes()` to retrieve entity data based on a user-specified set of entity attributes. This function accepts the following parameters as input:- **data_string:** A JSON document with the attribute data to search for- **response:** A memory buffer for returning the response document; if an error occurred, an error response is stored here.<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";

response = new StringBuffer();

int ret4 = engine.searchByAttributes(jsonData, response);
System.out.println("GET Entity: " + response.toString());
if(ret4!=0)
    System.out.println(engine.getLastException());
System.out.println(ret4)<jupyter_output><empty_output><jupyter_text>Replace the recordUse the `replaceRecord()` function to update or replace a record in the data repository (if record doesn't exist, a new record is added to the data repository. Like the above functions, `replaceRecord()` returns "0" upon success, and it can be called as many times as desired and from multiple threads at the same time. The `replaceRecord()` function accepts four parameters as input:- **dataSourceCode:** The name of the data source the record is associated with. This value is configurable to the system- **recordID:** The record ID, used to identify distinct records- **jsonData:** A JSON document with the attribute data for the record- **loadID:** The observation load ID for the record; value can be null and will default to dataSourceCode<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"ADAMS\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"653 STATE ROUTE 7\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;

int ret = engine.replaceRecord(dataSourceCode, recordID, jsonData, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret)<jupyter_output><empty_output><jupyter_text>Do ```getRecord()``` again to see the changes<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";
StringBuffer response = new StringBuffer();

int ret = engine.getRecord(dataSourceCode, recordID, response);

System.out.println("GET RECORD: " + response.toString());
System.out.println("");
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.print(ret)<jupyter_output><empty_output><jupyter_text>Export JSON Entity ReportThere are three steps to exporting resolved entity data from the G2Engine object in JSON format. First, use the `exportJSONEntityReport()` method to generate a long integer, referred to here as an 'exportHandle'. The `exportJSONEntityReport()` method accepts one parameter as input:- **flags**: An integer specifying which entity details should be included in the export. See the "Entity Export Flags" section for further details.Second, use the fetchNext() method to read the exportHandle and export a row of JSON output containing the entity data for a single entity. Note that successive calls of fetchNext() will export successive rows of entity data. The fetchNext() method accepts the following parameters as input:- **exportHandle:** A long integer from which resolved entity data may be read and exported- **response:** A memory buffer for returning the response document; if an error occurred, an error response is stored here.<jupyter_code>int flags = engine.G2_EXPORT_INCLUDE_ALL_ENTITIES;

long exportHandle = engine.exportJSONEntityReport(flags);

String response = engine.fetchNext(exportHandle);
System.out.println(response);

engine.closeExport(exportHandle);<jupyter_output><empty_output><jupyter_text>Export CSV Entity ReportThere are three steps to exporting resolved entity data from the G2Engine object in CSV format. First, use the `exportCSVEntityReport()` method to generate a long integer, referred to here as an 'exportHandle'. The `exportCSVEntityReport()` method accepts one parameter as input:- **flags:** An integer specifying which entity details should be included in the export. See the "Entity Export Flags" section for further details.Second, use the `fetchNext()` method to read the exportHandle and export a row of CSV output containing the entity data for a single entity. Note that the first call of `fetchNext()` may yield a header row, and that successive calls of `fetchNext()` will export successive rows of entity data. The `fetchNext()` method accepts the following parameters as input:- **exportHandle:** A long integer from which resolved entity data may be read and exported- **response:** A memory buffer for returning the response document; if an error occurred, an error response is stored here<jupyter_code>int flags = engine.G2_EXPORT_INCLUDE_ALL_ENTITIES;

long exportHandle = engine.exportCSVEntityReport(flags);

String response = engine.fetchNext(exportHandle);
System.out.println(response);

engine.closeExport(exportHandle);<jupyter_output><empty_output><jupyter_text>Finding PathsThe `FindPathByEntityID()` and `FindPathByRecordID()` functions can be used to find single relationship paths between two entities. Paths are found using known relationships with other entities.Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen.These functions have the following parameters:- **entityID1:** The entity ID for the starting entity of the search path- **entityID2:** The entity ID for the ending entity of the search path- **dataSourceCode1:** The data source for the starting entity of the search path- **recordID1:** The record ID for the starting entity of the search path- **dataSourceCode2:** The data source for the ending entity of the search path- **recordID2:** The record ID for the ending entity of the search path- **maxDegree:** The number of relationship degrees to search First you will need to create some records so that you have some that you can compare. Can you see what is the same between this record and the previous one?<jupyter_code>String dataSourceCode = "TEST";
String recordID = "2";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JOHN\", \"NAME_LAST\": \"SMITH\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"753 STATE ROUTE 8\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\"}";
String loadID = null;

int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);

StringBuffer recordID2 = new StringBuffer();
int ret2 = engine.addRecordWithReturnedRecordID(dataSourceCode, recordID2, jsonData, loadID);

System.out.println("New Record ID: " + recordID2.toString());
if(ret!=0)
    System.out.println(engine.getLastException());
if(ret2!=0)
    System.out.println(engine.getLastException());
System.out.println(ret)<jupyter_output><empty_output><jupyter_text>`FindPathByEntityID()`<jupyter_code>long entityID1 = 1;
long entityID2 = 2;
int maxDegree = 3;

StringBuffer response = new StringBuffer();

int ret_code = engine.findPathByEntityID(entityID1,entityID2,maxDegree,response);

System.out.println("Path result document: " + response.toString());
if(ret_code!=0)
    System.out.println(engine.getLastException());
System.out.print(ret_code)<jupyter_output><empty_output><jupyter_text>`FindPathByRecordID()`<jupyter_code>String dataSourceCode1 = new String("TEST");
String recordID1 = new String("1");
String dataSourceCode2 = new String("TEST");
String recordID2 = new String("2");

ret_code = engine.findPathByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,response);

System.out.println("Path result document: " + response.toString());
if(ret_code!=0)
    System.out.println(engine.getLastException());
System.out.println(ret_code)<jupyter_output><empty_output><jupyter_text>Finding Paths with ExclusionsThe `FindPathExcludingByEntityID()` and `FindPathExcludingByRecordID()` functions can be used to find single relationship paths between two entities. Paths are found using known relationships with other entities. In addition, it will find paths that exclude certain entities from being on the path.Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen. Additionally, entities to be excluded can also be specified by either Entity ID or by Record ID.When excluding entities, the user may choose to either (a) strictly exclude the entities, or (b) prefer to exclude the entities, but still include them if no other path is found. By default, entities will be strictly excluded. A "preferred exclude" may be done by specifying the G2_FIND_PATH_PREFER_EXCLUDE control flag.These functions have the following parameters:- **entityID1:** The entity ID for the starting entity of the search path- **entityID2:** The entity ID for the ending entity of the search path- **dataSourceCode1:** The data source for the starting entity of the search path- **recordID1:** The record ID for the starting entity of the search path- **dataSourceCode2:** The data source for the ending entity of the search path- **recordID2:** The record ID for the ending entity of the search path- **maxDegree:** The number of relationship degrees to search- **excludedEntities:** Entities that should be avoided on the path (JSON document)- **flags:** Operational flags<jupyter_code>String dataSourceCode = "TEST";
String recordID = "3";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"SAM\", \"NAME_LAST\": \"MILLER\", \"ADDR_TYPE\": \"HOME\", \"ADDR_LINE1\": \"753 STATE ROUTE 8\", \"ADDR_CITY\": \"FRESNO\", \"ADDR_STATE\": \"CA\", \"ADDR_POSTAL_CODE\": \"55073-1234\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;

int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);

StringBuffer recordID2 = new StringBuffer();
int ret2 = engine.addRecordWithReturnedRecordID(dataSourceCode, recordID2, jsonData, loadID);

System.out.println("New Record ID: " + recordID2.toString());
if(ret!=0)
    System.out.println(engine.getLastException());
if(ret2!=0)
    System.out.println(engine.getLastException());
System.out.println(ret)<jupyter_output><empty_output><jupyter_text>`FindPathExcludingByEntityID()`<jupyter_code>long entityID1 = 2;
long entityID2 = 3;
int maxDegree = 4;
String excludedEntities = new String("{\"ENTITIES\":[{\"ENTITY_ID\":\"1\"}]}");
int flags = G2Engine.G2_EXPORT_DEFAULT_FLAGS;

StringBuffer response = new StringBuffer();

int ret_code = engine.findPathExcludingByEntityID(entityID1,entityID2,maxDegree,excludedEntities,flags,response);
System.out.println("Path result document: " + response.toString());
if(ret_code!=0)
    System.out.println(engine.getLastException());
System.out.println(ret_code);<jupyter_output><empty_output><jupyter_text>`FindPathExcludingByRecordID()`<jupyter_code>String dataSourceCode1 = new String("TEST");
String recordID1 = new String("2");
String dataSourceCode2 = new String("TEST");
String recordID2 = new String("3");
String excludedRecords = new String("{\"RECORDS\":[{\"RECORD_ID\":\"1\",\"DATA_SOURCE\":\"TEST\"}]}");

ret_code = engine.findPathExcludingByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,excludedRecords,flags,response);

System.out.println("Path result document: " + response.toString());
if(ret_code!=0)
    System.out.println(engine.getLastException());
System.out.println(ret_code);<jupyter_output><empty_output><jupyter_text>Finding Paths with Required SourcesThe `FindPathIncludingSourceByEntityID()` and `FindPathIncludingSourceByRecordID()` functions can be used to find single relationship paths between two entities. In addition, one of the enties along the path must include a specified data source.Entities can be searched for by either Entity ID or by Record ID, depending on which function is chosen. The required data source or sources are specified by a json document list.Specific entities may also be excluded, using the same methodology as the `FindPathExcludingByEntityID()` and `FindPathExcludingByRecordID()` functions use.These functions have the following parameters:- **entityID1:** The entity ID for the starting entity of the search path- **entityID2:** The entity ID for the ending entity of the search path- **dataSourceCode1:** The data source for the starting entity of the search path- **recordID1:** The record ID for the starting entity of the search path- **dataSourceCode2:** The data source for the ending entity of the search path- **recordID2:** The record ID for the ending entity of the search path- **maxDegree:** The number of relationship degrees to search- **excludedEntities:** Entities that should be avoided on the path (JSON document)- **requiredDsrcs:** Entities that should be avoided on the path (JSON document)- **flags:** Operational flags<jupyter_code>long entityID1 = 2;
long entityID2 = 1;
int maxDegree = 4;
String excludedEntities = new String("{\"ENTITIES\":[{\"ENTITY_ID\":\"1\"}]}");
String requiredDsrcs = new String("{\"DATA_SOURCES\":[\"TEST\"]}");
int flags = 0;

StringBuffer response = new StringBuffer();

int ret_code = engine.findPathIncludingSourceByEntityID(entityID1,entityID2,maxDegree,excludedEntities,requiredDsrcs,flags,response);

System.out.println("Path result document: " + response.toString());

String dataSourceCode1 = new String("TEST");
String recordID1 = new String("2");
String dataSourceCode2 = new String("TEST");
String recordID2 = new String("3");
String excludedRecords = new String("{\"RECORDS\":[{\"RECORD_ID\":\"1\",\"DATA_SOURCE\":\"TEST\"}]}");

ret_code = engine.findPathIncludingSourceByRecordID(dataSourceCode1,recordID1,dataSourceCode2,recordID2,maxDegree,excludedRecords,requiredDsrcs,flags,response);

System.out.println("Path result document: " + response.toString());
if(ret_code!=0)
    System.out.println(engine.getLastException());
System.out.print(ret_code);<jupyter_output><empty_output><jupyter_text>Redo ProcessingRedo records are automatically created by Senzing when certain conditions occur where it believes more processing may be needed.  Some examples:* A value becomes generic and previous decisions may need to be revisited* Clean up after some record deletes* Detected related entities were being changed at the same time* A table inconsistency exists, potentially after a non-graceful shutdownFirst we will need to have a total of 6 data sources so let's add 4 more<jupyter_code>String dataSourceCode = "TEST";
String recordID = "4";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JANE\", \"NAME_LAST\": \"ADAMS\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret);

String dataSourceCode = "TEST";
String recordID = "5";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"LILY\", \"NAME_LAST\": \"OWENS\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret);

String dataSourceCode = "TEST";
String recordID = "6";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"AUGUST\", \"NAME_LAST\": \"Bauler\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret);

String dataSourceCode = "TEST";
String recordID = "7";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"JACK\", \"NAME_LAST\": \"MILLER\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret);

String dataSourceCode = "TEST";
String recordID = "8";
String jsonData =  "{\"NAME_TYPE\": \"PRIMARY\", \"NAME_FIRST\": \"LOGAN\", \"NAME_LAST\": \"WILLIAMS\", \"SSN_NUMBER\": \"111-11-1111\"}";
String loadID = null;
int ret = engine.addRecord(dataSourceCode, recordID, jsonData, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret);<jupyter_output><empty_output><jupyter_text>Counting the number of redosThis returns the number of redos within the processed records that are awaiting processing.<jupyter_code>long response = engine.countRedoRecords();
if(response<0)
    System.out.println(engine.getLastException());
System.out.print(response);<jupyter_output><empty_output><jupyter_text>Geting a redo recordGets a redo record so that it can be processed<jupyter_code>StringBuffer response_string = new StringBuffer();
int response = engine.getRedoRecord(response_string);
if(response==0 && response_string.length()>0)
    response = engine.process(response_string.toString());
if(response!=0)
    System.out.println(engine.getLastException());
System.out.print(response)
StringBuffer response_string = new StringBuffer();
int response = engine.getRedoRecord(response_string);
System.out.println(response_string);
System.out.println(response);<jupyter_output><empty_output><jupyter_text>Deleting Recordsuse `deleteRecord()` to remove a record from the data repository (returns "0" upon success) ; `deleteRecord()` can be called as many times as desired and from multiple threads at the same time. The `deleteRecord()` function accepts three parameters as input:- **dataSourceCode:** The name of the data source the record is associated with. This value is configurable to the system- **recordID:** The record ID, used to identify distinct records- **loadID:** The observation load ID for the record; value can be null and will default to dataSourceCode<jupyter_code>ret = engine.deleteRecord(dataSourceCode,recordID, loadID);
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.print(ret)<jupyter_output><empty_output><jupyter_text>Attempt to get the record again. It should error and give an output similar to "Unknown record" and a return value of -2.<jupyter_code>String dataSourceCode = "TEST";
String recordID = "1";
StringBuffer response = new StringBuffer();
int ret = engine.getRecord(dataSourceCode, recordID, response);

if(ret!=0)
    System.out.println(engine.getLastException());
System.out.println(ret)<jupyter_output><empty_output><jupyter_text>Purge RepositoryTo purge the G2 repository, use the aptly named `purgeRepository()` method. This will remove every record in your current repository.<jupyter_code>int ret = engine.purgeRepository();
if(ret!=0)
    System.out.println(engine.getLastException());
System.out.print(ret)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12155944458138197, 'fraction_numerical': 0.012077890066551638, 'mean_word_length': 6.159705882352942, 'pattern_counts': {'":': 69, '<': 104, '<?xml version=': 0, '>': 104, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '25', 'dir': 'jupyter-structured-clean-dedup', 'id': '589333', 'license': 'Apache-2.0', 'n_tokens_mistral': 7066, 'n_tokens_neox': 6821, 'n_words': 2384, 'path': 'notebooks/senzing-examples/java/Senzing-G2Engine(Java).ipynb', 'repo_name': 'SamMacy/docker-jupyter', 'size': '24342'}