<commit_before>.. include:: <isonum.txt>
.. _module_batch:

==============
Module: Batch
==============

.. important:: The Batch module requires ThreatConnect version 5.6 or higher.

The ThreatConnect TcEx Framework provides the :py:mod:`~tcex.tcex_ti_batch.TcExBatch` module to create, delete, and update both Groups and Indicators in the ThreatConnect platform. The App developer can dynamically build data objects, and the Batch module will write the data to the ThreatConnect API.

External ID (xid)
=================
The batch JSON data requires an **xid** value for all Groups and Indicators.  The xid is used internally in ThreatConnect for associations and for updating existing Groups.  The xid **must** be unique for the entire instance of ThreatConnect.

The :py:mod:`~tcex.tcex_ti_batch.TcExBatch` module provides the :py:meth:`~tcex.tcex_ti_batch.TcExBatch.generate_xid` method to assist in generating two types of xid. The first type is a unique xid based on UUID4.  No input is required to produce a unique xid.  The second type is a unique and reproducible xid value.  This is the **preferred** xid type, as it allows a Group to have the same xid on subsequent runs of the App.  To generate a unique and reproducible xid, either a string or array of value can be passed to the method (e.g., **myapp-adversary-5** or **['myapp', 'adversary', '222']**).

For Interface 1 and 2, the xid is optional; if not provided, a unique xid (UUID4) will be auto-generated.  A string value can also be passed if the xid is a known value (e.g., the id field from a remote source). Passing in an xid, when possible, is **best practice** and allows Groups and Indicators to be easily updated.

.. important:: It is best practice to provide a unique and reproducible xid value when creating Groups. If the xid created for a new Group matches the xid for an existing Group, the existing Group will be overwritten.

.. note:: In all of the examples below, the code to create the content is removed. Read more on how to create the content in the `Submit section <https://docs.threatconnect.com/en/latest/tcex/module_batch.html#submit>`__.

Groups
======
There are three interfaces to add Group Threat Intelligence data to the Batch Module.

Group Interface 1
-----------------
The first interface is for type-specific access.  This interface allows for passing of all the data in the method call or of only the required fields, with optional fields being set via property setters. All metadata (e.g., Attributes, Security Labels, and Tags) can be added to the group instance directly.

The example below passes all supported fields to the``adversary()`` method.

.. code-block:: python
    :linenos:
    :lineno-start: 1
    :emphasize-lines: 2-5,7

    batch = tcex.batch('MyOrg')
    adversary = batch.adversary('adversary-001', xid='my-xid-000')
    adversary.attribute('Description', 'Example Description', True)
    adversary.tag('Example Tag')
    adversary.security_label('TLP Green')
    # optional save method
    batch.save(adversary)

The example below passes only the required fields to **document()**.  The optional properties can then be set whenever required. The same interface is used for the Attribute.  The required Attribute properties are set first, and then optional values can be added.<commit_msg>Move submit note under groups section<commit_after>.. include:: <isonum.txt>
.. _module_batch:

==============
Module: Batch
==============

.. important:: The Batch module requires ThreatConnect version 5.6 or higher.

The ThreatConnect TcEx Framework provides the :py:mod:`~tcex.tcex_ti_batch.TcExBatch` module to create, delete, and update both Groups and Indicators in the ThreatConnect platform. The App developer can dynamically build data objects, and the Batch module will write the data to the ThreatConnect API.

External ID (xid)
=================
The batch JSON data requires an **xid** value for all Groups and Indicators.  The xid is used internally in ThreatConnect for associations and for updating existing Groups.  The xid **must** be unique for the entire instance of ThreatConnect.

The :py:mod:`~tcex.tcex_ti_batch.TcExBatch` module provides the :py:meth:`~tcex.tcex_ti_batch.TcExBatch.generate_xid` method to assist in generating two types of xid. The first type is a unique xid based on UUID4.  No input is required to produce a unique xid.  The second type is a unique and reproducible xid value.  This is the **preferred** xid type, as it allows a Group to have the same xid on subsequent runs of the App.  To generate a unique and reproducible xid, either a string or array of value can be passed to the method (e.g., **myapp-adversary-5** or **['myapp', 'adversary', '222']**).

For Interface 1 and 2, the xid is optional; if not provided, a unique xid (UUID4) will be auto-generated.  A string value can also be passed if the xid is a known value (e.g., the id field from a remote source). Passing in an xid, when possible, is **best practice** and allows Groups and Indicators to be easily updated.

.. important:: It is best practice to provide a unique and reproducible xid value when creating Groups. If the xid created for a new Group matches the xid for an existing Group, the existing Group will be overwritten.

Groups
======

.. note:: In all of the examples below, the code to create the content is removed. Read more on how to create the content in the `Submit section <https://docs.threatconnect.com/en/latest/tcex/module_batch.html#submit>`__.

There are three interfaces to add Group Threat Intelligence data to the Batch Module.

Group Interface 1
-----------------
The first interface is for type-specific access.  This interface allows for passing of all the data in the method call or of only the required fields, with optional fields being set via property setters. All metadata (e.g., Attributes, Security Labels, and Tags) can be added to the group instance directly.

The example below passes all supported fields to the``adversary()`` method.

.. code-block:: python
    :linenos:
    :lineno-start: 1
    :emphasize-lines: 2-5,7

    batch = tcex.batch('MyOrg')
    adversary = batch.adversary('adversary-001', xid='my-xid-000')
    adversary.attribute('Description', 'Example Description', True)
    adversary.tag('Example Tag')
    adversary.security_label('TLP Green')
    # optional save method
    batch.save(adversary)

The example below passes only the required fields to **document()**.  The optional properties can then be set whenever required. The same interface is used for the Attribute.  The required Attribute properties are set first, and then optional values can be added.
{'fraction_non_alphanumeric': 0.09163286765810424, 'fraction_numerical': 0.006309148264984227, 'mean_word_length': 4.2466509062253746, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4706769', 'n_tokens_mistral': 1872, 'n_tokens_neox': 1759, 'n_words': 918}