<commit_before># Current Version(Still in Development)

## Major Features and Improvements
*   Added TFX DSL IR compiler that encodes a TFX pipeline into a DSL proto.
*   Supported feature based split partition in ExampleGen.
*   Added the ConcatPlaceholder to tfx.dsl.component.experimental.placeholders.
*   Changed Span information as a property of ExampleGen's output artifact.
    Deprecated ExampleGen input (external) artifact.

## Bug fixes and other changes
*   Added Tuner component, which is still work in progress.
*   Added Tuner component to Iris e2e example.
*   Relaxed the rule that output artifact uris must be newly created. This is a
    temporary workaround to make retry work. We will introduce a more
    comprehensive solution for idempotent execution.
*   Made evaluator output optional (while still recommended) for pusher.
*   Moved BigQueryExampleGen to `tfx.extensions.google_cloud_big_query`.
*   Removed Tuner from custom_components/ as it's supported under components/
    now.
*   Added support of non tf.train.Example protos as internal data payload
    format by ImportExampleGen.

## Breaking changes

### For pipeline authors
*   Moved BigQueryExampleGen to `tfx.extensions.google_cloud_big_query`. The
    previous module path from `tfx.components` is not available anymore.
*   Updated beam pipeline args, users now need to set both `direct_running_mode`
    and `direct_num_workers` explicitly for multi-processing.
*   Added required 'output_data_format' execution property to
    FileBaseExampleGen.
*   Changed ExampleGen to take a string as input source directly instead of a
    Channel of external artifact:
    *   `input` Channel is deprecated. The use of `input` is valid but
        should change to string type `input_base` ASAP.<commit_msg>Added a release note for #2025<commit_after># Current Version(Still in Development)

## Major Features and Improvements
*   Added TFX DSL IR compiler that encodes a TFX pipeline into a DSL proto.
*   Supported feature based split partition in ExampleGen.
*   Added the ConcatPlaceholder to tfx.dsl.component.experimental.placeholders.
*   Changed Span information as a property of ExampleGen's output artifact.
    Deprecated ExampleGen input (external) artifact.

## Bug fixes and other changes
*   Added Tuner component, which is still work in progress.
*   Added Tuner component to Iris e2e example.
*   Relaxed the rule that output artifact uris must be newly created. This is a
    temporary workaround to make retry work. We will introduce a more
    comprehensive solution for idempotent execution.
*   Made evaluator output optional (while still recommended) for pusher.
*   Moved BigQueryExampleGen to `tfx.extensions.google_cloud_big_query`.
*   Removed Tuner from custom_components/ as it's supported under components/
    now.
*   Added support of non tf.train.Example protos as internal data payload
    format by ImportExampleGen.
*   Fixed the name of the usage telemetry when tfx templates are used.

## Breaking changes

### For pipeline authors
*   Moved BigQueryExampleGen to `tfx.extensions.google_cloud_big_query`. The
    previous module path from `tfx.components` is not available anymore.
*   Updated beam pipeline args, users now need to set both `direct_running_mode`
    and `direct_num_workers` explicitly for multi-processing.
*   Added required 'output_data_format' execution property to
    FileBaseExampleGen.
*   Changed ExampleGen to take a string as input source directly instead of a
    Channel of external artifact:
    *   `input` Channel is deprecated. The use of `input` is valid but
        should change to string type `input_base` ASAP.
{'fraction_non_alphanumeric': 0.05092592592592592, 'fraction_numerical': 0.0016339869281045752, 'mean_word_length': 4.073204419889502, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '554397', 'n_tokens_mistral': 982, 'n_tokens_neox': 932, 'n_words': 444}