<commit_before># coding=utf-8
# Licensed Materials - Property of IBM
# Copyright IBM Corp. 2017

"""
Python application support for IBM Streams.

Overview
########

IBM® Streams is an advanced analytic platform that allows user-developed applications to quickly ingest,
analyze and correlate information as it arrives from thousands of real-time sources.
Streams can handle very high data throughput rates, millions of events or messages per second.

With this API Python developers can build streaming applications that can be executed using IBM Streams,
including the processing being distributed across multiple computing resources (hosts or machines) for scalability.

IBM Streams is also available on IBM Cloud through :ref:`sas-main`

Creating Applications
#####################

Applications are created by declaring a flow graph contained
in a :py:class:`~Topology` instance.

For details see :py:mod:`streamsx.topology.topology`.

Microservices
#############

Publish-subscribe provides the abiltity to connect streams between independent
IBM Streams applications regardless of their implementation language.
This allows a `microservice approach <https://developer.ibm.com/streamsdev/2016/09/02/analytics-microservice-architecture-with-ibm-streams/>`_
where a Streams application acting as a service publishes one
or more streams. Subscriber services then subscribe to those streams
without requiring any knowledge of how a stream is published.

Publish-subscribe overview
==========================
Applications can publish streams to a topic name which can then be
subscribed to by other applications (or even the same application).
Publish-subscribe works across applications written in SPL and those written
using the Java/Scala and Python application APIs.<commit_msg>Add extension info<commit_after># coding=utf-8
# Licensed Materials - Property of IBM
# Copyright IBM Corp. 2017

"""
Python application support for IBM Streams.

Overview
########

IBM® Streams is an advanced analytic platform that allows user-developed applications to quickly ingest,
analyze and correlate information as it arrives from thousands of real-time sources.
Streams can handle very high data throughput rates, millions of events or messages per second.

With this API Python developers can build streaming applications that can be executed using IBM Streams,
including the processing being distributed across multiple computing resources (hosts or machines) for scalability.

IBM Streams is also available on IBM Cloud through :ref:`sas-main`

Creating Applications
#####################

Applications are created by declaring a flow graph contained
in a :py:class:`~Topology` instance.

For details see :py:mod:`streamsx.topology.topology`.

Extensions
##########

This package (`streamsx`) provides the core functionality to build
streaming applications in Python for Streams.

Additional `streamsx.*` packages are available that provide adapters to
external systems, analytics and streaming primitives. This include:

 * Apache Kafka integration - `streamsx.kafka <https://pypi.org/project/streamsx.kafka/>`_
 * Database integration - `streamsx.database <https://pypi.org/project/streamsx.database/>`_
 * Geospatial analytics- `streamsx.geospatial <https://pypi.org/project/streamsx.geospatial/>`_
 * IBM Event Streams integration - `streamsx.eventstreams <https://pypi.org/project/streamsx.eventstreams/>`_
 * MQTT integration - `streamsx.mqtt <https://pypi.org/project/streamsx.mqtt/>`_
 * Cloud Object Storage integration - `streamsx.objectstorage <https://pypi.org/project/streamsx.objectstorage/>`_
 * Streaming primitives - `streamsx.standard <https://pypi.org/project/streamsx.standard/>`_

A full list of available packages is at : https://pypi.org/search?q=streamsx

Microservices
#############

Publish-subscribe provides the abiltity to connect streams between independent
IBM Streams applications regardless of their implementation language.
This allows a `microservice approach <https://developer.ibm.com/streamsdev/2016/09/02/analytics-microservice-architecture-with-ibm-streams/>`_
where a Streams application acting as a service publishes one
or more streams. Subscriber services then subscribe to those streams
without requiring any knowledge of how a stream is published.

Publish-subscribe overview
==========================
Applications can publish streams to a topic name which can then be
subscribed to by other applications (or even the same application).
Publish-subscribe works across applications written in SPL and those written
using the Java/Scala and Python application APIs.
{'fraction_non_alphanumeric': 0.09185667752442997, 'fraction_numerical': 0.005646036916395223, 'mean_word_length': 5.460028050490884, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6348385', 'n_tokens_mistral': 1205, 'n_tokens_neox': 1111, 'n_words': 502}