<jupyter_start><jupyter_text>Connecting Spark with Cosmos DB Change feed  In this sample, we connect to Cosmos DB change feed through the Cosmos DB Java SDK and Spark. We will read new changes and count the number of new changes. In this specific example, the collection we are reading from contains documents with product information. To learn more about Azure Cosmos DB and change feed check out the [main page](https://docs.microsoft.com/azure/cosmos-db/introduction), or the [change feed page](https://docs.microsoft.com/azure/cosmos-db/change-feed).   ConfigurationConfiguring environment with Cosmos DB jar files available here on the [Github](https://github.com/Azure/azure-cosmosdb-spark/tree/master/releases/azure-cosmosdb-spark_2.1.0_2.11-0.0.4)<jupyter_code>%%configure
{ "name":"Spark-to-Cosmos_DB_Connector", 
  "executorMemory": "8G", 
  "executorCores": 2, 
  "numExecutors": 2, 
  "driverCores": 2,
  "jars": ["wasb:///example/jars/0.0.3.2/azure-documentdb-1.12.0.jar","wasb:///example/jars/0.0.3.2/azure-cosmosdb-spark-0.0.3-SNAPSHOT.jar", "wasb:///example/jars/0.0.3.2/azure-documentdb-rx-0.9.0-rc1.jar", "wasb:///example/jars/0.0.3.2/rxjava-1.3.0.jar" ],
  "conf": {
    "spark.jars.packages": "graphframes:graphframes:0.5.0-spark2.1-s_2.11",   
    "spark.jars.excludes": "org.scala-lang:scala-reflect"
   }
}<jupyter_output><empty_output><jupyter_text>Connecting to Cosmos DB change feed To start the Spark session with change feed, we need to specify some config options. **Endpoint**: your Cosmos DB url (i.e. https://youraccount.documents.azzure.com:443/)**Masterkey**: the key string for you Cosmos DB account**Database**: name of your existing database**Collection**: name of your existing collection from which you wish to read the change feed**ChangeFeedQueryName**: unique string for your query<jupyter_code># Adding variables 
rollingChangeFeed = False
startFromTheBeginning = False
useNextToken = True 

productsConfig = {
"Endpoint" : "https://youraccount.documents.azure.com:443/",
"Masterkey" : "<KEY>
"Database" : "AW",
"Collection" : "AWProducts", 
"ReadChangeFeed" : "true",
"ChangeFeedQueryName" : str(rollingChangeFeed) + str(startFromTheBeginning) + str(useNextToken),
"ChangeFeedStartFromTheBeginning" : str(startFromTheBeginning),
"ChangeFeedUseNextToken" : str(useNextToken),
"RollingChangeFeed" : str(rollingChangeFeed),
"ChangeFeedCheckpointLocation" : "./changefeedcheckpointlocation",
"SamplingRatio" : "1.0"
}<jupyter_output>Starting Spark application<jupyter_text>Loading data frame<jupyter_code>products = spark.read.format("com.microsoft.azure.cosmosdb.spark").options(**productsConfig).load()<jupyter_output><empty_output><jupyter_text>Initializing count of new products on change feed to be 0<jupyter_code>new_product_count = 0<jupyter_output><empty_output><jupyter_text>Rerun below section to check for new changes and increment count of new changes.<jupyter_code>products.show()
new_product_count += products.count() 
print new_product_count<jupyter_output>1
{'fraction_non_alphanumeric': 0.1310985728509791, 'fraction_numerical': 0.02057749751078659, 'mean_word_length': 5.928735632183908, 'pattern_counts': {'":': 9, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '578468', 'license': 'MIT', 'n_tokens_mistral': 1025, 'n_tokens_neox': 978, 'n_words': 229, 'path': 'samples/notebooks/Spark+and+Cosmos+DB+Change+Feed.ipynb', 'repo_name': 'kevlangdo/azure-cosmosdb-spark', 'size': '3066'}