Q:

Apache Spark vs Spring Cloud data flow

I'm new to big data processing and I'm reading about tools for stream processing and building data pipelines.
I found Apache Spark and Spring Cloud Data Flow. I want to know the main differences and the pros and cons of them.
Could anybody help me?

A:

They are 2 completely different tools. 
Spring Data Flow is a toolkit for building data integration and real-time data processing pipelines. This tool will help you to orchestrate data pipelines using Spring Boot Apps (Stream or Task). Under the hood, SCDF might use Spring Batch. Note this Spring Boot Apps can call Spark or Kafka applications to support Stream processing. 
Apache Spark is an engine for data processing, it is being highly used for data intensive processing and data science. It has libraries such as ML (Machine Learning), Graph (graph processing), integration with Apache Kafka (Spark Streaming), among others. 
For streaming, I highly recommend you to study Apache Kafka. 

