<commit_before>.. :author: Cask Data, Inc.
   :description: Getting Started with Cask Data Application Platform
         :copyright: Copyright © 2014 Cask Data, Inc.

.. _get-started:

===================================================
Getting Started with Cask Data Application Platform
===================================================

This chapter is a guide to help you get started with CDAP. At the end of this topic you will have the CDAP SDK up
and running in your development environment, and you will have built, deployed, run, and interacted with a sample
application.

Introduction to the CDAP SDK
============================

The CDAP Software Developers Kit (SDK) is all you need to develop CDAP applications in your development environment
(for example, your laptop or work station). It includes:

- A Standalone CDAP that can run on a single machine in a single JVM. It provides all of
  the CDAP APIs without requiring a Hadoop cluster, using alternative, fully functional
  implementations of CDAP features. For example, application containers are implemented as
  Java threads instead of YARN containers.
- The CDAP console, a web-based graphical user interface to interact with the Standalone CDAP
  and the applications it runs.
- The complete CDAP documentation, including this document and the Javadocs for the CDAP APIs.
- A set of tools, datasets and example applications that help you get familiar with CDAP, and
  can also serve as templates for developing for own applications.

System Requirements and Dependencies for the SDK
------------------------------------------------

The CDAP SDK runs on Linux, Unix, MacOS and Windows, and only has three requirements:

 - `JDK 6 or JDK 7 <http://www.oracle.com/technetwork/java/javase/downloads/index.html>`__ (required to run CDAP;
   note that $JAVA_HOME should be set)
 - `Node.js 0.8.16+ <http://nodejs.org>`__ (required to run the CDAP UI)
 - `Apache Maven 3.0+ <http://maven.apache.org>`__ (required to build CDAP applications)

Downloading and Setting Up the CDAP SDK
=======================================
The SDK is available as a binary on the Downloads section of the Cask Website.
[TODO: add link] Once downloaded, unzip it to a directory on your machine:
::

  $ tar -zxvf cdap-sdk-2.5.0.zip
  $ cd cdap-sdk-2.5.0

**Running CDAP SDK** ::

    $ ./bin/cdap.sh start

Or, if you are using Windows, use the batch script cdap.bat to start the SDK.

Once CDAP is started successfully, you can see the CDAP console running at localhost:9999, and you can
head there to deploy example applications and interact with CDAP.

Creating an Application
=======================

The best way to start developing a CDAP application is using the Maven archetype::

  $ mvn archetype:generate \
    -DarchetypeCatalog=https://repository.cask.co/content/groups/releases/archetype-catalog.xml \
    -DarchetypeGroupId=co.cask.cdap \
    -DarchetypeArtifactId=cdap-app-archetype \
    -DarchetypeVersion=2.5.0``

This creates a Maven project with all required dependencies and Maven plugins as well as a simple
application template that you can modify to develop your application. You can import this Maven project
into your favorite IDE, such as Eclipse or IntelliJ, and you are ready to start developing your first
CDAP application.

.. _examples:

First Steps<commit_msg>add instruction for the CDAP VM<commit_after>.. :author: Cask Data, Inc.
   :description: Getting Started with Cask Data Application Platform
         :copyright: Copyright © 2014 Cask Data, Inc.

.. _get-started:

===================================================
Getting Started with Cask Data Application Platform
===================================================

This chapter is a guide to help you get started with CDAP. At the end of this topic you will have the CDAP SDK up
and running in your development environment, and you will have built, deployed, run, and interacted with a sample
application.

Introduction to the CDAP SDK
============================

The CDAP Software Developers Kit (SDK) is all you need to develop CDAP applications in your development environment
(for example, your laptop or work station). It includes:

- A Standalone CDAP that can run on a single machine in a single JVM. It provides all of
  the CDAP APIs without requiring a Hadoop cluster, using alternative, fully functional
  implementations of CDAP features. For example, application containers are implemented as
  Java threads instead of YARN containers.
- The CDAP console, a web-based graphical user interface to interact with the Standalone CDAP
  and the applications it runs.
- The complete CDAP documentation, including this document and the Javadocs for the CDAP APIs.
- A set of tools, datasets and example applications that help you get familiar with CDAP, and
  can also serve as templates for developing for own applications.

System Requirements and Dependencies
------------------------------------

The CDAP SDK runs on Linux, Unix, MacOS and Windows, and only has three requirements:

- `JDK 6 or JDK 7 <http://www.oracle.com/technetwork/java/javase/downloads/index.html>`__ (required to run CDAP;
  note that $JAVA_HOME should be set)
- `Node.js 0.8.16+ <http://nodejs.org>`__ (required to run the CDAP UI)
- `Apache Maven 3.0+ <http://maven.apache.org>`__ (required to build CDAP applications)

.. highlight:: console

Download and Setup
==================

There are three ways to download the CDAP SDK: as a binary zip file, an Virtual Machine image, or a Docker image.

- The zip file is available on the Downloads section of the Cask Website. TODO - insert link.
  Once downloaded, unzip it to a directory on your machine::

    $ tar -zxvf cdap-sdk-2.5.0.zip

- To use the Virtual Machine image, follow these steps:

  + Download and install Oracle VirtualBox or VMWare player to your environment.
  + Download The CDAP Standalone Virtual Machine (.OVA File) from the cask.download.com (TODO - insert link).
  + Import the Virtual Machine into VirtualBox or VMWare Player.
  + The CDAP Standalone Virtual Machine has been configured and setup so you can be productive immediately.
    The Virtual Machine is configured as follows:

    * CDAP VM is configured with 4GB Default RAM (recommended).
    * The VM has Ubuntu Desktop Linux as the operating system for ease of use.
    * No password is required to enter the Machine, however, should you need to install or
      remove software, the admin user and password are both “<PASSWORD>”.
    * 10GB disk space available for you to build your first CDAP project.
    * IntelliJ and Eclipse IDE are installed and will start when the machine starts.
    * The CDAP SDK is installed under ``/Software/cadp-sdk-2.5.0``.
    * The Standalone CDAP will start automatically when the virtual machine starts.
    * The Firefox browser starts when the machine starts. Its default home page is the CDAP Console
      (http://localhost:9999). You may also install your favorite browser.
    * Maven is also installed and configured to work for CDAP.
    * The Java JDK and Node JS are installed.

Starting the Standalone CDAP
----------------------------

Use the ``cdap.sh`` script to start and stop the Standalone CDAP::

  $ cd cdap-sdk-2.5.0
  $ ./bin/cdap.sh start
  ...
  $ ./bin/cdap.sh stop

Or, if you are using Windows, use the batch script cdap.bat to start the SDK.

Once CDAP is started successfully, you can see the CDAP console running at localhost:9999, and you can
head there to deploy example applications and interact with CDAP.

Creating an Application
=======================

The best way to start developing a CDAP application is using the Maven archetype::

  $ mvn archetype:generate \
    -DarchetypeCatalog=https://repository.cask.co/content/groups/releases/archetype-catalog.xml \
    -DarchetypeGroupId=co.cask.cdap \
    -DarchetypeArtifactId=cdap-app-archetype \
    -DarchetypeVersion=2.5.0``

This creates a Maven project with all required dependencies and Maven plugins as well as a simple
application template that you can modify to develop your application. You can import this Maven project
into your favorite IDE, such as Eclipse or IntelliJ, and you are ready to start developing your first
CDAP application.

.. _examples:

First Steps
{'fraction_non_alphanumeric': 0.112009656004828, 'fraction_numerical': 0.00724200362100181, 'mean_word_length': 3.8006952491309387, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6952461', 'n_tokens_mistral': 2255, 'n_tokens_neox': 2103, 'n_words': 1096}