<commit_before># lambda-demo

A Clojure AWS Lambda demo.

Reads events from Kinesis stream and stores them in S3.

## Installation

* Create IAM role with following rights: Lambda, Kinesis read, Cloudwatch log write, S3 write
* Create S3 Bucket
* Create Kinesis stream
* Open 'src/lambda-demo/env.clj' and configure setup based on your environment
* Build "zip" file

```
lein uberjar
```

* Create Lambda function with name "LambdaDemo", uberjar as zip file, handler as "lambda-demo.LambdaFn", role the one you created before
* Set your Kinesis stream as an event source (use e.g. 10 as buffer size)
* Enter some test data via repl

```
lein repl
```

```clojure
(use 'lambda-demo.kinesis-util)

(insert-test-data "some-data" 50)
```

* Check your S3 bucket. There should be few nicely sorted files
* Profit!!!

If you make any changes to to Lambda code you can redeploy your Lambda function by running:

```
./update-lambda <your-region>
```

## License

Copyright © 2015 FIXME

Distributed under the Eclipse Public License either version 1.0 or (at
your option) any later version.<commit_msg>Update installation instructions<commit_after># lambda-demo

A Clojure AWS Lambda demo.

Reads events from Kinesis stream and stores them in S3.

## Installation

* Clone the repo
* Setup Lambda environment in `project.clj`.
  You have to at least change the S3 bucket (and possibly region if you don't want to use Ireland datacenter)
* Create Kinesis stream
* Open `src/lambda-demo/env.clj` and configure setup based on your environment
* Install Lambda by running

  $ lein lambda install test

* This will create a Lambda function with name LambdaDemo with basic setup.
* Configure Lambda function to use Kinesis stream as an event source.
* Enter some test data via repl

  $ lein repl


```clojure
(use 'lambda-demo.kinesis-util)

(insert-test-data "some-data" 50)
```

* Check your S3 bucket. There should be a bunch of sorted files.
* Profit!!!

If you make any changes to to Lambda code you can redeploy your Lambda function by running:

```
./update-lambda <your-region>
```

## License

Copyright © 2015-2016 <NAME>

Distributed under the Eclipse Public License.
{'fraction_non_alphanumeric': 0.08260267651130596, 'fraction_numerical': 0.012459621596677434, 'mean_word_length': 4.161904761904762, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7089781', 'n_tokens_mistral': 685, 'n_tokens_neox': 643, 'n_words': 298}