<commit_before><pre>
======================================================================
               __________  ____  ____  _________
              / ____/ __ \/ __ \/ __ \/ ____/   |
             / / __/ /_/ / / / / / /_/ / /   / /| |
            / /_/ / ____/ /_/ / _, _/ /___/ ___ |
            \____/_/    \____/_/ |_|\____/_/  |_|
                  The Greenplum Query Optimizer
              Copyright (c) 2015, Pivotal Software, Inc.
            Licensed under the Apache License, Version 2.0
======================================================================
</pre>

Welcome to GPORCA, the Greenplum Next Generation Query Optimizer!

GPORCA supports various build types: debug, release with debug info, release.
On x86 systems, GPORCA can also be built as a 32-bit or 64-bit library. You'll
need CMake 3.0 or higher to build GPORCA. Get it from cmake.org, or your
operating system's package manager.

# First Time Setup

## Clone GPORCA

```
git clone https://github.com/greenplum-db/gporca.git
cd gporca
```

## Pre-Requisites

GPORCA uses the following libraries:
1) GPOS - Greenplum's OS Abstraction Layer
2) GP-Xerces - Greenplum's patched version of Xerces-C 3.1.X

### Installing GPOS

[GPOS is available here](https://github.com/greenplum-db/gpos). The GPOS README
gives instructions for building and installing GPOS. Note that the build type
(e.g. DEBUG vs. RELEASE) for GPOS and GPORCA should match (mixing and matching
can lead to errors).

### Installing GP-Xerces

[GP-XERCES is available here](https://github.com/greenplum-db/gp-xerces). The GP-XERCES README
gives instructions for building and installing.

## Build GPORCA

Go into `gporca` directory:

```
mkdir build
cd build
cmake ../
make
sudo make install
```

## Test GPORCA

To run all GPORCA tests, simply use the `ctest` command from the build directory
after `make` finishes.

```
ctest
```

Much like `make`, `ctest` has a -j option that allows running multiple tests in
parallel to save time. Using it is recommended for faster testing.

```
ctest -j7
```

By default, `ctest` does not print the output of failed tests. To print the
output of failed tests, use the `--output-on-failure` flag like so (this is
useful for debugging failed tests):

```
ctest -j7 --output-on-failure
```

To run a specific individual test, use the `gporca_test` executable directly.

```
./server/gporca_test -U CAggTest
```

To run a specific minidump, for example for `../data/dxl/minidump/TVFRandom.mdp`:
```
./server/gporca_test -d ../data/dxl/minidump/TVFRandom.mdp
```

Note that some tests use assertions that are only enabled for DEBUG builds, so
DEBUG-mode tests tend to be more rigorous.

## [Experimental] Concourse
GPORCA contains a series of pipeline and task files to run various sets of tests
on [concourse](concourse.ci). You can learn more about deploying concourse with
[bosh at bosh.io](bosh.io).<commit_msg>Add instructions for creating a minidump and adding it to the test suite.<commit_after><pre>
======================================================================
               __________  ____  ____  _________
              / ____/ __ \/ __ \/ __ \/ ____/   |
             / / __/ /_/ / / / / / /_/ / /   / /| |
            / /_/ / ____/ /_/ / _, _/ /___/ ___ |
            \____/_/    \____/_/ |_|\____/_/  |_|
                  The Greenplum Query Optimizer
              Copyright (c) 2015, Pivotal Software, Inc.
            Licensed under the Apache License, Version 2.0
======================================================================
</pre>

Welcome to GPORCA, the Greenplum Next Generation Query Optimizer!

GPORCA supports various build types: debug, release with debug info, release.
On x86 systems, GPORCA can also be built as a 32-bit or 64-bit library. You'll
need CMake 3.0 or higher to build GPORCA. Get it from cmake.org, or your
operating system's package manager.

# First Time Setup

## Clone GPORCA

```
git clone https://github.com/greenplum-db/gporca.git
cd gporca
```

## Pre-Requisites

GPORCA uses the following libraries:
1) GPOS - Greenplum's OS Abstraction Layer
2) GP-Xerces - Greenplum's patched version of Xerces-C 3.1.X

### Installing GPOS

[GPOS is available here](https://github.com/greenplum-db/gpos). The GPOS README
gives instructions for building and installing GPOS. Note that the build type
(e.g. DEBUG vs. RELEASE) for GPOS and GPORCA should match (mixing and matching
can lead to errors).

### Installing GP-Xerces

[GP-XERCES is available here](https://github.com/greenplum-db/gp-xerces). The GP-XERCES README
gives instructions for building and installing.

## Build GPORCA

Go into `gporca` directory:

```
mkdir build
cd build
cmake ../
make
sudo make install
```

## Test GPORCA

To run all GPORCA tests, simply use the `ctest` command from the build directory
after `make` finishes.

```
ctest
```

Much like `make`, `ctest` has a -j option that allows running multiple tests in
parallel to save time. Using it is recommended for faster testing.

```
ctest -j7
```

By default, `ctest` does not print the output of failed tests. To print the
output of failed tests, use the `--output-on-failure` flag like so (this is
useful for debugging failed tests):

```
ctest -j7 --output-on-failure
```

To run a specific individual test, use the `gporca_test` executable directly.

```
./server/gporca_test -U CAggTest
```

To run a specific minidump, for example for `../data/dxl/minidump/TVFRandom.mdp`:
```
./server/gporca_test -d ../data/dxl/minidump/TVFRandom.mdp
```

Note that some tests use assertions that are only enabled for DEBUG builds, so
DEBUG-mode tests tend to be more rigorous.

## Adding tests

Most of the regression tests come in the form of a "minidump" file.
A minidump is an XML file that contains all the input needed to plan a query,
including information about all tables, datatypes, and functions used, as well
as statistics. It also contains the resulting plan.

A new minidump can be created by running a query on a live GPDB server:

1. Run these in a psql session:

   ```
set client_min_messages='log';
set optimizer=on;
set optimizer_enumerate_plans=on;
set optimizer_minidump=always;
set optimizer_enable_constant_expression_evaluation=off;
   ```

2. Run the query in the same psql session. It will create a minidump file
   under the "minidumps" directory, in the master's data directory:

   ```
$ ls -l ~/data-master/minidumps/
total 12
-rw------- 1 heikki heikki 10818 Jun 10 22:02 Minidump_20160610_220222_4_14.mdp
   ```

3. Run xmllint on the minidump to format it better, and copy it under the
   data/dxl/minidump directory:

   ```
xmllint --format ~/data-master/minidumps/Minidump_20160610_220222_4_14.mdp > data/dxl/minidump/MyTest.xml
   ```

5. Add it to the test suite, in server/src/unittest/gpopt/minidump/CICGTest.cpp

   ```
--- a/server/src/unittest/gpopt/minidump/CICGTest.cpp
+++ b/server/src/unittest/gpopt/minidump/CICGTest.cpp
@@ -217,6 +217,7 @@ const CHAR *rgszFileNames[] =
                "../data/dxl/minidump/EffectsOfJoinFilter.mdp",
                "../data/dxl/minidump/Join-IDF.mdp",
                "../data/dxl/minidump/CoerceToDomain.mdp",
+               "../data/dxl/minidump/Mytest.mdp",
                "../data/dxl/minidump/LeftOuter2InnerUnionAllAntiSemiJoin.mdp",
 #ifndef GPOS_DEBUG
                // TODO:  - Jul 14 2015; disabling it for debug build to reduce testing time
   ```


## [Experimental] Concourse
GPORCA contains a series of pipeline and task files to run various sets of tests
on [concourse](concourse.ci). You can learn more about deploying concourse with
[bosh at bosh.io](bosh.io).
{'fraction_non_alphanumeric': 0.13793555148022008, 'fraction_numerical': 0.014016243122871365, 'mean_word_length': 3.1950549450549453, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 8, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 3}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3969977', 'n_tokens_mistral': 2560, 'n_tokens_neox': 2387, 'n_words': 844}