<commit_before># INSTALLATION INSTRUCTIONS
## for Ubuntu 18.04.4-server

### -1/ Installer and Manual install instructions

Make sure you are reading the parsed version of this Document. When in doubt [click here](https://misp.github.io/MISP/INSTALL.ubuntu1804/).

To install MISP on a fresh Ubuntu 18.04, all you need to do is the following:

```bash
# Please check the installer options first to make the best choice for your install
wget -O /tmp/INSTALL.sh https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh
bash /tmp/INSTALL.sh

# This will install MISP Core
wget -O /tmp/INSTALL.sh https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh
bash /tmp/INSTALL.sh -c
```

### 0/ MISP Ubuntu 18.04-server install - status
-------------------------
!!! notice
    Installer tested working by [@SteveClement](https://twitter.com/SteveClement) on 20200501 (works with **Ubuntu 18.10/19.04** too)

!!! notice
    This document also serves as a source for the [INSTALL-misp.sh](https://github.com/MISP/MISP/blob/2.4/INSTALL/INSTALL.sh) script.
    Which explains why you will see the use of shell *functions* in various steps.
    Henceforth the document will also follow a more logical flow. In the sense that all the dependencies are installed first then config files are generated, etc...

!!! notice
    If the next line is `[!generic/core.md!]()` [click here](https://misp.github.io/MISP/INSTALL.ubuntu1804/).

{!generic/core.md!}

### 1/ Minimal Ubuntu install
-------------------------

#### Install a minimal Ubuntu 18.04-server system with the software:
- OpenSSH server
- This guide assumes a user name of 'misp' with sudo working

#### Make sure your system is up2date
```bash
# <snippet-begin 0_apt-upgrade.sh>
aptUpgrade () {
  debug "Upgrading system"
  checkAptLock

  # If we run in non-interactive mode, make sure we do not stop all of a sudden
  if [[ "${PACKER}" == "1" || "${UNATTENDED}" == "1" ]]; then
    export DEBIAN_FRONTEND=noninteractive
    export DEBIAN_PRIORITY=critical
    sudo -E apt-get -qy -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" upgrade
    sudo -E apt-get -qy autoclean
  else
    sudo apt-get upgrade -qy
  fi
}
# <snippet-end 0_apt-upgrade.sh>
```

{!generic/sudo_etckeeper.md!}

{!generic/ethX.md!}

#### install postfix, there will be some questions.
```bash
# <snippet-begin postfix.sh>
sudo apt-get install postfix dialog -qy
# <snippet-end postfix.sh>
```

!!! notice
    Postfix Configuration: Satellite system<br />
    change the relay server later with:
    ```bash
    sudo postconf -e 'relayhost = example.com'
    sudo postfix reload
    ```

{!generic/globalVariables.md!}

### 2/ Install LAMP & dependencies
------------------------------
Once the system is installed you can perform the following steps.
```bash
# <snippet-begin 0_installCoreDeps.sh>
installCoreDeps () {
  debug "Installing core dependencies"
  # Install the dependencies: (some might already be installed)
  sudo apt-get install curl gcc git gpg-agent make python python3 openssl redis-server sudo vim zip unzip virtualenv libfuzzy-dev sqlite3 moreutils -qy

  # Install MariaDB (a MySQL fork/alternative)
  sudo apt-get install mariadb-client mariadb-server -qy

  # Install Apache2
  sudo apt-get install apache2 apache2-doc apache2-utils -qy

  # install Mitre's STIX and its dependencies by running the following commands:
  sudo apt-get install python3-dev python3-pip libxml2-dev libxslt1-dev zlib1g-dev python-setuptools -qy

  sudo apt install expect -qy
}
# <snippet-end 0_installCoreDeps.sh>

# <snippet-begin 0_installDepsPhp72.sh>
# Install Php 7.2 dependencies
installDepsPhp72 () {
  debug "Installing PHP 7.2 dependencies"
  PHP_ETC_BASE=/etc/php/7.2
  PHP_INI=${PHP_ETC_BASE}/apache2/php.ini
  checkAptLock
  sudo apt install -qy \
  libapache2-mod-php \
  php php-cli \
  php-dev \
  php-json php-xml php-mysql php7.2-opcache php-readline php-mbstring php-zip \
  php-redis php-gnupg \
  php-gd

  for key in upload_max_filesize post_max_size max_execution_time max_input_time memory_limit
  do
      sudo sed -i "s/^\($key\).*/\1 = $(eval echo \${$key})/" $PHP_INI
  done
}
# <snippet-end 0_installDepsPhp72.sh>
```

### 3/ MISP code
------------
```bash
# <snippet-begin 1_mispCoreInstall.sh>
installCore () {
  debug "Installing ${LBLUE}MISP${NC} core"
  # Download MISP using git in the /var/www/ directory.
  sudo mkdir ${PATH_TO_MISP}
  sudo chown ${WWW_USER}:${WWW_USER} ${PATH_TO_MISP}
  cd ${PATH_TO_MISP}
  # Make git ignore filesystem permission differences for submodules
  $SUDO_WWW git submodule foreach --recursive git config core.filemode false

  # Make git ignore filesystem permission differences
  $SUDO_WWW git config core.filemode false

  # Create a python3 virtualenv
  $SUDO_WWW virtualenv -p python3 ${PATH_TO_MISP}/venv

  # make pip happy
  sudo mkdir /var/www/.cache/
  sudo chown $WWW_USER:$WWW_USER /var/www/.cache

  cd ${PATH_TO_MISP}/app/files/scripts
  $SUDO_WWW git clone https://github.com/CybOXProject/python-cybox.git
  $SUDO_WWW git clone https://github.com/STIXProject/python-stix.git
  $SUDO_WWW git clone https://github.com/MAECProject/python-maec.git

  # install mixbox to accommodate the new STIX dependencies:
  $SUDO_WWW git clone https://github.com/CybOXProject/mixbox.git
  cd ${PATH_TO_MISP}/app/files/scripts/mixbox
  $SUDO_WWW ${PATH_TO_MISP}/venv/bin/pip install .
  cd ${PATH_TO_MISP}/app/files/scripts/python-cybox<commit_msg>fix: [installer] Added missing checkout<commit_after># INSTALLATION INSTRUCTIONS
## for Ubuntu 18.04.4-server

### -1/ Installer and Manual install instructions

Make sure you are reading the parsed version of this Document. When in doubt [click here](https://misp.github.io/MISP/INSTALL.ubuntu1804/).

To install MISP on a fresh Ubuntu 18.04, all you need to do is the following:

```bash
# Please check the installer options first to make the best choice for your install
wget -O /tmp/INSTALL.sh https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh
bash /tmp/INSTALL.sh

# This will install MISP Core
wget -O /tmp/INSTALL.sh https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh
bash /tmp/INSTALL.sh -c
```

### 0/ MISP Ubuntu 18.04-server install - status
-------------------------
!!! notice
    Installer tested working by [@SteveClement](https://twitter.com/SteveClement) on 20200501 (works with **Ubuntu 18.10/19.04** too)

!!! notice
    This document also serves as a source for the [INSTALL-misp.sh](https://github.com/MISP/MISP/blob/2.4/INSTALL/INSTALL.sh) script.
    Which explains why you will see the use of shell *functions* in various steps.
    Henceforth the document will also follow a more logical flow. In the sense that all the dependencies are installed first then config files are generated, etc...

!!! notice
    If the next line is `[!generic/core.md!]()` [click here](https://misp.github.io/MISP/INSTALL.ubuntu1804/).

{!generic/core.md!}

### 1/ Minimal Ubuntu install
-------------------------

#### Install a minimal Ubuntu 18.04-server system with the software:
- OpenSSH server
- This guide assumes a user name of 'misp' with sudo working

#### Make sure your system is up2date
```bash
# <snippet-begin 0_apt-upgrade.sh>
aptUpgrade () {
  debug "Upgrading system"
  checkAptLock

  # If we run in non-interactive mode, make sure we do not stop all of a sudden
  if [[ "${PACKER}" == "1" || "${UNATTENDED}" == "1" ]]; then
    export DEBIAN_FRONTEND=noninteractive
    export DEBIAN_PRIORITY=critical
    sudo -E apt-get -qy -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" upgrade
    sudo -E apt-get -qy autoclean
  else
    sudo apt-get upgrade -qy
  fi
}
# <snippet-end 0_apt-upgrade.sh>
```

{!generic/sudo_etckeeper.md!}

{!generic/ethX.md!}

#### install postfix, there will be some questions.
```bash
# <snippet-begin postfix.sh>
sudo apt-get install postfix dialog -qy
# <snippet-end postfix.sh>
```

!!! notice
    Postfix Configuration: Satellite system<br />
    change the relay server later with:
    ```bash
    sudo postconf -e 'relayhost = example.com'
    sudo postfix reload
    ```

{!generic/globalVariables.md!}

### 2/ Install LAMP & dependencies
------------------------------
Once the system is installed you can perform the following steps.
```bash
# <snippet-begin 0_installCoreDeps.sh>
installCoreDeps () {
  debug "Installing core dependencies"
  # Install the dependencies: (some might already be installed)
  sudo apt-get install curl gcc git gpg-agent make python python3 openssl redis-server sudo vim zip unzip virtualenv libfuzzy-dev sqlite3 moreutils -qy

  # Install MariaDB (a MySQL fork/alternative)
  sudo apt-get install mariadb-client mariadb-server -qy

  # Install Apache2
  sudo apt-get install apache2 apache2-doc apache2-utils -qy

  # install Mitre's STIX and its dependencies by running the following commands:
  sudo apt-get install python3-dev python3-pip libxml2-dev libxslt1-dev zlib1g-dev python-setuptools -qy

  sudo apt install expect -qy
}
# <snippet-end 0_installCoreDeps.sh>

# <snippet-begin 0_installDepsPhp72.sh>
# Install Php 7.2 dependencies
installDepsPhp72 () {
  debug "Installing PHP 7.2 dependencies"
  PHP_ETC_BASE=/etc/php/7.2
  PHP_INI=${PHP_ETC_BASE}/apache2/php.ini
  checkAptLock
  sudo apt install -qy \
  libapache2-mod-php \
  php php-cli \
  php-dev \
  php-json php-xml php-mysql php7.2-opcache php-readline php-mbstring php-zip \
  php-redis php-gnupg \
  php-gd

  for key in upload_max_filesize post_max_size max_execution_time max_input_time memory_limit
  do
      sudo sed -i "s/^\($key\).*/\1 = $(eval echo \${$key})/" $PHP_INI
  done
}
# <snippet-end 0_installDepsPhp72.sh>
```

### 3/ MISP code
------------
```bash
# <snippet-begin 1_mispCoreInstall.sh>
installCore () {
  debug "Installing ${LBLUE}MISP${NC} core"
  # Download MISP using git in the /var/www/ directory.
  sudo mkdir ${PATH_TO_MISP}
  sudo chown ${WWW_USER}:${WWW_USER} ${PATH_TO_MISP}
  cd ${PATH_TO_MISP}
  ${SUDO_WWW} git clone https://github.com/MISP/MISP.git ${PATH_TO_MISP}
  ${SUDO_WWW} git submodule update --progress --init --recursive
  # Make git ignore filesystem permission differences for submodules
  $SUDO_WWW git submodule foreach --recursive git config core.filemode false

  # Make git ignore filesystem permission differences
  $SUDO_WWW git config core.filemode false

  # Create a python3 virtualenv
  $SUDO_WWW virtualenv -p python3 ${PATH_TO_MISP}/venv

  # make pip happy
  sudo mkdir /var/www/.cache/
  sudo chown $WWW_USER:$WWW_USER /var/www/.cache

  cd ${PATH_TO_MISP}/app/files/scripts
  $SUDO_WWW git clone https://github.com/CybOXProject/python-cybox.git
  $SUDO_WWW git clone https://github.com/STIXProject/python-stix.git
  $SUDO_WWW git clone https://github.com/MAECProject/python-maec.git

  # install mixbox to accommodate the new STIX dependencies:
  $SUDO_WWW git clone https://github.com/CybOXProject/mixbox.git
  cd ${PATH_TO_MISP}/app/files/scripts/mixbox
  $SUDO_WWW ${PATH_TO_MISP}/venv/bin/pip install .
  cd ${PATH_TO_MISP}/app/files/scripts/python-cybox
{'fraction_non_alphanumeric': 0.12904676258992806, 'fraction_numerical': 0.016546762589928057, 'mean_word_length': 3.574660633484163, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 21, 'lorem ipsum': 0, 'www.': 6, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5869615', 'n_tokens_mistral': 4103, 'n_tokens_neox': 3702, 'n_words': 1111}