<commit_before># Changelog

## 0.3

### Web App
* Reduce JS dependencies to JQuery (stable) only
* Add functional tests, increase unit test coverage
* Rewrite database layer (db.py) using SQLAlchemy declarative ORM
* Automate dev. setup with Vagrant and integrate with Travis CI
* Store more info in db and less on filesystem
  * "flagged" sources
  * metadata for new UI features (starring, etc.)
  * metadata for simpler/more efficient views in journalist.py
* Do not set headers in the web app (handle by production config.)
* Add 2fac auth for journalist interface
* Allow OSSEC emails to be encrypted with admin GPG key
* Install app server, monitor server, and hardening via deb package
* UI refresh on source and journalist interfaces
* New UX for journalists:
  * "quick filter" box for codenames
  * "download unread" link
  * star sources
  * more detailed source listings
* Normalize submission timestamps to that of the most recent submission
to minimize metadata that could be used for correlation

### Environment
* Add egress host firewall rules
* Add google-authenticator apache module and basic auth for access to
document interface
* Encrypt bodies of OSSEC email alerts (add postfix+procmail to monitor
server)
* Create apparmor profiles for chrooted interface Tor process
* Update interface apparmor profiles for changes to application code
* Change installation method to deb packages

## 0.2.1

### Web App
* Fix for flagging errors
* Validate journalist messages
* Add logging using standard Python library.
* Add delete collection
* Replace bcrypt with scrypt
* Clear referer on external links

### Environment
* Set maximum request body size in CONFIG_OPTIONS
* Add security-related HTTP headers to Apache config
* Remove mysql database, replace w/ sqlite. Update sqlite apparmor profile.
* Add outbound iptables rule for source/document groups

## 0.2

* Various documentation improvements

### Web App
* Remove javascript dependency in source interface
* Add warning to source interface about using javascript (with Gritter)<commit_msg>Update changelog for 0.3<commit_after># Changelog

## 0.3

### Web App

This is a high-level overview of some of the more significant changes between SecureDrop 0.2 and 0.3. For the complete set of changes, diff the tags.

* Reduce JS dependencies to JQuery (stable) only
* Add functional tests, increase unit test coverage
* Rewrite database layer (db.py) using SQLAlchemy declarative ORM
* Automate dev. setup with Vagrant and integrate with Travis CI
* Store more info in db and less on filesystem
  * "flagged" sources
  * metadata for new UI features (starring, etc.)
  * metadata for simpler/more efficient views in journalist.py
* Do not set headers in the web app (handle by production config.)
* Add 2fac auth for journalist interface
* Allow OSSEC emails to be encrypted with admin GPG key
* Install app server, monitor server, Python dependencies, and custom configuration via deb packages
* UI refresh on source and journalist interfaces
* New UX for journalists:
  * "quick filter" box for codenames
  * "download unread" link
  * star sources
  * multi-select actions for sources (delete, star, unstar) and submissions (download, delete)
  * more detailed source listings
* Normalize submission timestamps to that of the most recent submission
to minimize metadata that could be used for correlation
* Handle journalist authentication in the Document Interface instead of relying entirely on Authenticated Tor Hidden Services.
* Document Interface supports two-factor authentication via Google Authenticator or Yubikey
  * These logins are hardened in a manner similar to that of the `google-authenticator` PAM module: tokens may only be used once, logins are rate limited, etc.
  * If you are using TOTP, the window is expanded from 1 period to 3 in order to help the situation where the server and client's clocks are skewed
* Add Admin Interface so privileged "admin" users may add, edit, or delete other users on the Document Interface
* Requests are automatically encrypted with an ephemeral key as they are buffered onto disk to mitigate forensic attacks
* The haveged "high water" mark has been raised to maintain a higher average level of entropy on the system and minimize the appearance of the "flag for reply" flow
* Secure removel (via `srm`) of data has been moved to an async worker to prevent hanging the interface when deleting large files or collections
* New dedicated section of Source Interface for replies, instead of using flashed messages
* Change default codename length from 8 words to 7 words, maintains a sufficient security level while hopefully improving usability for sources
* Add recommendations for storing and memorizing the codename to the codename generation page
* Improve the quality of journalist designations generated by reducing the adjectives and nouns lists to a smaller subset of common words
* Use ntpd to continuously update the server time (especially important when using TOTP for two-factor authentication)
* Move Document Interface to port 80 so we don't have to keep remembering to type ":8080"
* We no longer ASCII-armor submissions when they are encrypted. This was unnecessary and bloated the size of the submissions, which is important to avoid because downloading large submissions over Tor is very slow.
* Flask now uses X-Send-File for downloads, which fixed some reported issues are large downloads not finishing or being corrupted.

### Environment

* Add egress host firewall rules
* Add google-authenticator apache module and basic auth for access to
document interface
* Encrypt bodies of OSSEC email alerts (add postfix+procmail to monitor
server)
* Create apparmor profiles for chrooted interface Tor process
* Update interface apparmor profiles for changes to application code
* Change installation method to use Ansible playbook and deb packages
* Split securedrop repo into 3 separate repos for securedrop-specific code (the application, Python dependencies, and custom configuration) and the upstream packages that we maintain (OSSEC and the hardened grsecurity kernel for Ubuntu)
* Add variety of development and testing environments for developers and researchers to use with Vagrant
* Reduce OSSEC email alert noise through whitelisting errors that are reported by the default configuration but that we have investigated and determined to be safe to ignore
* Document a thoroughly tested network firewall configuration with pfSense
* Reboot the machine automatically every 24 hours to reduce the potential for plaintext to remain in memory
* Add KeePassX password database template and document its use for journalists and admins
* Add secure backup and recovery scripts
* Add migration scripts
* Major improvements to the installation and user documentation, including lots of detail, testing, and the addition of TOC

## 0.2.1

### Web App
* Fix for flagging errors
* Validate journalist messages
* Add logging using standard Python library.
* Add delete collection
* Replace bcrypt with scrypt
* Clear referer on external links

### Environment
* Set maximum request body size in CONFIG_OPTIONS
* Add security-related HTTP headers to Apache config
* Remove mysql database, replace w/ sqlite. Update sqlite apparmor profile.
* Add outbound iptables rule for source/document groups

## 0.2

* Various documentation improvements

### Web App
* Remove javascript dependency in source interface
* Add warning to source interface about using javascript (with Gritter)
{'fraction_non_alphanumeric': 0.040191642267766836, 'fraction_numerical': 0.004657971785999467, 'mean_word_length': 4.830100853374709, 'pattern_counts': {'":': 1, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1941276', 'n_tokens_mistral': 1818, 'n_tokens_neox': 1704, 'n_words': 1063}