<commit_before>[![KOReader](https://raw.githubusercontent.com/koreader/koreader.github.io/master/koreader-logo.png)](https://koreader.rocks)

[Download](https://github.com/koreader/koreader/releases) •
[Wiki](https://github.com/koreader/koreader/wiki) •
[Developer docs](http://koreader.rocks/doc/) •
[Forum](http://www.mobileread.com/forums/forumdisplay.php?f=276) •
[Gitter **Chat**][gitter-link]

[![Build Status][circleci-badge]][circleci-link]
[![Coverage Status][coverage-badge]][coverage-link]
[![AGPL Licence][licence-badge]](COPYING)


KOReader is a document viewer application, originally created for Kindle
e-ink readers. It currently runs on Kindle, Kobo, PocketBook, Ubuntu Touch
and Android devices. Developers can also run a KOReader emulator
for development purposes on desktop PCs with Linux, Windows and
Mac OSX.

Main features for users
-----------------------

* supports multi-format documents including:
  * paged fixed-layout formats: PDF, DjVu, CBT, and CBZ
  * reflowable e-book formats: ePub, fb2, mobi, doc, chm and plain text
  * scanned PDF/DjVu documents can also be reflowed with built-in K2pdfopt
* use StarDict dictionaries / Wikipedia to lookup words
* highlights can be exported to Evernote cloud account
* highly customizable reader view and typesetting
  * setting arbitrary page margins / line space
  * choosing external fonts and styles
  * built-in multi-lingual hyphenation dictionaries
* supports adding custom online OPDS catalogs
* calibre integration
  * search calibre metadata on your koreader device
  * send ebooks from calibre library to your koreader device wirelessly
  * browser calibre library and download ebooks via calibre OPDS server
* can share ebooks with other koreader devices wirelessly
* various optimizations for e-ink devices
  * paginated menus without animation
  * adjustable text contrast
* multi-lingual user interface
* online Over-The-Air software update

Highlights for developers
--------------------------

* frontend written in Lua scripting language
  * multi-platform support through a single code-base
  * you can help develop KOReader in any editor without compilation
  * high runtime efficiency through LuaJIT acceleration
  * light-weight self-contained widget toolkit with small memory footprint
  * extensible with plugin system
* interfaced backends for documents parsing and rendering
  * high quality document backend libraries like MuPDF, DjvuLibre and CREngine
  * interacting with frontend via LuaJIT FFI for best performence
* in active development
  * with contributions from developers around the world
  * continuous integration with CircleCI
  * with unit tests (busted), static code analysis (luacheck) and code coverage test (luacov/coveralls)
  * automated nightly builds available at http://build.koreader.rocks/download/nightly/
* free as in free speech
  * licensed under Affero GPL v3
  * all dependencies are free software

Check out the [KOReader wiki](https://github.com/koreader/koreader/wiki) to learn
more about this project.

Building Prerequisites
======================

These instructions for how to get and compile the source are intended for a Linux
OS. Windows users are suggested to develop in a [Linux VM][linux-vm] or use Wubi.

If you only want to work with Lua frontend stuff, you can grab the AppImage and
run it with `--appimage-extract`.

To get and compile the source you must have `patch`, `wget`, `unzip`, `git`,
`cmake` and `luarocks` installed, as well as a version of `autoconf`
greater than 2.64. You also need `nasm` and of course a compiler like `gcc`
or `clang`. If you want to cross-compile for other architectures, you need a proper
cross-compile toolchain. Your GCC should be at least version 4.8.

Users of Debian and Ubuntu can install the required packages using:
```
sudo apt-get install build-essential git patch wget unzip \
gettext autoconf automake cmake libtool nasm luarocks \
libssl-dev libffi-dev libsdl2-dev libc6-dev-i386 xutils-dev linux-libc-dev:i386 zlib1g:i386
```

If you are running Fedora, be sure to install the package `libstdc++-static`.

That's all you need to get the emulator up and running with `./kodev build` and `./kodev run`.

Cross compile toolchains are available for Ubuntu users through these commands:
```
# for Kindle
sudo apt-get install gcc-arm-linux-gnueabi g++-arm-linux-gnueabi
# for Kobo and Ubuntu touch
sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf
# for Win32
sudo apt-get install gcc-mingw-w64-i686 g++-mingw-w64-i686
```

The packages `pkg-config-arm-linux-gnueabihf` and `pkg-config-arm-linux-gnueabi` may
block you from building for Kobo or Kindle. Remove them if you get an ld error,
`/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/
ld: cannot find -lglib-2.0`

**NOTE:** In the specific case of Kindle & Kobo targets, while we make some effort to support these Linaro/Ubuntu TCs,
they do *not* exactly target the proper devices. While your build will go fine, this may lead to runtime failure.
As time goes by, and/or the more bleeding-edge your distro is, the greater the risk for mismatch gets.
Thankfully, we have a distribution-agnostic solution for you: [koxtoolchain](https://github.com/koreader/koxtoolchain)!
This will allow you to build the *exact* same TCs used to build the nightlies, thanks to the magic of [crosstool-ng](https://github.com/crosstool-ng/crosstool-ng).

On Mac OS X you may need to install the following tools using [Homebrew](https://brew.sh/):
```
brew install nasm binutils libtool autoconf automake cmake makedepend sdl2 lua@5.1 luarocks gettext pkg-config wget md5sha1sum
echo 'export PATH="/usr/local/opt/gettext/bin:$PATH"' >> "$HOME"/.bash_profile
```

If you run into a gettext error while building glib, try `brew link --force gettext` to override the built-in Mac OS BSD gettext with GNU GetText.

The KOReader Android build requires `ant`, `openjdk-8-jdk` and `p7zip-full`. A compatible version of the Android NDK and SDK will be downloaded automatically by `./kodev build android` if no NDK or SDK is provided in environment variables. For that purpose you can use `NDK=/ndk/location SDK=/sdk/location ./kodev build android`.

Users of Debian Jessie first need to configure the `backports` repository:
```
sudo echo "deb http://ftp.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/backports.list
sudo apt-get update
```
For both Ubuntu and Debian, install the packages:
```
sudo apt-get install ant openjdk-8-jdk
```
Users on Debian finally need to remove JRE version 7:
```
sudo apt-get remove openjdk-7-jre-headless
```

In order to build KOReader package for Ubuntu Touch, the `click` package management
tool is needed, Ubuntu users can install it with:
```
sudo apt-get install click
```

You might also need SDL library packages if you want to compile and run
KOReader on Linux PC. Fedora users can install `SDL` and `SDL-devel` package.
Ubuntu users probably need to install the `libsdl2-dev` package:

Getting the source
==================
<commit_msg>add a note about building in mojave<commit_after>[![KOReader](https://raw.githubusercontent.com/koreader/koreader.github.io/master/koreader-logo.png)](https://koreader.rocks)

[Download](https://github.com/koreader/koreader/releases) •
[Wiki](https://github.com/koreader/koreader/wiki) •
[Developer docs](http://koreader.rocks/doc/) •
[Forum](http://www.mobileread.com/forums/forumdisplay.php?f=276) •
[Gitter **Chat**][gitter-link]

[![Build Status][circleci-badge]][circleci-link]
[![Coverage Status][coverage-badge]][coverage-link]
[![AGPL Licence][licence-badge]](COPYING)


KOReader is a document viewer application, originally created for Kindle
e-ink readers. It currently runs on Kindle, Kobo, PocketBook, Ubuntu Touch
and Android devices. Developers can also run a KOReader emulator
for development purposes on desktop PCs with Linux, Windows and
Mac OSX.

Main features for users
-----------------------

* supports multi-format documents including:
  * paged fixed-layout formats: PDF, DjVu, CBT, and CBZ
  * reflowable e-book formats: ePub, fb2, mobi, doc, chm and plain text
  * scanned PDF/DjVu documents can also be reflowed with built-in K2pdfopt
* use StarDict dictionaries / Wikipedia to lookup words
* highlights can be exported to Evernote cloud account
* highly customizable reader view and typesetting
  * setting arbitrary page margins / line space
  * choosing external fonts and styles
  * built-in multi-lingual hyphenation dictionaries
* supports adding custom online OPDS catalogs
* calibre integration
  * search calibre metadata on your koreader device
  * send ebooks from calibre library to your koreader device wirelessly
  * browser calibre library and download ebooks via calibre OPDS server
* can share ebooks with other koreader devices wirelessly
* various optimizations for e-ink devices
  * paginated menus without animation
  * adjustable text contrast
* multi-lingual user interface
* online Over-The-Air software update

Highlights for developers
--------------------------

* frontend written in Lua scripting language
  * multi-platform support through a single code-base
  * you can help develop KOReader in any editor without compilation
  * high runtime efficiency through LuaJIT acceleration
  * light-weight self-contained widget toolkit with small memory footprint
  * extensible with plugin system
* interfaced backends for documents parsing and rendering
  * high quality document backend libraries like MuPDF, DjvuLibre and CREngine
  * interacting with frontend via LuaJIT FFI for best performence
* in active development
  * with contributions from developers around the world
  * continuous integration with CircleCI
  * with unit tests (busted), static code analysis (luacheck) and code coverage test (luacov/coveralls)
  * automated nightly builds available at http://build.koreader.rocks/download/nightly/
* free as in free speech
  * licensed under Affero GPL v3
  * all dependencies are free software

Check out the [KOReader wiki](https://github.com/koreader/koreader/wiki) to learn
more about this project.

Building Prerequisites
======================

These instructions for how to get and compile the source are intended for a Linux
OS. Windows users are suggested to develop in a [Linux VM][linux-vm] or use Wubi.

If you only want to work with Lua frontend stuff, you can grab the AppImage and
run it with `--appimage-extract`.

To get and compile the source you must have `patch`, `wget`, `unzip`, `git`,
`cmake` and `luarocks` installed, as well as a version of `autoconf`
greater than 2.64. You also need `nasm` and of course a compiler like `gcc`
or `clang`. If you want to cross-compile for other architectures, you need a proper
cross-compile toolchain. Your GCC should be at least version 4.8.

Users of Debian and Ubuntu can install the required packages using:
```
sudo apt-get install build-essential git patch wget unzip \
gettext autoconf automake cmake libtool nasm luarocks \
libssl-dev libffi-dev libsdl2-dev libc6-dev-i386 xutils-dev linux-libc-dev:i386 zlib1g:i386
```

If you are running Fedora, be sure to install the package `libstdc++-static`.

That's all you need to get the emulator up and running with `./kodev build` and `./kodev run`.

Cross compile toolchains are available for Ubuntu users through these commands:
```
# for Kindle
sudo apt-get install gcc-arm-linux-gnueabi g++-arm-linux-gnueabi
# for Kobo and Ubuntu touch
sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf
# for Win32
sudo apt-get install gcc-mingw-w64-i686 g++-mingw-w64-i686
```

The packages `pkg-config-arm-linux-gnueabihf` and `pkg-config-arm-linux-gnueabi` may
block you from building for Kobo or Kindle. Remove them if you get an ld error,
`/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/../../../../arm-linux-gnueabihf/bin/
ld: cannot find -lglib-2.0`

**NOTE:** In the specific case of Kindle & Kobo targets, while we make some effort to support these Linaro/Ubuntu TCs,
they do *not* exactly target the proper devices. While your build will go fine, this may lead to runtime failure.
As time goes by, and/or the more bleeding-edge your distro is, the greater the risk for mismatch gets.
Thankfully, we have a distribution-agnostic solution for you: [koxtoolchain](https://github.com/koreader/koxtoolchain)!
This will allow you to build the *exact* same TCs used to build the nightlies, thanks to the magic of [crosstool-ng](https://github.com/crosstool-ng/crosstool-ng).

On Mac OS X you may need to install the following tools using [Homebrew](https://brew.sh/):
```
brew install nasm binutils libtool autoconf automake cmake makedepend sdl2 lua@5.1 luarocks gettext pkg-config wget md5sha1sum
echo 'export PATH="/usr/local/opt/gettext/bin:$PATH"' >> "$HOME"/.bash_profile
```

If you run into a gettext error while building glib, try `brew link --force gettext` to override the built-in Mac OS BSD gettext with GNU GetText.

*Note:* in Mojave (10.14) you need to set a minimum deployment version higher than 10.04. Otherwise you'll get the error `ld: library not found for -lgcc_s.10.4`.
```
export MACOSX_DEPLOYMENT_TARGET=10.09
```

The KOReader Android build requires `ant`, `openjdk-8-jdk` and `p7zip-full`. A compatible version of the Android NDK and SDK will be downloaded automatically by `./kodev build android` if no NDK or SDK is provided in environment variables. For that purpose you can use `NDK=/ndk/location SDK=/sdk/location ./kodev build android`.

Users of Debian Jessie first need to configure the `backports` repository:
```
sudo echo "deb http://ftp.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/backports.list
sudo apt-get update
```
For both Ubuntu and Debian, install the packages:
```
sudo apt-get install ant openjdk-8-jdk
```
Users on Debian finally need to remove JRE version 7:
```
sudo apt-get remove openjdk-7-jre-headless
```

In order to build KOReader package for Ubuntu Touch, the `click` package management
tool is needed, Ubuntu users can install it with:
```
sudo apt-get install click
```

You might also need SDL library packages if you want to compile and run
KOReader on Linux PC. Fedora users can install `SDL` and `SDL-devel` package.
Ubuntu users probably need to install the `libsdl2-dev` package:

Getting the source
==================

{'fraction_non_alphanumeric': 0.09566912686255397, 'fraction_numerical': 0.008007241331290906, 'mean_word_length': 4.395567242674681, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 9, 'https://': 16, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5022708', 'n_tokens_mistral': 4530, 'n_tokens_neox': 4179, 'n_words': 1770}