<commit_before>sbt-assembly
============

*Deploy fat JARs. Restart processes.*

sbt-assembly is a sbt plugin originally ported from codahale's assembly-sbt, which I'm guessing was inspired by Maven's assembly plugin. The goal is simple: Create a fat JAR of your project with all of its dependencies.

Requirements
------------

* sbt
* The burning desire to have a simple deploy procedure.

Reporting Issues & Contributing
-------------------------------

Before you email me, please read [Issue Reporting Guideline](CONTRIBUTING.md) carefully. Twice. (Don't email me)

Setup
-----

### Using Published Plugin

For sbt 0.13.6+ add sbt-assembly as a dependency in `project/assembly.sbt`:

```scala
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.4")
```

For older sbt 0.13.x, see [sbt-assembly 0.11.2](https://github.com/sbt/sbt-assembly/tree/0.11.2).

For sbt 0.12, see [sbt-assembly 0.9.2](https://github.com/sbt/sbt-assembly/tree/0.9.2).

(You may need to check this project's tags to see what the most recent release is.)

Usage
-----

Since sbt-assembly is now an auto plugin that's triggered for all projects with `JvmPlugin`, it shouldn't require extra setup to include `assembly` task into your project.
See [migration guide](Migration.md) for details on how to upgrade from older sbt-assembly.

### Applying the plugin to multi-project build.sbt

For example, here's a multi-project `build.sbt`:

```scala
lazy val commonSettings = Seq(
  version := "0.1-SNAPSHOT",
  organization := "com.example",
  scalaVersion := "2.10.1",
  test in assembly := {}
)

lazy val app = (project in file("app")).
  settings(commonSettings: _*).
  settings(
    mainClass in assembly := Some("com.example.Main"),
    // more settings here ...
  )

lazy val utils = (project in file("utils")).
  settings(commonSettings: _*).
  settings(
    assemblyJarName in assembly := "utils.jar",
    // more settings here ...
  )
```

In the above example, both the `app` project and the `utils` project do not run tests during assembly. The `app` project sets a main class whereas the `utils` project sets the name of its jar file.

### assembly task

Now you'll have an awesome new `assembly` task which will compile your project,
run your tests, and then pack your class files and all your dependencies into a
single JAR file: `target/scala_X.X.X/projectname-assembly-X.X.X.jar`.

    > assembly

If you specify a `mainClass in assembly` in build.sbt (or just let it autodetect
one) then you'll end up with a fully executable JAR, ready to rock.

Here is the list of the keys you can rewire for `assembly` task. 

    assemblyJarName               test                          mainClass
    assemblyOutputPath            assemblyMergeStrategy         assemblyOption
    assemblyExcludedJars          assembledMappings

For example the name of the jar can be set as follows in build.sbt:

```scala
assemblyJarName in assembly := "something.jar"
```

To skip the test during assembly,

```scala
test in assembly := {}
```

To set an explicit main class,

```scala
mainClass in assembly := Some("com.example.Main")
```

Excluding an explicit main class from your assembly requires something a little bit different though

```
packageOptions in assembly ~= { pos =>
  pos.filterNot { po =>
    po.isInstanceOf[Package.MainClass]
  }
}
```

### Merge Strategy

If multiple files share the same relative path (e.g. a resource named
`application.conf` in multiple dependency JARs), the default strategy is to
verify that all candidates have the same contents and error out otherwise.
This behavior can be configured on a per-path basis using either one
of the following built-in strategies or writing a custom one:

* `MergeStrategy.deduplicate` is the default described above
* `MergeStrategy.first` picks the first of the matching files in classpath order
* `MergeStrategy.last` picks the last one
* `MergeStrategy.singleOrError` bails out with an error message on conflict
* `MergeStrategy.concat` simply concatenates all matching files and includes the result
* `MergeStrategy.filterDistinctLines` also concatenates, but leaves out duplicates along the way
* `MergeStrategy.rename` renames the files originating from jar files
* `MergeStrategy.discard` simply discards matching files

The mapping of path names to merge strategies is done via the setting
`assemblyMergeStrategy` which can be augmented as follows:

```scala
assemblyMergeStrategy in assembly := {
  case PathList("javax", "servlet", xs @ _*)         => MergeStrategy.first
  case PathList(ps @ _*) if ps.last endsWith ".html" => MergeStrategy.first
  case "application.conf"                            => MergeStrategy.concat
  case "unwanted.txt"                                => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}
```

**NOTE**:
- `assemblyMergeStrategy in assembly` expects a function. You can't do `assemblyMergeStrategy in assembly := MergeStrategy.first`!
- Some files must be discarded or renamed otherwise to avoid breaking the zip (due to duplicate file name) or the legal license. Delegate default handling to `(assemblyMergeStrategy in assembly)` as the above pattern matching example.

By the way, the first case pattern in the above using `PathList(...)` is how you can pick `javax/servlet/*` from the first jar. If the default `MergeStrategy.deduplicate` is not working for you, that likely means you have multiple versions of some library pulled by your dependency graph. The real solution is to fix that dependency graph. You can work around it by `MergeStrategy.first` but don't be surprised when you see `ClassNotFoundException`.

Here is the default:

```scala
  val defaultMergeStrategy: String => MergeStrategy = { 
    case x if Assembly.isConfigFile(x) =>
      MergeStrategy.concat
    case PathList(ps @ _*) if Assembly.isReadme(ps.last) || Assembly.isLicenseFile(ps.last) =>
      MergeStrategy.rename
    case PathList("META-INF", xs @ _*) =>
      (xs map {_.toLowerCase}) match {
        case ("manifest.mf" :: Nil) | ("index.list" :: Nil) | ("dependencies" :: Nil) =>
          MergeStrategy.discard
        case ps @ (x :: xs) if ps.last.endsWith(".sf") || ps.last.endsWith(".dsa") =>
          MergeStrategy.discard
        case "plexus" :: xs =>
          MergeStrategy.discard
        case "services" :: xs =>
          MergeStrategy.filterDistinctLines
        case ("spring.schemas" :: Nil) | ("spring.handlers" :: Nil) =>
          MergeStrategy.filterDistinctLines
        case _ => MergeStrategy.deduplicate
      }
    case _ => MergeStrategy.deduplicate
  }
```

Custom `MergeStrategy`s can find out where a particular file comes
from using the `sourceOfFileForMerge` method on `sbtassembly.AssemblyUtils`,
which takes the temporary directory and one of the files passed into the
strategy as parameters.

### Shading

sbt-assembly can shade classes from your projects or from the library dependencies.
Backed by [Jar Jar Links](https://code.google.com/archive/p/jarjar/wikis/CommandLineDocs.wiki), bytecode transformation (via ASM) is used to change references to the renamed classes.

```scala
    assemblyShadeRules in assembly := Seq(
      ShadeRule.rename("org.apache.commons.io.**" -> "shadeio.@1").inAll
    )<commit_msg>Add note about MergeStrategy providing plugins<commit_after>sbt-assembly
============

*Deploy fat JARs. Restart processes.*

sbt-assembly is a sbt plugin originally ported from codahale's assembly-sbt, which I'm guessing was inspired by Maven's assembly plugin. The goal is simple: Create a fat JAR of your project with all of its dependencies.

Requirements
------------

* sbt
* The burning desire to have a simple deploy procedure.

Reporting Issues & Contributing
-------------------------------

Before you email me, please read [Issue Reporting Guideline](CONTRIBUTING.md) carefully. Twice. (Don't email me)

Setup
-----

### Using Published Plugin

For sbt 0.13.6+ add sbt-assembly as a dependency in `project/assembly.sbt`:

```scala
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.4")
```

For older sbt 0.13.x, see [sbt-assembly 0.11.2](https://github.com/sbt/sbt-assembly/tree/0.11.2).

For sbt 0.12, see [sbt-assembly 0.9.2](https://github.com/sbt/sbt-assembly/tree/0.9.2).

(You may need to check this project's tags to see what the most recent release is.)

Usage
-----

Since sbt-assembly is now an auto plugin that's triggered for all projects with `JvmPlugin`, it shouldn't require extra setup to include `assembly` task into your project.
See [migration guide](Migration.md) for details on how to upgrade from older sbt-assembly.

### Applying the plugin to multi-project build.sbt

For example, here's a multi-project `build.sbt`:

```scala
lazy val commonSettings = Seq(
  version := "0.1-SNAPSHOT",
  organization := "com.example",
  scalaVersion := "2.10.1",
  test in assembly := {}
)

lazy val app = (project in file("app")).
  settings(commonSettings: _*).
  settings(
    mainClass in assembly := Some("com.example.Main"),
    // more settings here ...
  )

lazy val utils = (project in file("utils")).
  settings(commonSettings: _*).
  settings(
    assemblyJarName in assembly := "utils.jar",
    // more settings here ...
  )
```

In the above example, both the `app` project and the `utils` project do not run tests during assembly. The `app` project sets a main class whereas the `utils` project sets the name of its jar file.

### assembly task

Now you'll have an awesome new `assembly` task which will compile your project,
run your tests, and then pack your class files and all your dependencies into a
single JAR file: `target/scala_X.X.X/projectname-assembly-X.X.X.jar`.

    > assembly

If you specify a `mainClass in assembly` in build.sbt (or just let it autodetect
one) then you'll end up with a fully executable JAR, ready to rock.

Here is the list of the keys you can rewire for `assembly` task. 

    assemblyJarName               test                          mainClass
    assemblyOutputPath            assemblyMergeStrategy         assemblyOption
    assemblyExcludedJars          assembledMappings

For example the name of the jar can be set as follows in build.sbt:

```scala
assemblyJarName in assembly := "something.jar"
```

To skip the test during assembly,

```scala
test in assembly := {}
```

To set an explicit main class,

```scala
mainClass in assembly := Some("com.example.Main")
```

Excluding an explicit main class from your assembly requires something a little bit different though

```
packageOptions in assembly ~= { pos =>
  pos.filterNot { po =>
    po.isInstanceOf[Package.MainClass]
  }
}
```

### Merge Strategy

If multiple files share the same relative path (e.g. a resource named
`application.conf` in multiple dependency JARs), the default strategy is to
verify that all candidates have the same contents and error out otherwise.
This behavior can be configured on a per-path basis using either one
of the following built-in strategies or writing a custom one:

* `MergeStrategy.deduplicate` is the default described above
* `MergeStrategy.first` picks the first of the matching files in classpath order
* `MergeStrategy.last` picks the last one
* `MergeStrategy.singleOrError` bails out with an error message on conflict
* `MergeStrategy.concat` simply concatenates all matching files and includes the result
* `MergeStrategy.filterDistinctLines` also concatenates, but leaves out duplicates along the way
* `MergeStrategy.rename` renames the files originating from jar files
* `MergeStrategy.discard` simply discards matching files

The mapping of path names to merge strategies is done via the setting
`assemblyMergeStrategy` which can be augmented as follows:

```scala
assemblyMergeStrategy in assembly := {
  case PathList("javax", "servlet", xs @ _*)         => MergeStrategy.first
  case PathList(ps @ _*) if ps.last endsWith ".html" => MergeStrategy.first
  case "application.conf"                            => MergeStrategy.concat
  case "unwanted.txt"                                => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}
```

**NOTE**:
- `assemblyMergeStrategy in assembly` expects a function. You can't do `assemblyMergeStrategy in assembly := MergeStrategy.first`!
- Some files must be discarded or renamed otherwise to avoid breaking the zip (due to duplicate file name) or the legal license. Delegate default handling to `(assemblyMergeStrategy in assembly)` as the above pattern matching example.

By the way, the first case pattern in the above using `PathList(...)` is how you can pick `javax/servlet/*` from the first jar. If the default `MergeStrategy.deduplicate` is not working for you, that likely means you have multiple versions of some library pulled by your dependency graph. The real solution is to fix that dependency graph. You can work around it by `MergeStrategy.first` but don't be surprised when you see `ClassNotFoundException`.

Here is the default:

```scala
  val defaultMergeStrategy: String => MergeStrategy = { 
    case x if Assembly.isConfigFile(x) =>
      MergeStrategy.concat
    case PathList(ps @ _*) if Assembly.isReadme(ps.last) || Assembly.isLicenseFile(ps.last) =>
      MergeStrategy.rename
    case PathList("META-INF", xs @ _*) =>
      (xs map {_.toLowerCase}) match {
        case ("manifest.mf" :: Nil) | ("index.list" :: Nil) | ("dependencies" :: Nil) =>
          MergeStrategy.discard
        case ps @ (x :: xs) if ps.last.endsWith(".sf") || ps.last.endsWith(".dsa") =>
          MergeStrategy.discard
        case "plexus" :: xs =>
          MergeStrategy.discard
        case "services" :: xs =>
          MergeStrategy.filterDistinctLines
        case ("spring.schemas" :: Nil) | ("spring.handlers" :: Nil) =>
          MergeStrategy.filterDistinctLines
        case _ => MergeStrategy.deduplicate
      }
    case _ => MergeStrategy.deduplicate
  }
```

Custom `MergeStrategy`s can find out where a particular file comes
from using the `sourceOfFileForMerge` method on `sbtassembly.AssemblyUtils`,
which takes the temporary directory and one of the files passed into the
strategy as parameters.

#### Third Party Merge Strategy Plugins

Support for special-case merge strategies beyond the generic scope can be
provided by companion plugins, below is a non-exhaustive list:

* Log4j2 Plugin Caches (`Log4j2Plugins.dat`):
  <https://github.com/idio/sbt-assembly-log4j2>

### Shading

sbt-assembly can shade classes from your projects or from the library dependencies.
Backed by [Jar Jar Links](https://code.google.com/archive/p/jarjar/wikis/CommandLineDocs.wiki), bytecode transformation (via ASM) is used to change references to the renamed classes.

```scala
    assemblyShadeRules in assembly := Seq(
      ShadeRule.rename("org.apache.commons.io.**" -> "shadeio.@1").inAll
    )
{'fraction_non_alphanumeric': 0.10233898532269955, 'fraction_numerical': 0.0053615709402855035, 'mean_word_length': 3.5984591679506934, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 44, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1901548', 'n_tokens_mistral': 4282, 'n_tokens_neox': 4096, 'n_words': 1665}