<commit_before><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<!--
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.    
-->
<html>
<head>
	<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1">
	<script type="text/javascript">var xookiConfig = {level: 0};</script>	
	<script type="text/javascript" src="xooki/xooki.js"></script>
</head>
<body>
	<textarea id="xooki-source">
Here are some recommendations and best practices we have gathered throughout our experience and consultancies with our customers.

<h1>Add module descriptors for all your modules</h1>
In Ivy world, module descriptors are ivy files, which are basically simple xml files describing both what the module produce as artifact and its dependencies.

It is a good practice to write or download module descriptors for all the modules involved in your development, even for your third party dependencies, and even if they don't provide themselves such module descriptors.

First it will seem like an extra work, and require time. But when you will have several modules using the same third party library, and than you will only need to add one line to your ivy file to get this library and all its own dependencies that you really need (if you have good module descriptors in your repository, especially with the use of module <a href="concept.html#configurations">configurations</a>). It will also be very helpful when you want to upgrade a dependency. One single change in your module ivy file and you will get the updated version with its updated (or not) dependencies.

Therefore we recommend to add ivy files for all the modules in your repository, you can even enforce this rule by setting the allownomd attribute to false on your <a href="configuration/resolvers.html">resolvers</a>. Hence you shouldn't need to use the dependency artifact inclusion/exclusion/specification feature of Ivy, which should only be used in very specific cases.

<h1>Use your own enterprise repository</h1>
This is usually not a valid recommendation for open source projects, but for the enterprise world we strongly suggest to avoid relying on a public repository like maven ibiblio or ivyrep. Why? Well, there are a couple of reasons:
<ul>
<li>control</li> The main problem with this kind of public repositories is that you don't have control over the repository. This means that if a module descriptor is broken you cannot easily fixed it. Sure you can use a chain between a shared repository and the public one and put your fixed module descriptor in the shared repository so that it hide the one on the public repository, but this makes repository browsing and maintenance cumbersome. 
Even more problematic is the possible updates of the repository. We know that versions published in such repositories should be stable and not be updated, but we also frequently see that a module descriptor is buggy, or an artifact corrupted. We even see sometimes a new version published with the same name as the preceding one because the previous one was simply badly packaged. This can occur even to the best, it occured to us with Ivy 1.2 :-) But then we decided to publish the new version with a different name, 1.2a. But if the repository manager allow such updates, this means that what worked before can break. It can thus break your build reproducibility.
<li>reliability</li> Ibiblio maven repository is not particularly well known for its reliability (we often experience major slow down or even complete break of the site), and ivyrep is only supported by a small company (yes we are only a small company!). So slow down and site hang occurs also. And if the repository you rely on is down, this can cause major slow down in your development or release process.
<li>accuracy</li> a public repository usually contains much more than what you actually need (except maybe ivyrep which certainly features much less than what you need :-)). Is it a problem? We think so. We think that in an enterprise environment the libraries you use should step through some kind of validation process before being used in every projects of your company. And what better way to do so? Setup an enterprise repository with only the libraries you actually want to use. This will not only ensure a better quality of your application dependencies, but help to have the same versions everywhere, and even help when declaring your module dependencies, if you use a tool like IvyDE, the code completion will only show relevant information about your repository, with only the libraries you actually want to see.
</ul>
Note that it's not because you use an enterprise repository that you have to build it entirely by hand. Ivy features an [[ant:install]] task which can be used to install modules from a repository to another one, so it can be used to selectively install modules from a public repository to your enterprise repository, where you will then be able to ensure control, reliability and accuracy.

<h1>Always use patterns with at least organisation and module</h1>
Ivy is very flexible and can accomodate a lot of existing repositories, using the concept of <a href="concept.html#pattern">patterns</a>. But if your repository doesn't exist yet, we strongly recommend to always use the organisation and the module name in your pattern, even for private repository where you put only your own modules (which all the same organisation). Why? Because Ivy listing feature rely on the token it can find in the pattern. If you have no organisation token in your pattern, Ivy won't be able to list the (only?) organisation in your repository. And this can be a problem for code completion in IvyDE, for example, but also for repository wide tasks like [[ant:install]] or [[ant:repreport]].

<h1>Public ivysettings.xml with public repositories</h1>
If you create a public repository, provide an url to corresponding <a href="configuration.html">ivysettings.xml</a>. It's pretty easy to do, and if someone want to leverage your repository, he will just have to call [[ant:configure]] with the url of your ivysettings.xml, or <a href="configuration/include.html">include</a> it in its own configuration file, which makes it really easy to combine several public repositories.

<h1>Dealing with integration versions</h1>
Very often especially when working in a team or with several modules, you will need to rely on intermediate, non finalized versions of your modules. These versions are what we call integration versions, because their main objective is to be integrated with other modules to make and test an application or a framework. 

If you follow the continuous integration paradigm across modules, these integration versions can be produced by a continuous integration server, very frequently.

So, how can you deal with these, possibly numerous, integration versions?

There are basically two ways to deal with them, both ways being supported by Ivy:
<ul>
<li>use a naming convention like a special suffix</li> the idea is pretty simple, each time you publish a new integration of your module you give the same name to the version (in maven world this is for example 1.0-SNAPSHOT). The dependency manager should then be aware that this version is special because it changes over time, so that it does not trust its local cache if it already has the version, but check the date of the version on the repository and see if it hass changed. In Ivy this is supported using the <a href="ivyfile/dependency.html">changing attribute</a> on a dependency or by configuring the <a href="configuration/resolvers.html">changing pattern</a> to use for all your modules.
<li>create automatically a new version for each</li> in this case you use either a build number or a timestamp to publish each new integration version with a new version name. Then you can use one of the numerous ways in Ivy to <a href="ivyfile/dependency.html">express a version constraint</a>. Usually selecting the very latest one (using 'latest.integration' as version constraint) is enough.
</ul>

So, which way is the best? As often, it depends on your context, and if one of the two was really bad it wouldn't be supported in Ivy :-)

But usually we recommend to use the second one, because using a new version each time you publish a new version better fits the version identity paradigm, and can make <b>all</b> your builds reproducible, even integration one. And this is interesting because it enables, with some work in your build system, to introduce a mechanism to promote an integration build to a more stable status, like a milestone or a release. 

Imagine you have a customer which comes on a monday morning and asks your latest version of your software, for testing or demonstration purpose. Obviously he needs it for the afternoon :-) Now if you have a continuous integration process and a good tracking of your changes and your artifacts, it may occur that you are actually able to fulfill his request without needing the use of a dolorean to give you some more time :-) But it may occur also that your latest version stable enough to be used for the purpose of the customer was actually built a few days ago, because the very latest just break a feature or introduce a new one you don't want to deliver. In this case, you can deliver this 'stable' integration build if you want, but be sure that a few days, or weeks, or even months later, the customer will ask for a bug fix on this demo only version. Why? Because it's a customer, and we all know how they are :-)

So, with a build promotion feature of any build in your repository, the solution would be pretty easy: when the customer ask for the version, you not only deliver the integration build, but you also promote it to a milestone status, for example. this promotion indicates that you should keep track of this version in a long period, to be able to come back to it and create a branch if needed.

Unfortunately Ivy does not by its own allow to have such reproducible builds out of the box, simply because Ivy is a dependency manager, not a build tool. But if you publish only versions with a distinct name and use Ivy features like versions constraint replacement during the publication or recursive delivery of modules, it can really help.<commit_msg>slight review of best practices: talk about security as argument for enterprise repository + some minor updates<commit_after><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<!--
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.    
-->
<html>
<head>
	<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1">
	<script type="text/javascript">var xookiConfig = {level: 0};</script>	
	<script type="text/javascript" src="xooki/xooki.js"></script>
</head>
<body>
	<textarea id="xooki-source">
Here are some recommendations and best practices we have gathered throughout our experience and consultancies with our customers.

<h1>Add module descriptors for all your modules</h1>
In Ivy world, module descriptors are ivy files, which are basically simple xml files describing both what the module produce as artifact and its dependencies.

It is a good practice to write or download module descriptors for all the modules involved in your development, even for your third party dependencies, and even if they don't provide themselves such module descriptors.

First it will seem like an extra work, and require time. But when you will have several modules using the same third party library, and than you will only need to add one line to your ivy file to get this library and all its own dependencies that you really need (if you have good module descriptors in your repository, especially with the use of module <a href="concept.html#configurations">configurations</a>). It will also be very helpful when you want to upgrade a dependency. One single change in your module ivy file and you will get the updated version with its updated (or not) dependencies.

Therefore we recommend to add ivy files for all the modules in your repository, you can even enforce this rule by setting the allownomd attribute to false on your <a href="configuration/resolvers.html">resolvers</a>. Hence you shouldn't need to use the dependency artifact inclusion/exclusion/specification feature of Ivy, which should only be used in very specific cases.

<h1>Use your own enterprise repository</h1>
This is usually not a valid recommendation for open source projects, but for the enterprise world we strongly suggest to avoid relying on a public repository like maven ibiblio or ivyrep. Why? Well, there are a couple of reasons:
<ul>
<li>control</li> The main problem with this kind of public repositories is that you don't have control over the repository. This means that if a module descriptor is broken you cannot easily fixed it. Sure you can use a chain between a shared repository and the public one and put your fixed module descriptor in the shared repository so that it hide the one on the public repository, but this makes repository browsing and maintenance cumbersome. 
Even more problematic is the possible updates of the repository. We know that versions published in such repositories should be stable and not be updated, but we also frequently see that a module descriptor is buggy, or an artifact corrupted. We even see sometimes a new version published with the same name as the preceding one because the previous one was simply badly packaged. This can occur even to the best, it occurred to us with Ivy 1.2 :-) But then we decided to publish the new version with a different name, 1.2a. But if the repository manager allow such updates, this means that what worked before can break. It can thus break your build reproducibility.
<li>reliability</li> Maven repository is not particularly well known for its reliability (we often experience major slow down or even complete break of the site), and ivyrep is only supported by a small company (yes we are only a small company!). So slow down and site hang occurs also. And if the repository you rely on is down, this can cause major slow down in your development or release process.
<li>accuracy</li> a public repository usually contains much more than what you actually need. Is it a problem? We think so. We think that in an enterprise environment the libraries you use should step through some kind of validation process before being used in every projects of your company. And what better way to do so? Setup an enterprise repository with only the libraries you actually want to use. This will not only ensure a better quality of your application dependencies, but help to have the same versions everywhere, and even help when declaring your module dependencies, if you use a tool like IvyDE, the code completion will only show relevant information about your repository, with only the libraries you actually want to see.
<li>security</li> the artifacts you download from a module repository are often executable, and are thus involved in security concerns. Imagine a hacker replacing commons-lang by another version containing a virus? If you rely on a public repository to build your software, you expose it to a security risk. You can read more about that in this <a href="http://www.fortifysoftware.com/servlet/downloads/public/fortify_attacking_the_build.pdf">Forrester article</a>.
</ul>
Note that it's not because you use an enterprise repository that you have to build it entirely by hand. Ivy features an [[ant:install]] task which can be used to install modules from a repository to another one, so it can be used to selectively install modules from a public repository to your enterprise repository, where you will then be able to ensure control, reliability and accuracy.

<h1>Always use patterns with at least organisation and module</h1>
Ivy is very flexible and can accomodate a lot of existing repositories, using the concept of <a href="concept.html#pattern">patterns</a>. But if your repository doesn't exist yet, we strongly recommend to always use the organisation and the module name in your pattern, even for private repository where you put only your own modules (which all the same organisation). Why? Because Ivy listing feature rely on the token it can find in the pattern. If you have no organisation token in your pattern, Ivy won't be able to list the (only?) organisation in your repository. And this can be a problem for code completion in IvyDE, for example, but also for repository wide tasks like [[ant:install]] or [[ant:repreport]].

<h1>Public ivysettings.xml with public repositories</h1>
If you create a public repository, provide an url to corresponding <a href="configuration.html">ivysettings.xml</a>. It's pretty easy to do, and if someone want to leverage your repository, he will just have to load it with [[ant:settings]] with the url of your ivysettings.xml, or <a href="configuration/include.html">include</a> it in its own configuration file, which makes it really easy to combine several public repositories.

<h1>Dealing with integration versions</h1>
Very often especially when working in a team or with several modules, you will need to rely on intermediate, non finalized versions of your modules. These versions are what we call integration versions, because their main objective is to be integrated with other modules to make and test an application or a framework. 

If you follow the continuous integration paradigm across modules, these integration versions can be produced by a continuous integration server, very frequently.

So, how can you deal with these, possibly numerous, integration versions?

There are basically two ways to deal with them, both ways being supported by Ivy:
<ul>
<li>use a naming convention like a special suffix</li> the idea is pretty simple, each time you publish a new integration of your module you give the same name to the version (in maven world this is for example 1.0-SNAPSHOT). The dependency manager should then be aware that this version is special because it changes over time, so that it does not trust its local cache if it already has the version, but check the date of the version on the repository and see if it hass changed. In Ivy this is supported using the <a href="ivyfile/dependency.html">changing attribute</a> on a dependency or by configuring the <a href="configuration/resolvers.html">changing pattern</a> to use for all your modules.
<li>create automatically a new version for each</li> in this case you use either a build number or a timestamp to publish each new integration version with a new version name. Then you can use one of the numerous ways in Ivy to <a href="ivyfile/dependency.html">express a version constraint</a>. Usually selecting the very latest one (using 'latest.integration' as version constraint) is enough.
</ul>

So, which way is the best? As often, it depends on your context, and if one of the two was really bad it wouldn't be supported in Ivy :-)

But usually we recommend to use the second one, because using a new version each time you publish a new version better fits the version identity paradigm, and can make <b>all</b> your builds reproducible, even integration one. And this is interesting because it enables, with some work in your build system, to introduce a mechanism to promote an integration build to a more stable status, like a milestone or a release. 

Imagine you have a customer which comes on a monday morning and asks your latest version of your software, for testing or demonstration purpose. Obviously he needs it for the afternoon :-) Now if you have a continuous integration process and a good tracking of your changes and your artifacts, it may occur that you are actually able to fulfill his request without needing the use of a dolorean to give you some more time :-) But it may occur also that your latest version stable enough to be used for the purpose of the customer was actually built a few days ago, because the very latest just break a feature or introduce a new one you don't want to deliver. In this case, you can deliver this 'stable' integration build if you want, but be sure that a few days, or weeks, or even months later, the customer will ask for a bug fix on this demo only version. Why? Because it's a customer, and we all know how they are :-)

So, with a build promotion feature of any build in your repository, the solution would be pretty easy: when the customer ask for the version, you not only deliver the integration build, but you also promote it to a milestone status, for example. this promotion indicates that you should keep track of this version in a long period, to be able to come back to it and create a branch if needed.

Unfortunately Ivy does not by its own allow to have such reproducible builds out of the box, simply because Ivy is a dependency manager, not a build tool. But if you publish only versions with a distinct name and use Ivy features like versions constraint replacement during the publication or recursive delivery of modules, it can really help.
{'fraction_non_alphanumeric': 0.04435876983071878, 'fraction_numerical': 0.0028361251440219798, 'mean_word_length': 4.569348469891412, 'pattern_counts': {'":': 0, '<': 115, '<?xml version=': 0, '>': 115, 'https://': 0, 'lorem ipsum': 0, 'www.': 5, 'xml': 8}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2604979', 'n_tokens_mistral': 5231, 'n_tokens_neox': 5002, 'n_words': 3564}