<commit_before>SimMetrics [![Build Status](https://travis-ci.org/mpkorstanje/simmetrics.svg)](https://travis-ci.org/mpkorstanje/simmetrics)
==========

SimMetrics is a java library of Similarity or Distance Metrics, e.g. Levenshtein distance, Cosine similarity, that provide float based similarity measures between String data. All metrics return consistent measures rather than unbounded similarity scores. 

## Usage ##

For a quick and easy use [StringMetrics](./simmetrics-core/src/main/java/org/simmetrics/StringMetrics.java) contains a collection of well known string metrics.

```

	String str1 = "This is a sentence. It is made of words";
	String str2 = "This sentence is similair. It has almost the same words";
	
	StringMetric metric = StringMetrics.cosineSimilarity();
	
	float result = metric.compare(str1, str2); //0.4472

```

The [StringMetricBuilder](./simmetrics-core/src/main/java/org/simmetrics/StringMetricBuilder.java) is a convenience tool to build string metrics. Any class implementing StringMetric, ListMetric, SetMetric can be used to build a string metric.

For a terse syntax use `import static org.simmetrics.StringMetricBuilder.with;`

```
	String str1 = "This is a sentence. It is made of words";
	String str2 = "This sentence is similair. It has almost the same words";

	StringMetric metric =
			with(new CosineSimilarity<String>())
			.simplify(new Case.Lower(Locale.ENGLISH))
			.simplify(new NonWordCharacter())
			.tokenize(new Whitespace())
			.build();

	float result = metric.compare(str1, str2); //0.5590
```

## Maven ##

SimMetrics is available from Maven Central. Include the dependency.

```
	<dependency>
		<groupId>com.github.mpkorstanje</groupId>
		<artifactId>simmetrics</artifactId>
		<version>3.0.0</version><commit_msg>Bump version in readme. Shorter readme.<commit_after>SimMetrics [![Build Status](https://travis-ci.org/mpkorstanje/simmetrics.svg)](https://travis-ci.org/mpkorstanje/simmetrics)
==========

SimMetrics is a java library of Similarity or Distance Metrics, e.g. Levenshtein distance, Cosine similarity, that provide float based similarity measures between String data. All metrics return consistent measures rather than unbounded similarity scores. 

## Usage ##

For a quick and easy use [StringMetrics](./simmetrics-core/src/main/java/org/simmetrics/StringMetrics.java) contains a collection of well known string metrics.

```

	String str1 = "This is a sentence. It is made of words";
	String str2 = "This sentence is similair. It has almost the same words";
	
	StringMetric metric = StringMetrics.cosineSimilarity();
	
	float result = metric.compare(str1, str2); //0.4472

```

The [StringMetricBuilder](./simmetrics-core/src/main/java/org/simmetrics/StringMetricBuilder.java) is a convenience tool to build string metrics. Any class implementing StringMetric, ListMetric, SetMetric can be used to build a string metric.

For a terse syntax use `import static org.simmetrics.StringMetricBuilder.with;`

```
	String str1 = "This is a sentence. It is made of words";
	String str2 = "This sentence is similair. It has almost the same words";

	StringMetric metric =
			with(new CosineSimilarity<String>())
			.simplify(new Case.Lower(Locale.ENGLISH))
			.simplify(new NonWordCharacter())
			.tokenize(new Whitespace())
			.build();

	float result = metric.compare(str1, str2); //0.5590
```

## Maven ##

SimMetrics is available from Maven Central. Include the dependency.

```
	<dependency>
		<groupId>com.github.mpkorstanje</groupId>
		<artifactId>simmetrics</artifactId>
		<version>3.0.1</version>
{'fraction_non_alphanumeric': 0.11266816143497758, 'fraction_numerical': 0.011771300448430493, 'mean_word_length': 4.756451612903226, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1606891', 'n_tokens_mistral': 1132, 'n_tokens_neox': 1037, 'n_words': 342}