<commit_before>---
layout: post
title: Making bits faster
description: "Bit operations are fast, but they can still be made faster"
modified: 2016-12-19
tags: [.NET, cookit, performance, similarity, bit operations]
image:
  feature: data/2016-12-19-Making-bits-faster/logo.jpg
---

This post was inspired by a [discussion on Reddit](https://www.reddit.com/r/programming/comments/5i2x5r/using_bit_masks_for_highperformance_calculations/) that followed my [previous post](http://indexoutofrange.com/Using-bit-operations-for-performance-optimizations/)

In this post, I will cover a suggestion by `BelowAverageITGuy` that cut down the total execution time by almost one hour.
<!--MORE-->

## Saga

Before I go further here are some link to the previous posts on the problem of calculating similarities and then optimizing it grew to few post. Here are all of them:

- [How I calculate similarities in cookit?](/How_I_calculate_similarities_in_cookit)
- [How to calculate 17 billion similarities](/How-to-calculate-17-billion-similarities)
- [Independent code in performance optimizations](/Independent-code-in-performance-optimizations)
- [Using bit masks for high performance calculatons](/Using-bit-operations-for-performance-optimizations)
- [Making bits faster](/Making-bits-faster/)

## Recap 

The last implementation of `GetNonZeroIndexes` looked like this:

```csharp
public IList<int> GetNonZeroIndexes()
{
    List<int> ret= new List<int>();
    for (var i = 0; i < _array.Length; i++)
    {<commit_msg>Link to discussion and to reddit account<commit_after>---
layout: post
title: Making bits faster
description: "Bit operations are fast, but they can still be made faster"
modified: 2016-12-19
tags: [.NET, cookit, performance, similarity, bit operations]
image:
  feature: data/2016-12-19-Making-bits-faster/logo.jpg
---

This post was inspired by a [discussion on Reddit](https://www.reddit.com/r/programming/comments/5i2x5r/using_bit_masks_for_highperformance_calculations/) that followed my [previous post](http://indexoutofrange.com/Using-bit-operations-for-performance-optimizations/)

In this post, I will cover a [suggestion](https://www.reddit.com/r/programming/comments/5i2x5r/using_bit_masks_for_highperformance_calculations/db5ujwc/) by [BelowAverageITGuy](https://www.reddit.com/user/BelowAverageITGuy) that cut down the total execution time by almost one hour.
<!--MORE-->

## Saga

Before I go further here are some link to the previous posts on the problem of calculating similarities and then optimizing it grew to few post. Here are all of them:

- [How I calculate similarities in cookit?](/How_I_calculate_similarities_in_cookit)
- [How to calculate 17 billion similarities](/How-to-calculate-17-billion-similarities)
- [Independent code in performance optimizations](/Independent-code-in-performance-optimizations)
- [Using bit masks for high performance calculatons](/Using-bit-operations-for-performance-optimizations)
- [Making bits faster](/Making-bits-faster/)

## Recap 

The last implementation of `GetNonZeroIndexes` looked like this:

```csharp
public IList<int> GetNonZeroIndexes()
{
    List<int> ret= new List<int>();
    for (var i = 0; i < _array.Length; i++)
    {
{'fraction_non_alphanumeric': 0.10754364089775562, 'fraction_numerical': 0.016209476309226933, 'mean_word_length': 5.00936329588015, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 11, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4809276', 'n_tokens_mistral': 1033, 'n_tokens_neox': 943, 'n_words': 306}