<commit_before>ToDo:
6)
# TODO:  FIX: this test fails due to floating point precision error: estimated threshold is -19.0418 but '-19.0418'.to_f * 10000 = -190417.99999999997
# A workaround exists: we can use fractions, i.e. ('-19.0418'.to_r * 10000).to_f = -190418.0 but it obscures code and being used uncarefully can involve huge slowdown.
# I think, it'd be used only at input to workaround discretization issue
#
#  def test_process_large_pvalue_floating_point_error
#    pvalue, threshold, real_pvalue = nil, nil, nil
#    assert_nothing_raised {
#      pvalue, threshold, real_pvalue = Helpers.find_threshold_output('KLF4_f2.pwm -p 0.8').strip.split("\t")
#    }
#    assert_equal '0.8', pvalue
#    assert_equal Helpers.obtain_pvalue_by_threshold("KLF4_f2.pwm #{threshold}"), real_pvalue
#  end
7)thresholds and thresholds_weak should return a collection (Array or Hash) when block not given
  merge this two methods into one parametrized method
8)(TODO: for theoretically consistency, while making small inconsistences to old calculations)
  When we work with strong threshold, we round matrix up(in order to overrate threshold comparing to real thus taking underrated pvalue) and take upper bound of discrete-thresholds fork.
  When we are estimating lower bound of threshold (weak threshold) we take lower bound of fork of discrete thresholds. But we should ALSO (not done yet) take matrix discreted down! This'd allow us give exact answer on a question in which range real threshold should lay with given P-value, now we correctly estimate only lower bound of threshold(upper bound of P-value)
9) '[--pcm] - treat the input file as Position Count Matrix. PCM-to-PWM transformation to be done internally.' - insert this option to all tools helpstrings

Specs and tests:<commit_msg>done feature removed from todo<commit_after>ToDo:
6)
# TODO:  FIX: this test fails due to floating point precision error: estimated threshold is -19.0418 but '-19.0418'.to_f * 10000 = -190417.99999999997
# A workaround exists: we can use fractions, i.e. ('-19.0418'.to_r * 10000).to_f = -190418.0 but it obscures code and being used uncarefully can involve huge slowdown.
# I think, it'd be used only at input to workaround discretization issue
#
#  def test_process_large_pvalue_floating_point_error
#    pvalue, threshold, real_pvalue = nil, nil, nil
#    assert_nothing_raised {
#      pvalue, threshold, real_pvalue = Helpers.find_threshold_output('KLF4_f2.pwm -p 0.8').strip.split("\t")
#    }
#    assert_equal '0.8', pvalue
#    assert_equal Helpers.obtain_pvalue_by_threshold("KLF4_f2.pwm #{threshold}"), real_pvalue
#  end
7)thresholds and thresholds_weak should return a collection (Array or Hash) when block not given
  merge this two methods into one parametrized method
8)(TODO: for theoretically consistency, while making small inconsistences to old calculations)
  When we work with strong threshold, we round matrix up(in order to overrate threshold comparing to real thus taking underrated pvalue) and take upper bound of discrete-thresholds fork.
  When we are estimating lower bound of threshold (weak threshold) we take lower bound of fork of discrete thresholds. But we should ALSO (not done yet) take matrix discreted down! This'd allow us give exact answer on a question in which range real threshold should lay with given P-value, now we correctly estimate only lower bound of threshold(upper bound of P-value)

Specs and tests:
{'fraction_non_alphanumeric': 0.07055749128919861, 'fraction_numerical': 0.036875725900116146, 'mean_word_length': 4.684818481848184, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3531553', 'n_tokens_mistral': 1077, 'n_tokens_neox': 961, 'n_words': 434}