<commit_before>---
layout: single
title:  "The canary judge&mdash;how does it work?"
sidebar:
  nav: guides
---

{% include toc %}

> Note: Automated canary analysis in Spinnaker is designed to support pluggable
> judges. This document describes how the default judge (NetflixACAJudge) works.

To assess the quality of a canary deployment against a baseline, metrics from
both deployments are compared in order to check for significant degradation.
This is done in two phases:

* Metric collection (retrieval)

  This phase retrieves the key metrics from the baseline and canary deployments.
  These metrics are typically stored in a time-series database, and include a
  set of tags or annotations that identify which deployment the data was
  collected from (canary or baseline).

  This phase is performed by Kayenta, not by the judge. Besides the default
  judge, it is possible to plug in a custom judge, and metric collection is not<commit_msg>Update index.md (#1978)<commit_after>---
layout: single
title:  "The canary judge&mdash;how does it work?"
sidebar:
  nav: guides
---

{% include toc %}

> __Note__: Automated canary analysis in Spinnaker is designed to support pluggable
> judges. This document describes how the default judge (NetflixACAJudge) works.

To assess the quality of a canary deployment against a baseline, metrics from
both deployments are compared in order to check for significant degradation.
This is done in two phases:

* Metric collection (retrieval)

  This phase retrieves the key metrics from the baseline and canary deployments.
  These metrics are typically stored in a time-series database, and include a
  set of tags or annotations that identify which deployment the data was
  collected from (canary or baseline).

  This phase is performed by Kayenta, not by the judge. Besides the default
  judge, it is possible to plug in a custom judge, and metric collection is not
{'fraction_non_alphanumeric': 0.04911180773249739, 'fraction_numerical': 0.0020898641588296763, 'mean_word_length': 4.349162011173185, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4718383', 'n_tokens_mistral': 499, 'n_tokens_neox': 477, 'n_words': 277}