<commit_before># AVClass and AVClass2

AVClass and AVClass2 are Python tools to tag / label malware samples. 
You give them as input the AV labels for a large number of malware samples (e.g., VirusTotal JSON reports) 
and they output tags extracted from the AV labels of each sample. 
The original AVClass only outputs family names (i.e., family tags). 
By default, it outputs the most likely family for each sample (e.g., *zbot*, *virut*). 
It can also output a ranking of all alternative family names it found for each sample.
The newer AVClass2, in addition to family names, also outputs other tags capturing the malware class (e.g., *worm*, *ransomware*, *grayware*), behaviors (e.g., *spam*, *ddos*), and file properties (e.g., *packed*, *themida*, *bundle*, *nsis*). 

A quick example helps illustrating the differences. If you run AVClass2 on our example input file:

```shell
$./avclass2/avclass2_labeler.py -lb examples/malheurReference_lb.json -p
```

the output on stdout is:

```
aca2d12934935b070df8f50e06a20539 33 CLASS:grayware|10,CLASS:grayware:adware|9,FILE:os:windows|8,FAM:adrotator|8,CLASS:downloader|3,FAM:zlob|2
67d15459e1f85898851148511c86d88d 37 CLASS:dialer|23,FILE:os:windows|9,FAM:adultbrowser|8,CLASS:dialer:porndialer|7,CLASS:grayware|6,CLASS:grayware:tool|3,FAM:target|2
```
which means sample *aca2d12934935b070df8f50e06a20539* 
was flagged by 33 AV engines and 10 of them agree it is *grayware*, 9 that it is more specifically *adware*, 
8 mention that it runs on *windows*, another 8 that it is the *adrotator* family, 
3 that it is a *downloader*, and 2 that it belongs instead to the *zlob* family.
Sample *67d15459e1f85898851148511c86d88d* is flagged by 37 AV engines and 23 of them 
consider it a *dialer*, 8 that it belongs to the *adultbrowser* family, and so on.

If you instead run AVClass on the same input file:

```shell
$./avclass/avclass_labeler.py -lb examples/malheurReference_lb.json
```

the output looks like this:

```
aca2d12934935b070df8f50e06a20539 adrotator
67d15459e1f85898851148511c86d88d adultbrowser
``` 

which simply reports the most common family name for each sample.

In a nutshell, that is the main difference between both tools. 
Of course, there are more options for both tools, 
which you can read about in their corresponding README files. 


## Which tool should I use?

You should use AVClass2. It is the newer tool, extracts more information 
from the input AV labels, and has a compatibility mode to be used for 
family labeling in the same way that the original AVClass. 

AVClass is no longer updated and may be deprecated in the near future. 
For example, we add family aliases and generic terms from time to time 
for AVClass2, but we are not currently adding them for AVClass.

The main reason to keep the original AVClass around is because in research, 
reproducibility can be important. 
Other researchers may have their own reasons to compare against an older tool. 
However, as a researcher, if you compare against the original AVClass, 
you should also include AVClass2 in the comparison.

The only benefit of the original AVClass is that it is slightly 
faster than AVClass2 since it extracts less info. 
It is unlikely that the lower runtime makes a significant difference 
for most users, but worth mentioning it in case you process many 
millions of samples and do not require the extra info AVClass2 provides. 

## Evaluating and comparing with AVClass/AVClass2

Other researchers may want to independently evaluate AVClass/AVClass2 and
to compare it with their own approaches.
We encourage such evaluation, feedback on limitations, and proposals for
improvement.
However, we have observed a number of common errors in such evaluations that 
should be avoided. Below is the quick summary. 
A more detailed explanation is in the [evaluation page](EVALUATION.md)

1. AVClass2 has superseeded AVClass, so your evaluation should include AVClass2, not only the original AVClass. 
2. For malware labeling, please Use AVClass2 compatibility mode (-c command line option).
3. Tagging more samples is not an evaluation goal by itself, the tags need to be accurate. For example, it is known that allowing tags from a single AV engine or ignoring generic tags will enable tagging more samples, but it will introduce incorrect tags. 
4. You need ground truth to evaluate the accuracy/precision/recall of AVClass/AVClass2 tagging.
5. You should evaluate scalability (runtime and memory usage) as well, since that is a major design goal of AVClass/AVClass2
6. Note that AVClass2 and AVClass may not provide the same family tag for all samples when run on the same AV labels.
7. AVClass2/AVClass are not malware detection tools, please do not try to evaluate them for that scenario. 

## References

The design and evaluation of AVClass is detailed in our 
[RAID 2016 paper](https://software.imdea.org/~juanca/papers/avclass_raid16.pdf):

> <NAME>, <NAME>, <NAME>, and <NAME>. 
AVClass: A Tool for Massive Malware Labeling. 
In Proceedings of the International Symposium on Research in 
Attacks, Intrusions and Defenses,
September 2016.

The design and evaluation of AVClass2 is detailed in our
[ACSAC 2020 paper](https://arxiv.org/pdf/2006.10615.pdf):

> <NAME>, <NAME>. 
AVClass2: Massive Malware Tag Extraction from AV Labels. 
In proceedings of the Annual Computer Security Applications Conference, 
December 2020.

## Why are AVClass and AVClass2 useful?

Because a lot of times security researchers want to extract family and other 
information from AV labels, but this process is not as simple as it looks, 
especially if you need to do it for large numbers (e.g., millions) of samples. 
Some advantages of AVClass and AVClass2 are:<commit_msg>More typos fixed<commit_after># AVClass and AVClass2

AVClass and AVClass2 are Python tools to tag / label malware samples. 
You give them as input the AV labels for a large number of malware samples (e.g., VirusTotal JSON reports) 
and they output tags extracted from the AV labels of each sample. 
The original AVClass only outputs family names (i.e., family tags). 
By default, it outputs the most likely family for each sample (e.g., *zbot*, *virut*). 
It can also output a ranking of all alternative family names it found for each sample.
The newer AVClass2, in addition to family names, also outputs other tags capturing the malware class (e.g., *worm*, *ransomware*, *grayware*), behaviors (e.g., *spam*, *ddos*), and file properties (e.g., *packed*, *themida*, *bundle*, *nsis*). 

A quick example helps illustrating the differences. If you run AVClass2 on our example input file:

```shell
$./avclass2/avclass2_labeler.py -lb examples/malheurReference_lb.json -p
```

the output on stdout is:

```
aca2d12934935b070df8f50e06a20539 33 CLASS:grayware|10,CLASS:grayware:adware|9,FILE:os:windows|8,FAM:adrotator|8,CLASS:downloader|3,FAM:zlob|2
67d15459e1f85898851148511c86d88d 37 CLASS:dialer|23,FILE:os:windows|9,FAM:adultbrowser|8,CLASS:dialer:porndialer|7,CLASS:grayware|6,CLASS:grayware:tool|3,FAM:target|2
```
which means sample *aca2d12934935b070df8f50e06a20539* 
was flagged by 33 AV engines and 10 of them agree it is *grayware*, 9 that it is more specifically *adware*, 
8 mention that it runs on *windows*, another 8 that it is the *adrotator* family, 
3 that it is a *downloader*, and 2 that it belongs instead to the *zlob* family.
Sample *67d15459e1f85898851148511c86d88d* is flagged by 37 AV engines and 23 of them 
consider it a *dialer*, 8 that it belongs to the *adultbrowser* family, and so on.

If you instead run AVClass on the same input file:

```shell
$./avclass/avclass_labeler.py -lb examples/malheurReference_lb.json
```

the output looks like this:

```
aca2d12934935b070df8f50e06a20539 adrotator
67d15459e1f85898851148511c86d88d adultbrowser
``` 

which simply reports the most common family name for each sample.

In a nutshell, that is the main difference between both tools. 
Of course, there are more options for both tools, 
which you can read about in their corresponding README files. 


## Which tool should I use?

You should use AVClass2. It is the newer tool, extracts more information 
from the input AV labels, and has a compatibility mode to be used for 
family labeling in the same way that the original AVClass. 

AVClass is no longer updated and may be deprecated in the near future. 
For example, we add family aliases and generic terms from time to time 
for AVClass2, but we are not currently adding them for AVClass.

The main reason to keep the original AVClass around is because in research, 
reproducibility can be important. 
Other researchers may have their own reasons to compare against an older tool. 
However, as a researcher, if you compare against the original AVClass, 
you should also include AVClass2 in the comparison.

The only benefit of the original AVClass is that it is slightly 
faster than AVClass2 since it extracts less info. 
It is unlikely that the lower runtime makes a significant difference 
for most users, but worth mentioning it in case you process many 
millions of samples and do not require the extra info AVClass2 provides. 

## Evaluating and comparing with AVClass/AVClass2

Other researchers may want to independently evaluate AVClass/AVClass2 and
to compare it with their own approaches.
We encourage such evaluation, feedback on limitations, and proposals for
improvement.
However, we have observed a number of common errors in such evaluations that 
should be avoided. Below is the quick summary. 
A more detailed explanation is in the [evaluation page](EVALUATION.md)

1. AVClass2 has superseeded AVClass, so your evaluation should include AVClass2, not only the original AVClass. 
2. For malware labeling, please use AVClass2 compatibility mode (-c command line option).
3. Tagging more samples is not an evaluation goal by itself, the tags need to be accurate. For example, it is known that allowing tags from a single AV engine or ignoring generic tags will enable tagging more samples, but it will introduce incorrect tags. 
4. You need ground truth to evaluate the accuracy/precision/recall of AVClass/AVClass2.
5. You should also evaluate scalability (runtime and memory usage) since that is a major design goal of AVClass/AVClass2
6. Note that AVClass2 and AVClass may not provide the same family tag for all samples when run on the same AV labels.
7. AVClass2/AVClass are not malware detection tools, please do not try to evaluate them for that scenario. 

## References

The design and evaluation of AVClass is detailed in our 
[RAID 2016 paper](https://software.imdea.org/~juanca/papers/avclass_raid16.pdf):

> <NAME>, <NAME>, <NAME>, and <NAME>. 
AVClass: A Tool for Massive Malware Labeling. 
In Proceedings of the International Symposium on Research in 
Attacks, Intrusions and Defenses,
September 2016.

The design and evaluation of AVClass2 is detailed in our
[ACSAC 2020 paper](https://arxiv.org/pdf/2006.10615.pdf):

> <NAME>, <NAME>. 
AVClass2: Massive Malware Tag Extraction from AV Labels. 
In proceedings of the Annual Computer Security Applications Conference, 
December 2020.

## Why are AVClass and AVClass2 useful?

Because a lot of times security researchers want to extract family and other 
information from AV labels, but this process is not as simple as it looks, 
especially if you need to do it for large numbers (e.g., millions) of samples. 
Some advantages of AVClass and AVClass2 are:
{'fraction_non_alphanumeric': 0.05899833930600472, 'fraction_numerical': 0.041080325146403286, 'mean_word_length': 4.729594391587381, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 19, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3536284', 'n_tokens_mistral': 3546, 'n_tokens_neox': 3178, 'n_words': 1631}