<commit_before>---
title: LogShark Plugins: ART (Vizqlserver Activity and Resource Tracing)
---

In this section:

* TOC
{:toc}


### What is it?
The ART workbook is built from Vizqlserver Activity and Resource Tracing (ART) logging.  The ART logging captures CPU and memory utilization for each vizqlserver activity as well as the sponsorship and depth of the activitties and additional contextual metadata about the activity.

**Note: Many of the views in the workbook make use of additional fields that were added to the ART logging in the 2018.3 release.  For Servers on versions prior to 2018.3 some functionality in the dashboards may be missing(Such as the ability to identify the different request Types**
 

### Glossary of Important Terms to Understand ART Workbook Data and Dashboards

-  **Inclusive Vs Exclusive Resource Usage**: ART Resource usage is tracked for each activity as inclusive and exclusive resource usage.  The **Inclusive** resource usage shows the memory/CPU utilization for the activity plus all sponsee or child activities that are called by the activity.  The **Exclusive** resource usage shows only the memory/CPU utilization for the specific activity and excludes the sponsee or child activities.
- **Sponsorship**: The request is given a Root Unique ID and then each activity that is called for the request will be the sponsee of the Sponsor activity that called the activity.  The sponsor-Sponsee relationship of activities can also be thought of as a parent child relationship.  Every activity which is not the root always can identify two other nodes in the tree: its direct parent (sponsor) and the root.
- **Depth**: The request will be at Depth 0 and each sponsee called by a sponsor activity will increase the depth by 1 so the Depth is used to show the depth of the chain of Sponsor-Sponsee activities.

### When to use it?

- Identify resource intensive requests in cases of high CPU/memory usage by the Vizqlserver process
- Get an idea of the total number of vizqlserver requests being made and what type of requests are accounting for the majority of the activity
- For resource intensive requests investigate which components and activities in the request account for the bulk of the resource usage
- For long running requests investigate which components and activities account for the bulk of the time spent
- Identify requests that error and see which component and activity had the error and what the error text was
- Comparing loads of the same dashboard to identify where additional time is spent on slower loads


### Activity Naming Conventions

```xml
component-name.activity-name[.activity-name ...]
```

- **component-name:** the name of the unit that logically groups together related activities. Everything up to the first dot is the component-name.
- **activity-name**: the name of the functional unit of work. Everything after the first dot is the activity-name.


## Memory Tracking
Activity memory tracking statistics include bytes allocated and deallocated, and the number of times allocations and deallocations occurred.

When an Activity begins, a MemoryScope object is pushed onto a stack of MemoryScopes in Thread-Local Storage (TLS). Memory allocations and deallocations on that thread will update values in the stack of MemoryScopes. When an Activity ends, its MemoryScope is popped out of TLS.

## Accuracy
- **Windows**: It is accurate to the extent that heap use occurs in a module linked to tabmemory; in other words, any untracked modules represent blind spots. 
- **Linux**: Linux should provide greater accuracy because the overrides are seen process wide. 


## CPU Thread Time Tracking

CPU Thread Time tracking provides separate values for user and kernel time in Linux while Windows only uses User Time.  Thread Time is reported in units of milliseconds, and includes a count of the number of threads involved. It represents the amount of time an Activity ran on a processor/core. Time units are normalized across different platforms/processors.

## Dashboards

### Memory Allocations

**Views:**
-  ***Allocations by View*** This sheet is a plot with Time of the request on the X axis and size of
inclusive allocations on the Y axis
   - Color: Colored by request description type(I.E. Bootstrap, Categorical filter, etc.)
   - Mark Size: Size of inclusive allocations for the request
-  ***Allocations Gantt*** This sheet is a depth flame chart with Timestamp on the Column and ART Depth and
Activity name on the rows
   - Mark Size: Gantt bars sized based on elapsed time
   - Mark Color: Gantt bars colored on exclusive allocation bytes for the activity 
   
   <img src="./assets/MemoryAllocations.png" alt="Sample Memory Allocations Screenshot">
   ![Sample Memory Allocations Screenshot](./assets/MemoryAllocations.png)

**Quick Filters:**
- Site Dropdown The Site dropdown filter can be used to show only requests from a specific site
-  Begin Timestamp range filter The begin timestamp range filter can be used to filter to requests that started in a specific range of time
-  Workbook+View Dropdown to filter to requests for a specific Workbook and View
-  Art Description Dropdown To filter to specific request types

**Selection Filters:**
-  Select any mark in the plot view to populate the Allocations flame chart with the requests activities 

**Notes:**
If you are investigating Vizqlserver high memory usage issues having the customer gather Perfmon data using the template on our 'Perfmon for customers' page will help narrow down to the specific time the memory spikes and allow you to narrow the timeframe you are looking at in the dashboard

**Use Cases:**
- Identify requests that are causing high vizqlserver memory usage
- For high memory usage requests the allocations gantt will be useful to identify which component and activity are allocating most of the memory


### CPU Time

**Views:**
- ***CPU by View*** This sheet is a plot with Time of the request on the X axis and Inclusive CPU time on the Y axis
   - Color: Colored by request description type(I.E. Bootstrap, Categorical filter, etc.)
   - Mark Size: Size based on Inclusive CPU time for request
- ***CPU Gantt*** This sheet is a depth flame chart with Timestamp on the Column and ART Depth and Activity name on the rows
   - Mark Size: Gantt bars sized based on elapsed time
   - Mark Color: Gantt bars colored on Exclusive CPU utilization percentage for activity
<commit_msg>Update logshark_art.md<commit_after>---
title: LogShark Plugins: ART (Vizqlserver Activity and Resource Tracing)
---

In this section:

* TOC
{:toc}


### What is it?
The ART workbook is built from Vizqlserver Activity and Resource Tracing (ART) logging.  The ART logging captures CPU and memory utilization for each vizqlserver activity as well as the sponsorship and depth of the activitties and additional contextual metadata about the activity.

**Note: Many of the views in the workbook make use of additional fields that were added to the ART logging in the 2018.3 release.  For Servers on versions prior to 2018.3 some functionality in the dashboards may be missing(Such as the ability to identify the different request Types**
 

### Glossary of Important Terms to Understand ART Workbook Data and Dashboards

-  **Inclusive Vs Exclusive Resource Usage**: ART Resource usage is tracked for each activity as inclusive and exclusive resource usage.  The **Inclusive** resource usage shows the memory/CPU utilization for the activity plus all sponsee or child activities that are called by the activity.  The **Exclusive** resource usage shows only the memory/CPU utilization for the specific activity and excludes the sponsee or child activities.
- **Sponsorship**: The request is given a Root Unique ID and then each activity that is called for the request will be the sponsee of the Sponsor activity that called the activity.  The sponsor-Sponsee relationship of activities can also be thought of as a parent child relationship.  Every activity which is not the root always can identify two other nodes in the tree: its direct parent (sponsor) and the root.
- **Depth**: The request will be at Depth 0 and each sponsee called by a sponsor activity will increase the depth by 1 so the Depth is used to show the depth of the chain of Sponsor-Sponsee activities.

### When to use it?

- Identify resource intensive requests in cases of high CPU/memory usage by the Vizqlserver process
- Get an idea of the total number of vizqlserver requests being made and what type of requests are accounting for the majority of the activity
- For resource intensive requests investigate which components and activities in the request account for the bulk of the resource usage
- For long running requests investigate which components and activities account for the bulk of the time spent
- Identify requests that error and see which component and activity had the error and what the error text was
- Comparing loads of the same dashboard to identify where additional time is spent on slower loads


### Activity Naming Conventions

```xml
component-name.activity-name[.activity-name ...]
```

- **component-name:** the name of the unit that logically groups together related activities. Everything up to the first dot is the component-name.
- **activity-name**: the name of the functional unit of work. Everything after the first dot is the activity-name.


## Memory Tracking
Activity memory tracking statistics include bytes allocated and deallocated, and the number of times allocations and deallocations occurred.

When an Activity begins, a MemoryScope object is pushed onto a stack of MemoryScopes in Thread-Local Storage (TLS). Memory allocations and deallocations on that thread will update values in the stack of MemoryScopes. When an Activity ends, its MemoryScope is popped out of TLS.

## Accuracy
- **Windows**: It is accurate to the extent that heap use occurs in a module linked to tabmemory; in other words, any untracked modules represent blind spots. 
- **Linux**: Linux should provide greater accuracy because the overrides are seen process wide. 


## CPU Thread Time Tracking

CPU Thread Time tracking provides separate values for user and kernel time in Linux while Windows only uses User Time.  Thread Time is reported in units of milliseconds, and includes a count of the number of threads involved. It represents the amount of time an Activity ran on a processor/core. Time units are normalized across different platforms/processors.

## Dashboards

### Memory Allocations

**Views:**
-  ***Allocations by View*** This sheet is a plot with Time of the request on the X axis and size of
inclusive allocations on the Y axis
   - Color: Colored by request description type(I.E. Bootstrap, Categorical filter, etc.)
   - Mark Size: Size of inclusive allocations for the request
-  ***Allocations Gantt*** This sheet is a depth flame chart with Timestamp on the Column and ART Depth and
Activity name on the rows
   - Mark Size: Gantt bars sized based on elapsed time
   - Mark Color: Gantt bars colored on exclusive allocation bytes for the activity 
   
   <img src="./assets/MemoryAllocations.png" alt="Sample Memory Allocations Screenshot">

   ![Sample Memory Allocations Screenshot](./assets/MemoryAllocations.png)

   ![Sample Memory Allocations Screenshot](/assets/MemoryAllocations.png)

**Quick Filters:**
- Site Dropdown The Site dropdown filter can be used to show only requests from a specific site
-  Begin Timestamp range filter The begin timestamp range filter can be used to filter to requests that started in a specific range of time
-  Workbook+View Dropdown to filter to requests for a specific Workbook and View
-  Art Description Dropdown To filter to specific request types

**Selection Filters:**
-  Select any mark in the plot view to populate the Allocations flame chart with the requests activities 

**Notes:**
If you are investigating Vizqlserver high memory usage issues having the customer gather Perfmon data using the template on our 'Perfmon for customers' page will help narrow down to the specific time the memory spikes and allow you to narrow the timeframe you are looking at in the dashboard

**Use Cases:**
- Identify requests that are causing high vizqlserver memory usage
- For high memory usage requests the allocations gantt will be useful to identify which component and activity are allocating most of the memory


### CPU Time

**Views:**
- ***CPU by View*** This sheet is a plot with Time of the request on the X axis and Inclusive CPU time on the Y axis
   - Color: Colored by request description type(I.E. Bootstrap, Categorical filter, etc.)
   - Mark Size: Size based on Inclusive CPU time for request
- ***CPU Gantt*** This sheet is a depth flame chart with Timestamp on the Column and ART Depth and Activity name on the rows
   - Mark Size: Gantt bars sized based on elapsed time
   - Mark Color: Gantt bars colored on Exclusive CPU utilization percentage for activity

{'fraction_non_alphanumeric': 0.04597612629957643, 'fraction_numerical': 0.0018482864844050829, 'mean_word_length': 4.377225672877847, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3483017', 'n_tokens_mistral': 3132, 'n_tokens_neox': 2967, 'n_words': 1949}