<jupyter_start><jupyter_text>Neuromatch Academy: Week 2, Day 2, Tutorial 1 Modern Convnets__Content creators:__ <NAME>, <NAME>, <NAME>, <NAME> (based on an initial version by <NAME>)__Content reviewers:__ <NAME>, <NAME>, <NAME>. __Content editors:__ <NAME>, <NAME>__Production editors:__ <NAME>, <NAME>, <NAME>, <NAME>.   --- Tutorial Objectives:1. Learn about modern CNNs and Transfer Learning.2. Understand how architectures incorporate ideas we have about the world.3. Know what ResNets are and how they are built.4. Learn to recognize opportunities for transfer learning and domain adaptation.5. Understand the speed vs. accuracy trade-off.6. Application of modern CNN in facial recognition and ethical aspects of it.<jupyter_code>#@markdown Tutorial slides
# you should link the slides for all tutorial videos here (we will store pdfs on osf)

from IPython.display import HTML
HTML('<iframe src="https://docs.google.com/presentation/d/1R8KxHw5CHEjTuInXIMlZ9lx0nQBZ8lqW/" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>')<jupyter_output><empty_output><jupyter_text>--- Setup<jupyter_code>#@title Install facenet - a model used to do facial recognition
!pip -q install facenet-pytorch
# Import libraries
import copy
import glob
import random
import time
import os

import ipywidgets as widgets
import matplotlib.patches as mpatches
import matplotlib.pyplot as plt
import numpy as np
import sklearn.decomposition
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision
import tqdm
import urllib
from facenet_pytorch import MTCNN, InceptionResnetV1
from matplotlib.colors import ListedColormap
from IPython import display
from PIL import Image
from torchvision.datasets import ImageFolder
from torchvision.models import AlexNet
from torchvision.utils import make_grid
from torchvision import transforms

import requests
from io import BytesIO
#@title Setup GPU device
seed = 522
random.seed(522)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
torch.cuda.manual_seed(seed)
np.random.seed(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")<jupyter_output><empty_output><jupyter_text>--- Section 1:Modern CNNs and Transfer Learning<jupyter_code>#@title Video 1: Modern CNNs and Transfer Learning
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="SCsc0yHqfig", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>Images are high dimensional. That is to say that `image_length` * `image_width` * `image_channels` is a big number, and multiplying that big number by a normal sized fully-connected layer leads to a ton of parameters to learn. Last week we learned about convolutional neural networks, one way of working around high dimensionality in images and other domains. The widget below calculates the parameters required for a single convolutional or fully connected layer that operates on an image of a certain height and width. Adjust the sliders to gain an intuition for how different model and data characteristics affect the number of parameters your model need to fit.Note: these classes are designed to show parameter scaling in the first layer of a network, to be actually useful they would need more layers, an activation function, etc.<jupyter_code>class FullyConnectedNet(nn.Module):
  def __init__(self):
    super(FullyConnectedNet, self).__init__()

    image_width = 128
    image_channels = 3
    self.input_size = image_channels * image_width ** 2

    self.fc1 = nn.Linear(self.input_size, 256)

  def forward(self, x):
    x = x.view(-1, self.input_size)
    return self.fc1(x)
class ConvNet(nn.Module):
  def __init__(self):
    super(ConvNet, self).__init__()

    self.conv1 = nn.Conv2d(in_channels=3,
                            out_channels=256,
                            kernel_size=(3, 3),
                            padding=1)

  def forward(self, x):
    return self.conv1(x)<jupyter_output><empty_output><jupyter_text>Exercise 1.1 Calculate FCNN Parameters<jupyter_code>def get_fcnn_parameter_count() -> int:
  """
  Calculate the number of parameters used by the fully connected network.
  Hint: Casting the result of fc_net.parameters() to a list may make it
        easier to work with

  Returns:
      param_count: The number of parameters in the network
  """
  ####################################################################
  # Fill in all missing code below (...),
  # then remove or comment the line below to test your function
  raise NotImplementedError("Calculate the number of parameters in the fully connected network")
  ####################################################################

  fc_net = FullyConnectedNet()

  fc_net_parameters = ...
  param_count = ...
  for layer in fc_net_parameters:
    current_layer_params = None
    for dimension in layer.shape:
      if current_layer_params is None:
        current_layer_params = dimension
      else:
        current_layer_params *= dimension
    param_count += current_layer_params

  # Alternatively, there's a convenient torch function to count the number of items in a tensor:
  fc_net_parameters = fc_net.parameters()

  param_count = 0
  for layer in fc_net_parameters:
    param_count += torch.numel(layer)

  return param_count

### Uncomment below to test your function
#print(get_fcnn_parameter_count())
# to_remove solution
def get_fcnn_parameter_count() -> int:
  """
  Calculate the number of parameters used by the fully connected network.
  Hint: Casting the result of fc_net.parameters() to a list may make it
        easier to work with

  Returns:
      param_count: The number of parameters in the network
  """

  fc_net = FullyConnectedNet()

  fc_net_parameters = fc_net.parameters()
  param_count = 0
  for layer in fc_net_parameters:
    current_layer_params = None
    for dimension in layer.shape:
      if current_layer_params is None:
        current_layer_params = dimension
      else:
        current_layer_params *= dimension
    param_count += current_layer_params

  # Alternatively, there's a convenient torch function to count the number of items in a tensor:
  fc_net_parameters = fc_net.parameters()

  param_count = 0
  for layer in fc_net_parameters:
    param_count += torch.numel(layer)

  return param_count

### Uncomment below to test your function
print(get_fcnn_parameter_count())<jupyter_output><empty_output><jupyter_text>Exercise 1.2 Calculate CNN Parameters<jupyter_code>def get_cnn_parameter_count() -> int:
  """
  Calculate the number of parameters used by the convolutional network.
  Hint: Casting the result of cnn_net.parameters() to a list may make it
        easier to work with

  Returns:
      param_count: The number of parameters in the network
  """
  ####################################################################
  # Fill in all missing code below (...),
  # then remove or comment the line below to test your function
  raise NotImplementedError("Calculate the number of parameters in the fully connected network")
  ####################################################################

  convnet = ConvNet()
  convnet_parameters = ...

  param_count = ...
  for layer in convnet_parameters:
    param_count += torch.numel(layer)

  return param_count

### Uncomment below to test your function
#print(get_cnn_parameter_count())
# to_remove solution
def get_cnn_parameter_count() -> int:
  """
  Calculate the number of parameters used by the convolutional network.
  Hint: Casting the result of cnn_net.parameters() to a list may make it
        easier to work with

  Returns:
      param_count: The number of parameters in the network
  """

  convnet = ConvNet()
  convnet_parameters = list(convnet.parameters())

  param_count = 0
  for layer in convnet_parameters:
    param_count += torch.numel(layer)

  return param_count

### Uncomment below to test your function
print(get_cnn_parameter_count())<jupyter_output><empty_output><jupyter_text>Check your resultsThe widget below calculates the number of parameters in a FCNN and CNN with the same architecture as our models above. Our models had an input image that was 128x128, and used 256 filters (or 256 nodes in the FCNN case).Note how few parameters the convolutional networks take, especially as you increase the input image size.<jupyter_code># @title Parameter Calculator

def calculate_parameters(filter_count, image_width, fcnn_nodes):
    # Convnet math: Implement how parameters scale as a function of image size between convnets and FCNN

    filter_width = 3
    image_channels = 3

    # Assuming a square, RGB image
    image_area = image_width ** 2
    image_volume = image_area * image_channels

    # If we're using padding=same, the output of a convnet will be the same shape as the original
    # image, but with more features
    fcnn_parameters = image_volume * fcnn_nodes
    cnn_parameters = image_channels * filter_count * filter_width ** 2

    # Add bias
    fcnn_parameters += fcnn_nodes
    cnn_parameters += filter_count

    print('CNN parameters: {}'.format(cnn_parameters))
    print('Fully Connected parameters: {}'.format(fcnn_parameters))

    return None

_ = widgets.interact(calculate_parameters,
                     filter_count=(16, 512, 16),
                     image_width=(16, 512, 16),
                     fcnn_nodes=(16, 512, 16))<jupyter_output><empty_output><jupyter_text>---  Section 2: The History of Convnets Convolutional neural networks have been around for a long time. [The first CNN model](https://www.rctn.org/bruno/public/papers/Fukushima1980.pdf) was published in 1980, and was based on ideas in neuroscience that [predated it by decades](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/). Why is it then that [AlexNet](https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html), a CNN model published in 2012, is generally considered to mark the start of the deep learning revolution?Watch the video below to get a better idea of the role that hardware has played in progressing deep learning<jupyter_code>#@title Video 2: History of convnets
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="ToK4egKEAxE", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>Exercise 2.1 How to improve CNNs?Based on previous classes and our discussion of convnets so far, what additions might we make to improve CNNs?What causes them to have the parameter efficiency we saw in the previous section and lecture?Discuss this with your group for ~10 minutes, then record your answers in the table below:<jupyter_code>parameter_efficiency = '' #@param {type:"string"}

t1 = time.time()
# to_remove explanation

"""
Improvements on vanilla CNNs include skip connections to improve gradient flow, special
architectures like U-nets that are good for image segmentation, or maybe finding a way to get
rotational invariance.

The parameter efficiency of CNNs comes from sharing weights across the entire input instead of
having each weight correspond to a particular pixel.
"""<jupyter_output><empty_output><jupyter_text>--- Section 3: Big and Deep Convnets<jupyter_code>#@title Video 3: AlexNet & VGG
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="aIUySbimbkY", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>Introduction to AlexNetAlexNet arguably marked the start of the current age of deep learning.It incorporates a number of the defining characteristics of successful DL today: deep networks, GPU-powered paralellization, and building blocks encoding task-specific priors.In this section you'll have the opportunity to play with AlexNet and see the world through its eyes.<jupyter_code># @title Import Alexnet
# This cell gives you the `alexnet` model as well as the
# `input_image` and `input_batch` variables used below

alexnet = AlexNet()

state_dict = torch.hub.load_state_dict_from_url("https://s3.amazonaws.com/pytorch/models/alexnet-owt-4df8aa71.pth")
alexnet.load_state_dict(state_dict=state_dict)

url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
try: urllib.URLopener().retrieve(url, filename)
except: urllib.request.urlretrieve(url, filename)

input_image = Image.open(filename)
preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)
input_batch = input_tensor.unsqueeze(0) # create a mini-batch as expected by the model

# move the input and model to GPU for speed if available
if torch.cuda.is_available():
    input_batch = input_batch.to(device)
    alexnet.to(device)<jupyter_output><empty_output><jupyter_text>What does AlexNet learn?This code visualizes the top-layer filters learned by AlexNet.What do these filters remind you of?<jupyter_code>with torch.no_grad():
  params = list(alexnet.parameters())
  fig, axs = plt.subplots(8, 8)
  for filter_index in range(params[0].shape[0]):
    row_index = filter_index % 8
    col_index = filter_index // 8

    filter = params[0][filter_index,:,:,:]
    filter_image = filter.permute(1, 2, 0)
    scaled_image = (filter_image + 1) / 2
    scaled_image = scaled_image.round()
    grey_image = scaled_image.mean(dim=2)
    axs[row_index, col_index].imshow(scaled_image.cpu())
    axs[row_index, col_index].axis('off')
  plt.show()<jupyter_output><empty_output><jupyter_text>Exercise 3.1: Filter Similarity<jupyter_code>#@markdown What do these filters remind you of?
filter_response = '' #@param {type:"string"}

t2 = time.time()
# to_remove explanation

"""
Some of the filters look like edge detectors, as they are color insensitive and consist of
vertical or horizontal patterns.
"""<jupyter_output><empty_output><jupyter_text>What does AlexNet see?One way of visualizing CNNs is to look at the output of individual filters for a given image. Below is a widget that lets you examine the outputs of various filters used in AlexNet.<jupyter_code># @title Image Widget Code
def alexnet_intermediate_output(net, image):
    return F.relu(net.features[0](image))

def browse_images(input_batch):
    intermediate_output = alexnet_intermediate_output(alexnet, input_batch)
    n = intermediate_output.shape[1]

    def view_image(i):
        with torch.no_grad():
            channel = intermediate_output[0, i,:].squeeze()
            plt.figure(figsize=(6,6))
            plt.imshow(channel.cpu())
            plt.title('Filter {}'.format(i))
            plt.axis('off')

            plt.show()

    widgets.interact(view_image, i=(0,n-1))
plt.imshow(input_image)
plt.axis('off')
plt.show()
browse_images(input_batch)<jupyter_output><empty_output><jupyter_text>Exercise 3.2 Filter PurposeWhat do these filters appear to be doing? Note that different filters play different roles so there are several good answers.<jupyter_code>filter_roles = '' #@param {type:"string"}

t3 = time.time()
# to_remove explanation

"""
Based on the areas that are highlighted in the outupt, some filters seem to be detecting edges,
while others seem to react to the white color of the dog or the green of the background.
"""<jupyter_output><empty_output><jupyter_text>Further ReadingIf the question "what are neural network filters looking for" is at all interesting to you, or if you like geometric art, you'll enjoy [this post](https://distill.pub/2017/feature-visualization/) creating images that maximize output of various CNN neurons. There is also a good article showing what the space of images looks like as models train [here](https://distill.pub/2020/grand-tour/). --- Section 4: Convnets After AlexNet<jupyter_code>#@title Video 4: Going deeper: Inception and ResNets
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="Cnpgf2A4wVQ", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>In this section we'll be working with a state of the art CNN model called [ResNet](https://arxiv.org/abs/1512.03385). ResNet has two particularly interesting features. First, it uses skip connections to avoid the vanishing gradient problem. Second, each block (collection of layers) in a ResNet can be treated as learning a residual function.Mathematically, a neural network can be thought of as a series of operations that maps an input (like an image of a dog) to an output (like the label "dog"). In math-speak a mapping from an input to an output is called a function.In the functional interpretation of machine learning, there is a ground truth function in your brain somewhere that maps images of dogs to the label "dog", and neural networks are just elaborate ways of learning that function. If you were to subtract out the true function from the function learned by a network, you'd be left with the residual error or "residual function". ResNet tries to learn the original function, then the residual function, then the residual of the residual, and so on with creative use of skip layers.In this section we'll run several images through a pre-trained ResNet and see what happens.<jupyter_code># @title Download imagenette
!rm -rf imagenette*
!wget https://s3.amazonaws.com/fast-ai-imageclas/imagenette2-320.tgz
!tar -xf imagenette2-320.tgz
!rm -r imagenette2-320.tgz
#@title Set Up Textual ImageNet labels
dict_map={0: 'tench, Tinca tinca',
 1: 'goldfish, Carassius auratus',
 2: 'great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias',
 3: 'tiger shark, Galeocerdo cuvieri',
 4: 'hammerhead, hammerhead shark',
 5: 'electric ray, crampfish, numbfish, torpedo',
 6: 'stingray',
 7: 'cock',
 8: 'hen',
 9: 'ostrich, Struthio camelus',
 10: 'brambling, Fringilla montifringilla',
 11: 'goldfinch, Carduelis carduelis',
 12: 'house finch, linnet, Carpodacus mexicanus',
 13: 'junco, snowbird',
 14: 'indigo bunting, indigo finch, indigo bird, Passerina cyanea',
 15: 'robin, American robin, Turdus migratorius',
 16: 'bulbul',
 17: 'jay',
 18: 'magpie',
 19: 'chickadee',
 20: 'water ouzel, dipper',
 21: 'kite',
 22: 'bald eagle, American eagle, Haliaeetus leucocephalus',
 23: 'vulture',
 24: 'great grey owl, great gray owl, Strix nebulosa',
 25: 'European fire salamander, Salamandra salamandra',
 26: 'common newt, Triturus vulgaris',
 27: 'eft',
 28: 'spotted salamander, Ambystoma maculatum',
 29: 'axolotl, mud puppy, Ambystoma mexicanum',
 30: 'bullfrog, Rana catesbeiana',
 31: 'tree frog, tree-frog',
 32: 'tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui',
 33: 'loggerhead, loggerhead turtle, Caretta caretta',
 34: 'leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea',
 35: 'mud turtle',
 36: 'terrapin',
 37: 'box turtle, box tortoise',
 38: 'banded gecko',
 39: 'common iguana, iguana, Iguana iguana',
 40: 'American chameleon, anole, Anolis carolinensis',
 41: 'whiptail, whiptail lizard',
 42: 'agama',
 43: 'frilled lizard, Chlamydosaurus kingi',
 44: 'alligator lizard',
 45: 'Gila monster, Heloderma suspectum',
 46: 'green lizard, Lacerta viridis',
 47: 'African chameleon, Chamaeleo chamaeleon',
 48: 'Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis',
 49: 'African crocodile, Nile crocodile, Crocodylus niloticus',
 50: 'American alligator, Alligator mississipiensis',
 51: 'triceratops',
 52: 'thunder snake, worm snake, Carphophis amoenus',
 53: 'ringneck snake, ring-necked snake, ring snake',
 54: 'hognose snake, puff adder, sand viper',
 55: 'green snake, grass snake',
 56: 'king snake, kingsnake',
 57: 'garter snake, grass snake',
 58: 'water snake',
 59: 'vine snake',
 60: 'night snake, Hypsiglena torquata',
 61: 'boa constrictor, Constrictor constrictor',
 62: 'rock python, rock snake, Python sebae',
 63: 'Indian cobra, Naja naja',
 64: 'green mamba',
 65: 'sea snake',
 66: 'horned viper, cerastes, sand viper, horned asp, Cerastes cornutus',
 67: 'diamondback, diamondback rattlesnake, Crotalus adamanteus',
 68: 'sidewinder, horned rattlesnake, Crotalus cerastes',
 69: 'trilobite',
 70: 'harvestman, daddy longlegs, Phalangium opilio',
 71: 'scorpion',
 72: 'black and gold garden spider, Argiope aurantia',
 73: 'barn spider, Araneus cavaticus',
 74: 'garden spider, Aranea diademata',
 75: 'black widow, Latrodectus mactans',
 76: 'tarantula',
 77: 'wolf spider, hunting spider',
 78: 'tick',
 79: 'centipede',
 80: 'black grouse',
 81: 'ptarmigan',
 82: 'ruffed grouse, partridge, Bonasa umbellus',
 83: 'prairie chicken, prairie grouse, prairie fowl',
 84: 'peacock',
 85: 'quail',
 86: 'partridge',
 87: 'African grey, African gray, Psittacus erithacus',
 88: 'macaw',
 89: 'sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita',
 90: 'lorikeet',
 91: 'coucal',
 92: 'bee eater',
 93: 'hornbill',
 94: 'hummingbird',
 95: 'jacamar',
 96: 'toucan',
 97: 'drake',
 98: 'red-breasted merganser, Mergus serrator',
 99: 'goose',
 100: 'black swan, Cygnus atratus',
 101: 'tusker',
 102: 'echidna, spiny anteater, anteater',
 103: 'platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus',
 104: 'wallaby, brush kangaroo',
 105: 'koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus',
 106: 'wombat',
 107: 'jellyfish',
 108: 'sea anemone, anemone',
 109: 'brain coral',
 110: 'flatworm, platyhelminth',
 111: 'nematode, nematode worm, roundworm',
 112: 'conch',
 113: 'snail',
 114: 'slug',
 115: 'sea slug, nudibranch',
 116: 'chiton, coat-of-mail shell, sea cradle, polyplacophore',
 117: 'chambered nautilus, pearly nautilus, nautilus',
 118: 'Dungeness crab, Cancer magister',
 119: 'rock crab, Cancer irroratus',
 120: 'fiddler crab',
 121: 'king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica',
 122: 'American lobster, Northern lobster, Maine lobster, Homarus americanus',
 123: 'spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish',
 124: 'crayfish, crawfish, crawdad, crawdaddy',
 125: 'hermit crab',
 126: 'isopod',
 127: 'white stork, Ciconia ciconia',
 128: 'black stork, Ciconia nigra',
 129: 'spoonbill',
 130: 'flamingo',
 131: 'little blue heron, Egretta caerulea',
 132: 'American egret, great white heron, Egretta albus',
 133: 'bittern',
 134: 'crane',
 135: 'limpkin, Aramus pictus',
 136: 'European gallinule, Porphyrio porphyrio',
 137: 'American coot, marsh hen, mud hen, water hen, Fulica americana',
 138: 'bustard',
 139: 'ruddy turnstone, Arenaria interpres',
 140: 'red-backed sandpiper, dunlin, Erolia alpina',
 141: 'redshank, Tringa totanus',
 142: 'dowitcher',
 143: 'oystercatcher, oyster catcher',
 144: 'pelican',
 145: 'king penguin, Aptenodytes patagonica',
 146: 'albatross, mollymawk',
 147: 'grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus',
 148: 'killer whale, killer, orca, grampus, sea wolf, Orcinus orca',
 149: 'dugong, Dugong dugon',
 150: 'sea lion',
 151: 'Chihuahua',
 152: 'Japanese spaniel',
 153: 'Maltese dog, Maltese terrier, Maltese',
 154: 'Pekinese, Pekingese, Peke',
 155: 'Shih-Tzu',
 156: 'Blenheim spaniel',
 157: 'papillon',
 158: 'toy terrier',
 159: 'Rhodesian ridgeback',
 160: 'Afghan hound, Afghan',
 161: 'basset, basset hound',
 162: 'beagle',
 163: 'bloodhound, sleuthhound',
 164: 'bluetick',
 165: 'black-and-tan coonhound',
 166: 'Walker hound, Walker foxhound',
 167: 'English foxhound',
 168: 'redbone',
 169: 'borzoi, Russian wolfhound',
 170: 'Irish wolfhound',
 171: 'Italian greyhound',
 172: 'whippet',
 173: 'Ibizan hound, Ibizan Podenco',
 174: 'Norwegian elkhound, elkhound',
 175: 'otterhound, otter hound',
 176: 'Saluki, gazelle hound',
 177: 'Scottish deerhound, deerhound',
 178: 'Weimaraner',
 179: 'Staffordshire bullterrier, Staffordshire bull terrier',
 180: 'American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier',
 181: 'Bedlington terrier',
 182: 'Border terrier',
 183: 'Kerry blue terrier',
 184: 'Irish terrier',
 185: 'Norfolk terrier',
 186: 'Norwich terrier',
 187: 'Yorkshire terrier',
 188: 'wire-haired fox terrier',
 189: 'Lakeland terrier',
 190: 'Sealyham terrier, Sealyham',
 191: 'Airedale, Airedale terrier',
 192: 'cairn, cairn terrier',
 193: 'Australian terrier',
 194: 'Dandie Dinmont, Dandie Dinmont terrier',
 195: 'Boston bull, Boston terrier',
 196: 'miniature schnauzer',
 197: 'giant schnauzer',
 198: 'standard schnauzer',
 199: 'Scotch terrier, Scottish terrier, Scottie',
 200: 'Tibetan terrier, chrysanthemum dog',
 201: 'silky terrier, Sydney silky',
 202: 'soft-coated wheaten terrier',
 203: 'West Highland white terrier',
 204: 'Lhasa, Lhasa apso',
 205: 'flat-coated retriever',
 206: 'curly-coated retriever',
 207: 'golden retriever',
 208: 'Labrador retriever',
 209: 'Chesapeake Bay retriever',
 210: 'German short-haired pointer',
 211: 'vizsla, Hungarian pointer',
 212: 'English setter',
 213: 'Irish setter, red setter',
 214: 'Gordon setter',
 215: 'Brittany spaniel',
 216: 'clumber, clumber spaniel',
 217: 'English springer, English springer spaniel',
 218: 'Welsh springer spaniel',
 219: 'cocker spaniel, English cocker spaniel, cocker',
 220: 'Sussex spaniel',
 221: 'Irish water spaniel',
 222: 'kuvasz',
 223: 'schipperke',
 224: 'groenendael',
 225: 'malinois',
 226: 'briard',
 227: 'kelpie',
 228: 'komondor',
 229: 'Old English sheepdog, bobtail',
 230: 'Shetland sheepdog, Shetland sheep dog, Shetland',
 231: 'collie',
 232: 'Border collie',
 233: 'Bouvier des Flandres, Bouviers des Flandres',
 234: 'Rottweiler',
 235: 'German shepherd, German shepherd dog, German police dog, alsatian',
 236: 'Doberman, Doberman pinscher',
 237: 'miniature pinscher',
 238: 'Greater Swiss Mountain dog',
 239: 'Bernese mountain dog',
 240: 'Appenzeller',
 241: 'EntleBucher',
 242: 'boxer',
 243: 'bull mastiff',
 244: 'Tibetan mastiff',
 245: 'French bulldog',
 246: 'Great Dane',
 247: 'Saint Bernard, St Bernard',
 248: 'Eskimo dog, husky',
 249: 'malamute, malemute, Alaskan malamute',
 250: 'Siberian husky',
 251: 'dalmatian, coach dog, carriage dog',
 252: 'affenpinscher, monkey pinscher, monkey dog',
 253: 'basenji',
 254: 'pug, pug-dog',
 255: 'Leonberg',
 256: 'Newfoundland, Newfoundland dog',
 257: 'Great Pyrenees',
 258: 'Samoyed, Samoyede',
 259: 'Pomeranian',
 260: 'chow, chow chow',
 261: 'keeshond',
 262: 'Brabancon griffon',
 263: 'Pembroke, Pembroke Welsh corgi',
 264: 'Cardigan, Cardigan Welsh corgi',
 265: 'toy poodle',
 266: 'miniature poodle',
 267: 'standard poodle',
 268: 'Mexican hairless',
 269: 'timber wolf, grey wolf, gray wolf, Canis lupus',
 270: 'white wolf, Arctic wolf, Canis lupus tundrarum',
 271: 'red wolf, maned wolf, Canis rufus, Canis niger',
 272: 'coyote, prairie wolf, brush wolf, Canis latrans',
 273: 'dingo, warrigal, warragal, Canis dingo',
 274: 'dhole, Cuon alpinus',
 275: 'African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus',
 276: 'hyena, hyaena',
 277: 'red fox, Vulpes vulpes',
 278: 'kit fox, Vulpes macrotis',
 279: 'Arctic fox, white fox, Alopex lagopus',
 280: 'grey fox, gray fox, Urocyon cinereoargenteus',
 281: 'tabby, tabby cat',
 282: 'tiger cat',
 283: 'Persian cat',
 284: 'Siamese cat, Siamese',
 285: 'Egyptian cat',
 286: 'cougar, puma, catamount, mountain lion, painter, panther, Felis concolor',
 287: 'lynx, catamount',
 288: 'leopard, Panthera pardus',
 289: 'snow leopard, ounce, Panthera uncia',
 290: 'jaguar, panther, Panthera onca, Felis onca',
 291: 'lion, king of beasts, Panthera leo',
 292: 'tiger, Panthera tigris',
 293: 'cheetah, chetah, Acinonyx jubatus',
 294: 'brown bear, bruin, Ursus arctos',
 295: 'American black bear, black bear, Ursus americanus, Euarctos americanus',
 296: 'ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus',
 297: 'sloth bear, Melursus ursinus, Ursus ursinus',
 298: 'mongoose',
 299: 'meerkat, mierkat',
 300: 'tiger beetle',
 301: 'ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle',
 302: 'ground beetle, carabid beetle',
 303: 'long-horned beetle, longicorn, longicorn beetle',
 304: 'leaf beetle, chrysomelid',
 305: 'dung beetle',
 306: 'rhinoceros beetle',
 307: 'weevil',
 308: 'fly',
 309: 'bee',
 310: 'ant, emmet, pismire',
 311: 'grasshopper, hopper',
 312: 'cricket',
 313: 'walking stick, walkingstick, stick insect',
 314: 'cockroach, roach',
 315: 'mantis, mantid',
 316: 'cicada, cicala',
 317: 'leafhopper',
 318: 'lacewing, lacewing fly',
 319: "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
 320: 'damselfly',
 321: 'admiral',
 322: 'ringlet, ringlet butterfly',
 323: 'monarch, monarch butterfly, milkweed butterfly, Danaus plexippus',
 324: 'cabbage butterfly',
 325: 'sulphur butterfly, sulfur butterfly',
 326: 'lycaenid, lycaenid butterfly',
 327: 'starfish, sea star',
 328: 'sea urchin',
 329: 'sea cucumber, holothurian',
 330: 'wood rabbit, cottontail, cottontail rabbit',
 331: 'hare',
 332: 'Angora, Angora rabbit',
 333: 'hamster',
 334: 'porcupine, hedgehog',
 335: 'fox squirrel, eastern fox squirrel, Sciurus niger',
 336: 'marmot',
 337: 'beaver',
 338: 'guinea pig, Cavia cobaya',
 339: 'sorrel',
 340: 'zebra',
 341: 'hog, pig, grunter, squealer, Sus scrofa',
 342: 'wild boar, boar, Sus scrofa',
 343: 'warthog',
 344: 'hippopotamus, hippo, river horse, Hippopotamus amphibius',
 345: 'ox',
 346: 'water buffalo, water ox, Asiatic buffalo, Bubalus bubalis',
 347: 'bison',
 348: 'ram, tup',
 349: 'bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis',
 350: 'ibex, Capra ibex',
 351: 'hartebeest',
 352: 'impala, Aepyceros melampus',
 353: 'gazelle',
 354: 'Arabian camel, dromedary, Camelus dromedarius',
 355: 'llama',
 356: 'weasel',
 357: 'mink',
 358: 'polecat, fitch, foulmart, foumart, Mustela putorius',
 359: 'black-footed ferret, ferret, Mustela nigripes',
 360: 'otter',
 361: 'skunk, polecat, wood pussy',
 362: 'badger',
 363: 'armadillo',
 364: 'three-toed sloth, ai, Bradypus tridactylus',
 365: 'orangutan, orang, orangutang, Pongo pygmaeus',
 366: 'gorilla, Gorilla gorilla',
 367: 'chimpanzee, chimp, Pan troglodytes',
 368: 'gibbon, Hylobates lar',
 369: 'siamang, Hylobates syndactylus, Symphalangus syndactylus',
 370: 'guenon, guenon monkey',
 371: 'patas, hussar monkey, Erythrocebus patas',
 372: 'baboon',
 373: 'macaque',
 374: 'langur',
 375: 'colobus, colobus monkey',
 376: 'proboscis monkey, Nasalis larvatus',
 377: 'marmoset',
 378: 'capuchin, ringtail, Cebus capucinus',
 379: 'howler monkey, howler',
 380: 'titi, titi monkey',
 381: 'spider monkey, Ateles geoffroyi',
 382: 'squirrel monkey, Saimiri sciureus',
 383: 'Madagascar cat, ring-tailed lemur, Lemur catta',
 384: 'indri, indris, Indri indri, Indri brevicaudatus',
 385: 'Indian elephant, Elephas maximus',
 386: 'African elephant, Loxodonta africana',
 387: 'lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens',
 388: 'giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca',
 389: 'barracouta, snoek',
 390: 'eel',
 391: 'coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch',
 392: 'rock beauty, Holocanthus tricolor',
 393: 'anemone fish',
 394: 'sturgeon',
 395: 'gar, garfish, garpike, billfish, Lepisosteus osseus',
 396: 'lionfish',
 397: 'puffer, pufferfish, blowfish, globefish',
 398: 'abacus',
 399: 'abaya',
 400: "academic gown, academic robe, judge's robe",
 401: 'accordion, piano accordion, squeeze box',
 402: 'acoustic guitar',
 403: 'aircraft carrier, carrier, flattop, attack aircraft carrier',
 404: 'airliner',
 405: 'airship, dirigible',
 406: 'altar',
 407: 'ambulance',
 408: 'amphibian, amphibious vehicle',
 409: 'analog clock',
 410: 'apiary, bee house',
 411: 'apron',
 412: 'ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin',
 413: 'assault rifle, assault gun',
 414: 'backpack, back pack, knapsack, packsack, rucksack, haversack',
 415: 'bakery, bakeshop, bakehouse',
 416: 'balance beam, beam',
 417: 'balloon',
 418: 'ballpoint, ballpoint pen, ballpen, Biro',
 419: 'Band Aid',
 420: 'banjo',
 421: 'bannister, banister, balustrade, balusters, handrail',
 422: 'barbell',
 423: 'barber chair',
 424: 'barbershop',
 425: 'barn',
 426: 'barometer',
 427: 'barrel, cask',
 428: 'barrow, garden cart, lawn cart, wheelbarrow',
 429: 'baseball',
 430: 'basketball',
 431: 'bassinet',
 432: 'bassoon',
 433: 'bathing cap, swimming cap',
 434: 'bath towel',
 435: 'bathtub, bathing tub, bath, tub',
 436: 'beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon',
 437: 'beacon, lighthouse, beacon light, pharos',
 438: 'beaker',
 439: 'bearskin, busby, shako',
 440: 'beer bottle',
 441: 'beer glass',
 442: 'bell cote, bell cot',
 443: 'bib',
 444: 'bicycle-built-for-two, tandem bicycle, tandem',
 445: 'bikini, two-piece',
 446: 'binder, ring-binder',
 447: 'binoculars, field glasses, opera glasses',
 448: 'birdhouse',
 449: 'boathouse',
 450: 'bobsled, bobsleigh, bob',
 451: 'bolo tie, bolo, bola tie, bola',
 452: 'bonnet, poke bonnet',
 453: 'bookcase',
 454: 'bookshop, bookstore, bookstall',
 455: 'bottlecap',
 456: 'bow',
 457: 'bow tie, bow-tie, bowtie',
 458: 'brass, memorial tablet, plaque',
 459: 'brassiere, bra, bandeau',
 460: 'breakwater, groin, groyne, mole, bulwark, seawall, jetty',
 461: 'breastplate, aegis, egis',
 462: 'broom',
 463: 'bucket, pail',
 464: 'buckle',
 465: 'bulletproof vest',
 466: 'bullet train, bullet',
 467: 'butcher shop, meat market',
 468: 'cab, hack, taxi, taxicab',
 469: 'caldron, cauldron',
 470: 'candle, taper, wax light',
 471: 'cannon',
 472: 'canoe',
 473: 'can opener, tin opener',
 474: 'cardigan',
 475: 'car mirror',
 476: 'carousel, carrousel, merry-go-round, roundabout, whirligig',
 477: "carpenter's kit, tool kit",
 478: 'carton',
 479: 'car wheel',
 480: 'cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM',
 481: 'cassette',
 482: 'cassette player',
 483: 'castle',
 484: 'catamaran',
 485: 'CD player',
 486: 'cello, violoncello',
 487: 'cellular telephone, cellular phone, cellphone, cell, mobile phone',
 488: 'chain',
 489: 'chainlink fence',
 490: 'chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour',
 491: 'chain saw, chainsaw',
 492: 'chest',
 493: 'chiffonier, commode',
 494: 'chime, bell, gong',
 495: 'china cabinet, china closet',
 496: 'Christmas stocking',
 497: 'church, church building',
 498: 'cinema, movie theater, movie theatre, movie house, picture palace',
 499: 'cleaver, meat cleaver, chopper',
 500: 'cliff dwelling',
 501: 'cloak',
 502: 'clog, geta, patten, sabot',
 503: 'cocktail shaker',
 504: 'coffee mug',
 505: 'coffeepot',
 506: 'coil, spiral, volute, whorl, helix',
 507: 'combination lock',
 508: 'computer keyboard, keypad',
 509: 'confectionery, confectionary, candy store',
 510: 'container ship, containership, container vessel',
 511: 'convertible',
 512: 'corkscrew, bottle screw',
 513: 'cornet, horn, trumpet, trump',
 514: 'cowboy boot',
 515: 'cowboy hat, ten-gallon hat',
 516: 'cradle',
 517: 'crane',
 518: 'crash helmet',
 519: 'crate',
 520: 'crib, cot',
 521: 'Crock Pot',
 522: 'croquet ball',
 523: 'crutch',
 524: 'cuirass',
 525: 'dam, dike, dyke',
 526: 'desk',
 527: 'desktop computer',
 528: 'dial telephone, dial phone',
 529: 'diaper, nappy, napkin',
 530: 'digital clock',
 531: 'digital watch',
 532: 'dining table, board',
 533: 'dishrag, dishcloth',
 534: 'dishwasher, dish washer, dishwashing machine',
 535: 'disk brake, disc brake',
 536: 'dock, dockage, docking facility',
 537: 'dogsled, dog sled, dog sleigh',
 538: 'dome',
 539: 'doormat, welcome mat',
 540: 'drilling platform, offshore rig',
 541: 'drum, membranophone, tympan',
 542: 'drumstick',
 543: 'dumbbell',
 544: 'Dutch oven',
 545: 'electric fan, blower',
 546: 'electric guitar',
 547: 'electric locomotive',
 548: 'entertainment center',
 549: 'envelope',
 550: 'espresso maker',
 551: 'face powder',
 552: 'feather boa, boa',
 553: 'file, file cabinet, filing cabinet',
 554: 'fireboat',
 555: 'fire engine, fire truck',
 556: 'fire screen, fireguard',
 557: 'flagpole, flagstaff',
 558: 'flute, transverse flute',
 559: 'folding chair',
 560: 'football helmet',
 561: 'forklift',
 562: 'fountain',
 563: 'fountain pen',
 564: 'four-poster',
 565: 'freight car',
 566: 'French horn, horn',
 567: 'frying pan, frypan, skillet',
 568: 'fur coat',
 569: 'garbage truck, dustcart',
 570: 'gasmask, respirator, gas helmet',
 571: 'gas pump, gasoline pump, petrol pump, island dispenser',
 572: 'goblet',
 573: 'go-kart',
 574: 'golf ball',
 575: 'golfcart, golf cart',
 576: 'gondola',
 577: 'gong, tam-tam',
 578: 'gown',
 579: 'grand piano, grand',
 580: 'greenhouse, nursery, glasshouse',
 581: 'grille, radiator grille',
 582: 'grocery store, grocery, food market, market',
 583: 'guillotine',
 584: 'hair slide',
 585: 'hair spray',
 586: 'half track',
 587: 'hammer',
 588: 'hamper',
 589: 'hand blower, blow dryer, blow drier, hair dryer, hair drier',
 590: 'hand-held computer, hand-held microcomputer',
 591: 'handkerchief, hankie, hanky, hankey',
 592: 'hard disc, hard disk, fixed disk',
 593: 'harmonica, mouth organ, harp, mouth harp',
 594: 'harp',
 595: 'harvester, reaper',
 596: 'hatchet',
 597: 'holster',
 598: 'home theater, home theatre',
 599: 'honeycomb',
 600: 'hook, claw',
 601: 'hoopskirt, crinoline',
 602: 'horizontal bar, high bar',
 603: 'horse cart, horse-cart',
 604: 'hourglass',
 605: 'iPod',
 606: 'iron, smoothing iron',
 607: "jack-o'-lantern",
 608: 'jean, blue jean, denim',
 609: 'jeep, landrover',
 610: 'jersey, T-shirt, tee shirt',
 611: 'jigsaw puzzle',
 612: 'jinrikisha, ricksha, rickshaw',
 613: 'joystick',
 614: 'kimono',
 615: 'knee pad',
 616: 'knot',
 617: 'lab coat, laboratory coat',
 618: 'ladle',
 619: 'lampshade, lamp shade',
 620: 'laptop, laptop computer',
 621: 'lawn mower, mower',
 622: 'lens cap, lens cover',
 623: 'letter opener, paper knife, paperknife',
 624: 'library',
 625: 'lifeboat',
 626: 'lighter, light, igniter, ignitor',
 627: 'limousine, limo',
 628: 'liner, ocean liner',
 629: 'lipstick, lip rouge',
 630: 'Loafer',
 631: 'lotion',
 632: 'loudspeaker, speaker, speaker unit, loudspeaker system, speaker system',
 633: "loupe, jeweler's loupe",
 634: 'lumbermill, sawmill',
 635: 'magnetic compass',
 636: 'mailbag, postbag',
 637: 'mailbox, letter box',
 638: 'maillot',
 639: 'maillot, tank suit',
 640: 'manhole cover',
 641: 'maraca',
 642: 'marimba, xylophone',
 643: 'mask',
 644: 'matchstick',
 645: 'maypole',
 646: 'maze, labyrinth',
 647: 'measuring cup',
 648: 'medicine chest, medicine cabinet',
 649: 'megalith, megalithic structure',
 650: 'microphone, mike',
 651: 'microwave, microwave oven',
 652: 'military uniform',
 653: 'milk can',
 654: 'minibus',
 655: 'miniskirt, mini',
 656: 'minivan',
 657: 'missile',
 658: 'mitten',
 659: 'mixing bowl',
 660: 'mobile home, manufactured home',
 661: 'Model T',
 662: 'modem',
 663: 'monastery',
 664: 'monitor',
 665: 'moped',
 666: 'mortar',
 667: 'mortarboard',
 668: 'mosque',
 669: 'mosquito net',
 670: 'motor scooter, scooter',
 671: 'mountain bike, all-terrain bike, off-roader',
 672: 'mountain tent',
 673: 'mouse, computer mouse',
 674: 'mousetrap',
 675: 'moving van',
 676: 'muzzle',
 677: 'nail',
 678: 'neck brace',
 679: 'necklace',
 680: 'nipple',
 681: 'notebook, notebook computer',
 682: 'obelisk',
 683: 'oboe, hautboy, hautbois',
 684: 'ocarina, sweet potato',
 685: 'odometer, hodometer, mileometer, milometer',
 686: 'oil filter',
 687: 'organ, pipe organ',
 688: 'oscilloscope, scope, cathode-ray oscilloscope, CRO',
 689: 'overskirt',
 690: 'oxcart',
 691: 'oxygen mask',
 692: 'packet',
 693: 'paddle, boat paddle',
 694: 'paddlewheel, paddle wheel',
 695: 'padlock',
 696: 'paintbrush',
 697: "pajama, pyjama, pj's, jammies",
 698: 'palace',
 699: 'panpipe, pandean pipe, syrinx',
 700: 'paper towel',
 701: 'parachute, chute',
 702: 'parallel bars, bars',
 703: 'park bench',
 704: 'parking meter',
 705: 'passenger car, coach, carriage',
 706: 'patio, terrace',
 707: 'pay-phone, pay-station',
 708: 'pedestal, plinth, footstall',
 709: 'pencil box, pencil case',
 710: 'pencil sharpener',
 711: 'perfume, essence',
 712: 'Petri dish',
 713: 'photocopier',
 714: 'pick, plectrum, plectron',
 715: 'pickelhaube',
 716: 'picket fence, paling',
 717: 'pickup, pickup truck',
 718: 'pier',
 719: 'piggy bank, penny bank',
 720: 'pill bottle',
 721: 'pillow',
 722: 'ping-pong ball',
 723: 'pinwheel',
 724: 'pirate, pirate ship',
 725: 'pitcher, ewer',
 726: "plane, carpenter's plane, woodworking plane",
 727: 'planetarium',
 728: 'plastic bag',
 729: 'plate rack',
 730: 'plow, plough',
 731: "plunger, plumber's helper",
 732: 'Polaroid camera, Polaroid Land camera',
 733: 'pole',
 734: 'police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria',
 735: 'poncho',
 736: 'pool table, billiard table, snooker table',
 737: 'pop bottle, soda bottle',
 738: 'pot, flowerpot',
 739: "potter's wheel",
 740: 'power drill',
 741: 'prayer rug, prayer mat',
 742: 'printer',
 743: 'prison, prison house',
 744: 'projectile, missile',
 745: 'projector',
 746: 'puck, hockey puck',
 747: 'punching bag, punch bag, punching ball, punchball',
 748: 'purse',
 749: 'quill, quill pen',
 750: 'quilt, comforter, comfort, puff',
 751: 'racer, race car, racing car',
 752: 'racket, racquet',
 753: 'radiator',
 754: 'radio, wireless',
 755: 'radio telescope, radio reflector',
 756: 'rain barrel',
 757: 'recreational vehicle, RV, R.V.',
 758: 'reel',
 759: 'reflex camera',
 760: 'refrigerator, icebox',
 761: 'remote control, remote',
 762: 'restaurant, eating house, eating place, eatery',
 763: 'revolver, six-gun, six-shooter',
 764: 'rifle',
 765: 'rocking chair, rocker',
 766: 'rotisserie',
 767: 'rubber eraser, rubber, pencil eraser',
 768: 'rugby ball',
 769: 'rule, ruler',
 770: 'running shoe',
 771: 'safe',
 772: 'safety pin',
 773: 'saltshaker, salt shaker',
 774: 'sandal',
 775: 'sarong',
 776: 'sax, saxophone',
 777: 'scabbard',
 778: 'scale, weighing machine',
 779: 'school bus',
 780: 'schooner',
 781: 'scoreboard',
 782: 'screen, CRT screen',
 783: 'screw',
 784: 'screwdriver',
 785: 'seat belt, seatbelt',
 786: 'sewing machine',
 787: 'shield, buckler',
 788: 'shoe shop, shoe-shop, shoe store',
 789: 'shoji',
 790: 'shopping basket',
 791: 'shopping cart',
 792: 'shovel',
 793: 'shower cap',
 794: 'shower curtain',
 795: 'ski',
 796: 'ski mask',
 797: 'sleeping bag',
 798: 'slide rule, slipstick',
 799: 'sliding door',
 800: 'slot, one-armed bandit',
 801: 'snorkel',
 802: 'snowmobile',
 803: 'snowplow, snowplough',
 804: 'soap dispenser',
 805: 'soccer ball',
 806: 'sock',
 807: 'solar dish, solar collector, solar furnace',
 808: 'sombrero',
 809: 'soup bowl',
 810: 'space bar',
 811: 'space heater',
 812: 'space shuttle',
 813: 'spatula',
 814: 'speedboat',
 815: "spider web, spider's web",
 816: 'spindle',
 817: 'sports car, sport car',
 818: 'spotlight, spot',
 819: 'stage',
 820: 'steam locomotive',
 821: 'steel arch bridge',
 822: 'steel drum',
 823: 'stethoscope',
 824: 'stole',
 825: 'stone wall',
 826: 'stopwatch, stop watch',
 827: 'stove',
 828: 'strainer',
 829: 'streetcar, tram, tramcar, trolley, trolley car',
 830: 'stretcher',
 831: 'studio couch, day bed',
 832: 'stupa, tope',
 833: 'submarine, pigboat, sub, U-boat',
 834: 'suit, suit of clothes',
 835: 'sundial',
 836: 'sunglass',
 837: 'sunglasses, dark glasses, shades',
 838: 'sunscreen, sunblock, sun blocker',
 839: 'suspension bridge',
 840: 'swab, swob, mop',
 841: 'sweatshirt',
 842: 'swimming trunks, bathing trunks',
 843: 'swing',
 844: 'switch, electric switch, electrical switch',
 845: 'syringe',
 846: 'table lamp',
 847: 'tank, army tank, armored combat vehicle, armoured combat vehicle',
 848: 'tape player',
 849: 'teapot',
 850: 'teddy, teddy bear',
 851: 'television, television system',
 852: 'tennis ball',
 853: 'thatch, thatched roof',
 854: 'theater curtain, theatre curtain',
 855: 'thimble',
 856: 'thresher, thrasher, threshing machine',
 857: 'throne',
 858: 'tile roof',
 859: 'toaster',
 860: 'tobacco shop, tobacconist shop, tobacconist',
 861: 'toilet seat',
 862: 'torch',
 863: 'totem pole',
 864: 'tow truck, tow car, wrecker',
 865: 'toyshop',
 866: 'tractor',
 867: 'trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi',
 868: 'tray',
 869: 'trench coat',
 870: 'tricycle, trike, velocipede',
 871: 'trimaran',
 872: 'tripod',
 873: 'triumphal arch',
 874: 'trolleybus, trolley coach, trackless trolley',
 875: 'trombone',
 876: 'tub, vat',
 877: 'turnstile',
 878: 'typewriter keyboard',
 879: 'umbrella',
 880: 'unicycle, monocycle',
 881: 'upright, upright piano',
 882: 'vacuum, vacuum cleaner',
 883: 'vase',
 884: 'vault',
 885: 'velvet',
 886: 'vending machine',
 887: 'vestment',
 888: 'viaduct',
 889: 'violin, fiddle',
 890: 'volleyball',
 891: 'waffle iron',
 892: 'wall clock',
 893: 'wallet, billfold, notecase, pocketbook',
 894: 'wardrobe, closet, press',
 895: 'warplane, military plane',
 896: 'washbasin, handbasin, washbowl, lavabo, wash-hand basin',
 897: 'washer, automatic washer, washing machine',
 898: 'water bottle',
 899: 'water jug',
 900: 'water tower',
 901: 'whiskey jug',
 902: 'whistle',
 903: 'wig',
 904: 'window screen',
 905: 'window shade',
 906: 'Windsor tie',
 907: 'wine bottle',
 908: 'wing',
 909: 'wok',
 910: 'wooden spoon',
 911: 'wool, woolen, woollen',
 912: 'worm fence, snake fence, snake-rail fence, Virginia fence',
 913: 'wreck',
 914: 'yawl',
 915: 'yurt',
 916: 'web site, website, internet site, site',
 917: 'comic book',
 918: 'crossword puzzle, crossword',
 919: 'street sign',
 920: 'traffic light, traffic signal, stoplight',
 921: 'book jacket, dust cover, dust jacket, dust wrapper',
 922: 'menu',
 923: 'plate',
 924: 'guacamole',
 925: 'consomme',
 926: 'hot pot, hotpot',
 927: 'trifle',
 928: 'ice cream, icecream',
 929: 'ice lolly, lolly, lollipop, popsicle',
 930: 'French loaf',
 931: 'bagel, beigel',
 932: 'pretzel',
 933: 'cheeseburger',
 934: 'hotdog, hot dog, red hot',
 935: 'mashed potato',
 936: 'head cabbage',
 937: 'broccoli',
 938: 'cauliflower',
 939: 'zucchini, courgette',
 940: 'spaghetti squash',
 941: 'acorn squash',
 942: 'butternut squash',
 943: 'cucumber, cuke',
 944: 'artichoke, globe artichoke',
 945: 'bell pepper',
 946: 'cardoon',
 947: 'mushroom',
 948: '<NAME>',
 949: 'strawberry',
 950: 'orange',
 951: 'lemon',
 952: 'fig',
 953: 'pineapple, ananas',
 954: 'banana',
 955: 'jackfruit, jak, jack',
 956: 'custard apple',
 957: 'pomegranate',
 958: 'hay',
 959: 'carbonara',
 960: 'chocolate sauce, chocolate syrup',
 961: 'dough',
 962: 'meat loaf, meatloaf',
 963: 'pizza, pizza pie',
 964: 'potpie',
 965: 'burrito',
 966: 'red wine',
 967: 'espresso',
 968: 'cup',
 969: 'eggnog',
 970: 'alp',
 971: 'bubble',
 972: 'cliff, drop, drop-off',
 973: 'coral reef',
 974: 'geyser',
 975: 'lakeside, lakeshore',
 976: 'promontory, headland, head, foreland',
 977: 'sandbar, sand bar',
 978: 'seashore, coast, seacoast, sea-coast',
 979: 'valley, vale',
 980: 'volcano',
 981: 'ballplayer, baseball player',
 982: 'groom, bridegroom',
 983: 'scuba diver',
 984: 'rapeseed',
 985: 'daisy',
 986: "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
 987: 'corn',
 988: 'acorn',
 989: 'hip, rose hip, rosehip',
 990: 'buckeye, horse chestnut, conker',
 991: 'coral fungus',
 992: 'agaric',
 993: 'gyromitra',
 994: 'stinkhorn, carrion fungus',
 995: 'earthstar',
 996: 'hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa',
 997: 'bolete',
 998: 'ear, spike, capitulum',
 999: 'toilet tissue, toilet paper, bathroom tissue'}
# @title Map Imagenette Labels to Imagenet Labels
dir_to_imagenet_index = {
    'n03888257': 1,
    'n03425413': 571,
    'n03394916': 566,
    'n03000684': 491,
    'n02102040': 217,
    'n03445777': 574,
    'n03417042': 569,
    'n03028079': 497,
    'n02979186': 482,
    'n01440764': 701
}

dir_index_to_imagenet_label = {}
ordered_dirs = sorted(list(dir_to_imagenet_index.keys()))

for dir_index, dir_name in enumerate(ordered_dirs):
    dir_index_to_imagenet_label[dir_index] = dir_to_imagenet_index[dir_name]

dir_index_to_imagenet_label
# @title Prepare Imagenette Data
val_transform = transforms.Compose((transforms.Resize((256, 256)),
                                     transforms.ToTensor()))

imagenette_val = ImageFolder('imagenette2-320/val', transform=val_transform)

train_transform = transforms.Compose((transforms.Resize((256, 256)),
                                     transforms.ToTensor()))

imagenette_train = ImageFolder('imagenette2-320/train', transform=train_transform)
random_indices = random.sample(range(len(imagenette_train)), 400)
imagenette_train_subset = torch.utils.data.Subset(imagenette_train, random_indices)




# Subset to only one tenth of the data for faster runtime
random_indices = random.sample(range(len(imagenette_val)), int(len(imagenette_val) * .1))
imagenette_val = torch.utils.data.Subset(imagenette_val, random_indices)
imagenette_train_loader = torch.utils.data.DataLoader(imagenette_train_subset,
                                                      batch_size=16,
                                                      shuffle=True)

imagenette_val_loader = torch.utils.data.DataLoader(imagenette_val,
                                                    batch_size=16,
                                                    shuffle=False)

dataiter = iter(imagenette_val_loader)
images, labels = dataiter.next()

# show images
plt.figure(figsize=(8, 8))
plt.imshow(make_grid(images, nrow=4).permute(1, 2, 0))
# @title eval_imagenette function
def eval_imagenette(resnet, data_loader, dataset_length):
    resnet.eval()
    with torch.no_grad():
        loss_sum = 0
        total_1_correct = 0
        total_5_correct = 0
        total = dataset_length
        for batch in tqdm.notebook.tqdm(data_loader):
            images, labels = batch

            # Map the imagenette labels onto the network's output
            for i, label in enumerate(labels):
                labels[i] = dir_index_to_imagenet_label[label.item()]

            images = images.to(device)
            labels = labels.to(device)
            output = resnet(images)

            # Calculate top-5 accuracy
            # Implementation from https://github.com/bearpaw/pytorch-classification/blob/cc9106d598ff1fe375cc030873ceacfea0499d77/utils/eval.py
            batch_size = labels.size(0)

            _, predictions = output.topk(5, 1, True, True)
            predictions = predictions.t()

            top_k_correct = predictions.eq(labels.view(1, -1).expand_as(predictions))
            top_k_correct = top_k_correct.sum()

            predictions = torch.argmax(output, dim=1)
            top_1_correct = torch.sum(predictions == labels)
            total_1_correct += top_1_correct
            total_5_correct += top_k_correct

        top_1_acc = total_1_correct / total
        top_5_acc = total_5_correct / total

        return top_1_acc, top_5_acc
# @title Imagenette Train Loop
def imagenette_train_loop(model, optimizer, train_loader, loss_fn):
    loss_fn = nn.CrossEntropyLoss()
    for epoch in tqdm.notebook.tqdm(range(5)):
    # Set model to use the imagenette classifier head
        model.train()
        # Train on a batch of images
        for imagenette_batch in train_loader:
            images, labels = imagenette_batch

            # Convert labels from imagenette indices to imagenet labels
            for i, label in enumerate(labels):
                labels[i] = dir_index_to_imagenet_label[label.item()]

            images = images.to(device)
            labels = labels.to(device)
            output = model(images)
            optimizer.zero_grad()
            loss = loss_fn(output, labels)
            loss.backward()
            optimizer.step()

    return model<jupyter_output><empty_output><jupyter_text>This cell creates a ResNet model pretrained on [ImageNet](http://www.image-net.org/), a 1000 class image prediction dataset. The model is then trained to make predictions on [Imagenette](https://github.com/fastai/imagenette), a small subset of ImageNet classes that is useful for demonstrations and prototyping.<jupyter_code># Original network
top_1_accuracies = []
top_5_accuracies = []

# Instantiate a pretrained resnet model
resnet = torchvision.models.resnet18(pretrained=True).to(device)
resnet_opt = torch.optim.Adam(resnet.parameters(), lr=1e-4)
loss_fn = nn.CrossEntropyLoss()

imagenette_train_loop(resnet,
                      resnet_opt,
                      imagenette_train_loader,
                      loss_fn)

top_1_acc, top_5_acc = eval_imagenette(resnet,
                                       imagenette_val_loader,
                                       len(imagenette_val))
top_1_accuracies.append(top_1_acc.item())
top_5_accuracies.append(top_5_acc.item())<jupyter_output><empty_output><jupyter_text>Exercise 4.1Complete the function below that runs a batch of images through the trained ResNet and returns the Top 5 class predictions and their probabilities.<jupyter_code>def predict_top5(images):
  ''' Args:
          images: torch Tensor with dimensionality B x C x H x W.
      Output:
          top5_probs: torch Tensor (B, 5) with top 5 class probabilities
          top5_names: list of top 5 class names (B, 5)
  '''
  ####################################################################
  # Fill in all missing code below (...),
  # then remove or comment the line below to test your function
  raise NotImplementedError("Predict top 5")
  ####################################################################

  B = images.size(0)
  with torch.no_grad():
    # run images through model
    images = images.to(device)
    output = ...
    probs = ...

  _, top5_idcs = output.topk(5, 1, True, True)
  top5_idcs = top5_idcs.t().cpu().numpy()

  top5_probs = probs[torch.arange(B), top5_idcs]

  top5_names = []
  for b in range(B):
    temp = [dict_map[key].split(',')[0] for key in top5_idcs[:,b]]
    top5_names.append(temp)

  return top5_names, top5_probs
# to_remove solution
def predict_top5(images):
  ''' Args:
          images: torch Tensor with dimensionality B x C x H x W.
      Output:
          top5_probs: torch Tensor (B, 5) with top 5 class probabilities
          top5_names: list of top 5 class names (B, 5)
  '''
  B = images.size(0)
  with torch.no_grad():
    # run images through model
    images = images.to(device)
    output = resnet(images)
    probs = torch.nn.functional.softmax(output, dim=1).cpu().numpy()

  _, top5_idcs = output.topk(5, 1, True, True)
  top5_idcs = top5_idcs.t().cpu().numpy()

  top5_probs = probs[torch.arange(B), top5_idcs]

  top5_names = []
  for b in range(B):
    temp = [dict_map[key].split(',')[0] for key in top5_idcs[:,b]]
    top5_names.append(temp)

  return top5_names, top5_probs
# get batch of images
dataiter = iter(imagenette_val_loader)
images, labels = dataiter.next()
# retrieve top 5 predictions
top5_names, top5_probs  = predict_top5(images)
# visualize probabilities of top 5 predictions

fig, ax = plt.subplots(5, 2, figsize=(10, 20))

for i in range(5):
  ax[i, 0].imshow(np.moveaxis(images[i].numpy(), 0, -1))
  ax[i, 0].axis('off')

  ax[i, 1].bar(np.arange(5), top5_probs[:, i])
  ax[i, 1].set_xticks(np.arange(5))
  ax[i, 1].set_xticklabels(top5_names[i], rotation=30)

fig.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>Out-of-distribution examples The code below runs two out-of-distribution examples through the trained ResNet. Look at the predictions and discuss, why the model might fail to make accurate predictions on these images.<jupyter_code>response = requests.get('https://upload.wikimedia.org/wikipedia/commons/e/ec/Bonsai_locker_aufrechte_form.png')
image = Image.open(BytesIO(response.content)).resize((256, 256))
data = torch.from_numpy(np.asarray(image))[:, :, None].repeat(1, 1, 3) / 255.

response = requests.get('https://upload.wikimedia.org/wikipedia/en/a/a6/Pokémon_Pikachu_art.png')
image = Image.open(BytesIO(response.content)).resize((256, 256))
data2 = torch.from_numpy(np.asarray(image)[:, :, :3]) / 255.

images = torch.stack([data, data2]).permute(0, 3, 1, 2)
# retrieve top 5 predictions
top5_names, top5_probs  = predict_top5(images)
# visualize probabilities of top 5 predictions
fig, ax = plt.subplots(2, 2, figsize=(10, 10))

for i in range(2):
  ax[i, 0].imshow(np.moveaxis(images[i].numpy(), 0, -1))
  ax[i, 0].axis('off')

  ax[i, 1].bar(np.arange(5), top5_probs[:, i])
  ax[i, 1].set_xticks(np.arange(5))
  ax[i, 1].set_xticklabels(top5_names[i], rotation=30)

fig.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>--- Section 5: Efficiency: ResNeXt + MobileNet<jupyter_code>#@title Video 5: Improving efficiency
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="iaEc03aebxg", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>[ResNet vs ResNext](https://arxiv.org/abs/1611.05431)  ResNet vs. ResNeXtThe widget below calculates the number of parameters in a ResNet and ResNeXt.Note how many parameters the ResNeXt networks take, especially as you increase the number of filters.<jupyter_code># @title Parameter Calculator

def calculate_parameters(in_channels, resnet_channels, resnext_channels, num_paths):
    # ResNet math: Implement how parameters scale as a function of feature maps and path in ResNet vs ResNeXt

    out_channels = in_channels
    # resneXt channel size
    # d = resnet_channels // num_paths*2 if num_paths>0 else resnet_channels
    d = resnext_channels

    resnet_parameters = in_channels*resnet_channels + 3*3*resnet_channels*resnet_channels + resnet_channels*out_channels
    resnext_parameters = (in_channels*d + 3*3*d*d + d*out_channels)*num_paths

    # print('ResNeXt number of channels: {}'.format(d))
    print('ResNet parameters: {}'.format(resnet_parameters))
    print('ResNeXt parameters: {}'.format(resnext_parameters))

    return None

in_channels = widgets.FloatLogSlider(
    value=256,
    base=2,
    min=5, # max exponent of base
    max=10, # min exponent of base
    step=1, # exponent step
    description='In channels'
)
resnet_channels = widgets.FloatLogSlider(
    value=64,
    base=2,
    min=5, # max exponent of base
    max=10, # min exponent of base
    step=1, # exponent step
    description='ResNet channels'
)
resnext_channels = widgets.FloatLogSlider(
    value=4,
    base=2,
    min=1, # max exponent of base
    max=5, # min exponent of base
    step=1, # exponent step
    description='ResNeXt channels'
)
num_paths = widgets.FloatLogSlider(
    value=32,
    base=2,
    min=1, # max exponent of base
    max=7, # min exponent of base
    step=1, # exponent step
    description='Number of paths'
)
_ = widgets.interact(calculate_parameters,
                     in_channels=in_channels,
                     resnet_channels=resnet_channels,
                     resnext_channels=resnext_channels,
                     num_paths=num_paths)<jupyter_output><empty_output><jupyter_text>Exercise 5.1How do parameter savings depend on number of feature maps, 32 vs. 1024?<jupyter_code>parameter_savings_features = '' #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>How do parameter savings depend on number of paths, 4 vs. 64?<jupyter_code>parameter_savings_paths = '' #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Depthwise seperable convolutionsAnother way to reduce the computational cost of large models is the use of depthwise seperable convolutions.The widget below calculates the number of parameters of a regular convolution vs. depthwise seperable convolution.<jupyter_code># @title Parameter Calculator

def calculate_parameters(in_channels, filter_count, out_channels):
    # Convolution math: Implement how parameters scale as a function of feature maps and path in convolution vs depth-separable convolution

    conv_parameters = in_channels*filter_count*filter_count*out_channels
    dconv_parameters = in_channels*filter_count*filter_count + in_channels*out_channels

    print('Regular convolution parameters: {}'.format(conv_parameters))
    print('Depthwise convolution parameters: {}'.format(dconv_parameters))

    return None

_ = widgets.interact(calculate_parameters,
                    in_channels=(2, 64, 2),
                     filter_count=(2, 64, 2),
                     out_channels=(2, 64, 2))<jupyter_output><empty_output><jupyter_text>Exercise 5.2How do parameter savings depend the on number of feature maps, 4 vs. 64?<jupyter_code>parameter_savings = '' #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>--- Section 6:Transfer Learning<jupyter_code>#@title Video 6: Transfer Learning
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="H--3iLNkcvw", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>The most common way large image models are trained in practice is via a type of transfer learning called "fine-tuning". Fine-tuning involves a network pretrained on a large dataset like ImageNet, then retraining that network on your task of choice. While training a network twice sounds like a strange thing to do, the model ends up training faster on the target domain. There are also other benefits such as [robustness to noise](https://arxiv.org/pdf/1901.09960.pdf) that are the subject of [active research](https://arxiv.org/abs/2008.11687).In this section we will demonstrate fine-tuning by taking a model trained on ImageNet and teaching it to classify Pokemon.  Download and prepare the data<jupyter_code># @title Download Data
!git clone --quiet https://github.com/ben-heil/cis_522_data.git
!tar -xzf cis_522_data/archive.tar.gz
!tar -xzf cis_522_data/faces.tar.gz
# List the different Pokemon
!ls small_pokemon_dataset/
# @title Determine number of classes
num_classes = 0
for folders in os.listdir('small_pokemon_dataset/'):
    num_classes += 1
print(num_classes, 'types of Pokemon')
# @title Display Example Images
train_transform = transforms.Compose((transforms.Resize((256, 256)),
                                     transforms.ToTensor()))

pokemon_dataset = ImageFolder('small_pokemon_dataset', transform=train_transform)

image_count = len(pokemon_dataset)
train_indices = []
test_indices = []
for i in range(image_count):
    # Put ten percent of the images in the test set
    if random.random() < .1:
        test_indices.append(i)
    else:
        train_indices.append(i)

pokemon_test_set = torch.utils.data.Subset(pokemon_dataset, test_indices)
pokemon_train_set = torch.utils.data.Subset(pokemon_dataset, train_indices)

pokemon_train_loader = torch.utils.data.DataLoader(pokemon_train_set,
                                                   batch_size=16,
                                                   shuffle=True,)
pokemon_test_loader = torch.utils.data.DataLoader(pokemon_test_set,
                                                  batch_size=16)

dataiter = iter(pokemon_train_loader)
images, labels = dataiter.next()

# show images
plt.imshow(make_grid(images, nrow=4).permute(1,2,0))<jupyter_output><empty_output><jupyter_text>Finetuning the ResnetIt is common in computer vision to take a large model trained on a large dataset (often ImageNet) and fine-tune it to perform a different task. In this case we're using a pre-trained ResNet model to classify types of Pokemon.<jupyter_code>resnet = torchvision.models.resnet18(pretrained=True)
num_ftrs = resnet.fc.in_features
# reset final fully connected layer, number of classes = types of Pokemon = 9
resnet.fc = nn.Linear(num_ftrs, num_classes)
resnet.to(device)
optimizer = torch.optim.Adam(resnet.parameters(), lr=1e-4)
loss_fn = nn.CrossEntropyLoss()
# @title Finetune ResNet
pretrained_accs = []
for epoch in range(10):
    # Train loop
    for batch in pokemon_train_loader:
        images, labels = batch
        images = images.to(device)
        labels = labels.to(device)

        optimizer.zero_grad()
        output = resnet(images)
        loss = loss_fn(output, labels)
        loss.backward()
        optimizer.step()

    # Eval loop
    with torch.no_grad():
        loss_sum = 0
        total_correct = 0
        total = len(pokemon_test_set)
        for batch in pokemon_test_loader:
            images, labels = batch
            images = images.to(device)
            labels = labels.to(device)
            output = resnet(images)
            loss = loss_fn(output, labels)
            loss_sum += loss.item()

            predictions = torch.argmax(output, dim=1)

            num_correct = torch.sum(predictions == labels)
            total_correct += num_correct

        # Plot accuracy
        pretrained_accs.append(total_correct / total)
        plt.plot(pretrained_accs)
        plt.xlabel('epoch')
        plt.ylabel('accuracy')
        plt.title('Pokemon prediction accuracy')
        display.clear_output(wait=True)
        display.display(plt.gcf())<jupyter_output><empty_output><jupyter_text>Train only linear readoutAnother possible way to make use of transfer learning is to take a pre-trained model and replace the last, linear layer which is also called the readout. Instead of fine-tuning the whole model as before, we train only the readout.<jupyter_code>resnet = torchvision.models.resnet18(pretrained=True)
for param in resnet.parameters():
  param.requires_grad = False
num_ftrs = resnet.fc.in_features
# reset final fully connected layer
resnet.fc = nn.Linear(num_ftrs, num_classes)
resnet.to(device)
optimizer = torch.optim.Adam(resnet.fc.parameters(), lr=1e-4)
loss_fn = nn.CrossEntropyLoss()
# @title Finetune readout of ResNet
linreadout_accs = []
for epoch in range(10):
    # Train loop
    for batch in pokemon_train_loader:
        images, labels = batch
        images = images.to(device)
        labels = labels.to(device)

        optimizer.zero_grad()
        output = resnet(images)
        loss = loss_fn(output, labels)
        loss.backward()
        optimizer.step()

    # Eval loop
    with torch.no_grad():
        loss_sum = 0
        total_correct = 0
        total = len(pokemon_test_set)
        for batch in pokemon_test_loader:
            images, labels = batch
            images = images.to(device)
            labels = labels.to(device)
            output = resnet(images)
            loss = loss_fn(output, labels)
            loss_sum += loss.item()

            predictions = torch.argmax(output, dim=1)

            num_correct = torch.sum(predictions == labels)
            total_correct += num_correct

        # Plot accuracy
        linreadout_accs.append(total_correct / total)
        plt.plot(linreadout_accs)
        plt.xlabel('epoch')
        plt.ylabel('accuracy')
        plt.title('Pokemon prediction accuracy')
        display.clear_output(wait=True)
        display.display(plt.gcf())<jupyter_output><empty_output><jupyter_text>Training ResNet from scratch<jupyter_code>resnet = torchvision.models.resnet18(pretrained=False)
num_ftrs = resnet.fc.in_features
# reset final fully connected layer
resnet.fc = nn.Linear(num_ftrs, num_classes)
resnet.to(device)
optimizer = torch.optim.Adam(resnet.parameters(), lr=1e-4)

loss_fn = nn.CrossEntropyLoss()
# @title Train ResNet from scratch
scratch_accs = []
for epoch in range(10):
    # Train loop
    for batch in pokemon_train_loader:
        images, labels = batch
        images = images.to(device)
        labels = labels.to(device)

        optimizer.zero_grad()
        output = resnet(images)
        loss = loss_fn(output, labels)
        loss.backward()
        optimizer.step()

    # Eval loop
    with torch.no_grad():
        loss_sum = 0
        total_correct = 0
        total = len(pokemon_test_set)
        for batch in pokemon_test_loader:
            images, labels = batch
            images = images.to(device)
            labels = labels.to(device)
            output = resnet(images)
            loss = loss_fn(output, labels)
            loss_sum += loss.item()

            predictions = torch.argmax(output, dim=1)

            num_correct = torch.sum(predictions == labels)
            total_correct += num_correct

        scratch_accs.append(total_correct / total)
        plt.plot(scratch_accs)
        plt.xlabel('epoch')
        plt.ylabel('accuracy')
        plt.title('Pokemon prediction accuracy')

        display.clear_output(wait=True)
        display.display(plt.gcf())<jupyter_output><empty_output><jupyter_text>Head to Head ComparisonStarting from a randomly initialized network works less well, especially in the case of small datsets. Note that the model converges more slowly and less evenly.<jupyter_code># @title Plot Accuracies
plt.plot(pretrained_accs, label='Pretrained: fine-tuning')
plt.plot(linreadout_accs, label='Pretrained: linear Readout')
plt.plot(scratch_accs, label='Trained from Scratch')
plt.title('Pokemon prediction accuracy')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Exercise 6.1First, we compare the Pretrained ResNet with the ResNet trained from scratch. Why might pretrained models outperform models trained from scratch? In what cases would you expect them to be worse?<jupyter_code>when_pretraining_works = '' #@param {type:"string"}

t5 = time.time()
# to_remove explanation

"""
1. The closer your pretraining and target data domains are, the better pretraining will work
2. The more pretraining data you have, the better pretraining will work
3. The better your model is able to take advantage of your pretraining data (that is to say
   the larger your model is assuming you have enough data), the better pretraing will work

Pretraining isn't necessarily always a benefit though. If you source domain is very different from
the domain you're trying to predict, your models might learn unhelpful features.

Additionally, if you have a lot of training data in your target domain, pretraining data might
cause your model to converge to a local minimum (this process is referred to as ossification in
the Scaling Laws for Transfer paper cited in the Further Reading section)
"""<jupyter_output><empty_output><jupyter_text>Exercise 6.2Second, take a look at the different transfer learning methods - fine-tuning the whole network and training only the readout. Why might fine-tuning the whole network outperform training only the readout? What are the benefits of training only the readout? In what cases would you expect a similar performance of both methods?<jupyter_code>linear_readout = '' #@param {type:"string"}
# to_remove explanation

"""
1. More weights are adjusted to the pretraining domain.
2. Since only one layer is trained, the training procedure is faster/less computationally expensive.
3. If your pretraining and target data domains are close.
"""<jupyter_output><empty_output><jupyter_text>Further ReadingSupervised pretraining as you've seen here is useful, but there are several other ways of using outside data to improve your models. The ones that are particularly popular right now are [contrastive learning](https://arxiv.org/pdf/2002.05709.pdf) techniques that learn features from unsupervised image data and all sorts of methods of masking input for training [transformer models](https://arxiv.org/pdf/1810.04805.pdf).There is also a [recent paper](https://arxiv.org/abs/2102.01293) that seeks to quantify the relationship between model size, pretraining dataset size, training dataset size, and performance. --- Section 7: Compute-performance tradeoff / Different backbones<jupyter_code>#@title Video 7: Speed-accuracy trade-off
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="Ejl4IXYp-2E", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>As the models got larger and the number of connections increased so did the computational costs involved. In the modern era of image processing, there is a tradeoff between model performance and computational cost. Models can reach extremely high performance on many problems, but achieving state of the art results requires [huge amounts of compute power](https://arxiv.org/pdf/1810.00736.pdf).![compute_vs_performance.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAArcAAAKECAYAAAAdcxJ0AAAABHNCSVQICAgIfAhkiAAAABl0RVh0U29mdHdhcmUAZ25vbWUtc2NyZWVuc2hvdO8Dvz4AACAASURBVHic7L15mGRFlTf8u/fmvtSWVV17V29sIjgDKtO2L5sIQjfCACONzAz0PGwCDz7Op6B+vKDQju84qAgqoI0ji9KM8okOjrwqiEAjOyh0YzdNd1V1VWXlvm837/L9kUT0vVlZW1ZEd1UTv+dpqMh778mIE78492TEiROSaZomBAQEBAQEBAQEBA4ByAe7AgICAgICAgICAgKsIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbCuRUQEBAQEBAQEDhkIJxbAQEBAQEBAQGBQwbveefWNM0pfzf6bLpn5iqHpSwez8xHzmKoE4t2NMLB1i2v72Yp62Dz82ByhDenD0Y7eH53s3VabO1oto+Wyjg7VNrRCIeavViMdVqoDpup02x4zzu3AgICAgICAgIChw6EcysgICAgICAgIHDIwHGwKyDAF6ZpQtd1GIYBSZKmXJvpM/J3/WcAGn7ejKxm68SiHfXPHIx2HCgdHirtsPJvNvmLvR2NnjkY7ZitTgv57mbrtNja0WwfLZVxBsxtTC32dizEpi/mdvCy6QdynMmyDFk+cPOpwrk9xBGJRPCTn/wEuVyO2QAGhHO71IzjodKO+byIF3s7Fuo4Hag6LeS7m63TYmtHs320VMYZIJzbxdwOXjb9QI0zwzBw7LHH4uyzz4bL5ZrSDh4Qzu0hjj179uChhx7CeeedB6/Xa7vWLPGBmQ3hfGQ1GrD1OFAvuGbbMVud5yrH+je5xtLIN5K7GJ2HmTCfFzGPOvHkwnR/k+9g2Q4ANrkz1aOZdjS652DrttH1ZsbZfHS7VMdZs7LqP5uOYweKC/ORz3ucseLCbO1fbONMlmW89NJLeOmll3DaaacJ51aAHbq7u7Fp0yb09vbaPm92AAOAruvI5/NobW2ddnA0YwxKpRI0TUMwGJxRznzbUS6Xoes6/H7/jIN5Pu0AgHw+D4fDweyHA/ksk8nA7/fD4XAs+AVDUK1WUSgU0NLSYjPmC9Ut4UJLSwtkWebOBaB557YRF5rVLWsuNPobAOWC0+lkpltN01AoFBAMBulSIQsuqKoKVVURCASY6VaSJBQKBUiSBJ/P17Ru66+bpkn7zOPxzLm+s8k3DAP5fB4ejwdOp5PZOAMOHBealWW9T1VVVCoVBAKBKRxbKBcAwO/3z6rD+ei2UChAURR4vV5muuXBBVLOZDLw+XzUWWShW03TkM/nEQwGoSjKvHQ4Excefvhh/PSnP23YP7wgNpQd4uBFpmq1ilgsBl3XmcrN5XJIp9MwDIO53EwmQw0ZCxADUywWmckEaj8cotEoVFVlKrdarSKRSEDTNOZy4/E48z4rFApIpVJcuMBaLuECefGyAm8usB6/xWIRiUSCi27z+TwzmQS8xm88Hke5XGYuNxaLoVKpMJVLxi9ru1AsFpFMJpnaXKA2fnlxoVAoMK2vYRhIJBJcuBCPx5lzQdM0LnaBtQ2fC8TMrUBT8Hg8WLlype3XHQuEQiEA7J3yzs7OaX/ZNwtJkqbMhrOAoihYvXo18+B7j8eDFStWcJPLmgvt7e1ob2/nwgWALceWGhfcbjeGhoaYy21tbZ0yA7hQSJKEnp4e5k4SAPT29jLnl6IoXHSrKAoXm0u4wFpua2srXSViiWXLlnHhQk9Pj21FiwVkWcby5cu5cYG1bnlx4WBAOLcCTUGSJC47H4lM1oOWGC3WDg2v3Z885Fp1wPLlIEkSF2PIgwu8VjKWMhdYy1UUhdv4ZS2Tp255yOUxznj0GZHFSw+CCwuprwmY5rv/NwBIgAkAMiTwGb8HA8K5FWgKmqYhl8vRmFtWKJVK0HXdFrO3WOVaY/bq4ywXKjeTySAQCMDhYDdESWwd69mUpcQF0zSpXGvMLQu5vLngdDqZybVygeVLvVKpQFVVBINBprolMbf1cZYLlWuNuWUFEmfp9XqZ9plhGDTmluWmHGvMLSvn2TRNygXWtpyEkfDiAsvxa425ZdlnhmEgm83aYm5nhGkCpg5U05AqY0A1DVRzkIwSIEmQIAOuDkDxQ3IP1P6WXQAkYIk6usK5FWgKqqoiGo0yX4LM5XJ0EwJL5HI5qKrK3KFJpVLw+XzMDWIkEoHL5WLq3Kqqing8Dp/Px/Slq6oqYrEYcy7k83mUSiUuXKhUKsxfjul0Gh6PhwsXnE4n0z4jcZZzfjnOEcViEel0Gn6/n+ksYyaTgaIoTPsMAB2/rJ3beDyOrq4u5s5tNBpFf38/c+eWcIFln1m5wBKZTAayLDOXS8Yvay4kEgmEQiHmzm00GkVvb+/Mck0TMCpAaRhIvwTk/wqocUCv1q5L5D9mbfZWUgBXK+BbCbQeB/jfBziD5MYlBeHcCjQFn8+Hww8/nEs8FY8lke7ubuYxtySeinX8l8PhwJFHHgkATGX7fD6sWbOGi9zDDjuMeb91dXVxWX7s7u4GwFYHsixjcHCQKxdYwuv1Ui6wREdHB/M4aUmS0N/fz1y3kiRh+fLlANhyweFwYPXq1Vzk8rC5Ho8Hq1evZt5nHR0d6OjoYN5vfX19XLgwODhIy6zkOxwOrFq1iqlMIvfwww+fWaahQSqNALH/AdJ/AWAAkGt+qvKu62eatZlZ0wTkd/u/mgHSrwGpVwDvANBzNhA8BlDYOf0HAsK5FRAQEBAQEBA4VKAXgcj/APEnAePdWVr53Vn5uTjZsgwYElAOA8M/gBQ8Cuj7VM3ZXSIQqcAEmoKu6ygWi8xTfKiqyjxtCpFbqVSYpwIrl8vM0zQZhoFiscglHQuPPuPFhWq1ilKpxLzPeHCMJxcKhQIXLhQKBS59Vi6XmcvloVtefWaaJorFIvPUWrzk8rALpmlC0zTm4xfYH9fNEkuVC43sgqTGgbH7gPjvAUOtzcjKTczKW5/L7QBGt9T+zyFTBQ8I5/Y9AMMwqOHSdR3JZBLVapWWrfnydF1HIpGgZRJLSJwBssElk8kgFovRuFMS5G+aJrLZLC0bhoFcLodsNkvrk81maZls8Mlms/SFOzExYbu/VCrZ8pKqqjqlbM2NW6lUbLn6DMPA6OgozbmoaRrS6TQ1OCS3HzFshmHYdGQYBtLpNNUJuV4ul5FKpVAoFJDNZlEul2GatZNocrkcSqUSbUM6nUY+n4dhGNQw5XI5ej2fz9McqaZpYu/evYjH4/TFUC6Xkclk6P2kTL6vXC7b2lytVpFKpWzlWCyGaDQKXdehaRpisRjVga7rSKfTNl5YdUTiCK06yGQyqFQqVHYjHRWLRdqGXC5nywObz+dtvKnPRZxIJDA+Pk7LpVIJ+XyetrlUKiGRSNDrhAf1ZStP4vE4MpkMUqkULVt5kEqlbOV4PE51QnhCyqZpIplMQtM0+vf4+Djtd6IDazmTydjK2WyW8qBRPtdCoYA9e/ZQbhaLRRv38/k8MpmMjftWHZTLZSSTScoDwhMyRqrVKpLJJH2+PucpKVt5UW8/rHk88/k83n77bdtYIjwh5XQ6beM6GUPkupUXQM1ejI2NIZfLUftC7AXhRb29sLaJtLnefpDvTqfTNh0RPpP76+2Fruu2sUXyz1p5MjIyQvuRtJnoxDRNJBIJ6vg1sheZTIbqlNhQYnMrlQoKhQKVTzZDEf2QDXipVIrKs9oLK08Mw4CmaZiYmEA8HrfpIJ1O29pYzxPrWCA6IToyTRNjY2OIRqMwTXMKT+rfK/U2lly3jgViIzOZDLLZLC2TNuXzeRv3CS+s9sKqAzKWSDkejyMcDtM2q6pqy9Nbnye83l4QHdW/VyYnJ+nBMVaekLFhHSupVGpG+5HL5egPkWg0ing8Tvu9VukUMPZjIPUqYOhsNoPJ724qK40D++4Dctupg0t0YB0r9TaTR67yOVX7gH+jwAGH9ZczMXyEbMQwkgFNyoScAGy/EIljBgCDg4OQJAmlUsmWTLr+F3C5XLa9rEqlks2Qkxk6AGhra0NHR4ftuqqqtjrrum5zkjRNszlR5MQscr9hGHC5XOjq6oIsy3SmgtxP2mz9dW2dLaufSbXqqLu7G62trSiVSjanp1Qq2XRQr6NKpWKbPbSWZVlGd3c3dF2ndSRtbKQza5ut91sNv67r0HUdfX19cDgc0HXdZhRJv1p5YX2eOB3Wl1exWES1WoXb7cbAwADK5bKNN4VCwVauVCpTeFKvE2sbvV6v7XSySqXSkBcEjXhgLZPTd1pbW6l+rW2crs1WHtTzhPBClmW6gcjqxBSLRRsPiM6sZasOSqWSjReapqGnpwdut7uhjkh5Op4QHVlf0IVCAU6nE/39/bSfCOpniut10Mh+FAoFqpNAIIBgMGibUSInzVnL1jYXCoUp9sKqg3K5jGAwiPb2dvp8PQ/qy1Yuz8ST7u5u+P1+m70gPLG20arj+pnOel7IsoyWlha6maxeR/Vl0sZ6e2HlCdHZ4OAgfD5fQ/sx01iqn0UlZQBwuVzo6emxXa9fjakfK410VCqVbGVFUdDT00NPLaxvc30brTOcROf1bVJVFZ2dnejo6JhiP8rlckNeWLnfyH6Q66FQyLaBt95+1L9X6sdC/WoIaQPhWP17pn72lciztqn+XUzeKyTGn7QbACS9BER+BeR21m5uZrZ2JkgSoCaAia1AJQLAnHUskPfMwXBuJZP1msESg3WTEfm70WfTPTNXOSxlzeeZ559/Hps3b8aWLVuYHr+70GP+FqoPHjo8GO04UDoknwP2vltq7bC2YTb5i70djZ45kO0A7HrkbfcWotu59DvPdjTbR0tlnAFzG1OLvR0LselWkOwcJFMLccRbWloA7J+V7ejooD9iDMNAuVymKcVaWlqgKAqd1e7o6KDlbDZL0/DVf7dhGFBVFU6nk2avqK8jcYJbW1tr9xhVIP47YOIXNPlBPq8i4HMCjro5TNPcP6NL/m70WaNnjHdz47Z9ABjcBDiCs+rYNE089NBD2Lp1Kx588EGqw+nG9nze1zNBzNwKNAVd1+kyPEtomgZVVZnLrVarqFarTOWaZi1+k0c8FY+4RcMwUKlUuMTc8qivpmlc4qSr1Spzjgku1MBj/JI+46HbSqXCTS7rOGlecnlwwTRroQisxy+w35azBNEteUfccMMN+Kd/+ida9wceeACbN2+m97/++uvo6enBs88+Sz+LRCL4x3/8R5x//vn46Ec/irvuuguapuEvf/kLVq9eje3btwOohT98+tOfxtjYWMO6ZDIZfPnLX0Y0Gm143TAM3HLLLVi7di2SyWTN8SyPAdHf0XuSqTIu/OKTGIuwPQ4csgQoCpB5A8i8XHN0FymEcyvQFMrlMvbs2cP85RiPxzE2NsbcIMZiMVv8JguYpomJiQlbbBsLaJqG3bt3Mz/zvlQqYWRkhPnLvFwuY3h4mDkX6mNuWSEej3Phwvj4eO1lwxC6rmP37t22ZXUWKJfLXLiQSqUwOjrKnAuTk5OIxWJMZQJAOBxm3meapmHv3r22mHoW0HUd77zzDhcuDA8PM3cYU6kURkZGmI/fSCTCnQsulwuvvfYannrqqSn36bqOhx9+GJs2bcKjjz5Kf2z8+Mc/xtFHH41nnnkGTzzxBO699168/fbbGB8fRyAQwDe+8Y051aNUKuHNN9+c9nqxWMS2bduwcuVKvPLKK7U8tvEnAS3LPhRhWphA7Amg0tgBXwwQqcAEmoLX68Vhhx3G/MhCcm74fJcgZgOPPLeSJGFgYIB5XR0OB4444gjmR236fD6sWrWK6cEQQI0Lq1evZl7frq4uLlzgwTFJsufJZAVFUbhwwev1cuECyXPL0i5IkoS+vj5m8qzgNX55jAdFUXD44Ydz4wLLAyeAGhfa2tqY67enp4epPALCBUmS4PF4sGnTJtx555045ZRTbPel02m8+uqr+NGPfoQLL7wQyWQSXV1d8Hg8GB4eRqFQQG9vL+6++24sX74clUoF69atQyQSwY4dOxAKhaiseDyOr3zlK9izZw+GhoZw44034tFHH8Wbb76JL3zhC3jwwQen1PONN97A0NAQzjzzTPzmN7/BGWsHIOXeAKTZx9xbbydw0/dfQSqv4qgVbfjmv56AXz+7D0+/EsYLb0bxkQ9041//8Vhc943nUNUNfOCwDoxFivj+l9bhqVcmcM8jf4WqGfjHM9fgok8AyLwCuM/EYjzkQczcCjQFWZbhdDqZGy5ZluFwOJjLVRSFy/npDoeDuYMvSRKcTic3uTz6bKlwQZJqZ6fzkOtwOJg7HkuNCzzGGQBuul1qfcbT3rAeD7xs+YHiwoc//GE4HA4899xzCIVCdFPnzp070dbWhqGhIfT19eGtt94CAFx44YUoFos455xzcP/99+OII46A3++Hw+GA3+/Hddddh9tuuw1er5eeKvbTn/4UbW1teOCBB7Bq1Sps2bIFGzZswFFHHYVbbrllSj0Nw8Bjjz2Gj3/84zjppJPw17/+FYnhZ4Hq3Gbzt7+TwoaThvCfN5+IPRM5vP7XBCqqgT++OonvfH4trrvo/Xjwf97GmuUtuO+rJ6OzzYNwooiJeBH3/Pwt3PqZ4/GfN5+IO7a+iYlovpYarJqf/YsPAoRzK9AUyG58HnlueeVGZB0jbJrmlB2/LMArtynZ8bxU8tzy4AKJreMhtz5bBAsQLrAOH+DFBV7jt35nPAuQ8ctDLq/cpry4wCPPbbVatWUbYAXeXCD19fl8uPrqq/Hd736XOr6GYeCJJ57A0NAQXn75ZaxYsQK//vWvYRgGent7cdddd+Ff//Vf8atf/QpXX301TQMGAGeccQZGRkYwOjpKZ8lfffVVjI2N4cc//jF27dqF7du3w+VyQVEUeL1ePP/887j66qtx1VVX4f7770cmk8ELL7yA1tZWjI+PI53J4K9/fhrbd8XxmVufwdW3PoN7H/krNL1xX15w+mqsXzeAcKKEQklDLF2r2wcO68CH3r8Mywda8ObuJE48rhdt7R6c+Le98LsVxJMlvDOWw+/+NI6f/Ho3svkqxqJFoDIBVBdnaIJwbgWagqqqXGJj0+k0IpEIc7mpVIrm5WUF0zQRjUZt+WdZgOTltaatYYFKpYKJiQnmL8dKpYKxsTHmjhIvLqTT6Wk3azQL3lxgfeiEqqpcuJDL5RAOh5k7SvF4HOl0mplMgmg0asuRywKapmF8fJx5bKyu6xgbG2NuF1RVxfj4OHMuZLNZhMNh5uM3kUgw3+cA1PZl1I/fdevWoVwuY9euXQBqtuPpp59GMpnEvffei2QyiRdffBETExO4++67EY1GcdZZZ+HBBx+EaZr47//+b2prnE4nrrzySjz00EO2H8GhUAiDg4M46aSTcNlll9m+v6enByeffDJOOeUUHHnkkdi9ezei0Sgef/xx3HnnnQj6PXj86R3obPfi1A/34dQP9eGY1dMff/3LJ/fiuv/4Ex57egRVbf/kScDvolkVDKNBsgQJUBQJAz1+DPYG8IV/PhYr+4O1k9CKe5tTOGeImFuBpuDxeLBy5UrmS2QkHom1XBK/yVKuLMvo6+vjsrS7Zs0aLrGxQ0NDzGPrPB4PVqxYwXypkAcXJElCZ2cn85hbWZbR29vLjQusdevxeLhwoa2tDa2trcz7rLu7m7luAaCvr4+5rXE6nVzGg8Ph4BInTcYvSy5IkoT29naa7ooluru7mcoj6O3tpam5KpUK8vk8nE4nLr30Unz+85/HP/zDP2Dnzp1QVRVbtmyBw+GApmn46Ec/ir179+K1115DNpvF5z73OWiahnK5jI6ODvT19VGn+ZxzzsGWLVsQj8cBAKtXr4bL5cL555+PRx55BLt27cIxxxwD06wdNrRixQqsWLGC/kC45ZZb8KlPfQo33ngjqlUNrzz7C9x84/+DgNeJfzhjNU3rlUzWVik13QC0d39oyhJ+8eQwrrrgKHzg8BBeeLPxprzukBdvvZPGmR8ZxGu74ihUdLQFXFgW8uIjH+hGf5cfF/+/T+LMdYM1T7iaapw+7CBDzNwKNAVZluFyubi8zFkbbyKX9csGgC0XISvwioeUJIlLny1FLvCQy5MLPOIsefUZj5hbnrrlZRdY9xkvubxj8VmDZ5w0kVupVGi2i0984hPo6OhAtVrFM888g1NOOYXepygKTj/9dDz11FO4/vrr8cwzz+CEE07A2rVrcfjhh+P000+Hw+FAIpEAALjdblx44YV05vayyy7Dtm3bcNZZZ+Gee+7Bxz72MbS2tmJoaAgbN26cckLbq6++ilNPPRUA4HQ68IHVflQ1YMfeqasaY9EC/vmmP+KMa3+DM679DX786E6cdHwfrr/9RVz+1aeRL1WRyk49bviaC9+HR57ci/Wf/b947I+jcMgShnqD+KczD8PFNz6FEy9/DEesaENXuweACagxwGR7bDELiEMcDlCiapay5vMMr0McyC9LYhRZtYMsjVmdDxb9QY4gJhscFprwm+hB0zS6GaFZOY0+0zSNOgmskpnruk51AMz/II7p2sGTC6Zp2l6Q9Qm/Z5Lf6DrhgmmaNidsoeOM1HchXGj0N5GrKAo96YmFbg3DgK7rtg0/PMYZC91KktTQLsxXt/XXTbOWi5VsfpprfeeqW1mWqVwWugVqOV7JpjKWXLCO34XIqh9nB5sL89Et4YKiKCgWi3A6ndT2kFAQwhWrratWq/QUTDJjK8syPB4PZFmmebUDgQAkSaJlr9dL+7RSqcDpdMLj8QCohYpUKhUEAgHa16ZZi+t3u93vnrRpQkk+DnXPz+ByKoAi7T+QwQQqZQ26btB2upwKFIeMUqkKRZGgyDIkCZAlCZpuwOGp6fPlP0fgcMg4YqgVT70axk9+vRv333oyZIeMSlmDaQIej1JLO2aaQPB9MFd8BpLinZYL4hAHgSWDSqXCJZ9lJpNZMjG3ALjFWY6MjDDfNEHipHnE3O7bt29JcYGcec8Kprm0Ym5JnPRSiLkFaimTWOejBWo5U1nH8hqGgfHxceaxsYZhYN++fVxi8QUX7Fzwer22H9Verxcejwcej4dmOiBwuVzweDyQpNpqSEtLCwKBABwOBwzDwOTkpO1+kj2B/PBxu91oaWmB1+ulDhz5zDpLL8syfD4f3dgWTyQBU4LL0+AUMlmC2+eEL+CC991/irvmAHv9Tri8TihuBbJTBpwyHO79M+GTySI+980/4UvffQnf+emb2PiJ1bX73pXp8TsBZfG7jiLmVqApuN1uDA4OMl8ia21tRUtLy7x/pc2Gtra2pn79zYZly5YxlynLMpYvX8586dzlcqG/v5+5XLfbjYGBgfc0FyRJ4saFwcHBJcOFYDAIn8/HLRafNZYtW8a8riQWn7VuZVnGwMAA86V+nlzw+/3MxwSJmWcNXuOXR1y3LMvoaG8D0grzFLMb/tdyHLG8Dam8io6gG2uGWmapjAOLMc+tcG4FmgLPOEvWMoHar2UeDo3T6WRuaMkMAACmsmVZtp1VzlLuUuGCNWyApQ4OBBdYwsoFliDhE6z7jZduiaPImgs8xi+Ry8MBYy2XLO8risK834gtZwkrF3jJZW9vHICjBZAcgMFw1l2Rcdjqdvtn09ZdAlzLAIm9jVooFv/cssCihGEY9BxultB1nfnyGFCLWyQxnKxA4q14nCGvqirz5TxefbaUuEDiLHlwQdO09zwXdF3nIpeXbnnJJXGYS0ku6/HAkws87MJS5ILpaK05lhxmsucMR2DRZUoAhHMr0CRKpRLefvtt5oM2FotxieWNxWIYHx9nbsDHxsboTlhW0DQNu3btQrFYZCq3VCrhnXfeYX7QQKlUwu7du5m/GOLxOBcuRKNR7Nu3j6lM0zSxb98+5lzQdR27du1injO1XC5z4UIqlcLIyAjzPLcTExPMcxMDwL59+2haJlbQNA179uxhnj9X13W8/fbbth30LEDsgqqy3fGeSqUwPDzM3LkNh8OIRCJMZQLA2NgY4vE40/rquo69e/dy4cKuXbuQ1doAVwjAQXJunT7Av+bgfPcsEGEJAk2BZ55bHrGxnZ2dMAyD+dIbj3gqcjY9Oe6RFUieWx55MleuXMklz217+/QJyRcilzUIF3iEZpBcmCxB8tyy5kJbWxvd4c0SSynPrcPh4KJbRVGwcuVK5naBR55boMaFYDDIvN+6u7vn7IDWsgxoME0DJgyQ2NDaO0au/YMCQKJ5bllCURQMDQ0xt42KomDVqlW10ITiGqA4wlT+nGCagHMZ4OKTd3ihEM6tQFMgxwPyiK3j8RIjMXusnVu3280l/oukiWEdG8tDrqIodLcwS/A4l55nPCRvLrCElQssQXKQLpVxRhxF1lwgKZ1Yy+Vhc0naKtZ95nA4uMTHulyuGWWapgHDVFGtRlEs7UCh+FdU9TQMswyrc6vIHjiVILyeI+D3vQ8uVy9kaX+KLxbgyTGv11uTGTgSSG4DDLbZdeaEwJGAM3jgv3cOEM6tQFPQdR2qqtKce6xAYiFZz1SR2C+WzjOJe2K9GcEwDKiqyjxhO4mzZL15hMTWuVyuRc8FEgtoGAbTPjsQXGA5+2PlAus+03WdOcdUVWWuWxLPLMsy01lWIpfko2Upl+RDXexcIONM13XmExYkfKLeLpimBlWNIFd4DdnCyyir46jqBRiGNn0+XgBS7iU4FB98rn4EA8ch6PsgXK4eSNLCuXZAuOA/DGh5H5B69cDFvhom4O4E2o6vD7Sm0AAAIABJREFUbWg7mDG/00DE3Ao0BV55bpPJJCYnJ7nJZR1zOzk5ySW36d69e5nnNi2Xy9zy3PLgQiqVwsTEBHO5iUSCCxd45UwdHh7mltuUdZx0Nptl3memaSIWizGPZwZquU1Zj19d1zE2NsY8TlrXdYyMjDCPxSd5qlnbhWw2y3yfA1CLxbdywTQNqGoUseQvsHf83zEWewDZ4ltQtRwAS/hBg3+yJAMwoekFZEs7MRZ7CHsn/g9iyZ+jok7ANBc2PggXWMdJ27jg8APt/wtw+GpO54FC+zrAM3Dgvm+eEDO370FYjY317/oTW+qfsV4nuU0bpXppVG50Ikz9/ZIkTZuDdLrnpzs1qP7Uk/b2dnoCTaPrjXRSf1pPo/qQPJnTyZtNp43+JnlurbF1c9Fxo3J9n1nzb9Z//1x00ug73G43+vv7bXqYrg6NZEyns9bWVlvMXrM6qC+3tbXR+Ou5cm0u5a6uroYn68xWp5m+T1EUDA4O0mXu6eo33XdMJ9/lcqGvr2/K+J0rT6cbG8FgkJ7MNJuMuXwfQSgUmnOfkM9msxcA0NXVZdPBfLnfSL6iKOjt7W0wszh9P89UZwJFUdDf3w+Px8PU5hIu1J8KOVubZxvfwWCQhlHMpZ/nai/IvgwTJmDqyOReQiz5CAqV0Xefn/+cXe07JUgAKtUYwolfIJX7E3o6zkNL8COQ5OnTedXr2dqn9VyYqw2c6V4id3Bw8N1wO0DyHwa0fRiI/3HebZ8XiPPsGwLa/w6mpEyZtZ2OJwcaYub2PQArwQzDQLFYpDMrpmkin8/TX+2GYaBQKNCyaZoolUoNyz6fDwBQLBbpUpFp1o4ItJYrlYptFrJcLtOyadaWc8vlMkzTpMuZ1vtVVbXVWdd1lEol2iZN02xlXddRKBTo/WTZze12Q5IkGIaBUqlkk5fP5+ksFmkjKZP7rdcLhQKq1So8Hg8cDgfK5TJdRidttu4+LpVKqFQq9Do5XpGgUqnQMnlBkvutbSQgxzwSeZqmIZ/P23RQrxNVVanjYRgGcrmcTUdWnUzHE6tOSJtJ/DUpk/sJL0gd6nUyHU+mM4ZEZ6TN1WrVNiPSqM3FYtFWzufzcDqd8Hq9tDxdmw3DsLWZ8Mo620nKxEnQNI1mICDySJmMDauOyuUy7ff6MmkzyR1LdGhtU6VSmTIWrDrUNM02FghPJEmCz+ejXLbaB6tO6nVE+tXKA6v9UBTFlsKN8KqeF9Y21tuPRjwhx5sSHVnbTOyHVWfWfpxubAC1TZayLNt0pOs6crmcTSczjQ0ylqabBa+3J1b7QbhsHQukzfW8qVQq8Pv9cDgcNvthtSdW+1JvLxrZTMMwoCgKXC7XFJvZqM0z2Vjr/aQOxJ7P5b1SPzaIzSQgvHC73XC7XTAMFfHUYxiZvPNdx1ZqyrGtR02GhLI6iZHJuxFLPgLdKM3JXtSXC4UCXC4XjT228qT+vULut46FfD5v4wHRAYm5pe9O2Q0sWw8Ej6g5mzxmcIlIhx/ovxCmK9TwXWq1kfW8OZAQzu17AOSMb6BGvkgkQgeMruuYmJigRoSUrYYxGo3SsmnWlggzmQyVW3/saDKZpE6HaZpIpVK2dDuJRMK2rFQoFJBMJqlhzufztuu5XA6RSIS2oVKp2JaVy+UyYrEYvV4ulzExMUENpa7rCIfDtKxpGqLRqK08Pj5ue1lMTk7SMmmjtTw5OWkz7olEwrYMmUwmbeVYLIZ0Ok3rnM1mkUql6PVMJmNb0q4/LrhYLCIWi9HrxWLRptNSqYSJiQlqVFRVxeTkJC1Xq1XaZuIYjo2N2RyxaDRKDauu64hGozbDaj1S1DAMxGIxargIL6zXI5EIcrkcrWM6nbbxJJVK2a6TNhM04oH12M18Po9wOEzL5XIZ0WjUxpN6XpCwDOKch8Nh2sZqtYrJyUnKi2q1iomJCaoTTdNsYwWoLWtbdRYOh2mbDMNAOBy2jYV4PE55QXRobVMsFrPxIpfLIZVK2XhjPTo4k8lMaaM1ZVahULDxoFgs0jKJ57UeD6qqqq3NqqpifHzc1sZ6+zE5OUn7Xdd1G09Im60/ZmOxmC01UiQSoWXTNBvyJJFI0DYnk0lqL0gb6+2J1V4QHZD7S6US5Ql5OVvtRX3IhqZptjaTstW+jI2NUR2RfiVtns5+WMML6u2FlSdEh4lEAoZhwDTNKfaivpzNZm3psorFok2HxGZa7Uu9/bDayGq1ajtKt54nhBdWHUxOTk4pkzaTNlnL0Wh0Ck/qeUB4omsZTES2YCz2IExTZ+LU1kOSZJgwEU78f5iYvBvl0iSi0ZjtvWG1F/U8ITojPyrI2LDyoJ4n9TaTvGeITurfrYlEgtoL09kGDFwCeHpqD7P2J00TcHiBvgsB3xoAEn2vWN8z1rGgqirC4TDz8Ke5QDIP1pzxIkGjJYWZlhnm8lkjOSxlzeeZ559/Hrfeeiu2bNmCvr4+6og0qhf5hQ3sXxqSJAm6rtNTh4hxJQ7m0NAQvZcsSzeSb5omXf4jlCPfaX0mkUigUqnY0rLU12mmOhN5hmHY6pxMJqGqKnp6euhn5Pp8dWKtUyQSgcfjQVtbG70fgE1n1vsbtcn6HbIsQ9M0jIyMoLe3l860kjZZl1CtdZytDWQmJBaL2ZajJUmyya/XyWz9CtScgXA4POWY2OnuJ3Wy6qgRLxKJBEqlEvr7+xv260x1mq7OhmEglUpBVVWaXqr+upU3M/HKWmeg9iJyu93o6OiYlw7J2CC8sfYJia3r7e2Fz+dryCPTNOdd50qlgkgkQpej56PDmcZGJpNBLpezcYx8/3zk11+PRqNwOBx0Sdr6neSZZuxFOByG2+1GW1vbFBs33diYzh6Q6+RHUSgUgt/vnzevrG2wfr9pmhgdHUV3dzeddbeGAtWP97naC2LLyfL5dDaUYK72IpPJIJvNTglZmguvGtkDYi80I4fJyI+QyD2H2mzt9O/F6T6f7/vUNA10BI9HT9flcDk7GvZ7o34lDm5HRwflwnx0OJ2ODMPA6OgoOjs74ff76f0SAJRHgfGfAPm9teQQ9Toxzf2fkb8bfUbvf/c/zjag5++B9r8DJDvvZqqzYRjYunUrHn74YTz44INoaWmxtaV+PM/V55kNIub2PQDrC9z6t9WYNLrXStr6591uN3p7e20vgJnkW50A8hJq9ExLS8sUIs+nzqRcX2er3EbX56MT8jxQy59rLVvbaJVRH49ar5d649/f32/bZVz/HfXPz6UNbrcb3d3dNFVTfb810slc+tXlcqG3t5fKbfT9VuejEa9I2YpgMIhAIDClX2fql9nqLMsyWltbYRiGTZ71eiNeNNJRfb+RuNBmdGiVVy+fcKFRnehLbYbvbNQml8uFnp4e2y7uuepwprERCARsMbfW75+P/HqddHR0NGz/XMdro+umWYvflGV5io7mMjas8q3ZCxRFQXd3N/2ht1BeWXlqdUCtz5P/N3qmvs31dXA6neju7m5ob+bLK2u/ES6Qa436dbo+qr++XycqEslfIZF7HpCkd6Nk+aJWLxmp/OtQHD9Hb+c/QZa9c+pXh8NBuTBfHc7EY1mW0dPTMzWjjiTB9AxCGrwciPwSyLwG6GrNWW1GVYYJyDLgGaw5tsGjAGl+7yEex3HPBcK5FWgKvHKb8spzS+L1WMomToLVYLMAyTsJNN5UsNjkLiUuSBK/s9555GK19tlSkMsjzy0we27TZkD6DGDPBV55bnmMs6WS59Y0daQyTyKW/i0AA9IBjKokDlwy8wzcjk6E2s+BJM3uPvHkAs1zO/Uq4O4C+i4C/IcDiT8C5XHA0OnlWUFidl1tQMc6oG1tTSaH8A9eEM6tQFMwzdqSKesXmXU5jrVcMnvISi5ZVmo067CY5bI+hYcXFxot17KQSf6xzjvJmws86rsUxhkAGrPHMr/rUhy/ZGmeNRd42EbW46yijiGa/BV0Y/9hDAcahqkimn4cft/74fMeMev9B5VjDj/Mjo9CCh4NZF4GMq8D5TCgl2qhB6YJwNwfdmC+K0dxA65OIHA40L4W8C6HiYMz+7oQLB03XGBRoVwuY3h4mHmgeCKRwMTEBPNZmng8jnA4zFSuaZoIh8O2DUAsoOs63nnnHS55bkdGRuiGB5ZylxoXeORi5cWFPXv2cMltOjw8zJwLmUwGY2NjzHUbiUS45Lnl1WcjIyNccpvu3buXCxd42IVMJoN9+/Yx4YJhVpFM/RoVLQEem8fmCkmSUdUyiKf+G4Y5u750Xcfo6Cg3LsyeS1kCXCGg83Rg1eeAVZ8Fej8JdKwFWt8P+NcAgTVA4Aig4wRg2ekwll8JY/UXgP5PA76VgMT26OADBTFzK9AUnE4nOjs7mc8CBgIB+Hw+5nKDwSDdrMMKklTLy8v63HBZlrFs2TLmZ707nU6EQiHm9eXJBZKqiSWCwSBzjvHkQldXly3nMQuQzVms6+vz+ZgfmyxJtZh51jwAajmPWZ5OBtT6rLOzk3nYB08udHR0LGouVMrDSBVexsGasbVDQjr/OjqKO+D3HTvjnbIsIxQKHXwuSBIguQDfSpjeFVPiYsn/dV1HpVKBW2LLsYMB4dwKNAWHw4HW1lbmSxU8zk4ncnnE3AYCAS5xlm1tbQDYxmk5HA60tLRM2djBQi4PLvCILySxagD7GDjeXGAJXn3mdruZH70LwLbTnBVInwHsY9utO8JZyuXRZzy4QOKZ2cShm4hn/i80vTB15/9BgmGqSKR/A5/3fZjpmF7eXOAV47/UQhAaQYQlCDQFEv/FenAZhsElJx7Jacm6viRfKEvw0i2J0+Ihd6lwgeiAdZ8BggsAv3HGo8+IbnlxYamMs8U+frVqArnCGwBwQLIjzIaa4ychV9oFtTo56/1Lqc8AcBkPBwPCuRVoCqVSCcPDw8wHQjKZZB4PCdTiLK2J3FmAJCbncTb9nj17bAdpsEC5XMbo6Cjz2LpSqYSRkRHmjmgqleLCBR6xvIQL1kT6LMAzznJ0dJQmoGcFXjG30WiUS8zt5OQk85hbTdOwb98+LnGWIyMj3LjA0i6YpolsNrtgLpimgWJlD1QtjcURkrAfmp5HqfQ2THP69um6zo0Lw8PDc4i5nR94cOFgQYQlCDQFp9NJk9WzhM/no8fkskQgEOAScxsMBpnHxpI4LR4xtzziQp1OJ9rb25nHRPp8Pm5L3Kxj4AgXeMRvdnR00LPpWcHhcNCDC1iCxEjzGL88Ym55jd/29nbmsbGKoqCjo4N5fR0OB9rb25nbBWuO22ZhmjoKpe2onUK2uDY2mTCQL/0FLS0fgSI1tieEC6zHL+ECj/hrHlw4GBDOrUBTIA4ND+eWR7wPOdGHx0YXHnFPJGk969jY9vZ25nJ5cYFH/LUkSfD5fADYx9zy5gJLWLnAEh6Ph/mPU/LDgUeeW57xkKzlkk2LPGJuWcsl+V09Hs+CdKCbZeSLb0KSZTA/TnaBkCAjX3obmpaG4uppfI8kceNCe3s78zHBgwsHC8K5FWga1lNJWMvlIZMHeMjlVVcim4fhElzgzwXWeuCV85gHhFy77PfKODO1EjS9CAkyzEXm3UqSDN0oQtdnDg1YShzjydsDDRFzK9AUyuUy89g6oBZnGYlEmA+yVCqFaDTKtL6GYSAajTKPuSXnhlcqFaZyK5UKxsfHmcdZ8uJCOp1mHidtmiblAuuYW15cGBkZYZ7zmHCBdWxdNptFOBxmHnMbi8WQTCaZySRyI5EIl5j58fFx5rGxJH6TNRdUVWVuF0zTRC6XW3BucVWPQDPYxquyhGGq0I3ctNd1XcfExARzLhiGgX379jHfl6GqKsbGxkTMrcB7Fw6HA8FgkEv6J9bxSUBtiZt1/KYkSfD7/Vxym7a2tjKP31QUhUuf8eQC6/hCYD8XWIInF9ra2pjrQVEULrljPR4Pl5hbHrmvgVr8NY+Y25aWFuYcI3KXil1gE54iQYK0yOZsrZBmzOAgyzKCwSAXe8OTCzzG2oGGcG4FmgKv2ByR53Z/bB3APuaWR/wXLy6IPLd2LrCElQsswSPPLU/dBoNBAHzir3nI5ZHnVlGURZ7ndumCJxfa2tq4xNzy4NjBwNJ3zwUOCnjmTOWRZ8961jkPuaxl8tTtUpPLWiaPPrPKZi1zqfXZUhpnvLjAQ7e85C7u8bv0HWPBhYMD4dwKNAUSs8fa+chkMszjIYFazG0sFmMevxmNRpHNZpnJBPbHU7GOuZ0uti6fzyMSiSCZTNJctZVKBZFIBJOTk5icnEQ8Hp8ij8Svjo6O4s9//jMSiURTcXuJRMIWk5bNZlEsFpHJZKbEXxuGgXg8bos7JPcbhjFrbJumaRgZGaFyDcNAKpXC5OQkisWi7btM00Q6nbbl7zUMA+l0mr4A0uk0IpEIcrkcIpEIstksNE1DPB5HNBqdcx9WKpWGPCqXy9ixYwey2SzTXJk84iwBIJfLYXJyknnMbTweZ56PFgCXOGlecZYklpdHnOXExASXmNuFcqF2+tdinkWUMFP9dF1HOBzmwoWxsbElEX99sCCcW4GmIEkSlxykiqJwibN0OBzc5PLICchDt436bHh4GBdeeCHOO+88nHbaaXjwwQcBAE888QSOPfZYnHrqqTjllFNw1VVXTZFXKpVw6aWX4tOf/jQ+97nP4YwzzsAPfvCDeR/mcO655+KrX/0qdSxvvfVWbN26tSEXdF3HNddcg7vvvpvOup133nl48cUX8Ze//AVf/epXp/0eXdfx0EMP4YYbbqB99sQTT+D888/HJZdcggsvvJA6mKZpYt++fTjrrLMQDoepjO3bt+NLX/oSVFXFyy+/jA0bNuCSSy7BRRddhNHRUZimiTvvvBNnnXUWNmzYgC9+8YuzOqW6ruO2227DzTffPOXac889hxtvvBHDw8O4/vrrmTliZNmYNcdkWYbT6eQS1806vhCopbFjPX7JOGMdt8hLrizL3Gz5QuW6HcvgVIJYnDO4JhTJBUUJTnvHUuMCSeEmwhIE3rNwu93o7u5mPriCwSBCoRDzwdXS0sL8oAFJkhAKhej59KygKAp6e3uZJ+h2uVzo7u62OQm33347zjrrLGzbtg0PPfQQbrnlFuqMnXbaadixYwfeeust/PznP28o0+1245vf/Caeeuop3H///XjggQewe/fuedXL5/Ph0UcfnfJcMBhEZ2enjQtOpxPXX389Hn74Yezduxf3338/1qxZg3Xr1mFychIjIyMNvyOfz+OWW27B17/+dfh8PnR0dKBYLOL222/H5s2b8Zvf/AZHHHEE/uu//gsA8Pvf/x4bN27E5OT+4zU1TcP3vvc9XHDBBQCAH/7wh/j617+Oxx9/HMcccwyefPJJZLNZ3Hvvvfj973+Pbdu2YefOnXj22WdnbP9LL72E2267reE1kpf3qKOOQl9f37T9MF+4XC4sW7aMucMYCATQ2dnJfJy1t7fT2EWWcjs6OmjcLSvIsoxly5bR2G6Wcru7u5kfQOJ0OplzgWywrB+/84bigcsRWpTL5KZpwu3sglOZPiZelmV0dXVx4UJPTw+Xd0RXVxeXH5IHGsK5FWgKiztOq7FcEfc0tb5utxvRaBSlUgkrV67E1q1b523YSH3Jj5JSqYREIoFrrrkGn/jEJ3D55ZdjdHQUqVQK1113Hc444wyceeaZeOGFFwAAoVAI559/Pr73ve/Z5Gqahvvuuw8bNmzAhg0b8Nhjj8EwDPzN3/wNTjnlFNxyyy24++67ceONN6JUKuG+++7D888/jx/96EdT6lipVHD88cfja1/7GuXC+Pg4ZFnGkUceCVmW8aEPfQivvfYadF2HYRj4zne+gxUrVlAZr7zyCuLxOP7u7/4Obrcb3/72t/GRj3wEiUQC4XAYxxxzDPr6+rBt2za0tLTQU/zK5bJtZvbmm2/G7bffDl3XkUqlcPvtt+Oyyy6j3/Pqq69i/fr1OPfcc3HffffR/jrzzDPxy1/+kll4goi5PXDj7L0ol4Utl+FGS+A4zHTE7cGCCQMB//shKzNPbizFPluMPybmC+HcvgdgfTGYpglN02zlarVKjRC5bjVK5GVPoGkaSqUS9u3bB8MwoGkaXYo2TRO6rk9bJvLq4xh1XacxjOFw2BbzQ76D1Hm+ZQCIx+OIxWJ04M5FB/U6q7+u6zrNk2nVEWlzvc7mqhOS29Qap0XaNFt5Oh2YpolCoYCxsTH6ebVaxWWXXYa33noLF1xwAR555BEcccQR9FShZ555BieffDJOOukk3H333VN0pOs6NE3D008/jYceegi33norBgcHsWLFCmzduhV+vx/3338/jjrqKPzwhz/ESy+9hB07duC+++7D5Zdfjueffx6qqkKSJKxfvx5vvvkm9uzZg2XLlgGohQA88MADuOOOO3DHHXfg1ltvRSKRgKIouPTSS/GHP/wBJ598Mvr7+xEMBnHRRRfh+OOPx8UXXzxFB6FQCBs2bIDT6UShUEAkEkEikYBhGDRrQE9PD5LJJKrVKs444wysXLkSkiRRHT733HNYtWoVzejh9/uxbds2XHzxxcjn8+jv70c6naYzgU888QQSiQQ+9KEPYePGjfjDH/6An/3sZ3juueewceNGmKaJ//zP/8QJJ5yA973vfZAkCbqu484778T69etxzz33oL29Hfl8HuVyGWvWrIGu69i+ffsU3tT3+2y8qFQqlAuNuF4/Nma6bh0b2WwWExMTU7jeaOxMV240NiYnJ5FMJm3XrW2qtyezjQWio0gkglQq1VSbp7ufxMYWCoWGOgLQ0MZa21Bvb3RdR7VapblN52Jj52ovCBcqlYqtTVYdW3U2V50QW261udO9V6bTkWkCXu+RUGS2M5QsIMMBn+coSFJtMqCRDkje2EKh0JBXM+mgUZn0M8l5XCgUGr5Hphsb1uvEkbX2q9UukOetda4vz/ddeiAhnNv3AAzDoGTTNM22WUnTNOzdu5c6UpqmYXh42DY7ND4+TsuGYWBiYgLpdJo6QePj47bNHpFIBOl0GsD+TVfWuEWySYlct24ccjqdKJfLiEQi9P50Oo19+/bRQWp1rAGgWCzaNrcVi0UMDw/TRNSapiESidDYOhI0T65Xq1Xs3buXbtQgG7pIsH79Rg7ifGazWbhcLjgcDrqpiAzi+o0qExMTiMfj9Ho6nUY0GqXXE4kEEokELVcqFduhE/l8HhMTE/Q62axB5BUKBezdu9e2Icyqs3K5jHA4DIfDAUmSoKoqdu/ejeXLl+MHP/gBLr30Utx333245ppr6Maqww47DLfddhvuvPNObNiwAa+99hquuuoqXHnllfjqV7+Kd955B8ViETt37sR9992HJ598Ep///OfR1taGV155BTt37sRdd92FHTt2YPv27Whvb4eqqvja176Gvr4+nHPOOTTGNRQK4dxzz8Wdd95Jl13D4TBGR0fxs5/9DD//+c8xMjKC8fFxmKaJ7du3w+Vy4dlnn6UOssPhoPq65557cPnll2PTpk144403KC/C4TCNBSSGmvCgUqmgUqlQw06cQDJWdu3aBafTadvQEwqF8K1vfQurV6/G3XffjcnJSaTTaWzbtg033XQTNm7cCK/Xi76+Plx77bW44oorcNlll6G7uxsvvvginnnmGXzyk5+kY2FiYgJvv/02PvjBD6K7uxsnnXQSZFmmuY+r1SrefPNN2s54PI6JiQlaLhQKmJiYoLzIZrMYGRmhbcpmsxgeHoZhGHC73ahWqxgZGaH3VyoV7N27F6qq2spkLDSyH8PDw8jlaonsZVlGLBaj9sQwDITDYZujNz4+bjuQgczqk+vxeNx2nfwQIfHXZKMjqXMmk7GFj2QyGYyOjlLuW9tMeEDshcvlQrVaxfDwMG1zqVTCO++8Q5+vVqs2e0F0YLUf77zzjs0+5PN5et0wDIyNjTW0H8TJqLcXk5OTtvjvyclJRKNRmkc4mUzaNnmSMpGXTCYxNjZGr+fzedvmLsITckod2WhptRfWNquqOuX63r17bTwYGxujOjQMA5FIhOY8rn+vEK4T3pBy/XsmkUjA7RqA192HxRV3a8Lt6oIiL4em7efJvn37qA7I2DIMA4qiQNd1G0/IxsP694713Tk8PEx5QXhCrpNVt1gsRq+Hw2FaBqa+ZwgvCA/S6TTdKwDU+tU6qVIoFGz2o1AoYM+ePfR5srGc8EJVVezZs4fai0qlYuPNgcTSD6wQmBXk5QjUNmb09PTQF4XD4UB/fz+N3XE4HOjr67Mlne7q6qJlSZKwbNky22afZcuW2WLsQqEQLZO4Nusv8lAoZKtfIBCgs2GBQAAej8c2GILBIHw+H3VOPR4Penp66Hd4PB5bHYgzQZbXFUXB6tWr4Xa76YYXa4wZ0QFxqkg8E2mzoihYtmwZLcuyjN7eXrhcLjidThiGQTemkJm+jo4O20aVrq4um45aWlpsv2atOQtlWcaqVaug6zqNV/P5fLbNVX6/37YhyOv1or+/n36ny+VCT0+PrTwwMEA3C0iShMHBQXzjG9/AlVdeifPPPx8nnngiLr30Uvz+97+Hoijo6urCcccdB0VRYJomPB4PTj/9dBiGgba2NvT29sLv9+OKK67ABz7wAVxyySV4+umncdxxxwEAVq1ahaOOOgpr1qxBR0cHjj76aPzgBz/AM888gy9/+cs48cQTcf3119M2b9y4ERdddBFaW1vh9/vh8/kQDAaxcuVKAMDmzZsxNDSESCSCLVu24N///d9x++2345e//CUuuOACSJIEp9MJh8OBY489Fu3t7VT3hCednZ3wer3o6OhAZ2cnnE4n8vk83G43xsfHMTAwQOPjyCZEondd19HW1gaPx4NqtYo//elPOP744+F2u3H22WfjpptuwurVqzE8PIzPfvaz+I//+A98+MMfps+bpglFUegM2SOPPIJYLIbNmzdjeHgYyWQS5557rm1Dls/ng8/ng8vlojMt1lzDwBnBAAAgAElEQVTQ7e3t0HXdxn3rJh5ySAHhut/vh8vlgsfjgc/ng67r6O3tpfc7nU709fXROjudTvT399vGQr39sNqLQCCAww8/nNoTWZapnoH99sM6Fnp7eylPG+X1DYVC6OjooDJDoVoMJqkzsR8EpExkkjY3shehUAi6riMQCNA6ut1uDA4O0ucdDofNXhAdWO3H4OCgzYauXLmSyiOxsuS6JEno7e2l/dzIXljjlklsv3XjlyzLNvtBVh+ITsgYIiAcsvLE6XTSd8PQ0BBUVbXZC2vsJeEFed7lcqG/v5+2UVEUdHd329o8NDREbZSiKDZeSZJkk094YS13dXVBURQosgOtgQ+jUB619fvBAtF7S+A4uF1tVCdOp7Phu9Xj8UBRFBiGYeMJsbFWnVlttiRJU3TW0dEBWZYpB8vlMtWHJEno7Oy01TUYDE7hSSAQoHUOBoM2exIIBLBy5UraD/XvUq/Xi4GBAVubiT9A2jwwMEDb6HQ6beP7QEI4t+8BEGeG/G1NYk+WV633+nw+eh2wH6xAdlNanycHJJDrhNjken0i7/rnrQ6EYRhTMhs4nU768iQOgvVlVn+/oijw+XxT6kxmNWRZtrVJlmWqA2ubrDqp1xmRT9pl3eTRaCd6/SES1hcbAOq8WNtsbVN9Vgaye5w8Y91N3khHiqLY+kWWZQQCAbz44osYGBjAxRdfDEVRoGka2tra6AxKtVqlv9pDoRAuuOAC2o5isUgdX7fbjWuuuQZf+tKXcMEFF+Cwww6Dpmn4+7//ezz66KPYuXMnvF4v3njjDVx55ZX427/9W2zevBm6rqNSqaBYLOLwww/HWWedhbvuugtf/OIXEQqF0N7ejo9//ONwOp3YtGkTzj//fNxzzz0YGBjAJz/5SbS3t+O6667DaaedZtPn2rVrsXbtWtp+0zThcDhsPCAv6927d8Pv9+OPf/wjTj/9dNvLRVEUWl69ejXS6TQURUGhUMANN9yAb33rWzjuuOPw6quvor+/H5VKBZ/5zGfwhS98AevWraOOyL59+3DHHXfg4YcfxvXXX4+TTz4Z1157LS6++GKYpolf/epX2L17N44++mh0dnZieHgYxx57LF5//XVa/1wuB5fLRcMlrLwhZTIWiB6IY2vlHflBRnhgHe+KosDv99PnSdnKm/qxMRd7Qfqmkb2oH1v19oKUyfdbN9GQ1Z6Z7EX9dUVR6Nix6oDUuV4HjeyFtc5W+0HaZK0jaaNVJ0Rn1vut9oLwtF6HZJLAai/IznmrDskPb4KZ7AWZvbXWkdiP+jZbeWHVWf1163uCXPf5fDO+B+ptqFXHbcETkcg8hUo1hsWQGsyptKKj5WNQFIetX61jo35skXdto361Pm+9Xv98PffrMyYQ3tS/Zwjq7QUZK9Z+tcqb63tmOntRz5MDCRGWINAUqtWqbZmdFYrFItLpNHO5+XwemUyGqVzTrMWVsc47SZafWJ/vrWkajTMlIGm3TjjhBJx++uk4/vjjceKJJwIAnnzySbz//e/H0UcfjQ9+8IO2sAgrCBfWrl2L5cuX4yc/+Qk2bdqEl19+GevXr8ddd92Fj33sYzj66KPx4osvYsOGDbj++utx0UUXwe/3o1KpoFAoQJIkbNq0iepzYGAA69atwznnnIPTTjsNp556Knbv3o3f/va3uOGGG+BwOHDKKafgiCOOwPe//30sX74ce/bsmTEdGFBbekun0wgEArjiiivw+c9/HmeffTYymQzOOeecaZ9bu3YtduzYgUKhgEAggFtvvRXXXXcdPv7xj+PXv/41rrjiCvzud7/D888/j3vvvRcbNmzA+vXr8fjjj+Pb3/42zj//fHzsYx/Dxo0b8Z3vfAcDAwM47rjjcNxxx2HFihXo7u5GKBTC5z73OXzzm9/Eeeedh927d6NcLtNlXdM0cfTRR8+366egERdYoFQqIZfLg3WIXTabpaENLJFOp5nLNQwDyWSSeZ5qkuOZLHuzgq7rzLlgmiZKpRKNk14oXK5l6Go7HRIUmAc1PMEEIKGr/RPwuJfPejdvLhyId8RShWQejEjfRQTrrxjyd6PPpntmrnJYyprPM88//zw2b96MLVu2oLe3l8l3AzWHJpVK0eV2Vu3I5XKoVqtob29veF+zOszlctB1Ha2trdPWd746AWqHQ7hcrinpwBaiD8MwEIvF0N7ebvvlvRB+AvsPCiBLW+S7rHGnZDlX0zSUy2UYhkFn/n0+n0135AUmSRKy2Sy6urroZgKydF+pVOjsgCRJ9DMyk6woCkqlkm3pvFAswOlwolKpoFyuwOv10FkNXdehqqptNqBSqdAYUrLkb50Zs+qkWq0ik8lAlmW6/F0ul6HrOl0+JDMXhmGgXC7TOEdVVXHVVVfhX/7lX/DRj34UhmHQGF2Xy0Vj1eqzTbjdbui6TtuoaRpUVaUzY6a5f+MImYEjDq3L5UIkEkF3dze++93vwul04rrrrmuKV1aoqopMJmNbDl/wODN0qOU85NwEnKVxoLA/bh6uANCyAmgZBNwtMCVlXuMjnU5DluUp6cDmOs4aXTdNk45fv98/Zx3OJl/TNKRSKQSDQTojy8KGmaaJWCyG1tbWKTN0c9HhdN+jqirS6TRCoZBtdWgh7yHTNFEsFlEqldDe3j6FY828zzQti9Hwt5At7gAgzUmH85E/l2cAEwHPKgz1fQFOZ2hWOSQjSiAQYMYFYL9z29LSMmUGvFlfgdjHVCqFjo4Omx1b6Dh76KGHsHXrVjz44IO244it79L5cm02iLAEgaZA8mSyXm4IBAJcljBIjCtL2ZJUi4Fi/fuQxOgBYCqbxNFZ5UqS/Qx4oh+Hw4FAIDCjzojDKUn7l9esclwu17TLpVaj5vV6a46iVkK+kkS6HIGqlyCZJkxJgrPiQZunGw5DglPxTFnmsn6ndYmsEZxOJ41LI8+Q5VJrnUj7rE6yy+XCpk2b8Itf/ILG0lrDTep1SGCapi2GnYSYWO+zxuVa2yFJEpYvX46xsTHs2LEDmzdvnrZt84HT6aRcWDBME6hkIE1sg3viJSAzAegqAGP/dUkGJAXwdwJdR0NafhIQHATkucXiWWPSWYGM31oV2ckmsZSs5ZL8uaztI+ECa9vo9/tty/ALhaIE0B36FKraPSipYRz48AQTbmcI3Z0boTja5/QEiTkH+HCB9ZjgwYWDBeHcCjQFMpMAgOlA4CGTyGXt3PKSy9pg8Za9EC6YpomKVkQktwejmTeRKcWg6WWYMGs+kQQAEhyyCy3eEIbajkV3cDU8Dn/TOl+IDtauXYve3t6GhxTw5EIwGMRXvvIV9PT0MJe9oPoaOhDfDux+DEi8jZpDK73bcURH73akqQP5ydq/yJ+BlacBy08GnL5pxVvr+l4fZ1a5POwYa7nsdSDB5z0SvaGNGIv+CKqegSQdmMhK0zTgUPzo6TgfAd8xmI9jzZsLrGXy4tiBhoi5FWgKvGJuC4UCUqkUc7k8Y25ZnxtuGAaXmNtqtcolnqpZLpimgVQpjFfH/wevTTyOeH4UmlEBJAmSJL8bDlH7v25WkSxM4PXx3+LVsceQLI7DaCKxu2mayOfzTcd1K4qCNWvWTAk94MmFWCwGj8eDwcFBZnJJny2IC7oKaexp4PV7gfhO1JxY2XJZQzhWF88qybV/hTiw42fAW1uBSrY2uzsNTLOWLrDR4RWaptGUWbFYzJaScC6oj7k1DAPj4+PYuXMndu7caUtvNVcYhoG33nrLlpIpHo/PWLdKpTLr4RxkKZqE4cRiMezatcuWZrEZ8LALpmnSw1zY2nIJTuex6Gw7Gw7FC9M0OMfgmjBNA4rswbL29WhtORGSNPed/4ZhIJFIcIu5ZS2XxNwejNRdrCGcW4GmYBgGTUzNEqqq0hx5LFGL3ywzd25LpRJzJxSoOfnW5NosoOs6CoUCc+eWyJ2Pbk3TRCS3By+PPYbJ3DvvzsrNbI4kSYYJE5HcMF4e+29MZHY21Z+ECyxhmibK5TLzzT7EGWf9stF1neYzbq5iBhB+AXjjp0AlDTSYyf7reBYXfevpxs9LUk3G3ieBXT8HtJn7o1KpNNTtU089he9///sAgLvuugu/+MUv5tWMUqlkk6uqKj772c/i2muvxRe/+EV86lOfwje/+c15jXHDMHD11Vfjf//v/031e8cdd2Dr1q3TPvPII4/g4YcfnlGuaZp0/L700ku46KKLcP311+Pcc8/FCy+80HRfElvO40cva5sLAMViGV7XiRhYdjlcjiBqDii/WUyH4kd/18UItZ8NWXbN/mCdjFKpxNyWEy7wsAs83usHAyIsQaApeDwerFixgvnSRUdHB5flkM7OTubLmrIso6+vj7khUBQFq1atAsB26cnj8WBoaIi5XK/XO28upEoT+Mvk71FQs5AlZV71kSQZBTWLN8NPwuMIIOQfmMezErcYuN7eXuZccDgclAssYeXCvGGaQHp3zbE1VNts7bwgSQAcwJ6nAHcIWH0WoDgb3Cahu7u7oW4nJiboDOlNN930bvXm1geSVMsjWv+Mw+HAzTffjHXr1mH37t245JJLsH79ehx77LFzkutwONDS0oInnngCb7311pyyWwwPD0/ZBFkPRVHoONuyZQv++Z//GZdccgkef/xx/Nu//RsefvhhW6qoucLtdmNoaIh5zG1raytaW1uZjglJkmieY2AdXM5lGJ38LsoqOcSDVRtqdXY7QhjsvRZ+7/uakqIoCl1xYR3XvWLFCub2htiFpR6SAAjnVkBA4ABD1Ur488TvUKg0HzMnQUKxmsfr4d9i3YoL4XH4Z39IYCr0KpAdBSJ/hhR7EzBrM0zS/8/ee8dJUWXv/+/qPLEn52EYBiTnIAgCgi5JAUEyogi4CIIgIsiiixJEBREcVFhEssKygosiKphFJMcZwgQmMjmHjlXfP5oqumFAQs+Pj/vj8TUvqa7q2zece+6pc899jt4PItpBUFMw+LsasLZqOPs5WCrloOg/xasbDiNKcDGvkoTMUmb0b8LonvU5npTPjHVHsIrf0fqhx5m/cDHnzp3jn//8JxaLhQ4dOjB37ly++OILdu7cqWSBmjBhAr169eLf//43ycnJdOjQgZMnTxIdHc348eNZsWIFu3fvxmw2M3ToUKZMmcIbb7xBeXk5GRkZJCYm8sILL/DMM8/csN6+vr5oNBrMZjPFxcVMmzaN9PR0QkJCmD9/PsHBwcycOZMLFy4giiL//Oc/6dmzJ0ajkaeffpqlS5eydu3aK11ns7Fu3Tq2bNkCwPjx42nVqhVffvklFouFFi1a0LNnT+X5vXv38t5777Fjxw60Wi3Dhg1j0qRJfPjhh4oB4m4D8q8CQRDwMDQgLmoeBSVfUlj6AzbRERYk3KaRKyGBJKFWe+Lv04kQ/4FotSHXHDa9h//7uGfc3sNtwWKxUFRUdE22oTuFTAXmbg9uaWmpkmHKXfWVJEeaS51Oh4+Pj1vKBMfWkEwF5nzK/k5RE/2Tu8q9WVkQRTuZpYkUV+Uq8bS3A/l7ZdUFpJecpkFg+5uKhZMkBy2cnKzCXTImiqJCK+VOWZDjr/39/W/LK3f9gm2QdxIStkLqH1BpcoQJOPeH6t8QGARNHoMGA8Az2OG1zTkC+Yk3bdgCimG78tn7OXShgPd2JTK0cwxztxzn9eEtaV3Xj3lfHOXgH7+z9pP1zJw5k7Zt2zJp0iR+/fVXqqurSUtLY8eOHZjNZvr3789jjz3G448/zqlTpxg8eDDHjx9Hkhz0h/v27eOTTz5BkiSeeOIJ+vfvr6S9XbFiBUlJScyaNUvJCOdM5We32/nuu+9IS0vj+++/JyIiggYNGrB582aio6N5//33+fe//80HH3zA448/zoULF9ixYwfHjh3j/PnzPPjgg9hsNrp168aPP/7IuXPnFPaT1NRUtm3bxubNmxEEgYEDB7J9+3b69OmDh4cH3bt3d+m35s2bK2lbo6OjSUhIoGHDhgqxvpzhbsKECbctHzL9U2Bg4DWx5LcLSXJQgVVVVREYGOhWXS6nn5Xnr1YbQGjQSHy976e4dB+lFcewiuWAeNkBe6PfdtB7SZf5azUqb/y8WuFvfBgPj/qohGt3Em4Fcszt1Vn07hSyXjAajW4t12KxuF0W7hb+2rW/h7sGURSxWCxuf5u12Wy1EsNqt9trpVyr1VorqQVlvlZ3QpKkWhmzW5EFq91Eeskpt+weOhZMiYzi09QxNsVDd3NGZW3ImMwR6W5ZkCQJs9ns3tg6axVC4mdwaBOUV1wZC0G48m8JBxNCfi78/DGk/ozQaToENYHc4457as0ND4Jdjc6Ng/EzGgj1N2C22zFbRc5nl9GlSSioYN7I1uSHeXHx4kV27tzJ3r17SUlJIS0tDUEQaNmypQtXd15enkKr5rwQX7x4kaioKKKjozGbzcTGxpKQkABAx44dCQ0NVfiOc3NzmTdvHgCRkZHMmjULgKKiIiVr3Lp16zAajZw6dYpLly7x1ltvkZaWRlFREfXr18fX15cpU6YwZswYnnzySSVrn5eXF8899xwrVqygRYsWygtQUlIS8fHxAKSnp2MymZQ2WK1Wpk+fTm5uLn5+frz66qs0adKE/fv3ExwcTOvWrRVDPDc3l5dffpnWrVvTr1+/2/Yu1pYury2da7PZrjGWVYIGT8N9eOjrEuSfS0XlCSqrE6gyp2G1lSJJdiThSvskQJBAENToNL546Ovg5dkYb89W6HURCILOLQa5JEkK97c7Ievy2ir3f8FLfc+4vYfbgl6vJzIy0q1eW7jyNu7umB9/f38kSXJrfQXBkQ/d3VCpVERHR7u9D/R6PREREbVS7s3JgkSpKZ9yUzEqQbgVu+i6EAQVFZYSSk15N2XcCoKAv//NcVTeWj3+IrJgM8HJtXDwU7BYXY8UXz0egtONtPNgegMefNlh2ArCLRm2ADpNDfRp8tosqKguK8GUn4pOp6NDhw4EBQXRsmVLWrVqxaFDh1z6wOGxuzmvmvNC7bwTIggCdevW5dlnn8VmsyneNbVazbBhw2jfvj3PP/8827dvp0mTJgiCQKNGjejatStmsxkPDw+CgoKIj4/n0KFDrFy5kh9//JF58+ah0+nQ6/X07duXNWvWEBAQoMT3+vn5KVkAmzRp4iI3Go2GIUOGUFVVhU6nw9/fn759+xIfH09ISAhPPfUU3t7elJWVsXDhQkJDQ5k7d+5N90VNkOevu2NufXx83LqLIeNKzO21UKn0GPR1MOjrECD+Das1H4stH7MlC6utAFmoJUlEpwlEp49Cpw5CqwtGrTI43XcXN69aGXd3Qq1WExUV5XZdrtPp3C4Ldwv32BLu4bbxvzAB7hS11Qf/i+VKkkSluRCbZMGdBOx2yUa5ueC2qMHcif/zYyaJcH4n/LHFYdjeSrEqIDcb9i+FyPvBI+CWjdurodOoiA33Zn9iHlWVFqavPsDJk6cJDQ0hKCiIBx54gN9++42ioiIA9u/fT1ZWFmlpaUiSpIQuVVdXu1AixcTEkJmZSVZWFnl5eaSkpNCkSc0HggwGAz169OBvf/sbnTt3djF+dTodEyZM4JtvviEpKYkmTZogiiKdOnWioqKCY8eOcejQITZt2kSvXr145ZVXOHv2LGazWaH2MhgMDBo0iE8//<KEY>+++oq3336b9957j9dff53Q0FAmTpzIlClTNP88VRZqfDI/htiw9O0f1Eep1o4xMkprG2LjkI4YOWGZoxHbico1ILDf6WJL6kkw72Z/kYajJeOXFTAZ4dIrYNSZbd4iCEKXyJgqC8e72BiOUyxzVq2l6vttiCFm4k8bTepFk6n8YVu7PwTGvLyC9Zu+5MtrH6BflH/nOiUlhWnTprF48WIef/xxysvLMRqNzJ8/n6+//prPP/+cFStWYLVamTJlCj///DPjx4/nyiuvDAoEiomJ4ZJLLmHDhg2AnyeJiYls2LBBozN+/HjtfrPZzIoVK8jJySEnJ6fVqYGKiooKXnrpJa1qmVoRDuCFF17gwIEDrF69ukMZBGLMItemhJBhEZCOYUiZYxMJPWkM3gYHUWdNwTZ0DFJDLe7SAvKWzCVv8RU0Ze9Gam6kx6U3Uf3Rq+QvuhxkCXPPNDJtAjEhBiZOnEhtbS379+9HFEV8Ph/l5eX84x//YNKkSbzwwgta0Jbb7eb5559n1KhRvPjii0EJ/AVBoKGhgbFjxzJo0CD++te/Yrfb2+y7eqKiwu12t6un1OqIqj9+SEgIffv2Zc6cOUydOpX+/fuzdetW1qxZQ9++fTnjjDOC+jRp0iTND/bcc8/F5XKxdOlS/ud//kcrNAD+ILbk5GRMJhMRERFBPv2RkZHs3bv3qO9DRUhIiO5G3YnO36PRtVqtugfxCoJAaGio7nRVXa735tIfgW63hG4cFwJ9ibrCd1HvydWVPrdw/P6bYWFhLFu2jOuuu461a9cyc+ZM1q1bhyzL/OUvf+Gll14iNDQUk8mEy+XinHPOQVEU+vTpw/LlyxEEgZ49e9LY2EhpaSlPP/004eHhFBQUsH37dg4ePEh1dTUGg4H+/fuTlpbGBRdcwCWXXMJll12mKcfQ0FCuvfZaVq5cyeTJkxEEgZqaGrKzs1mwYAGSJFFaWkp1dbXWd4fDwS233EJ2djaKovD42xso88qtgnUEswk5oDKR1NyMaOlYJggfsN/pYfxZF2FFQFz3MtjraFWEACA+Gi69Eia0v2vbVT63cGKy0MMairGT/rYAUScPoejdjxFNRgxmM6VffE3yheeBICA1O6ndsw9jeBiCIGLrnUpsQgJJXgO+Hw+jGEUUrwRHZODss8/mxRdfZOPGjZx99tnce++9TJw4kY8//hin08kNN9zAvHnzuPPOO0lNTQ0KIiwpKUEQBC6//HJGjx7N2LFj6dGjBykpKaxfv55x48axcOFCVq9ejcFgoKqqivz8/HaDFgMRGRnJ1q1b8Xg8nHXWWTz77LNceuml7N27l3Xr1rF06VIKCwspLCwkPT39mEbugDADN6aaeSTXQ73UvnuCMSqGyL9cSOjAYeQ9MJ+o08/FGJtASEoGqYuew2ALx12YgyUlAxno++wnuHL2Ub72KQaKTqYPz0Ddi16wYAGvv/665oJhNBq59957OeOMM7Db7bjdbmRZ5vXXXycxMZFly5Yxffp0fvrpJ04//XStT7GxsXz22WfAb9Wk2kJKSgr5+fnAbxXf1ApvLWEwGDAajdq86Nevn7ZxofqgiqJIdnY2u3fv5ocffiA/Px+j0ciZZ55JWloaZWVlKIpCdnY2ycnJlJeX88wzz7B8+XJ27tzJY489xtSpU1tVKFPh9Xo7vBvbVT636vi7Yu3RW9d0pc9tV6zrfwS6jdtuHBe6yue2trYWt9ute9qurvKzLC8vx2q1Href1pw5c3jkkUfo378/999/PzfeeCP//ve/SUxMxGAwEB4erh11S5LEq6++qrkSREdHYzQaGTduHH/961+RJIkVK1bw9NNPI4oi559/PpdccgmNjY1IkkRoaCg33XQTXq+XdevW8fe//51Vq1ZpfRk9ejQvv/wye/bs0fifnJzMokWLALjyyiuDcqCGhYWxdOlSHA4HVZWV1IRYqatvnYrJGNsDqcaO7PMhiCKeshLCho/sMI/y3F521SnkpU1iwh0D6LnzS8jaBY76Ix2JgJNHwLjzIX0AiuHoZuJ/m8+tACRZbUSHWKj1uDtl4saeMhJFUbBv3+FPR3beOcSdOhrJ5cSakox9644jOdcM9LzwXE7unUFUvRu5yYWy9RAZKTGY+vj9KUNCQli4cCH/+c9/mDt3Ltdddx0vvvgiPXv25OabbyYhIYH77ruP9evX+3/MPP44cXFx9OvXT9sJt1gszJ07V/Prvu+++1i1ahWrVq3i1ltvpU+fPsTGxnLqqafy/vvvc+utt7Y5rvj4eIYPHw74j9QXLlzIv/71L1auXMmMGTOYOXMmX3zxBenp6Xz44Yfae7ztttsYPHjwUXlmEGBctIkbeim8WOylTlKOeoQZktyHyNPOpXLjKlLveQHRaMJ1eD+GiGgq33qO5Jsfpvrj1zEnJGPLHIc1LYMJYV4SjLK2sz127FieeeYZysvLMZvNnHbaaXzxxRf069ePVatWMWzYMCwWC6tXr2bNmjWkp6cze/Zsli9fzrBhwzAajeTl5fHrr7+SlpaGzWY7avaXM888kyVLljBlyhS2bNlCQkICSUlJbd5rs9mwWCwUFRUxbNgwRowYgdvt5qOPPiIlJYWCggJGjBjB2WefjcfjQRAEnn/+ebxeL5mZmdx0003MmTMHgOXLl/Pcc8+xdu1aHA4HDQ0N3HLLLSxYsIC0tDReeuklfvzxRxwOh2Z8gz9Dy8iRIzs0H7vK57aiooKQkBBiYmJ0oylJEmVlZURFRemaDqzb5/bY6DZuu3FcMJlM9OjRo0t8Y/XO3Qd+37Cu+AV9ov7BsbGxLF++nPnz59PY2EhRURGXXXYZoijS2NhIdnY2VqsVURRJSUlh4MCBbS5qoigyf/58Lr74Yvbt28fIkSPZtGkTYWFhfPXVVzgcDjweD6+//jqLFy9m9uzZWgql+vp6mpqaCA0N5fLLL+fqq6/mzjvvJCUlBZvNhsPhICIigqeeeor33nsPg8FAeXk5TqeT9PR0ZFnG7XbzRbMXvyUVfNxrSc9Aam6mce8uDFYbzVl7SJxzbYd5pABuCd4tlPjJlsHIU/szaJyDMMmFUYQ+sWEYIqIR1CO6Yxw3q7KgN2JiYo5bvuIsofQNj2S7vbJTvmKi0Uj86WPoccapwG9lKgVjGINuu0n7DPxBa0OiY7DtrkLy+qBO4uLzTsM06bcI9TPPPJMzzzwTRVGYMWMGM2bM0K4pisLYsWODsnao96n/ttlsLFq0SDN2U1JSePDBB4PKacbGxvLoo48elVeZmZmMHDlS6/uwYcN4+umng9I1TZ48mQsvvLDN1FjHgtkgMDXBRJhRYGWRh0KPghgwrwxWGyG9MvynA6KBmPNnUfLcfUgOOykLHqLqo7XILifREy/CnJhK7N+uomb9s4Qe3Mq04YO5dOoUysrKSElJAfwG+vXXX8/bb7+NyWRi7ty5PPnkkyxdupSBAwdywQUX8N5773HRRRdpAVizZ89m9+7d7OIZSLwAACAASURBVN+/n5kzZ/LUU08xcuRIBgwY0OaY+vTpo+3OZmZmMnXqVB588EGioqK4++67CQ8P13gTFxdHWloa4Ne5559/Pt999x0XXnghUVFRLFu2jOeeew5JkrjllltISUkJeocZGRl4vV5KS0sBmDVrFoqisH//fvbv38/IkSMJDw9HkiROPvlkevbsSUZGBrfeeitPP/00GRkZXHLJJdhsNmRZ5quvvmLq1KnHfG+ArkULAnEi87c9iKJIXFyc7oaiKIokJCToTleNpfn/wedWUPROlPYnQ1fWWW55n160OvPMtm3bePjhh1m9enWrNDnH+93ACfVJD952BQ//iHGUl5fz6KOPsm/fPqxWKxdccAHz589n7969XH311VrOwfDwcFasWNHKuL3llluYOnWq5q/4wAMPAHDrrbeybNkysrKyiIiIYOnSpSQkJPDCCy+wdetWTCYT8+bNY/z48dxwww3MnTuXMWPG4HK5mD59OtOmTePaa6/lgw8+YN26dXi9XmbOnMmMGTPYsWMH9957L1dddRWzZs3S+PFhTQOPl9YgoVC5/nUMJhPx02cBYP/sY2o+/dg/llPHkjDzCmRFQRQEnIX5FD3zOP2feSmIX7Lir9uUGRrCLGssK/a6EPHHjJkEAaMAZgPcmxlKn0gD4u8gI10pCx8U/MqzB/Ygt6M/TqRPACZR5JF+IzjpX9uQD1f4q8WN6oP5xgsQzMYu13v/jfpCVuCnOi//KvGwp1HBpfjLsIiCEJTxpM2+4ZdREYg2CkyJMzAzyUysWWwli7+XDjveZ0pLS7nxxhtZs2aNVsjg91oby8rKuPHGG3nppZe0wDi9vrszfTrRcfxeMv3f2KejXVcUhbfeeov169ezbt06zf0k8DrQ6T4dC93G7e80gfWk1Zlnusq4BYLy4ek1DrVogfrXHp3OjkP9U32JTlQRqnxQ/99y9/ZE+KEowbkR9VLyLWu96yXrftoSX9Y7eajEjkdRtMVd++6Az9R/t/VZy2cU4HSbhb+ZY3guy6X586rGhU+G+QPNnJcagkHsmKINVKZ6zbNAXhyvLFQ7ndyzcwsH6uv85Yl11BeKonBqYhJLam3Ir32HIMuggNAjAsuiixDjI7tknunBW0EQ2vWzPNFFV31n9T7YXOPj/QoPh5wK7iNqTn2TLWUVwV9sO8IAo8INzOhpJjPSoPUVfpu3+s4z+ag67Hh0rizLyLLMxx9/THFxMQsWLDiqbuoobwP7rc6Jtui8/vrruFwuLbXbscbRGVnoDG8D+6onb0FfWVDbLfPy6jHPAteeztgux5KFP8K47XZL6MZxQU3z07t3b10jNu12Oy6Xi169enVamI+G6upq3G53UPTuiUJRFEpKSggNDdW1bKMkSRw+fJhevXrpmlrK5XJpR6V6HmepstA/uRexRpFS77HTlnUGA6xmcu0yLV0e1LeY45CYoHDMqlMqqqqqNFnQ6/hNUfy7XxaLJSgFU2cQZ7Vyae/+PJO1k0af7zjCy9rpGxAdYmFOQm/Ejf+LEuCSobg84PG1/3AnUVdXR319va6++Iri9203GAytUl6dKEpKSrBarcTGxjI10cz4WBN7G7z85JD5uUGi2qtQ7wO/PStgECDaIJBhFRgRYWRkpIF+NgPGgAIjkiRRVFREXFxchzI4dBSSJJGbm0vPnj11pavqBTXrgV6ora3F4XCQkpJyVNet2bNnd0onV1RUIAiClntXL5SWlhISEnLc87ctdKUs5OXlkZCQoGtaNJfLRUlJCampqbpnjvi90W3cduO4YDabiYuL6xKf265IMaYGZulJVxAEoqKidE/HovpThbSTIut4YTKZiI2N1b2/qiwIRhN9zSaKvVKHDc1jIVQU6GsOYXOD1KYvqiBApUvBJ/tdFDqCiIgI7YRAL6iycKLzYVxCMoWN9azLPYSsnLhfsKIoWA0GruozkL5bDuMttP+WZQIQdDOh/bDZbBiNRt31QkRERJekJ2o5f8NNAqdGmzg1Glw+hWZZoUk+sqsnCBgBm0EgzOg3dAVaZ8dQ/Sz1TjdnMBjo0aOH7nS7Si90pSx0BaKionT3NTUYDF0mC/Hx8e2WWj9emEwm4uLidJeFPwJ//hF04w+B0WgkKipKdyM0ML2Q3nSP52jjaBAEIShIQy+IoqgFhuhJW31nXUl3fJSN7c1uPPx2pNu49xfcpUUIgoApMgrLkGGIHYgcVoDh1hBsHiO7ft6KHGIjItUfTNNQmovi9RCWNhChxVDUMq29evWid+/egL+yks/nY8iQIZSWltK7d+92ZaGxsZH9+/cHlQLdvn07JSUl9OvXjyFDhgQt2D///DP9+/cnIiJCO9bbsWMHJ598MmazWSttmpqayqhRowB/qduffvqJ2tpaMjMzteAekygyLa0vdreLT0sKNP9bAG99Pc0VVUT2C67C5a1voLmomPD+/WjKyydiQD/Af3xuNRiYnpbBhHwn3s9+8dMKOApEZ6c0NQep3vMsLCxM93mmzl8Ing+qwWo1QqggEkf7R7Bt9UkUxaCjVz3727L0rh4wGo260xUEQcstrvd7UzNF6AlVxlToRV8QhC6TBbXQj57oCln4o/Dnz9TbjT8E6iKu9+QK9IPrCrp60+4qHvzZ6Ko0TwkLZVSoOShBftXbr9Pw3Wbchw9T/vYbFC65E9nrOSpNWVEIFwXGh9v4sdRN1ger2PPSIiSXP5F/xY4vKfr2fWQFRCG4Kq8gCOzcuZN77rkHt9uN3W7ngQceoKqqiry8PO6+++6g5PiBaGpq4h//+Af33XefNraNGzfy+OOPs2/fPu666y62bt2qXdu+fTsXXXQRxcXFGh/27dvHypUrAfjwww+599572bt3L4sXL+aTTz5BkiSWLFnCypUr+e6777jmmmvIy8vT+hAVYuHa/icxs3c/Qg0GZEVB9vnIf/s9it//pFWfnaXlFKx/DwGFim++py7rAIqiEGsO4do+g5hZDry33e9+cGTR2mkv5Pwvn2PO/65m+vy5PPXUU9TX1x/1nbTE999/z5w5c7QCBI2NjVx88cVBRRZaYs2aNdr9baG6uppbb71Vy6fscDi49NJLufTSS5k5cybLly8HYNeuXVx//fVMnz6dFStW4HQ6O9V36DqdAF2jF6Dr9aPeNLsiK0lX9fXPJgtdqcu7or+/N7qN224cFzweD2VlZborr/r6eqqrq3WfXHV1ddjtdl3pKopCdXU1DQ2tc7ueCNR65GrCd72gvrOOlPLtLN3S0lJkWSbWZGB2fBS9TIYgAzd6wl/pecPfSbv3YVzFhTgPZQPgczhw5uXirajQfEF9DfU05R0ms66KkCaBn6slDAJ4mxxU79sa9N2yAokhMoX5uRw+fBin04koisydO5fs7Gy2bdvGO++8Q3JyMqeffjp5eXnk5eWRlZXVpuzeeeed/PTTT9rOhdPp5Pnnn+ehhx5iyZIlXHzxxbz77rsAfPrppzz11FPaYlBVVUVVVRUrVqzQMl3k5eWxePFili5dyqWXXsrnn39OXV0dlZWVPPHEEzz55JMMGzaMTz75hIqKCqqqqgAweLycaQrjpsHD6BseSf2efdT+tBuOLGiKouCqttNcXIrviGEnmEwkTjyH4o0fMSQiilvSBzNpXy28sQVqmxECdpsbvB7MooH7J1/B3Xffxeeff8769es79d4dDgefffYZn3ziN7hlWWbnzp1UVFS0O8+ef/55rdJYS+Tn53PXXXfx/vvvaz8+CgsL8fl83Hzzzdxxxx3MmDEDWZb5n//5H0455RQWLVrEl19+yX/+859O9V1FVVVVp436Y0GSJC1Vnp5Q9UJg8QM90BV6QVEUGhsbqays1F2X19TUUFdXpytN6DpZqKio6BJZKCsr+1PIwh+FbreEbhwXRFHssvK7XeFbZzabMRqNuvfXYrF0WWlFvfnQVSWTA2VBFASG2yzc1COaf5TXUCP9trDJigIoCCYTiiDgs1dT/I9HUWQJ2eWkx2VXYRueSf69d5Bhs7KtsZkdw88nZsKVAPS+8BryN71J7JBTtR1IUYDv17/Al4d+QhQFhgwZwuLFiwkPD+fuu+9myZIliKLISy+9pO202u12fvnlF0466aRWPnZXXnkljY2NrFixAvjNcB80aBAA/fv35/333wcgLS2NVatWMWnSJMAvC7/++it79+7VSpTedttteDweDhw4wFdffcXcuXOJjY3ltddeQxAEPB4Pzc3NREdH88033/Dhhx/y/PPPs3z5cnw+Hw/edjcDQ53M3vIzoy+aQvYvezAKArU5ueQ8vxprUiKeOgd4fUSazQwZOhSnvJ7ZpR4yt+3Fu7cIQZaCDFsVYSEW+p8yHPO4M5g4cSI//PAD1113HXl5ebz22mu43W6uueYa+vTpQ2FhIa+99hp2u50LLriAiRMnAjB+/Hj+/e9/87e//U1zR7BarbhcLjZs2MAvv/zCuHHjmDx5Mh988AFFRUWsXLmSO++8s1XgzsaNGxk2bBi5ubnaZzk5OfTq1Yu0tDRCQkKIj49HlmVuv/12JkyYgMViYfz48fzyyy9cfPHFnZbdrpy/XZErtCvodpUuN5lMuvuEArrHIqjoKlmwWq1dJgtdVX632y2hG38aBKY78Xq92q6Voih4PJ6gXazA62o78Jec1+tFEASio6MRBCHouqIo+Hy+Vu3Asost25Ik4fP5/MEvVis2my3o+yRJCjrqlGW5Q+3AMZjNZsLCwrRUMoHXZVkO4kFLHrXFE0mSkGWZiIgIrFZr0JjVMbbkmTrGwDEH0lPvD/TTasmjo7U9Hs9ReQK/BU2o7709nrV8vqWcBPLIZDIRHR2t8cQoCEyIDuPepFh6GkVkRaH6w3cpffpx8pfciSkyitD+A6j74j+EpKaT8cQKes5bQOVrq/Hk5RLX1MBTy19m+IJncJrDUQPR4wafgigasGdtJTTGX20pwtdI1ufvsGbNK6xdu5ZffvmFrKwsBEFgypQp5Ofnk5mZSf/+/QkJCWHSpElkZGRw+eWXYzAYgsYkyzIjRozQFiJFUXA6nUEpiMxmM83NzQAMGTJES0Kv+m9mZWVplbVUHh06dIgFCxZQUFBAUlJS0Nz4z3/+Q1ZWFlOmTGHatGm43W4eeeQRdu/efaR6l8LKW+/j6piB3NrrZIZFxrJoUCZRP/zMxVfNYfGK5Vx62SxSLaE81SeTp8ReTAhP4aenXsO3Ox9RUaC9oDGrGcOQNOw1NXzzzTcMGzaM+vp67rrrLlJTUzn99NO5/PLLqa2t5fbbb8disTBx4kQWLlxIUVERAIMGDSIjI4MPPvhA8wO0Wq2sXbuW7du3M2XKFN58800++eQTzj77bHr06MGUKVM0/0aVRz6fj9tvv51LLrkEo9Go8SgrK4svvviCuXPncvHFF/PNN98AcP7552OxWKivr+ebb77RfkyocyFQdtvTH+APTrJYLG3qAxUt50ZL/dBSf4iiSFhYmJaRRNWBR9OpbemLljpUlmWio6Mxm81B+qItHauO+VhtRVEwGo1akGUgjwL7rI65PZ601BeAVmFRnTttrTOBY2hL57a1bqiFftS2+p0+n6/VGNtbZ1ryRPW5NZlMbcqJ2m6pL9qSi8DnJUkiIiJCM8iPts6o7ZY8aWttFUVRW38Dx9iRdaY9HgTyQd1YabmudGSdacmjwO//PdFt3P4fQKCS8nq95OXlaccZPp+PnJwc7ajQ4/Fw+PBh7ahdURSKiopobGwE/MJaXFxMVVWV5lNVWFio+ciBP6WK3W7Xni8vL6ekpCToulrZRlEU6urqtNrk4HchUK+r7YKCAm2SNDc3k5+fr42psbGRwsJCrd3U1MThw4e1SevxeMjLy9Oe93g8FBUVBV0/fPiwxgNZlsnPz9eOknw+H/ZqO83uJmpcdVQ326lx1eL0OfHJPi0NVH19vcaTsrKyoKOzQJ4pioLdbg/yPaysrAw6wqusrKSoqEgbU0NDA8XFxdr99fX1lJSUaPerY1bH6HQ6yc/P19oul4vCwkJNEblcLrKzs3G73doYi4qKguSisLAw6HpOTo4mF5IkUVxcHDTmoqIiHA4HAIKiMAIfK9LiGWA1EzlgEGJ0DL66WpJuuQuMJjxVldR8+SmHFlxL8T+fwFxj59Fxp5KRMZhp503gm4/fIWH4XwhExuT5FGxaj0/yIQKZFgdFhQVcdtllXHLJJezatYu6ujoUReHtt98mOjqaTZs2cfDgQZqbmykoKNDk7uGHH2bQoEEMHjyYDRs2aHKiGjZut5u8vDwt56IqJ4G7JYE8liSJXbt2aZHRkiSRn59PYmIiX375Jffffz9XXnmlNjc2bNjAk08+yUMPPURERARGo5F58+bx2muvccUVVxATE8P27H3k1Fcy1dsT38c/E36glMznNxNzsJTZ1Rau31LN5eVm4itdJD72Mb5nP8VW2cy+utI2d2s1KArfl+cw4rLJ9ExMpH///txwww1kZ2eTn59PRkaGVt508+bNiKJIbm4u6enpbNiwQdt19Xq9TJgwgfXr19PQ0EBsbCw5OTls2LCBU045BavVSkZGBps2bSImJoaQkBDcbjc///wz7777Lu+++y6fffaZxhOVZm1tLQCnnnoqzz77LGvWrOGRRx7h7rvvpqysTJtvt912G/Hx8fTv3197vq6uLui91NfXk5ub20pfqMZcU1MTubm5mj5wOp38+uuvWtvr9VJUVKTJhdfrpaCgIOj6oUOHgvRFYWGhpjMlSaKgoCConZubi8Ph0PRBRUWFNmZAG58qq+qYA11g1NK+gNZWr1dXV1NYWKhdb2ho0J4Hv74oKSkJ4kmg/gjUF4FzIVC/HD58WNMPkiRRWFio6Q9V9gP1R15enqYfFEXR9IfaLiwsDNIvRUVFQetKZWUlNTU12hgrKyuD5KaqqkrzfQd/KjLVXUpRFBwOR5AOra2tJT8/XzMenU6nJhfQep1pOWav10t+fr4mFx6PJ2jMgeuKqj8C5USSJEpKSrQfyyrPampqNJ7k5uZq64gqB+qYVTeVwPdeUlJCZWWlNsaamppWchIo+3V1deTl5Wk8U9cZdcz19fVB772xsZGcnByNR6qcqNfdbjc5OTnamFwuV5Bc/Z7odkv4PwCDwaD9EjOZTPTp00fbnTIajfTv3z/oet++fYMS/aelpQW1U1NT8Xq9FBcXk5ycTHp6etD3BeaSFQSBnj17Bv2iT05ODro/Ojpay7xQV1eH1+sNuic6Opro6GitD7b/x955x0lRpP//3T05b47ssiwLSxRYULIICBIOBDxFBQQ9ETGcGE7PCKd4IIcoxwkqIopExfD1MBENhBUERECSS9yc0+zMTurfH0O3M8uyLDB7nr/j42tf0tPdT1c99dRTTz311FMmE6mpqcq12WwOmm2aTCbS0tIUI0ROdVNWVkZUVBQ6nY7mzZsrPNDpdLRq1Up5XxRFWrRooXg4a7w1ZPlOc/SXY+Q7ivFKfmVp1VlpZo7lqqh2JMc3Q6PSKHVOSEgICitITk5WPIDAOUuyMTExyn15dhwXF6c8b7Vag3bz2my2IO+u2WymVatWSp2MRqNSB/Bni4iPj6ewsJC4uDj0ej1t27YNavdAnmg0GlJSUpTv15UTlUpFcnIy4FdohYWFNGvWTOG5IAiYLBYsokiaTsst1/XG1rc/n4ZZ2Pray1y/aCk1GjWpo29i2LjxpAoSrpxCjp4Gw+gn6TqwglPrl7P71T/T4+l3lXqGtc5Aawmn9OAO2iVE0D9Jz4qWLVmwYAFGo5GsrCzatm1LWVkZr7zyCosXL2bNmjUsWrSIuXPn0qJFC5xOJ0VFRTz00EPcc889Cn91Oh2pqank5eUpctGmTRvUajU1NTWYzWb0en3QYSgtWrRAo9Hg8/koKSlBr9crA7xKpWLjxo2MHDkSURTp0KEDlZWVWCwWvvvuO958801effVVunTpotA7evQoSUlJbNmyhREjRrBz507ytB7G71tDUWEhOY4KPti6CamkGuePJ3Hnq6kpOIm31gUOF6hVePBh1jSwdOvzgVqkV5/evLtsGWvXruWjjz5SBrTy8nJWrVqFIAi0adOGq666ih49erBkyRIefvhhYmNj+de//qXIyYABA9ixYwfr1q1DkiQlJdq6devYuXMnLpeLXr16KZ9PSkrixIkTSpzs4MGDFUNaphkREQH4j5G1WCyUlZURGRlJZWUlCQkJ1NbW8vDDDxMeHs4zzzwTlENU1ieyLNtsNiwWyzn6QhAECgoK0Ol0tGzZUpFdo9FIenq68rxWqz2nb7Ro0SLofps2bZT7giCg1WqVa5VKRUpKSpB+admyZZB+iIuLC7pu1qyZ8m9Zn4A/L29UVBQxMTFBXs6YmBjlWYDo6Oig3NtWqzWIR7L3UxRFXC4XlZWVpKamKjwwGAzn6MhAnWs0GoN0rFqtDtIXcqqqsrIy5ZjYli1bBi13JycnB40Tge+Lokjz5s2Dno+NjUUURYqLi5V0iYH3Y2Jigq4jIiIU7yb4V60Cj+6NiIhQZFWOEQ4PD1fqbDKZgrKqGI3GIB2r1WqDxlKtVhukc2WeyeU1GAy0adMmSIcG5oNWqVRB3xMEIUhOZDmQPeE5OTlERkYqoR+CIJCUlBQ01tbNvy7LjfzNwLEX/KEZoigqz9TtOxaLhdatWweNK4Htptfrz7nfsmVLdu/ezX8aV4zb/wEIghDUYWSFJBtTgZ6ouvcB5RQy+b5s9MlLTrJykO97PB72799PeXk5qampivAfOHAgyIObnp4e1DHAP9vduXMnOp0OlUpFampqkJKVyxwYw1T3NJXq6moOHjxIp06dMBqN+Hw+du/eTffu3RVe1K1TcXExYWFh6HQ6hQc+ycfJitOs3fEJ+48cxIcXAZHmnVNRaVWcOHWcH05l8k34twztfQPXNe+NXqU/h77Mw0DjNvCUocD7cnlsNltQSqW6dQw8lUa+rntSTd121Wg0yqAu3w/85vna+XxyI9+Xlx/lnJZ120gQBFJ1WobHhNP/T5P4w9o13FpwkhHXX8eylWuwFtby2Y7t7P35KHED9Bxd8wrpYx8mqkMvyn/5CelsvldneRHmxDQSeg1nz6t/5pZ77qRti2aoVCpOnDiB2WxmwYIFzJ49m8WLFzNw4EC6dOmCzWbj7rvvJjMzk5iYGKqqqqiqqiI+Pj4oNZpcx0AeWCwWevfuzYoVK+jfvz9r167lhhtuUPgqPy/HWXbr1o3ly5cr7+/atYu8vDzGjRvHqlWr6N27N3a7nb/+9a+MHz+eyMhITp8+TUREBEVFRXzyyScsWLCAl19+ma+++oo777yTm2+5BTGnlPUvvckH33/NyOadyHVWsC57P4kmG5vzD+PwuvwhQj4vZ6rLGJrUnnohSUgmHer0dDQ/5hAWFsbEiRNZvHgxX331FW3atCElJYXZs2djNpuZN28eRqORzMxM7r33Xv70pz8xdepUdu3aBfgnNlqtlnHjxnHvvfcqy8ZRUVFMnDiRYcOGsWnTpqClT5VKxeDBg4P4GCjLsk4Bf8zyXXfdRdeuXcnKylLkau7cufh8Pp588kmlDQP7RmCqrsC+EXhfkiQlF2vddr9Q37hQX7NarUpYQn3v142VPJ++CHwf/Eap3M/q9tXA8gTqfLnOgTwO5IdKpVLo1qXZEE/qez7QsDIYDEo563u/Lg8DdXrd+4E8ktNF1o1jrcvDuvQD7wXKhUy/bgzrhcbGC7WrfG2xWNBoNI2Sq/p4XB+P5PE3cBIl6+L6xplAngTyou44JIeoBE7EAnlU3zhzoXGiKfa6NAaqGTNmzPiPf/W/DIGMDxS0+u439rf66ISSVmPfyc7O5ttvv2XkyJH1npByqfVQqVTKLC/wWZ/Px5w5c1ixYgWnTp1i2bJlpKSkkJyczNNPP81HH31EVlYWe/fuJT4+Pmg5ESArK4t7770Xg8HArl27eP3110lOTiYtLa3RZT506BCjR4+mTZs2tGnTBqfTyfDhw3nwwQeDTuYKfGfixIn07NlTmelLksTJitN8mrWe919bRfbPp6mtrqWysIKE9EQqCytZv2AdNeV2Dm09QFbBCSLSoki2JKIW1efQb4i3jzzyCOnp6coMWlbgdY/fPV97CIIAkg+81eDIQagtAncJuCtBVFNRWcO//72OtLQ09u3bx4cffkjPnj1DJuuyLNTU1PDGG2+waNEidu7cSbt27bBYLOzZs4e2bdvSIiUFq9mEShTJOniASePHk5+bz+qVy8nKKyV5xD1YmqXhKM0j++u1VOceJ3XoBMyJaZT/8hOWZq2wxibSvWUCtvITdG6XTvfu3UlPT+ftt99m27ZtjBkzhqSkJN59911mz56N1WolPDwcl8vF3r17+cMf/sDevXs5c+YMPXr0qFceiouLOXPmDEOGDEGlUtGhQwdWrlzJ+vXr6dChAxMmTAjylnz33XcMHDiQuLg4bDYbb7zxBmPHjkWv19OhQwe++eYbPv30U1QqFX/961/Jzs5m8+bN5OXlsWXLFjZs2IDFYmHHjh307NmT4cOHExcXxyeffMKIESMIs9kwxUZRqfZRcjCLIZGtaGOLY2PuYTbnHSHeFE64zsj1CW2o8bh45eAm/trpBkzqX08YEgAkkIxa1KOv4Uy8jh/27mHs2LHKJpJ33nmH22+/naNHj7J582Z27NjB3r17ufXWW1m1ahVffvklp0+f5tChQ9x5550UFRWxY8cORo8ejdls5ocffuDAgQNMmzYNi8XCkiVLOHHiBO+//z79+/cnOTmZVatWUV1dTceOHes9fa+6uppPPvmEUaNGYbFYcLvdLFq0iNzcXD766CPGjRtHQkICd911F9dccw3Hjx8nMzMTj8ej5ApurEwLgj8Xa6ARWved89FpiL4gCOj1+qDJXWPL1NBvdek29p3Glre+DayXOg7JRo0caxqq8UwQXKDw5wAAIABJREFU/F5x2Vhs6J0L/V6XviwL5xvzLqUeTS0LdY3Xy+WtKIqNkoWG6NR3/8CBAxw4cIAxY8YEbQi8UJ+6EP2GIEiBU63/QQTOcuR/1/fb+d5pLJ1Q0rqYdzIzM5k5cyZvvfVW0HLq5XxbRn3nnLvdbtLT09m+fTuxsbEsXbqU3bt3889//pN77rmHUaNGMWLEiPN+/8cff+Shhx5SNo68/vrrbN68mWXLlqHRaCgtLcXj8Sjxe16vl7KyMlwuF2azGavVyo8//siECRPo0KEDb731FoIg0LFjR/bv34/BYKC6upqqqiqMRiNhYWGUl5dz7bXXsnjxYjIyMlCr1VS6qlj28/sU2otZ+8wK+t11PXGtEpQyf71kA1qDhh63XkvxqUL+/eIH3Ln4fgY370eP+G6oBFWjeduvXz9ee+01OnTooPwW+GzD7eFD8DmgNBNyP4bKU+Dz+q0ZQQXWZHbnt2bWm1/zzrvvUVpaSm5uruLFDoWsy7KwdOlStm7dygMPPMD7779PbW0t8+fPb7DuAMU1XjZku9ha4KW4VkJJsBDwjkaAaL3AwEQ11yXqsOmClW9j61FX3YWqn8nwer384x//ID4+nokTJ9arBxoq7/n6mtvrJLviMFHZKrRrD+I7+WtcnXS2HpIk8Xn2Qb4rOMbMjBtRi6LfUyvTtBkRb+lOVssS7FU+nIVqrr22H+CPL/zqq68YMGCAYrA7nU769etHVFQUdrudHTt2UFJSQkZGBq1atSInJ4eTJ08qm7nOnDnD3r17GTZsGIIgsHfvXrKyskhLS6Nr165IksSRI0c4fPgw/fr1Izw8/ByeOJ1OduzYQY8ePTAajXg8Hnbt2sXJkydJTk6mR48elJSUsGnTpqB+0r59ezp16vQfkYXG0K9LN1RjiryZ6GLk6kLflsMbZKP0cmidT35D1c8C9WNdA6w+OhdbTrmsoRyXG6J9KbIgI3AD48WW6UK8bUgWGqJT331Jkli1ahWrV69m+fLlQYdaBPbDi+XthXAlLOEKLgm1tbUUFBTQrFmzczwJNpuNnTt3csMNNzB27Nig5cfAHa2iKJ6ztFRTUxO0a9disSi7O9etW8eiRYuUWKfZs2fz9ddf8+KLL2K1WnG73axatQpAMVI3btzIoEGDkCRJiX16/PHHqa6uVrzMBw8eJD8/n/fff5+WLVsSHhHOnsKfKKgpwlnhwFnt4Oct+zn0zQE6Du5CVHI0hcfz6X5TL0RRxBRmwuv14bLXsjPvR9qEtyLSEHHRPJW97DExMXz66aeMGTOGa6+9FkmS2Lp1K59//jlt27Zl3LhxaDQaNm/ezKZNG8iIz+OPbU5TY3fz0Y5Cwk0aMg+VM7BzJNdlVLH6rY/5aZ+Gt95aTJeMruTm5NC9e3ecTicrVqwgKyuLQYMG0a9fPwoKCti0aRORkZF88803DBgwgOuvvz5IsRw6dIgdO3Zw1113IUkSH3/8MSaTiU6dOjF06FDi4+PJz89XUmbVB0mSkGpr8WbnYDpxilE+LwPaXsUxycTBKpFyF4CEgECMQaBduIqWNhURevGyD4yVJ0N1Y/QuB5Lk3+Sj1WqZMmUKf/7znxk9enRQjN+lwOfzYa+u4ZT9ezZnLScprA197v4DYd8V49l6GOxO5FPGKt1OPjn1IzO7jfQff+zzp15Dp0F1VXNcQ1uzXbWVfb9sJMrUjGFd7le+YzQaGTVqlMIP2UCVYTKZGDhwYNBviYmJSiwo+ONEbTYbhYWFxMbG0q1bN7p16xY0wMurKXUHfRkGg4EBAwYo91UqFT169FD2AchH0N52223ApQ16gZBjbuuGNlwO5NymYWFh9XqnLxXyZt7o6OiQ0nW5XBQVFREXFxe0knE5kCRJcSLExsaGNBVWcXExgiCcs2/hcvF7lIWoqKiQy4K8LyNUsvBb4YpxewWXBHn5oi5UKhWLFy9m2rRpvPLKK0yZMoUxY8Yo9++9915MJhNWq5UlS5bQtWvXc+gWFxcza9Ysampq+Oijj/j73/+O3W7n7bffZv78+bRq1Yqbb76ZH374gRUrVnDXXXcxfvx41q5dq+wU1Wg03HXXXbz00kv079+fpKQk1Go1b7/9NldffTXTpk3jyy+/ZM6cObz99tu89NJL3HfffURGRuLwODhYfASVqKKqpApBFIhoFonk8/HB0+8xYf5kKgvKMUX5Z6AqtQqdSUdttZNyUwUnKk8TYQhHuEgzrKioiGeffZYpU6bQq1cvHnjgAb744gsOHz7M3LlzefTRR1m1ahWVlZWkp6ezeNFc7hsMH325j8LDFv7YN44Zy44zrn8svdqH85clR3jT0p5BnSM4dPIkY1r/yPdnzGza8j1jx47l3nvvJTExkT59+jB79mwcDgctWrTgqaeeYurUqXTq1Inp06fTvHnzoPARo9HImjVr6N+/P3Fxcbz66qu88sordOnShaKiIp5//nm2bt3K66+/Xm89JY8H18FDOL7+DvfxU0geD4IgoNrwDR3joujaMgVBp0dCQjQY0TTvgCoiAurxtl0KmurcdI1Go8Q1y3G3lw0B3OpKdpz+P9w+F7+U/Ehe5XG69B1E+z6DMOwvwfdzNlJeOdYaLW8MnISAgKBTI0ZbEdsk4OoYw9GIXHaeeZ3i6hxUoppC+xl+yP4317e+G40qdOfeq1SqkB+/C5x3GfpyIS9xNwXdUOepliTpvEvGlwNRFJW9BqGmq9VqL/zgRaKpZEGn0zVJntvfkywIgtAksvBb4IpxewWXBI1GQ3R09DmdQBAEunXrxoYNG9iyZQsLFy5kz549zJo1C4AFCxYwdOhQBMEfM7RixQoWLFgAoMTRGY1GEhIS+PDDDxkyZAhDhgwhLy+PM2fO8PTTT6PT6fjpp5/Iz8/njjvuYPr06Xz77bfccccdtG3blh9//BGArl27kpCQwMaNG9FoNISFhZGbm8v+/fv54YcflJQrgfj22295+C8PU+woIeXqVlx7x0BunT0JjV6D5PVxZv8pTuz5BY1eg8flT2/i80l4XB5UWv9mgFNVZ7gqqh069cUnG7dYLDz22GOIosirr75KTU0NX375JbfccguDBw+mQ4cO5OXlsWrVKrqmuInV1nBdp0iWb8pjdJ9YEqI0TBmeREKsgZ1Hytmwu4RBXSMx6tXE+U6iK64EIpEkiS1btrBnzx4iIyMpKipi/fr1TJkyhcjISB588EGMRiPLli1T0s/IaXz69u1LQkIC+/fvV2Shc+fOyiDZqVMnTp48ydKlS3nxxRd/rZwk4auqovrLDTi37YSzhqp4Nrek5HLhOp2L+1TO2cclEARUWzOx3DwaTes0uEylKwhCvbHnlws572QojO9ACMC+vPVUOEsQBBAFEYenmm0nPmaffgtJaekkdmlNuK8ZFode8WQ6dR7K1KXkO49wquwjyor86YBUogpBAEkSyCrdS5uKI7SI6BSy8hqNxqB48VBAXg0KNW+Bc0IjQgGVShVyr6JMtz6de7nQaDRERUWFlK4g+DdYhtKrKKMpZEEQBMVjW1/IyqVCFMUmlYVQoylk4bfCFeP2Ci4J51MA+/fvZ+nSpcyePZuhQ4eSnJzMlClTlHyFBoNBMS4kSWLo0KF069YN8KctycnJwWq1MmnSJEaMGEGvXr344x//SEJCAhEREUybNo2oqChyc3Np06YNer2eVatWsWPHDp599lmeeeYZ4uLiAP9M/M4772TOnDlKXj5RFBk1ahTDhw+npqaG2traoI6ckZHB3/81m3XHN6Az6SjNKaaqqIKW3dNBFBE1Ij6PD1tcOJUFZcSlxeOpdVNrr8UUZsaHD7fXg8SlKcfIyMignargzz0oL2/LS8HL3n2HDd/t5sRRAx6vRNc0KwJg1KrQaUQQwKBTUVbp9tuQggCSD6HiF5D8G6xUKpXitTIYDEomi/Dw8KAsALW1tWzcuJFffvlF4dGwYcP47LPPyM7OZtCgQYiiSEVFBWazmVGjRnH11VfTo0ePIOPWW15O9fsfU3vwcJAcnfNv+f9n5ctbWEzFO8ux3PpHdB3bQxOcYBcqhHpQqHFX8EvxboV24DK8vbacQ4WZHC78Ho1Kh1qlPXtfxOt14fbV+k+FqyeGTgCcHgfHijNJCb8q5IZNUyDUdENpxPyn0JS8/T0ZNL+nsv7e8HuThfPhv3eUuIL/arhcLoqLi88ZHGJjY1m/fj0bNmzgxIkTbN++HZvNphhnZ86c4dChQxw6dIgzZ84QERFBeno66enpREVFUVtbq8TcRkZGctddd/Haa69hNBqJioqisLAQvV7Pv/71L6qrq/nHP/7Bjh076N27N1dddZWSsLqqqoqamho6dOiAWq3mp59+orKykquuuoqTJ0+i1+v5/vvv+frrrwF/DFNBQQF6vZ4WrVKJbB6NJcqGy+5k26pvyf8lj/xjuRQeyyM2LYHkjs05tuMw5fllHN1+iNi0eISzR2n5wxEuXTnI8ZuyQR4WFqYc+LBnzx5mzpyJUeXglj4xLH60A8/dnkrHFH+Kr6JKF6XVbvBBWZWbxGj9r85OQQCfB8FboywXyonlCwoKSE1Nrbc8JpOJF154gVWrVrFq1Sqio6Pp1q0bx44d4/PPP2fIkCGUlJTw1FNPsXXrVkU+AuPsfE4nNRu34Pr5yNmiNI4/giAgiCJSjRP7/32G+9Rpxei9FEiSP2m/nEA/VJAkf5J4+SCQEFGlpCaHqtqycxzWgnB2d7PgV+FunwuHu/rsXxUurz/XriicmxbqV+qQX3mcGndFyEpcU1NDWVlZUA7WUKCyslKR1VBBkvxJ7OWE86GCz+ejtLRUSeYfSrpFRUUhp+t2uykpKQlpon1J8p/uV1ZWFvIJhJzKL5QI7L+hLG9TykJxcXHI6Xo8npDLwm+FK57bK7hkBB4XKSM6OpqXX36ZBQsW4Ha7iY6OZubMmcpyx1tvvaXEI/bq1Yu5c+cGva/RaIKyOkyaNInbb7+dgoICHnvsMebNm8c777zDtddeS0pKCrfffjvz5s1j9erVpKSkMGLECPLz85XjYC0WC/fddx+7du3C5/Nxyy23cPr0aR566CEiIiJ49tlnEQSBESNGsGDBAmbNmoU2XINaUOOVvMS3TSK9T1s2v/Elokrk6jG9iEuLxxZro+h0EV+88n/ojDr63jHg1zqo1FR7ocLrQwQsagGt2HhzV5KkoE11o0ePZvr06YiiyKZNm7jpppto1cHBrDklSAJsP1jO9RmRCEBppZfZ758go6WVbw+U8e5jHREFyC52sO77IlQC4KlCEARuuukmZsyYQceOHfniiy94+umnG932ycnJJCYmcvLkSdq0aUNFRQXXXHMNs2fP5scff2THjh3cfvvtcoVwHzmG8/s9lzxwCIKAt7AYx9ffoR4bg2AyXRId+PVYzFBCkiTlJKTQ0fRRWH0cj+RR0sudD0KdfzemFAJQWVtClbMUo+byNr7JkGU31Ai1sRxIN9S06/bfUKIpeNtUdJtSFprCsxjq/htItynkt6lk7P8HwxaupAK77HQnjaUTSloX805TpQK73JQjl8uPpuCh/HuNx8HSA6socpSc973z0fJJPq5PGcw7Ra1ZWuJBI8JNYWomJGjoF67GqBbOSb3yf//3f/Tt2xev10tmZiYjR44E4KOPPqJ///5YrVZ++ukndu3aRcuWLenXrx/q43M4sOVDdh2tpHmMngFdIikqczFhzk9MG51Cfnkt3VrZ6JDmDwHZuKuYGoeXDs3N5Oh70Oe2+TidTr766iuKioro2rUrXbp0oby8nO+++47hw4ejUqn47LPPyMjICJIduez79u2jsrKSvn37KsZdZmYmhw4dIikpieuuuw6dTofP4aDy3ZW4Dh72e2EvVRYkCbRarJNuR9e+bVD87X9KRppapgP/7fHW8vmhf3Gk+Idz0v2EokwAgiAypPU9tIvte/a66fTeb83by6lHY/79n6hHU30bzl1N+T3W43w6/bcaU5qCh/+NZWroviRdSQV2BVfwXwG9SkfbqNbkn96KSmh8ChsJiQidBZeYwNJcDzh9uIHVpbWsPlHLkCQtr7bR09oU3O1GjRrlf1+SFMMWYMyYMUqn7tKlC507dw5QBiIdWlrp0NJ6jkv46tZWIiODN7Ndf3WU4s5LTU5Ewh9ne+ONNwYpkPDw8KAy1E0JFYhOnTopign8mxx69+5Nr169gt7x5Bf4wwkuUjmdA0FAcjpxHT2Gtk1rhCbKevDfA8Efux16J1XAF67gCq7gCv7/w5WY2yu4JMj58EK93FJdXU1paWmQ0RQKVFZWUl5e3qjyioJIt5hORBuiGl0OSZKQfBLd4jJYVWACx9nlOAEQBUDgy9Mu2mRWk1nmboiUkhvR5XKd/yHbVSAGG95GvYobe8Vi0DVgkKtUEBa63fFwYVlwZ51AsjsuypAqcTgY/sEKhn+wghvWLOOPH6/h+5wzSID7+El8jYy/TEhIYO3atQA4nU4mTJjA+vXrzxsLuGDBAnbu3HleetXV1UyZMkWJ1fZ4PMycOZPrr7+efv36sWLFiqDn33//fR555JHGVbpeNP3CmhDCb9jtdkpLS0OuF8rLy6msrAwpTUmSKC0tDXksrxwP6XQ6Q063sLAw5HTdbjfFxcUhj7m12+2UlJSEXJc3lSyUlZVRVVUV0vLKsuBwOEJGU6ZbUFDQJPHXoZaF3wpXjNsr+K9CqBVhXdqNWdoQBAGL1syIloOI0Ichnf3vvHTx0706vhMnve15+/RZxRD4KeHsX6WP2w86yHFe5uBv7QSmKP9S/VmYzGqmjkrGaDqPR1OSwBgF1qsu79sXCclu93/7Ij23Vp2OWf0GsmrkzdzeriOzMrdS7fEg2WugkcpXo9Hw+uuvU1ZWBvgN8YYMr2+++ea8A9GxY8d46KGH+OqrrxQamZmZ7N69W0l7tnjxYkpLS3G5XMybN4+XXnqJ8vLyi6q3AkHAoLU2uk+4a72snbuTeXd/wT+nfsXiv2ymNK+aPRtPMmfiOv459Stee3A9mZ/9gsd99nQjQYVW7Y9f3rZtG927d2fEiBEMHjyYhx9++KLLvmfPHoYOHUpubi7gP5hk6NCh533e5/Px3HPPNWhgZmVlcf/99ysGjcfjoU+fPgwfPpyRI0fywgsvIEkS33//PaNHj2bIkCE888wzSuq6hnCxS52NgSRJTarHmgKXsux7IQQuOV9B6NFU/P3/pc2uGLdXcEnQarXExMSEPIm0xWIhIiIi5IrWarUSHh7eaLqiIJJiTWZE6mCameIREPFJPr+ZKwX/GdQ6usd3waPvxYSDPvA0sJQsCJws8bLwlAuvr34lolKpiI2NbTgBuj4OEkeBSt04554EqNWQeCPoYhvxQuNxQVkQhEsKSRAFAbNWS4TBQEZcPAXVVbi8XnJLSnjm+ee57bbbmDdvHk6nk+LiYv76179y66238vDDDyvGVatWrYiKimL9+vUKXbPZjMVi4b333mP8+PE89NBDHD58mO3bt7N3717mzp3LiRMnzilPVlYWN910Ex07dlR+2759O927dycpKYlOnToRFRXFL7/8QmVlJRqNhsmTJ190vX+tv4p4a2tF5hoDR7WL625ry21P9mTMtKuxRhlxOTy06BjNbU/2ZPi9GWz94DAluX5jUq82EWGMRxAEnE4nMTExvPHGGyxYsIC8vDzeeOONi/bCHjlyhA8++EDZuJeXl3feZyVJYvny5efdeLRz507uv/9+Dh8+jMFgAPwZVywWCwsWLOCNN97g/vvvx+Fw8NRTTzFx4kQWLlzIiRMnGjwhD/zGQXh4OGaz+aLqdyHIOUjrO+TmciCKIjExMSGnK+csD+UpYoIgKBluQq3Lw8LClLjNUEGWBYvFEtLyynluZdkNJd2mlIWmOujmP4krxu0VXBICjbumoBtq1FdeSZJweSWKnT4KHf6/SpeE1+f304qCSGpYCjenj2RgUm+aWxKxaMyYNEZMGiNRhgjaR7WhZ7NhHPT2YMyPEjh8DQcyCoBPYlaOi/za+o2GRvFWUEHcHyChnz/v69mTVusniP+ZhOsgbgRSiLt9o8p7CW3q9no5UVHOz8WFLP1pLxlxCZg0GpZkbsMWH8+MGTPYs2cPX375JR9++CEFBQVMnz4ds9nMhg0bAL+hMXHiRD788EPsdjsWiwVJktizZw+ffPIJjz/+OL169eLpp58mIyOD9u3bM2XKFJKSks4pz5AhQxg2bFjQpCM7O5vmzZsD/pRpRqORgoICoqKiePDBB5Wcy5cCAYFYUwv0qsYnwhcEgfBYE9HJViITLag1/rY22XREJ1uJbxGGRq/27wxHItKYgFkXqbyv0+lISEggPT2dvn37cvDgQbxeL9999x0PPfQQjz32GAcPHgTg8OHDPProo9x9990sX75cOSb7hhtuYP369Zw5cyaobDU1Nbz66qvce++9vPnmm1RVVbF06VKKi4uZP39+vd7bLVu2MG7cOGJiYhT5OnToEPHx8TidTrxeLzabDa1Wy3PPPcfIkSNJTU3lmmuu4fjx4w3yqql0WCDt/2W6v8ey/l7K21R0m5IP/2n8/s3zK/hN4Ha7KS8vD/nM3G634/F4CAsLCynd6upqPB4PNpvN76XySBwo9bD2hIuN+R7cZ/tyc6PAqCQtNyRrSTSJCIJAmM5Gn8QeXBOfQamjHB9ear0SWQ4Tu6p0jD3ihjL32fjaRhRGEKDKy8FqLwn6c1/w+XyUlJQQFhbW8NGgajOk/tnvxT21Ftyu+o1crQ6Sx+CO/SMV5U7CwvQh9dLIshAZGVkvXUGnO3uIxMWFJlTW1rJk3x6OlhQTb7GyZOhI9CoVPxTmM9zrZf/+/ZhMJnbs2MF1113He++9x/bt23nwwQeDTu/JyMjgs88+Y+PGjZjNZux2Ozt37kSSJI4ePUp5eTn79+9Hr9ej1WqxWCyUlJSwevVqHA4H4eHhTJ48uV7PtNvtRq32n0xXUVFBbW1tSAcGqyGGKGM8udUnGxWz7PX42LLyZyzheqKTrPQe7T82+fD3udgrainLt9O8XRSR8RZERJrZ2qANOH5Xzl9dXFzMpk2b6NevH8XFxbz00ks88MADOBwOHnjgATZt2sTTTz9Nv3796Ny5M3PmzKFLly643W5iY2Np3749H374IWPHjlU8o2+//TZnzpxh8uTJvPXWW2i1WkaPHs3zzz/PhAkT6j3N6oknnuDYsWMsX76cmpoaJElSThicOXMmeXl5/PnPf2b06NH069cPgJKSEjZu3Mi0adMuyK/y8nK0Wm1IT9Lyer2Ul5djNpvR6S7+lMKG6JaUlGCz2UJK1+12U1FRoRzeEgpIkj/Prdx/QqnL5Ry3ofbelpeXo9FoMF1GmsG68Pl8lJeXYzQaQ+pllWN5rVZrSOl6PB4qKioICwv73Xtvr3hur+CS4PP5qK6uDvkMz+VyhTypukxX3ohR45aYv9/BNV9UMWd/LXtKfOwv8/+tO+Ph7m0Oen1RyY9Fbv8JT5w9c1ulI84UQ6I5gRohjsG74emfHFDu8fekxupvAZCgvIF9ZVVVVY3KESlprND8T5DxKiQNBlMkGMz+P1MEJA2CjPnQ/G7ckr5J2szr9TYYM6lJSUY0XPxgHGk0MvPaAXx+y3jKnA7OVFaAJOHx+cgtKCArK4vU1FRuuOEGhgwZwpw5c9i7dy9Dhgzh888/V+io1WomTZrEJ598gtvtxuVyKQc5ZGVlUV5ezn333Rf0bUEQzvmrDwkJCUoIRGlpKVVVVZflra0LncpAp4TrUQmqRrWbIAo0ax1BWtc4EtPDEVX+cofHmmjVNY72fZpxYn8hRbmVHN9TTubnWSxZskQ5nW7//v0MGzaMbt26kZyczNSpUzlw4AAulwudTodWq6WiooLc3FyioqLYtWsXFouFdevW0a5dO7xeL16vlzvuuIONGzdSWFionEi4efNmUlNTqaqqIjExkczMTGUQjYiI4NChQyxZsoQlS5aQmZkZVC+fz4fL5UKSJK677jrWrl3LypUrmTlzJv/85z+VwzOqq6u599576dSpE3369Lkgv+RTCkONmpqaJtmUI0/SQ4mm0uVut7tJdLnT6WySNnM4HCGnK2+s+73IgtfrDfmmut8KV4zb/yG43W5at25NSkoKrVq1onfv3qxcufKSaOl0OlJSUi4Yc/vSSy/Rr18/RWl89tlnTJgw4bzPO51OZs+eXa8xUVpaytSpU+nevTtdu3bl888/x+fzceTIEUaNGkW/fv0YOnQo2dnZyjsFBQX0798fu91ObGwsTo/ArD12ntzjBC+gEs7d+CXCmQqJjM12dha4611R98oeUjH0G1JEUSQ1NbXxM3JBBEtbaP1X6PE+Uq9PoNf/QY8PoPWTYGoNgoher6d58+Yhn5Hr9foGZUGdEI8qIf6SFWaU0cQ9nbry8q4dOFQCzZKTGTx4MI8//jgRERHY7XY++ugjamtrWbBgAc8995wSb+lwOPB4PFx11VWEh4ezadMmwsPD6dixIy1atOCRRx5h7NixZGVlBX0zJiaGBx98kCeeeIJ77rmnwXRo+/btw+124/V6sdvtpKSkXFI9z4dW0T1IDmvbqNBqURRo1S2Ozv2TadUlDpXa3yaxKTY6929Orxtb0bpbPCf2lGCtTSbneBGHDh1SjJCMjAwyMzNZuXIl33zzDV6vF5/PR35+Pp999hlbtmxh0KBBGI1GXn75Zfr06cNf/vIXRo4cSX5+vuL9TklJYcCAAXz66adK2bxeL9u2bWPdunWUlZUxcODAoLKXl5crpxcWFhYqvwuCgFarVVZzWrRoofA4KSkJrVZLVVUVbrebSZMm0b59e5555plGeTcTEhIIDw9vBGcbD1EUSUpKCqkHUKabkpISUi8z+GPmQ60XBEHAarXSrFmzkMfcRkdHExkZeeEHLxLx8fEh3+8hy0Ko47pFUaRkNLbAAAAgAElEQVRFixYhlwWdTtckY8RvgSvG7f8YJEliy5YtfP/997z00kvMnz+fb7755qLpXMijFYiTJ08GedIaMnKKi4vZv39/vfc+//xz7HY769evZ+HChcyZM4eioiLmz5/P0KFD+eyzz7juuuuUTTC7du3izjvv5NixY0qZPzjuZObPLrkS56nc2b9qiT9/X0N+zbmxsVoRuNSV/bPV15zv8xfB2zov+g3doL9fN3NdMt0LfrZhuoLZjP7qDP+GtkswcEVBYEx6O06Vl7FPI/LAk39l0aJFTJo0iVWrVtGlSxeaN2/O/Pnzueuuu1i6dCk33ngj4DeYXC4XGo2GO+64g4KCAgRBYODAgfh8PiZOnMjUqVPp1MmfHq1Tp04899xz/Pzzz40q28CBA7Hb7UyYMIFp06Zx4403BoVEXC4EQUCrNtAn5VaijPENZu1oDCSfRGm+nQ5JPZj+2FzmzvX/tWrVSvmeKIoMGDCAxMREli9fjs1mIzU1lenTp/PMM88giv6J0rvvvsvo0aP56KOPCAsLY/v27QiCQG1tLT6fj9GjR7N582aKi4sBiIqKYsyYMcydO5dBgwadowd69+6tlCcw17JcLkEQ8Hg8PP7443z11VeAP+bZbrej1+uZOXMmcXFxPProo42aGP5W/eF/jW6o8Xvkwf86H34L/P7N8ytoFAIHEpvNRnh4OL169WLgwIFs2rSJa6+9lrfffpuNGzditVqZOnUqbdu25f7772fo0KGsW7cOl8vFjBkziIuLY+7cuezevZvo6Ggee+wx0tPT2bx5M8uWLUMQBO68804lBu6WW27hrbfe4g9/+INSFtnjOn/+fCoqKhg8eDDjxo1jyZIlHD58mL///e889dRTyvPyRpXBgwdjtVpp06YNdrudoqIisrOzefDBBzGZTPTr149//OMfOBwOzpw5w4MPPsjs2bNxOp2U1rh5L8vld7teqO+eNXB3Ffv4NtfFLWm6oMMOEnXQI0xFZp6nYVr2KnhmOJjDQK2FxFYw6A5Ib0e6+VfrWN6RLggCPp+PyspKTCaTMoOWeSbHtMo8CSyTfP3kk09is9l44oknEASBZcuW4XQ6ufPOO3E4HMoysSwTp0+fZuPGjUyaNAlRFJUUM4Eys3XrVjZs2KAcA3zs2DFeeeUVioqK6Nu3L+PHj8dmsyGKIqdPn+a5555j6dKliqLUdmiH7tARavcdvADj/bCdTQMWb/KX1aTRsGbcJGJvvomYjC606tyZkpISoqOjiYuLIz4+nkWLFlFSUoLFYiEpKQlJknj//feJjfVnh+jatSvbtm1TMju8+uqrnDlzBoPBQHJyMpIkMW3aNEaPHk2zZs2CeCrzxOfz8fLLLxMVFQX4D8J4/fXXyc3NxePxBHlt5SX0jIyMBttMkiSlXeWjRWUvuM/no6KigghzM/qljmNz1jLKHIWIFzn47P/2DMVnqnDXeoi1tmD86HuptXvQmL1Bg5lcR6PRyLRp03jsscf44osviI+P57HHHsPlcmGxWDAYDFRXVzNt2jTi4+Ox2+107NiR7OxsSkpKcLvdpKSk0Lt3bxYvXowkSUyePJnZs2ezfft2fvnlFx5//HF/W9tsvPrqqzz88MPKhj8IPrHI4/HgdDqJjIxk4sSJPP/882zevJnjx48zZcoU7HY78+fP5+abb+bFF19EEAT69u3L0KFD621H+bq6uhq1Wo1Wqz1H9uvyRL4f2E5121Xuv7LBrdFoGnw/8LrufZkH8vuCIATpBfmbgX21bpkboy/ksCKz2YxKpQr6ZmN5UrcMkiRRW1uL2+1WPNh1y9gQD+vyIPAZOVWf0Wg8h++BfaehOte9liSJ6upqVCqVktlAvt8QvYbaVZL8xw/X1NQoslBfuzbEw4bKX1FRgdFoRK1WB+kLGYE0AuvQULt6PB6qq6uxWCyoVKqL4mFD178FVDNmzJjxm3z5vwiBsxT53/X9dr53GksnlLQa+052djZbtmxh5MiRmEwmXnvtNYYPH47NZkOj0XDq1Cm+/fZbWrduzcKFC3nuuedwuVyK92vhwoU4HA6mTp3K0aNHycnJobKykhUrVvDCCy/g8/nYsWMHcXFx/O1vf+Opp56idevWzJgxgz/84Q/s27eP8PBwcnNzsdlshIWFKfk0Z82axcCBAxk5ciRz5syhU6dOtGjRgszMTGbMmEFERATg78SyISNvCJozZw4xMTH07NmTdevWMXDgQKKjo/H5fKxYsYKePXvSu3dvYmJiWLNmDVd3zaDKGMtTh72/puo6n4EQ+LtPIloP3a1ORCQ0Go1fYZWXIml0fFnoPfedQDq1Tvj0Nbj/n9ChL1QUwVdvM3TwYO5pFYZG9CvCkpISampqMJlM+Hw+Tp48icvlwmq1IgiCkhxfNkyrq6spLy9XBg673U5eXh4Wi4Xly5fz73//m6uvvpqEhAS2bNlCfn4+aWlpinHidrs5ffo0ZrOZzMxMPv74Y3r37o1Op0OtVispnFQqFevXr+fxxx/H6/UyfPhwAP7yl7/QqlUrbr31Vl5//XU6duyIWq1m//79/O1vf+Obb75h7NixSJKETqdD0OlQJ8bjPn0GX0WlMh84n0yLgoBNp0ctikiAoFYTO2YEuqs6otXpsNls6PV67Ha7sklQVvIJCQmoVCqcTic+n0/hYW1tLS6XC61Wi8vlQq/X4/V6adasGRqNRklgnnJ26dflcpGdna0MTG63m5ycHGJjYxW+5+TkYLPZSEhIAKCwsBCtVoter1c2BgZu+igoKFB4IkmSkpTfZDIhSRL5+fm4XC5lubGiooLs7GwiwiOItCQQZUqitCYHu6vcP2gE6AZBFIhubiU60YJaq1L4aQk3kNQ2kuZtYxg5+CYeve9ZmsU3VzZT5efnY7VaCQsLo3379ng8HgwGA6mpqaSmpqLT6Rg2bBjNmjWjc+fO9OvXD4vFQs+ePUlJSUGtVjN+/HjS09PRaDRYrVYlrKZDhw60b9+elJQUUlNT6dKlC6IocuONN9KjRw9EUaRt27ZERUWRlpaGSqWitLQUp9Op8KC6upq4uDhSU1OVjBTt27cnLS2NUaNGkZGRQWVlJTfccAMdOnQgKioKq9VKx44diYqKorq6moKCAkUOampqKC4uxmQyUVFRgdPppKysDJPJpMhNdnY2VqsVURTxeDwUFBSg1/s3YsrXBoNBuX/69GkMBgMajQaPx8Px48fR6XQYDAZ8Ph95eXmKEe3z+cjJyUEURSXzRnFxMV6v97xyUlRUhN1ux+FwoNfrqaqqwm63K3JYVlZGTU2NYpiVlZVRVlambLqqrq5W6hioT0wmEx6Ph5ycHKqrq5U619bWUlRUhMFgQKVS4XK5yMvLw2w2I4oiTqeT3NxcDAaDoi/y8/MV/eHz+Thx4gRerxeLxRJUZ7lO+fn5AMp1Xl6ewhP5kAK3243BYEAQBIqLi5WYea/Xi8PhUPqxrEOrqqowm80IgkBFRQUVFRUKjyorKykrK1N0aEVFhSIHoihSXFxMaWmpMkl3Op0UFBQo6cHkOhuNRkUOAnngdrvJy8tTeOZ2u5WVBaPRiCAIQXLi9XopLCxEo9Eohm9eXh4+nw+DwYAkSWRnZyMIgsIj+YAFnU5HcXExlZWVeDwexdCX9Yn8vfLy8nPkRB435MlScXGxUsfKykpycnKIiIhApVJht9spKChQ2t1ut5Obm6v0JfkAn0A5kXWmWq2mtraWvLw8Tpw4wc8//8yYMWOCwoXOp/8ba/M0hCue2/8B1F1msFqtQR5BjUbDgQMH8Hg8/PTTT5SVlZGbm0tJSQkAt912G+3ataNjx44cPXqUwYMHY7fb+e6777jpppuUzSDZ2dkcOXJEMbTkJOrh4eE88sgjzJw5k+nTpyuGwy+//EJGRgY1NTWIosjhw4fp2LEjRqOR+Ph4NmzYwLZt23A6nQwYMICEhAQ8Hg+rV69m69atvPfee7jdbkRRVDqn7AWTFZRc/+iYWHIlHbjc/jjZi5hNFjklBI0OjcbPM1EU0RmM3GTWsTjPy4HCCwT1a/XQoqP/dLDkdnD4e0aV7kVLMp988ikbN24kLS2NW265hdLSUhYvXkxKSgr//ve/GTx4MGPHjsXlcrFs2TJOnjxJhw4duPXWWzEYDHzxxResX7+ehIQExo8fjyiK2Gw2evbsyccff0yPHj0A/6aqqKgocnNzmTFjBjU1Ndx8882Eh4fzwQcfKCm1Jk6cqNTRZrNRUlJCZmYmEyZMYM+ePWg0GiXDwKhRo9BqtSxatIjY2FgEQeCzzz5jyJAhHD16VEnRJEMVHY3ltpuxf/o5rp+P/NoG9SktSQLJvwgvWEyYh9+ApmsX3AHtptfrCQsLU65lw0qWdbVaHXSt0WgICwtTvBIul4uwsDDFuyXLTeB1eHi4krFCpVKds4vYZrOhVqsRBEExeuU6C4KgTCJlWCwW5VoQBGUgDrwfmHHCYDDQsmVL/wRBEEgKa8+wNvezJ+dzDhXuoNbjOMsuCUGA+JQwBEW8/fwLizHRPCmJbs2G0Tb2WvCoUGvUJCYm4vV6lVjWiIgIbDYbdrsdlUqFWq3m+uuvV7x6cjou2dupVqvp1q0bbdq0UYyqxMREBg8erAxg0dHRDB06VOFJeno6zZo1QxRFRFFEkiS6dOmCRqNRDLu6cYQyDfl3s9nM1VdfrVzLm93k0AqHw4Hb7VbiHHU63XnlJDY2Frfbjd1uV/SFvMFNbhdZv8jtIgiCYgTK9yMiIhS5UKlUpKSkKDyQn68rF4GDvGxYyzCbzQq9QDkJ5E2gl042bgLlJlBOZfmRIWcFkf+dkpKC0+kM4oHFYlGu5b5Rty/J3xBFEavVGsSj+Ph4xXCUJOmcOlut1qC+UJdHFoslqA5Go1EJiZEkCafTGVSnujyQJ6WB14E81uv1igdTloW6PAiss1qtDsooIevIutd19YfJZFKM2UA5keUqsJ3r8kSexMvXspyIokizZs2C5BY4R5/IvKorF/IzOp0u6H2j0RgUc6vVaoN4oNVqg7JfqFSqoL5Ql0d1dfB/Eldibv8HELhsAr8at5Ikcfr0adq1a4coijgcDkpLSzGbzdx9993K4BAZGRmkdDp27MjKlSuprq7m5ptvZvXq1eh0OiWNkMPh4P7771eWbo1GI927dyciIoJ9+/YpqXe8Xi9lZWVUVlYyYsQIMjIylCUUk8mEwWAgLCyMuLg4oqOjEQSBr776itdff53Vq1eTlJSkKAOv14soihQUFBAZGRmUlFwQBExmU+OzGdQDrVanKB25fIkGNa+3NYDxAnlmCbhnNJDSKo3yM8f5YfduFi5cyLhx48jPz+fNN9/E4XAwb9488vLyuPvuu/nnP//J4cOH+fjjj9m1axd33303mzZt4vPPP+fAgQO88cYbjB07FofDwaJFi5QB749//CN79uxRPA+yN2T69Om0adOGIUOGKKmSBg0aRPv27Rk9evQ5ijc5OZkXX3yRtLQ0ZbDIzs6mtLSUZ599lqlTp/K3v/0No9GI2Wzm73//OwMHDlSUmuxVARBEEU1CPNbxYzENvR4x7Gwqn7OGrGLQ+nx+Y02rRde+DbY7x6Pv1QO1Xh9ET3fWgxs44AYqd41GEzRAywOyPNhqNJpzBqPAAbquMSsP8IEDZuBEUTZC5MFIHvADB6fA9FDytWyw1L2W6xiYAF5AIMKYQP+WkxjV/hE6JwwkwhCPVqVHEH4dqFWiGqPGQqI1jX4txnLTVU/SKfEGdGojOr0uyGipOyEIHLDVanW9RkzdAT1QPwQO0KIoYjabg+6bzWalTvLzgddGozGozrK3Vl7uNJlMiudJ5lHg5i29Xh804MoDdKAcBE6G5TrL7S7XOZBHgcZtfdeBaftk/VC3L9U1WmSDU65zYJxwXTmR6ywbKgaDIUhu5GuZnsFgUIxXmQeBfUO+DuRJXSOlbh0DeVhXTuoaavJ14ASgrpzUraOspwKvA+WgrlzI13KdZB0kX+v1ekVOZO9n3ftynWWniGyMB/Kkbt9oSF/UvZZ5JI9rdfuS2WwOkpu6+iJwQiDLQaCcGI3GoHaX9UdgHQP1iSwXsk0g8yBQZwY+X7fvyBs7G5oEBfYFmWehPuypMbjiuf0fxr59+1i3bh0LFiygtraWuLg4pkyZQm5uLq+99tp5c6Fu3bqV06dP8/DDDzNo0CDmzZtH7969iY2NZfLkyWi1WqZOnaqkKZFjmR566CEmTZpERkYGJpOJiIgIRowYQefOnXn22WeVPJayAu/Tpw99+vRRBrUffviB559/njVr1ijplsLCwjCbzRw6dIi0tDS2bdumeLoC4XK5MRm9folvIAVXfbBqhHpngZIk0VlXy9edtVx3yA1lvoYNXL3Im+307Nmgoaamhk8//ZSuXbsiCALp6emsXLmSe+65B7VazcSJE7FYLCQmJio5M3Nzc6msrGTt2rWoVCpmz55NamoqgiDQqVMnFixYoMSjxcXFcfPNN7NgwQIyMjIoLy/n+PHjnD59mtTUVMCvLE+dOkVYWBgmkwmr1cp3332nZJu47rrr6k1rVVVVRVZWFvPnz6d9+/aMHTuWTz/9lFGjRjVqhi6YTJiGDkbf8xpqfz6M5+gvuHPzEeT4MKsFbWoKYnorhGaJaAIM2suFHLPn9XpDuoM50KMZytOIJElS4iwDjWqVqCYprD1JYe1xe2spd+Rjd1UgSf5DRFSCBpshBos2QjF6AyHHWdb1FF8u5NCPuh6ky4XdblcG91BCbrNQ5gqVJImqqipl+TmUdOuThctFU8iCJEm4XC5qa2t/d7IQ6v5bXV19jhc5FHTlmNsGT7K8SDSVXvgtcMW4/R/E5MmTqaysxG6386c//YmePXsiSRLvvfced9xxB1VVVYwZM+a8Cr9169bMnj2bNWvWIAgCo0ePJj09neHDhzN27Fi8Xi/du3fHZrMBKLulu3XrRosWLaipqSEsLIzJkyfzxBNPKJ6gli1bUlhYiCAI3HfffSxcuDDou2+88QalpaU899xzgD/cYcaMGUyZMoXp06ezevVqcnJyePvtt8+ZKTocDpIj3PQJF9la4G28F1cl0DdOg0l77guSJGG319BZryO/p4X5J5zMynNDpQ/kFUMB1CLc1ELLM61NpJsEtpQU0euabvz000/s3r2bmpoafD6fsgEvLCxMGRxljBo1Co1Gw8KFC5kxYwbz5s3D4/GwZ88efD4fPp+PwYMHBy3jjR07lptuuom0tDTAv1mguLiYVatWAf7MAHI8NPiXOQ8dOsTevXuV+/UZt9HR0bRs2ZLmzZuj0WhIT0/n2LFjF71xQLTZMPTsDtd0w+f1/hqHK4qgUlFZVUVtVRXRIT5i0ul0KhtdQmk019TUnONpDQXdyspKdDrdeQdHtagl2tycaFAmgnU3ddSFvHEk1CmK5BzCoTY85OXiUNO12+0YDIaQJ9iXDaVQGzRVVVUNysKlQDZo6oZGXC5qa2sVuqGEHI4QaroOhyMoPCYUkGVBpVI1mSyEEk0lC78Frhi3/0NQq9WsX78el8ulLIFERUUphuC//vUv8vLy0Gq1xMfHI4oi7777LjExMQDcfPPN1NbWEhUVxcqVKykqKsJoNBIbG4tareaRRx7h1ltvxefzkZiYiE6nY/Lkycr3RVFk6dKluN1uBEFg2LBhZGRk4HA4iI2NxWKxYLVaWbNmTb3Jv6dPn86jjz56zjJR7969WbVqFVVVVURGRiob0cC/bPzOO+8QFxeHSqXmjpa1bC12gO8ChtjZMINOkQIDm2nr3Z0unxsOYBVF/tbawJ9b6Dlc7aHGIwECNZVuntCLLO5gwKRTsfP77zl58iRdunQhPz+f/v378/TTT/Pzzz+zfft2wO9RrZsfctu2bbRo0YJly5bx2muvsWrVKlq2bEnv3r2ZNWsWWVlZrF+/XtnI4Ha7sdls3HjjjSxdupThw4djsVhISUnhhRdewGKxMGvWrKAlKI1Gwz333BPMhnoMoxYtWmAwGCgsLMRqtZKTk0OfPn0uaelJEARQqxFVqnMMTYvFEvKz3gVBCHleU/DLQihTgAXSbYpcoVqtloSEhJDTtVgsIffUAUF9OpSIiYkJeVlVKhXx8fEhpytvlgz1Eq9Go1H0fagghxX8nmRBPm1TnhyGAiqViri4uCaRhcTExJDSBILG/v/H3nmHSVFlf/9TnSf15MRkQBiyoKAgKIjCmhCXxYQBUYwosuawLMviLoqri0j4LUFXUVFXEAEFXAwoICBJQMAhzTA59EzPTE/Hqnr/aKvsnsQA1SLvzvd53KW6pk7de+vcU6fO/d5zznW0O7f/QxAEgZycHPU4MLoDfm5bp06dgiI+Skol8G+eUf4dGxur8h0VY2A2m5vIVwyRcp3CnQX/xE9PTw9qo06nUzcnNYaSnqnxOVmWSU1NJTU1Neg35R4ZGRnq7zfkmNhR4eP/Dv9cqrY5m6PwZ8Ph7/3C6BDR8kRX+aSAUSeQYhZIMZvUdlabjNRU25j98ksUFhaSl5fHmDFjyM3NxWq1Mn78eCRJ4ocffuC6665rIleB1+vlueeeY8iQIezcuZOxY8fSr18/nnzySaZOncqhQ4e44oorMBgM1NXVYbfb0el03HLLLcycOROv16vuVn/mmWeIiopSsyXExcWRl5fH8uXLGTNmTIt9DXwOv/vd73jmmWfo1q0bNpuNoUOHnvS6U8XZ2IRwJghVe0MhV9XbEMnWWm4oxyAUCJXsUDgdyvM6V8b4XJEZatmh1LFzzfY2h/ZUYPz/nwps48aNjBo1KmiDwZn2QxRFdbm0tRflqfZDKa1oNBpb/LvTGUMlbUx0uJn+SQYa3BLf1zTDkZXx794PF1hzSTgjM8z+ImYt3N/hcCBJUrNLToIgoNfryc3NJTExkdzcXO644w6uuuoqlXg/ZMgQBEFQc/haLBY1NZLBYCAnJ4fc3Fx69OhB3759EQSBa665hmHDhpGQkMBFF12EXq9n6NChXHvttRgMBjIyMujatStRUVFEREQwaNAgLrjgAuLj4+nZsyeJiYmkpaXxwAMPEBUVRXJyMj169MBqtQZ9CAT2IyoqitzcXDVydMEFF5CTk0N8fLzKE1Z0wWg00r17d5Xb29ozPJkuuFwuTCbTKc+jls4ru6xb07HTmWfg5+y1pgttlRP4b4WWYDAYgjZItkVWa/1QaAmNx/ZM55nb7W72mZ3J2AqCP3WVKIpnPLaBvyl8SEmSgnaQt6W9rclXlqJ1Op2aN7atbWrtN0UX9Hq9prqglFw1Go1NnOfTfYZKnlun06m5LiiljduqC22RD03nrxZjG0pdsNvtQbrQ1ja11o+26kJrcpo7v2/fPvbt29eeCqwdv30o+e2UPIpaoa6uDqfTqTmfqq6uDrfbTXh4OCnhev5+UQTD0zws+snNWpuocmTTLDrGphoZ39VM70QjAi1XVJNlGZvN1mQXbyBMJhO///3v1b9XJqgiMycnR3UCZVnGZDJx+eWXk5eXR2ZmJgMHDlSv6dGjBz179gySk5WVRXZ2dtA9+/fvH2QIBg0aBPgdcZfLxWWXXaY6dso9hwwZ0upSXIcOHdRcrkqU/pJLLlE3aJ04cULdNRsREdGktOrpoK6uTs39e6qGrTXU1taquqAVZFmmurqasLAwTTm3SulbJRevVvB6vWoeUy03pDgcDnWji5acPeVFrnW50erq6iaZCs4UkiRRXl5OUlKSps9M0YW0tDRNn5nH41EpZlo+s4aGhqBc3FrBbrcjCEJIdCEU/OuKigoSEhI014WysjJSU1M11QWv10t5ebnmunA2IMhnq3zEbwSNHQ7lhd/4t5auaascLWWdyjXfffcdM2bMYNGiRUHL9mdyb+CM2qTF2Go5hsdqRVw+f7+SwwTiwvSnLUurNml9799CPwBEt4jP7UPAf43OoMdg0SPomy6Ltibn19aRX2MMz1Y/TtamM7n36bbpt9aP031G54q9gLbNqd96P05m91qT/1vuR6hs+q8xz2RZ5r333mPZsmUsXbpUDYgFngdOuU0nQ3vkth3/88ix/uLMKhOtHdpB9EjY8qop+qaU4u/KqDpaBzLIkkx4jJnEPvFkDksl5cJELDHmUzZi7WhHO9rRjnYEot25bcdpQRRFXC6XWoJSK3i9XiRJalJR50zhdruRZVlTubLs55UFltHUApIk4XK5mlSPOVOIohhUrlJLuW63W634o0CWZeqLG9j770McXXcCV41/E5/ML1/mnjovNSfqObK2gPiuMfS9txtpl6SgN/mzPiglSbV8ZoE6phV+DV3QcplQkqRmn9mZwuv1quVBtZTrdrsBQvbMtE7T5HK5MBqNmj4zWZZVDmtgyr8zRSh0QZZlfD6fqgta23IInS5oOX/PNV1oyZafizi3W9+OswaFm6N1pLOurg6bzaa5XKW2upZyZdnPuXU4HJrJVOSWlpbi8Xg0lev1etUa9lrLbawLkihRsd/GF09s4cf3j+C0uREEAUH3y85s9VgnIIky5ftsfPXsdva9eQh3rZu6ujqqq6s1bSuERsdkWVbruGsJRReUF7pWUDi3WuuCw+GgqqpK8/lrt9upq6vTVCb4eZZaPzOFZ+lyuTSXW15efs7oQkNDQ8h0oba2VlOZ4NeF+vp6TdsbSl0oKyvTXK7P56OiokItwHQuoz1y247TgtlsJiMjQ/OvO6UGvNZyY2NjT4u30xqUtGVaQ6fTkZWVpfkYmM1m0tLSQiI3PT1dlSvLMtV5dr597ntsR+xg8DuwJ4NOr8PT4OX7+T8iyzLdb+tCTIz2aWlCoQuCIIQsz20odMFkMgU9M61gtVqDynFqhfj4eE3lKQhVnttQzDMl57HWG31MJhNpaWmayhWEX/Lcnmu6EMgDPVOEUhcyMzM1191Q6MLZQrtz247TgiAIQalNtEKocuwp9cO1W94GUZRwuSFM720AACAASURBVCXqGjz4RL8xjDDriYgwYTScft5IZWz999EuiiAIQpADqqXcQF1wVrjYNmsPtqO1CIaf79dqXeJgW<KEY>xEE<KEY>//<KEY>  Exercise 7.1: Compare accuracy and training speed of different modelsChoose two pretrained models from torchvision.models and fine-tune them.Don't forget to create the dictionary containing the names of the models and the corresponding models.If you want to try out more than two models, just add them to the dictionary and add their learning rates.<jupyter_code># @title Imagenette Train Loop
def train_loop(model, optimizer, train_loader, loss_fn):

    times = []

    for epoch in tqdm.notebook.tqdm(range(5)):
        model.train()
        t_start = time.time()
        # Train on a batch of images
        for imagenette_batch in train_loader:
            images, labels = imagenette_batch

            # Convert labels from imagenette indices to imagenet labels
            for i, label in enumerate(labels):
                labels[i] = dir_index_to_imagenet_label[label.item()]

            images = images.to(device)
            labels = labels.to(device)
            output = model(images)
            optimizer.zero_grad()
            loss = loss_fn(output, labels)
            loss.backward()
            optimizer.step()
            torch.cuda.synchronize()
            times+= [time.time() - t_start]

    return np.mean(times)
model1 = ...
model2 = ...
model3 = ...

models = {'ResNet18':model1, 'AlexNet':model2, 'VGG':model3}
learning_rates = [1e-4, 1e-4, 1e-4]

times, top_1_accuracies = [], []
# to_remove solution
model1 = torchvision.models.resnet18(pretrained=True)
model2 = torchvision.models.alexnet(pretrained=True)
model3 = torchvision.models.vgg16(pretrained=True)

models = {'ResNet18':model1, 'AlexNet':model2, 'VGG':model3}
learning_rates = [1e-4, 1e-4, 1e-4]

times, top_1_accuracies = [], []
for (name, model), lr in zip(models.items(), learning_rates):

  print(name, lr)

  model.cuda()
  model.aux_logits = False  # only important for googlenet

  optimizer = torch.optim.Adam(model.parameters(), lr=lr)
  loss_fn = nn.CrossEntropyLoss()

  model_time = train_loop(model, optimizer, imagenette_train_loader, loss_fn)
  times.append(model_time)

  top_1_acc, _ = eval_imagenette(model, imagenette_val_loader,
                                 len(imagenette_val))
  top_1_accuracies.append(top_1_acc.item())
# @title Plot accuracies vs. training speed
def get_parameter_count(model) -> int:
    return sum([torch.numel(p) for p in model.parameters()])

def plot_acc_speed(times, accs, models):
    ti = [t*1000 for t in times]
    for i, model in enumerate(list(models.keys())):
        scale = get_parameter_count(models[model])*1e-6
        plt.scatter(ti[i], accs[i], s=scale, label=model)
    plt.grid(True)
    plt.xlabel('speed [ms]')
    plt.ylabel('accuracy')
    plt.title('Accuracy vs. speed')
    plt.legend()

plot_acc_speed(times, top_1_accuracies, models)<jupyter_output><empty_output><jupyter_text>Exercise 7.2Look at the plot above.It shows the training speed vs. the accuracy of the models you chose.The training speed is measured as the mean time the training takes per epoch.The size of the marker visualizes the number of parameters of the model.Which model seems to be the best for this task and why?Explain your conclusion based on speed, accuracy and number of parameters.<jupyter_code>best_model = '' #@param {type:"string"}
# to_remove explanation

"""
This task is just to see if they understand the plot.
"""<jupyter_output><empty_output><jupyter_text>Exercise 7.3How does the speed correlate with the accuracy? Are faster models also more accurate?<jupyter_code>accuracy_vs_speed = '' #@param {type:"string"}
# to_remove explanation

"""
also depends on the models they chose.
E.g. if we compare VGG with ResNet the ResNet is both faster and more accurate.
AlexNet is a bit faster than ResNet but not as accurate.
"""<jupyter_output><empty_output><jupyter_text>Exercise 7.4Is there a correlation between the number of parameters and the training time? Explain!<jupyter_code>num_params_vs_speed = '' #@param {type:"string"}
# to_remove explanation

"""
The more parameters, the longer the training time.
"""<jupyter_output><empty_output><jupyter_text>--- Section 8: Facial Recognition<jupyter_code>#@title Video 8: Face Recognition using CNNs
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="3q4fKmimZm8", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>One application of large CNNs is **facial recognition**. The problem formulation in facial recognition is a little different from the image classification we've seen so far. In facial recognition we don't want to have a fixed number of individuals that the model can learn. If that were the case then to learn a new person it would be necessary to modify the output portion of the architecture and retrain to account for the new person.Instead, we train a model to learn an **embedding** where images from the same individual are close to each other in a embedded space, and images corresponding to different people are far apart. To achieve this, facial recognitions typically use a triplet loss that compares and two images from the same individual (the "anchor" and "positive" images) and a negative image from a different individual (the "negative" image). The loss requires the distance between the anchor and negative points to be greater than a margin $\alpha$ + the distance between the anchor and positive points.  View and transform the dataA well-trained facial recognition system should be able to map different images of the same individual relatively close together. We will load 15 images of three individuals (maybe you know them - then you can see that your brain is quite well in facial recognition).After viewing the images, we will transform them: MTCNN detects the face and crops the image around the face. Then we stack all the images together in a tensor.<jupyter_code># @title Display Images
# @markdown Here are the source images of <NAME>, <NAME>, and <NAME>
train_transform = transforms.Compose((transforms.Resize((256, 256)),
                                     transforms.ToTensor()))

face_dataset = ImageFolder('faces', transform=train_transform)

image_count = len(face_dataset)

face_loader = torch.utils.data.DataLoader(face_dataset,
                                          batch_size=45,
                                          shuffle=False)

dataiter = iter(face_loader)
images, labels = dataiter.next()

# show images
plt.figure(figsize=(15,15))
plt.imshow(make_grid(images, nrow=15).permute(1,2,0))
# @title Image Preprocessing Function
def process_images(image_dir: str, size = 256):
    """
    This function returns two tensors for the given image dir: one usable for inputting into the
    facenet model, and one that is [0,1] scaled for visualizing

    Parameters:
        image_dir: The glob corresponding to images in a directory

    Returns:
        model_tensor: A image_count x channels x height x width tensor scaled to between -1 and 1,
                      with the faces detected and cropped to the center using mtcnn
        display_tensor: A transformed version of the model tensor scaled to between 0 and 1
    """
    mtcnn = MTCNN(image_size=size, margin=32)
    images = []
    for img_path in glob.glob(image_dir):
        img = Image.open(img_path)
        # Normalize and crop image
        img_cropped = mtcnn(img)
        images.append(img_cropped)

    model_tensor = torch.stack(images)
    display_tensor = model_tensor / (model_tensor.max() * 2)
    display_tensor += .5

    return model_tensor, display_tensor<jupyter_output><empty_output><jupyter_text>Now that we have our images loaded, we need to preprocess them. To make the images easier for the network to learn, we crop them to include just faces.<jupyter_code>bruce_tensor, bruce_display = process_images('faces/bruce/*.jpg')
neil_tensor, neil_display = process_images('faces/neil/*.jpg')
pam_tensor, pam_display = process_images('faces/pam/*.jpg')


display_tensor = torch.cat((bruce_display, neil_display, pam_display))

plt.figure(figsize=(15,15))
plt.imshow(make_grid(display_tensor, nrow=15).permute(1, 2, 0, ))<jupyter_output><empty_output><jupyter_text>Embedding with a pretrained network We load a pretrained facial recognition model called [FaceNet](https://github.com/timesler/facenet-pytorch). It was trained on the [VGGFace2](https://github.com/ox-vgg/vgg_face2) dataset which contains 3.31 million images of 9131 individuals.We use the pretrained model to calculate embeddings for all of our input images.<jupyter_code>resnet = InceptionResnetV1(pretrained='vggface2').eval().to(device)
# Calculate embedding
resnet.classify = False
bruce_embeddings = resnet(bruce_tensor.to(device))
neil_embeddings = resnet(neil_tensor.to(device))
pam_embeddings = resnet(pam_tensor.to(device))<jupyter_output><empty_output><jupyter_text>Think!- What are the dimensions of one input image?- What are the dimensions of one stack of images (e.g. bruce_tensor)?- What are the dimensions of the corresponding embedding (e.g. bruce_embeddings)?- What would be the dimensions of the embedding of one input image? We cannot show 512-dimensional vectors visually, but using **Principal Component Analysis (PCA)** we can project the 512 dimensions onto a 2-dimensional space while preserving the maximum amount of data variation possible. This is just a visual aid for us to understand the concept. If you would do any caluclation, like distances between two images, this would be done with the whole 512-dimensional embedding vectors.<jupyter_code>embedding_tensor = torch.cat((bruce_embeddings, neil_embeddings, pam_embeddings)).to(device = 'cpu')
pca = sklearn.decomposition.PCA(n_components=2)
pca_tensor = pca.fit_transform(embedding_tensor.detach().numpy())
colors = ['blue'] * 15 + ['orange'] * 15 + ['magenta'] * 15

plt.scatter(pca_tensor[:,0], pca_tensor[:,1], c=colors, marker = 'x')
green_patch = mpatches.Patch(color='blue', label='<NAME>')
orange_patch = mpatches.Patch(color='orange', label='<NAME>')
purple_patch = mpatches.Patch(color='magenta', label='<NAME>')

plt.title('PCA Representation of the Image Embeddings')
plt.legend(handles=[green_patch, orange_patch, purple_patch])<jupyter_output><empty_output><jupyter_text>Great! The images corresponding to each individual are separated from each other in the embedding space!If <NAME> wants to unlock his phone with facial recognition, the phone takes the image from the camera, calculates the embedding and checks if it is close to the registered embeddings corresponding to <NAME>. --- Section 9: Ethics: bias/discrimination due to pre-training datasetsPopular facial recognition datasets like VGGFace2 and CASIA-WebFace consist primarily of caucasian faces. As a result, even state of the art facial recognition models [substantially underperform](https://openaccess.thecvf.com/content_ICCV_2019/papers/Wang_Racial_Faces_in_the_Wild_Reducing_Racial_Bias_by_Information_ICCV_2019_paper.pdf) when attempting to recognize faces of other races.Given the implications that poor model performance can have in fields like security and criminal justice, it's very important to be aware of these limitations if you're going to be building facial recognition systems.In this example we will work with a small subset from the [UTKFace](https://susanqq.github.io/UTKFace/) dataset with 49 pictures of black women and 49 picture of white women. We will use the same pretrained model as in Section 8, see and discuss the consequences of the model being trained on an imbalanced dataset.<jupyter_code>#@title Video 9: Ethical aspects
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="9i8fQwd5fak", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video
# @title Download Data
!git clone --quiet https://github.com/richardvogg/face_sample.git
!unzip -q face_sample/face_sample2.zip<jupyter_output><empty_output><jupyter_text>Load, view and transform the data<jupyter_code>black_female_tensor, black_female_display = process_images('face_sample2/??_1_1_*.jpg', size = 200)
white_female_tensor, white_female_display = process_images('face_sample2/??_1_0_*.jpg', size = 200)<jupyter_output><empty_output><jupyter_text>We can check the dimensions of these tensors and see that for each group we have images of size 200x200 and three channels (RGB) of 49 individuals.<jupyter_code>print(white_female_tensor.shape)
print(black_female_tensor.shape)
# @title Visualize some example faces
display_tensor = torch.cat((white_female_display[:15], black_female_display[:15]))

plt.figure(figsize=(12,12))
plt.imshow(make_grid(display_tensor, nrow = 15).permute(1, 2,0,))<jupyter_output><empty_output><jupyter_text>Calculate embeddingsWe use the same pretrained facial recognition network as in section 8 to calculate embeddings.<jupyter_code>resnet.classify = False
black_female_embeddings = resnet(black_female_tensor.to(device))
white_female_embeddings = resnet(white_female_tensor.to(device))<jupyter_output><empty_output><jupyter_text>Calculate the distance between each pair of image embeddings in our tensor and visualize all the distances. Remember that two embeddings are vectors and the distance between two vectors is the Euclidean distance.<jupyter_code>#@title Function to calculate pairwise distances
def calculate_pairwise_distances(embedding_tensor: torch.tensor):
    """
    This function calculates the distance between each pair of image embeddings in a tensor

    Parameters:
        embedding_tensor: A num_images x embedding_dimension tensor

    Returns:
        distances: A num_images x num_images tensor containing the pairwise distances between each
                   image embedding
    """

    distances = torch.cdist(embedding_tensor, embedding_tensor)

    return distances
#@title Visualize the distances

embedding_tensor = torch.cat((black_female_embeddings, white_female_embeddings)).to(device = 'cpu')

distances = calculate_pairwise_distances(embedding_tensor)

plt.figure(figsize=(8,8))
plt.imshow(distances.detach().numpy())
plt.annotate('Black female', (2,-0.5), fontsize=24, va='bottom')
plt.annotate('White female', (52,-0.5), fontsize=24, va='bottom')
plt.annotate('Black female', (-0.5, 45), fontsize=24, rotation=90, ha='right')
plt.annotate('White female', (-0.5, 90), fontsize=24, rotation=90, ha='right')
plt.colorbar()
plt.axis('off')<jupyter_output><empty_output><jupyter_text>Exercise 9.1What do you observe? The faces of which group are more similar to each other for the Face Detection algorithm?<jupyter_code>observation = '' #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Exercise 9.2- What does it mean in real life applications that the WSS is smaller for the embeddings of one group?- Can you come up with example situations/applications where this has a negative impact?- What could you do to avoid these problems?<jupyter_code>ethics_discussion = '' #@param {type:"string"}<jupyter_output><empty_output><jupyter_text>Lastly, to show the importance of the dataset which you use to pretrain your model, look at how much space white men and women take in different embeddings. FairFace is a dataset which is specifically created with completely balanced classes. [Image Source](https://arxiv.org/abs/1908.04913)  Summary<jupyter_code>#@title Video 10: Summary and Outlook
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="MdD6DzqLrLY", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)

video<jupyter_output><empty_output><jupyter_text>Bonus: Within Sum of SquaresWe can try to put this observation in numbers. For this we work with the embeddings.We want to calculate the centroid of each group, which is the average of the 49 embeddings of the group. As each embedding vector has a dimension of 512, the centroid will also have this dimension.Now we can calculate how far away the observations of each group $S_i$ are from the centroid $\mu_i$. This concept is known as Within Sum of Squares (WSS) from cluster analysis.$ \text{WSS} = \sum_{x\in S_i} ||x - \mu_i||^2$where ||.|| is the Euclidean norm.If the WSS is small, all elements of a group are close to each other. If WSS is larger, they are further away from each other.<jupyter_code>def wss(group):
  """
    This function returns the sum of squared distances of the N vectors of a
    group tensor (N x K) to its centroid (1 x K).

    Parameters:
        group: A image_count x embedding_size tensor

    Returns:
        sum_sq: A 1x1 tensor with the sum of squared distances.

    Hints:
        - to calculate the centroid, torch.mean() will be of use.
        - We need the mean of the N=49 observations. If our input tensor is of size
          N x K, we expect the centroid to be of dimensions 1 x K.
          Use the axis argument within torch.mean
    """
  centroid = torch.mean(group, axis = 0)
  distance = torch.linalg.norm(group - centroid.view(1,-1), axis = 1)
  sum_sq = torch.sum(distance**2)
  return sum_sq

print("Black female embedding WSS: " + str(round(wss(black_female_embeddings).item(), 2)))
print("White female embedding WSS: " + str(round(wss(white_female_embeddings).item(), 2)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08293402844328593, 'fraction_numerical': 0.09328401119331468, 'mean_word_length': 7.860628361166148, 'pattern_counts': {'":': 0, '<': 243, '<?xml version=': 0, '>': 248, 'https://': 38, 'lorem ipsum': 0, 'www.': 9, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 29, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '53', 'dir': 'jupyter-structured-clean-dedup', 'id': '82372', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 86583, 'n_tokens_neox': 77310, 'n_words': 9529, 'path': 'tutorials/W2D2_ModernConvnets/W2D2_Tutorial1.ipynb', 'repo_name': 'haltakov/course-content-dl', 'size': '375388'}