<jupyter_start><jupyter_text>Motivating example: Interactive parallel face detection with OpenCV To get a sense of what IPython.parallel might be used for,we start with an example.We will revisit pieces of this example as we learn about the different components of IPython.<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import sys, os, re, time

import numpy as np

import ipyparallel as parallel<jupyter_output><empty_output><jupyter_text>First, initialize OpenCV for simple facial detection<jupyter_code>HAAR_CASCADE_PATH = "haarcascade_frontalface_default.xml"
# if you have opencv installed via homebrew, this would be in
# /usr/local/share/OpenCV/haarcascades/
# If via Conda, it will be in:
HAAR_CASCADE_PATH = os.path.join(sys.prefix, 'share', 'OpenCV', 'haarcascades', HAAR_CASCADE_PATH)

import cv
storage = cv.CreateMemStorage()
cascade = cv.Load(HAAR_CASCADE_PATH)<jupyter_output><empty_output><jupyter_text>Then define a few functions for extracting faces from images<jupyter_code>def extract_faces(image, faces):
    """Returns any faces in an image in a list of numpy arrays"""
    import numpy as np
    A = np.frombuffer(image.tostring(), dtype=np.uint8).reshape((image.height, image.width, image.nChannels))
    A = A[:,:,::-1]
    face_arrays = []
    for face in faces:
        Aface = A[face[1]:face[1]+face[3],face[0]:face[0]+face[2]]
        face_arrays.append(Aface)
    return face_arrays


def detect_faces(filename):
    """Loads an image into OpenCV, and detects faces

    returns None if no image is found,
    (filename, [list of numpy arrays]) if there are faces
    """
    
    image = cv.LoadImage(filename)
    faces = []
    detected = cv.HaarDetectObjects(image, cascade, storage, 1.2, 2, cv.CV_HAAR_DO_CANNY_PRUNING, (100,100))
    if detected:
        for (x,y,w,h),n in detected:
            faces.append((x,y,w,h))
    if faces:
        return filename, extract_faces(image, faces)<jupyter_output><empty_output><jupyter_text>Since we don't trust the network, we can just build a list of images from anywhere on our filesystem.Any list of images will do.For instance, you can use the path to the 'Thumbnails' directory in your iPhoto library,which vary from ~320x240 - 1024x768.<jupyter_code>pictures_dir = os.path.join('..', 'images', 'portrait')<jupyter_output><empty_output><jupyter_text>This will search `pictures_dir` for any JPEGs or PNGs.See the [download images](download impages.ipynb) notebook for a quick way to populate a folder with images from Wikimedia Commons with a certain tag.<jupyter_code>import glob
pictures = []
for directory, subdirs, files in os.walk(pictures_dir):
    for fname in files:
        if fname.lower().endswith(('.jpg', '.png')):
            pictures.append(os.path.join(directory, fname))<jupyter_output><empty_output><jupyter_text>Let's test our output<jupyter_code>for p in pictures:
    found = detect_faces(p)
    if found:
        break

filename, faces = found
for face in faces:
    plt.figure()
    plt.imshow(face)<jupyter_output><empty_output><jupyter_text>Hey, that looks like a face!  Now in parallel First, we connect our parallel Client<jupyter_code>rc = parallel.Client(profile='py2')
all_engines = rc[:]
view = rc.load_balanced_view()<jupyter_output><empty_output><jupyter_text>Then we initialize OpenCV on all of the engines (identical to what we did above)<jupyter_code>here = os.getcwd()
%px %cd $here
%%px
HAAR_CASCADE_PATH = "haarcascade_frontalface_default.xml"
import os, sys
HAAR_CASCADE_PATH = os.path.join(sys.prefix, 'share', 'OpenCV', 'haarcascades', HAAR_CASCADE_PATH)

import cv
storage = cv.CreateMemStorage()
cascade = cv.Load(HAAR_CASCADE_PATH)<jupyter_output><empty_output><jupyter_text>and make sure `extract_faces` is defined everywhere<jupyter_code>all_engines.push(dict(
    extract_faces=extract_faces,
))<jupyter_output><empty_output><jupyter_text>Now we can iterate through all of our pictures, and detect and display any faces we find<jupyter_code>tic = time.time()

amr = view.map_async(detect_faces, pictures[:1000], ordered=False)
nfound = 0
for r in amr:
    if not r:
        continue
    filename, faces = r
    nfound += len(faces)
    print "%i faces found in %s" % (len(faces), filename)
    for face in faces:
        plt.imshow(face)
        plt.show()

toc = time.time()

print "found %i faces in %i images in %f s" % (nfound, len(amr), toc-tic)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10573881608676006, 'fraction_numerical': 0.008133755083596927, 'mean_word_length': 4.871352785145889, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '402745', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1536, 'n_tokens_neox': 1475, 'n_words': 427, 'path': 'parallel/examples/Parallel face detection.ipynb', 'repo_name': 'minrk/ipython-cse17', 'size': '4426'}