<jupyter_start><jupyter_text>Test Linear RegressionThis notebook is an example that will test the generalization capability of a regression to word vectors. There are three corpora involved. Required Data1. The _word vector corpora_  * Examples: New York Times, Wikipedia Text8  * Data: Pretrained word vectors (word2vec, etc.)2. The _training corpora_  * Examples: IAPR-TC12, MSCOCO, Visual Genome  * Data:    - Training image features and text labels    - Testing image features and text labels <-- Used as validation data3. A _testing corpora_ with a different vocabulary  * Examples: MSCOCO, Visual Genome, etc.  * Data: Training and testing image features Imports<jupyter_code>import numpy as np
import matplotlib.pylab as plt
import sys

## Should probably update to PYTHONPATH
sys.path.append('/data/fs4/home/kni/attalos/')
# ls /data/fs4/home/kni/attalos/
from attalos.dataset.dataset import Dataset
imdir = '/data/fs4/teams/attalos/features/image/'
txdir = '/data/fs4/teams/attalos/features/text/'

## Import word vector load in
import attalos.imgtxt_algorithms.util.readw2v as rw2v

## Import linear regression
import attalos.imgtxt_algorithms.linearregression.LinearRegression as linreg

## Import evaluation code (right now, using Octave soft evaluation)
# from attalos.evaluation.evaluation import Eval
from oct2py import octave
octave.addpath('../evaluation/')

reload(linreg)
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Load the word vectors in<jupyter_code>F = rw2v.ReadW2V('/data/fs4/teams/attalos/wordvecs/text9Bvin.bin')
vectors = F.readlines()<jupyter_output><empty_output><jupyter_text>Load training corpora in<jupyter_code>train_dataset = Dataset(imdir+'iaprtc12_train_inception.hdf5', txdir+'iaprtc12_train_text.json.gz')
test_dataset = Dataset(imdir+'iaprtc12_test_inception.hdf5', txdir+'iaprtc12_test_text.json.gz')
D = open('linearregression/data/iaprtc_dictionary.txt').read().splitlines()
reverse_D = dict()
for i in range(len(D)): reverse_D[D[i]] = i 
    
xTr, yTrWds = train_dataset.get_next_batch(17565)
xTe, yTeWds = test_dataset.get_next_batch(1962)
data = np.load('linearregression/data/iaprtc_alexfc7.npz')
train_ims = [ im.split('/')[-1] for im in open('linearregression/data/iaprtc_trainlist.txt').read().splitlines() ]

test_ims_full = [ im for im in open('linearregression/data/iaprtc_testlist.txt').read().splitlines() ]
train_ims_full = [ im for im in open('linearregression/data/iaprtc_trainlist.txt').read().splitlines() ]<jupyter_output><empty_output><jupyter_text>Create probability distribution for words<jupyter_code>wordvecs = np.zeros((len(D), 200))
for i, word in enumerate(D):
    if vectors.has_key( word ):
        wordvecs[i] = vectors[word]
    else:
        print "{}: {}".format(word,i)
        
distvecs = 1 / (1 + np.exp( - 0.1*wordvecs.dot(wordvecs.T) ) )
# distvecs = np.tanh( 0.1*wordvecs.dot(wordvecs.T) )
distvecs = distvecs / np.linalg.norm( distvecs, axis=1 )

wdvecs=dict()
for i, word in enumerate(D):
    wdvecs[word] = distvecs[i]
    
plt.stem( wdvecs['shore'] )<jupyter_output>bedcover: 15
table-cloth: 259
tee-shirt: 261<jupyter_text>Convert multi-hot vectors to distribution vectors1. wordvecs -> distvecs2. onehot -> distvecs  Convert all images in the training set to distribution vectors<jupyter_code>yTr = np.zeros( (len(yTrWds), len(D)) )
for i, imtaglist in enumerate(yTrWds):
    for tag in imtaglist:
        # if tag in ['tee-shirt','bedcover','table-cloth']:
        #     continue
        yTr[i]+=wdvecs[tag]
    if yTr[i].sum() == 0:
        print "vector "+str(i)+" has no tags"
yTe = np.zeros( (len(yTeWds), len(D)) )
for i, imtaglist in enumerate(yTeWds):
    for tag in imtaglist:
        yTe[i,reverse_D[tag]] = 1<jupyter_output><empty_output><jupyter_text>Load testing corpora in -------------------------------  Train and validate<jupyter_code>print xTr.shape
print yTr.shape
mp_solution = linreg.LinearRegression(normX = True, normY = True)
mp_solution.train(xTr, yTr)
yHat = mp_solution.predict(xTe)<jupyter_output>(17565, 2048)
(17565, 291)
Building W matrix = Y \ X = Y^T X (X X^T)^-1<jupyter_text>Test  Evaluate the regression<jupyter_code>[precision, recall, f1score] = octave.evaluate(yTe.T, yHat.T, 5)
print precision
print recall
print f1score<jupyter_output>0.234626174307
0.230973034904
0.232785273152<jupyter_text>Visualize<jupyter_code># Randomly select an image
i=np.random.randint(0, yTe.shape[1])

# Run example
imname='linearregression/images/'+test_ims_full[i]+'.jpg';
print "Looking at the "+str(i)+"th image: "+imname
im=plt.imread(imname)

# Prediction
ypwords = [D[j] for j in yHat[i].argsort()[::-1] [ 0:(yHat[i]>0.2).sum() ] ]
# Truth
ytwords = [D[j] for j in np.where(yTe[i] > 0.5)[0] ]

plt.imshow(im)
print 'Predicted: '+ ', '.join(ypwords)
print 'Truth:     '+ ', '.join(ytwords)

plt.figure()
plt.stem( yHat[i] )<jupyter_output>Looking at the 121th image: linearregression/images/02/2843.jpg
Predicted: 
Truth:     desert, front, man, middle, plane, woman
{'fraction_non_alphanumeric': 0.1374824894936962, 'fraction_numerical': 0.027816690014008404, 'mean_word_length': 5.014440433212997, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '178729', 'license': 'Apache-2.0', 'n_tokens_mistral': 1948, 'n_tokens_neox': 1841, 'n_words': 412, 'path': 'attalos/imgtxt_algorithms/TestLR-FullDist-GoogleNet.ipynb', 'repo_name': 'karllab41/attalos', 'size': '4997'}