Q:

how to use tf.metrics.recall_at_k properly?

I am a bit confused therefore I need your help and leading! 
I am having a sample dataset. I have 2 sources and 4 targets as shown in the following matrix. Each cell is representing a score from a source and a target. 

Ground T Label for S1=T1, 
Ground T Label for S2=T3
If I am not wrong, the recall@k is calculated like the following: 
the recall@1 result for S1 and S2 should be 0 + 0 = 0 (Since S1 and S2 do  not have the highest top-1 scores to the Ground T labels)
Likewise, the recall@2 result for S1 and S2 should be 0 + 0 = 0 (Since S1 and S2's top 2 results are still not containing the Ground T labels)
Finally, the recall@3 result for S1 and S2 should be 1 + 1 = 1 (Since S1 and S2's top 3 results are now containing the Ground T labels)
If I am correct, could you please lead me how to implement this toy example in tensorflow (estimator API) by using tf.metrics.recall_at_k or tf.metrics.recall_at_top_k (I do not know the exact differences between them) to have the same result as I showed?
Thank you so much

A:

First, let's talk about the difference between tf.metrics.recall_at_k and tf.metrics.recall_at_top_k. 
If you look at open source code, you will find precision_at_k is a simple wrapper around precision_at_top_k. precision_at_k applies tf.nn.top_k first, and then calls precision_at_top_k. Documentation shows that precision_at_k expects a float tensor of logits values, but precision_at_top_k expects integer tensor the predictions to be the indices of the top k classes.
So if your value is a logit score values, you should use precision_at_k.
There are also some mistakes in the calculation method. When you calculate recall@1 result for S1 and S2 should be (0 + 0)/2=0 (Since S1 has the highest top-1 score to T4 label not to the ground T1 and S2 has the highest top-1 score to T1 not to the Ground T3 label).
import tensorflow as tf
import numpy as np

y_true = np.array([ [0], [2]]).astype(np.int64)
y_true = tf.identity(y_true)
y_pred = np.array([[1, 2, 1, 4],
                   [3, 2, 1, 0]
                   ]).astype(np.float32)
y_pred = tf.identity(y_pred)
k = 1
_, update_recall = tf.metrics.recall_at_k(y_true, y_pred, k)

tmp_rank = tf.nn.top_k(y_pred, k)
stream_vars = [i for i in tf.local_variables()]

with tf.Session() as sess:
    sess.run(tf.local_variables_initializer())
    print("update_recall: ",sess.run(update_recall ))
    print("STREAM_VARS: ",(sess.run(stream_vars)))
    print("TMP_RANK: ",sess.run(tmp_rank))

#print
update_recall:  0.0
STREAM_VARS:  [0.0, 2.0]
TMP_RANK:  TopKV2(values=array([[4.],
       [3.]], dtype=float32), indices=array([[3],
       [0]], dtype=int32))

You can try to change the value of k to see the recall@k.

