Q:

Logging LSTM internal gate activations during recurrent timesteps

I am currently working with the recurrent LSTM cells in keras.
For analyzing my network I want to look at the i, c, o and f values in the LSTM cell during a time series prediction.
Essentially, I want to have a logging tensor, which logs the internal LSTM cell values during the recurrent calculations.
I already tried to simply print out these values, but I think since the code is only run on compilation and is not included in the graph, I don't get the expected behaviour. As far as I understood TF's internal tensors are declerativ and don' hold any data outside sessions.
I additionally tried to modify the complete LSTM class to have a logging variable, but I didn't get that to work. I currently lag the complete knowledge about the code structure to find a way to output these values.
Basically I want to log these 4 values, located in the call function of LSTMCell:
i = self.recurrent_activation(x_i + K.dot(h_tm1_i ,self.recurrent_kernel_i))
f = self.recurrent_activation(x_f + K.dot(h_tm1_f,self.recurrent_kernel_f))
c = f * c_tm1 + i * self.activation(x_c + K.dot(h_tm1_c, self.recurrent_kernel_c))
o = self.recurrent_activation(x_o + K.dot(h_tm1_o, self.recurrent_kernel_o)

Is there a simple built-in way in keras, to log internal LSTM cell data during the recurrent/unrolled calculations? If not, what would be the best way, and place to start looking at? Is there a different approach or library for adding a logging functionality?

A:

As of now, I was not able to implement gate activation logging in keras.
I think the best solution for achieving logging in keras is to split the LSTM layer into two intermediate layers and have a print out layer in between. This involves implementing two costum layers.
However, I switched from an keras based implementation to matlab. The matlab neural network implementation and structure allows for easy code edits, and implementing a logging feature is straight forward.

