<jupyter_start><jupyter_text>Set initial weights for last layer......so that only the last two states (S3 and S4 in write-up) contribute to transcription<jupyter_code>tau_layer_weights = model.layers[9].get_weights()[0].copy()
tau_layer_weights.shape
tau_layer_initial_weights = np.array([0,0,1,1])
tau_layer_initial_weights = tau_layer_initial_weights.reshape(4,1)
model.layers[9].set_weights([tau_layer_initial_weights])
model.layers[9].get_weights()[0]<jupyter_output><empty_output><jupyter_text>The following show non-trainable and trainable weights in the softmax layer<jupyter_code># these weights appear to be in reverse order. 
print(model.layers[8].get_weights())<jupyter_output>[array([[ 0.89298546,  0.80297136,  0.8113314 ,  0.8452251 ],
       [ 0.02258557,  0.8962004 ,  0.81296504,  0.2399528 ],
       [-0.5816989 , -0.7589378 ,  0.90861726, -0.25431544]],
      dtype=float32)]<jupyter_text>Set softmax layer weights manually, will be kept frozen during trainingNote that the bias weights here may be interpreted as gamma<jupyter_code># the following are the input weights to the softmax array.
# the first two arrays indicate 4 weights each coming out of the e_c and 
# e_r nodes. The last node indicates the gamma input, connecting only to
# microstate 4.

softmax_weights = np.array([[0,1,0,1],[0,0,1,1],[0,0,0,1]]) # this works

softmax_weights = [softmax_weights]

# set weights
model.layers[8].set_weights(softmax_weights)
# see set weights
model.layers[8].get_weights()
input_sequences_tensor2 = []
test_input_sequences_tensor2 = []

for _ in range(len(input_seqs_ohe)):
    temp = input_seqs_ohe[_].flatten()
    temp = np.append(temp,1)
    #input_seqs_ohe[_] = np.append(input_seqs_ohe[_],1)
    input_sequences_tensor2.append(temp)
    
for _ in range(len(test_seqs_ohe)):
    
    temp = test_seqs_ohe[_].flatten()
    temp = np.append(temp,1)
    test_input_sequences_tensor2.append(temp)
input_sequences_tensor2 = np.array(input_sequences_tensor2)
test_input_sequences_tensor2 = np.array(test_input_sequences_tensor2)
input_sequences_tensor2[0].shape
labels = np.array(y_train_ohe)
test_labels = np.array(y_test_ohe)
######
## Define custom function for keras
######

def log_poisson_loss(y_true, y_pred):
    return tf.nn.log_poisson_loss(y_true,y_pred)

def poisson_loss(y_true, y_pred):
    y_pred = ops.convert_to_tensor(y_pred)
    y_true = math_ops.cast(y_true, y_pred.dtype)
    return K.mean(y_pred - y_true * math_ops.log(y_pred + K.epsilon()), axis=-1)

model.compile(loss=log_poisson_loss,optimizer=Adam(lr=0.0005), metrics=['categorical_accuracy'])<jupyter_output><empty_output><jupyter_text>load model<jupyter_code># Load the model if the notebook is restarted
model = keras.models.load_model("trained_model_tareen_kinney_mlcb_2019.h5",
                                custom_objects={'log_poisson_loss':log_poisson_loss,
                                                'CustomConnected':CustomConnected})
history = model
predictions_on_test = model.predict(test_input_sequences_tensor2)
model.evaluate(test_input_sequences_tensor2,test_labels)<jupyter_output>10104/10104 [==============================] - 0s 21us/step<jupyter_text>First hidden layer weights (PWMs)Need to input sequence corresponding to CRPs binding site. i.e. fixed the shape of test_input_sequences_tensor<jupyter_code>crp_weights = model.layers[4].get_weights()[0].copy()
rnap_weights = model.layers[5].get_weights()[0].copy()
print(model.layers[4].get_weights()[1])
print(model.layers[5].get_weights()[1])
crp_weights = crp_weights.reshape(26,4)
rnap_weights = rnap_weights.reshape(41,4)


plt.figure(figsize=(12,3))
sns.heatmap(crp_weights.T,cmap='RdBu_r')
plt.show()
plt.figure(figsize=(16,3))
sns.heatmap(rnap_weights.T,cmap='RdBu_r')
#sns.heatmap(rnap_weights.T,cmap='RdBu_r')
plt.show()
import logomaker
crp_logo_df = pd.DataFrame(crp_weights,columns=['A','C','G','T'])

crp_df.columns = ['A','C','G','T']
logomaker.Logo(-crp_df,center_values=True, font_name='Arial Rounded MT Bold')
plt.tick_params(labelsize=14)
plt.title('CRP PSAM Reported in Kinney et al. 2010',fontsize=14)
plt.ylabel("$-\Delta \Delta G$ ($k_{B}T$)", labelpad=-1)
logomaker.Logo(-crp_logo_df,center_values=True, font_name='Arial Rounded MT Bold')
plt.title('CRP PSAM learned by neural network (full-wt data).',fontsize=14)
plt.tick_params(labelsize=14)
plt.ylabel("$-\Delta \Delta G$ ($k_{B}T$)", labelpad=-1)

plt.show()
#logomaker.Logo(rnap_logo_df,center_values=True, font_name='Arial Rounded MT Bold')
rnap_logo_df = pd.DataFrame(rnap_weights,columns=['A','C','G','T'])
rnap_df.columns = ['A','C','G','T']
logomaker.Logo(-rnap_df,center_values=True, font_name='Arial Rounded MT Bold')
plt.tick_params(labelsize=14)
plt.title('RNAP PSAM Reported in Kinney et al. 2010',fontsize=14)
plt.ylabel("$-\Delta \Delta G$ ($k_{B}T$)", labelpad=-1)
logomaker.Logo(rnap_logo_df,center_values=True, font_name='Arial Rounded MT Bold')
plt.tick_params(labelsize=14)
plt.title('RNAP PSAM learned by neural network (full-wt data).',fontsize=14)
plt.ylabel("$-\Delta \Delta G$ ($k_{B}T$)", labelpad=-1)
plt.tick_params(labelsize=14)
model.layers[6].get_weights()
e_i_model = Model(inputs=model.input,outputs=model.layers[6].output)
e_i_node_output = e_i_model.predict([test_input_sequences_tensor2])
e_i_node_output[0]
model.layers[8].get_weights()
model.layers[9].get_weights()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14680772086117297, 'fraction_numerical': 0.0425018559762435, 'mean_word_length': 6.382191780821918, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '653419', 'license': 'MIT', 'n_tokens_mistral': 2153, 'n_tokens_neox': 2004, 'n_words': 341, 'path': 'tareen_kinney_mlcb_2019_run_trained_model.ipynb', 'repo_name': 'jbkinney/19_mlcb', 'size': '5388'}