I am trying to create an autoencoder where a few inputs can affect the other inputs, but not the other way around. An example of this is given by the following picture. I looked into defining my own Keras layers, but don't know how to proceed. 
Keras create custom network with certain unconnected neurons
neural network;keras