Clearly, for this communication to happen, there must be a language in common between the two agents. The sensory data or the category models themselves can not be shared between the agents because both are using different sensors and hence different feature representations. The feature representations can not be concatenated because there is no alignment between the two sources of data from which the agents are learning. It is not clear how a robot with a Lidar sensor (say) can take advantage of an annotated but unaligned and disjoint set of RGB images (or their features) taken by a robot in a different environment. Additionally, due to security, privacy and/or bandwidth concerns, it may not be feasible for the agents to exchange the sensory data (or the featuresA lot of progress has been made on inverting features to visualize the original image [1]}, [2]}.). These concerns arise for a variety of applications such as robots operating in our homes (privacy, and perhaps bandwidth concerns), robots being used for defense or security purposes (security concerns), robots responding to disasters (bandwidth concerns), or agents in autonomous vehicles (bandwidth concerns).
