LLMs are now commonly used as parts of larger, more complex systems. It is crucial to understand how these models interact with information with varying degrees of trustworthiness, accuracy, and uniformity. Our analysis shows that further work is required to characterize the risks of using LLMs to answer questions given contextual information. In particular, we find that model behavior can be erratic and unpredictable when presented with information that exists at the margin of its prior beliefs.