Is it possible for a conference not to be published? I was quoting a paper from the ICLR 13 and noticed that I could not find any information about the publishing of the conference. So far I have only seen proceedings which published the papers of the conference in a book entitled 'proceedings of the xth international conference ...'

Q: Is it possible for a conference not to be published, i.e. the papers can only be found online (or via a database, e.g. arXiv). Is this a common thing? Why?
The important distinction that you want to pay attention to here is whether the proceedings are archival or not.  Fewer and fewer publications (conference or journal) are actually put onto dead trees: it's expensive and heavy and takes up lots of space in your office.

An archival publication is one whose proceedings are in the custody of an institution that guarantees their long-term preservation and availability.  This can be anything from a traditional journal to a large society like IEEE or ACS to a database like arXiv or even a memo collection like the IETF RFCs.  If it's archival, it's a "real" publication because it will be there if somebody goes looking for it in 20 years.  Prestige of the venue is a completely orthogonal question...

Anything that is not archival is transient, and might vanish at any moment.  This happens often with communities that don't have peer reviewed conferences (e.g., many biology communities).  This is fine if the point of what you wrote was to have a ticket to give a presentation or to have a discussion.  If it's not archived somewhere, though, it's not really a publication, because there is no persistent reference that can be followed.