Once a cat is done playing in your yard, they will leave a thank you gift – usually silver fish but sometimes gold fish (in-game currencies) which can be used to buy upgrades for your garden. I am saving for the Yard Expansion which will double the amount of usable space. There are also many different remodeling options once the yard has been expanded.
Currently there are over 40 different cats to collect and rarer cats require certain combinations of food/toys. Each cat also has one memento to share ("proof that we have become a cat's friend"). So far it doesn't seem like you get a reward for taking pictures of the cats, but it is another fun thing to collect.
There is a very extensive Neko Atsume Wiki that I recommend checking out if you have any questions about the game and a frequented Neko Atsume community on Reddit if you're looking to connect with other players.
Available for free download on both iOS or Android devices.