How to Harvest Cashmere. Cashmere is the down that grows on a cashmere goat during late summer an autumn to protect the goat with extra warmth during winter. It's the short, soft fibre growing close to the skin, underneath the more coarse hair. Getting the cashmere for use in human products is a case of accurate timing and good after-care for the goats to ensure that they don't suffer from the cold. Here is how to harvest cashmere.