Tell the player in advance when they are going to get a refill. Maybe the first or second time they will assume there is a boss battle between now and then, but after that they should get the hint.

Alternatively, make using the weapon so fun that they don't WANT to save it.

How to prevent the "Too awesome to use" syndrome
In my experience, cloister tiles in Carcassonne are often "too lucky". If you draw a cloister tile in the beginning of the game, it will typically still require an investment of quite a bit of "meeple" time to obtain the full 9 points, which makes it a fair trade-off. However, after about half of the game, it's relatively likely that you can "parachute" a cloister tile in some spot and get 8 or 9 points immediately. This adds a lot of randomness to the game.

What house rules work well to diminish this effect?
