Against a calling station no... you have to show them a hand, against the weak tight this works really nicely. When you push on a draw at any stakes, you have to be prepared that they will call, you are bluffing and it will fail a good deal of the time when you are called, but it works every time that they fold
In your instance he had a hand he was willing to play for all the money, you had set yourself up for the bluff to work really nicely with the preflop raise, it just didn't work that time against that opponent.
$50 NL, I call a raise from the big blind with 35d and pick up a gutshot straight and flush draw on a K high flop. I lead, he raises I push. My thinking is AK KK AA call, QQ-88 missed big Ace that doesn't have the flush draw folds almost all of the time, there is also the slim chance he will mistake my aggression for a set and fold everything except KK. Adding in the folding equity even though he had raised preflop was so high, that pushing on a draw where if he has a one pair hand and having about a 45% chance of making a legit hand, makes this a positive ev bluff with 5 high on the flop.
Losing a buy in isn't the worse thing in the world, and you are shown to be a gambler win or lose, which can lead to more action later and that is always nice. I don't like to make these moves as big stack against a big stack, or with a fresh buy in. I want to do it when I'm well ahead of the game, especially against someone with less than a full buy in, or against a player that I know I have a fair shot of pushing around. When I get to about $120 the variance I am used to is about $20-$30 swings either way in the .25/.50 game, and this continues until I quit playing, I usually leave with a triple or better stack. So there is room to gamble it up and make the all in on a draw move at the lower levels, you just have to pick proper spots to do it.
Edit:
I should add that I don't advocate making a habit of pushing on a draw, but that there are times where it is a good play.