Some signs you might want to pass on an idea:
1. There are no competitors.
If your problem doesn't have a successful competitor, really evaluate the idea. There is no better signal that an idea is worth doing than a competitor who is already making money from it.
2. Costs to acquire customers will be high.
You can often get a sense of this by looking at the average CPC bids in Google Adwords.
3. Long sales cycle
It's very hard to get a bootstrapped startup off the ground that has to deal with long sales cycles. SaaS products targeting large enterprise clients or government agencies fit this profile.
I am going to answer a question about when to decide to scrap an idea.
1. If you discover that someone is already successfully solving the problem. This is often the most common reason people give up on ideas since you have to check for competition before moving to customer validation. A lot of ideas are not original, even if they seem so to us. If you think you can improve on execution, or be different in some meaningful to customers way you can still pursue the idea, at least through validation stage.
2. If you discover that the problem is temporary and is poised to be solved by market trends or changes in industry practices or law very soon.
3. In B2B space, if you research prospective clients and discover that the companies or even the industry in general is in decline or being disrupted (not related to your idea). In B2C, if you realize that consumers are replacing whatever you want to improve with something else completely.