It's all to easy for us to get a bit neurotic about getting a great image and finding nothing great to photograph. While you could 'work' the scene harder in the belief that it's there, you only need to find it, but perhaps it's more productive to admit that while there may be an image here, it isn't here, today, and it's time to move on. A frequent occurrence is to come home with a good shot that has nothing to do with what you went out to take, that the spot you anticipated would work well doesn't, but 10 minutes later you stumble on something really good.
Some photographers will wait for hours for the right light, others move on in the hope that the time they could have spent sitting for the right circumstances is better spent looking for another image.
I guess if you know this is going to be a fabulous image if the light is right, it's worth waiting just about any amount of time. If you think it might be a good image if you wait, then it's perhaps better to cut and run.