"Uh, what exactly do you mean by that?"
If you've got a fixed use case to build something, you don't need to make a generalized implementation. Just build the dirty implementation with hard coded variables and values first. First, make it work. IFF you need more flexibility farther down the road, you can always come back and refactor the code. I've noticed that when I make my first iteration pass on solving a problem, it's certainly not going to be the last one, and it's certainly not the 100% perfect & correct solution. Only around the ~10th pass does it approach intended perfection, and that's only because all of the actual uses for a system have been implemented and generalized appropriately.
So, in short, do this for new systems:
1. Hard code the first implementation. Make it work. Move on.
2. Later, if you have a new variation, you can come back and just barely generalize the first implementation to be good enough. Make it right. Move on.
3. IFF you identified a performance problem empirically, you can optimize the problem. Make it fast.
Gradually, your hard coded solution rises to meet the exact level of flexibility your system needs to support, AS NEEDED. No more, no less. It's very lean. Everything which has been built is actively being used. Best of all, you save a lot of time.
"Why do this? It seems hackish and like a 'cowboy coder'".
The problem with "big design up front" is that the designer has a really difficult time accounting for factors which are only discoverable during implementation. The engineering tendency / instinct is to just say, "I can't predict every problem the system will encounter, therefore, I will try to engineer this solution to be as flexible as possible to account for future problems." For example, if your "need" is to build a dog house for your dog, you shouldn't let your design account for the future possibility of having multiple dogs, or your dog gaining weight, or switching from a small dog to a large dog. Just build the damned dog house today for the dog you have and be done with it, and if its not good enough later, change it then.
I understand what you mean. Thing is, when you build up this "right amount of flexibility" along the road, you might essentially make it useless.
IE, the case for flexibility isn't that given the current use case, if it could be considered flexible enough. Because if the current case was that set in stone, flexibility wouldn't be necessary, you'd just hardcode everything. The idea of flexibility is for allowing modifications in the future.
Now, if you build it along the road, in each iteration you're essentially saying "Hey, this is how the code should have been in the first place for accommodating the changes I'm making right now!". That way you're not saving time at all, you're just postponing the refactoring.
I like to think of this "thinking ahead" as a skill like any other, something you train and develop.
If you make a solution that for the current set of requirements can be considered as "over engineered", but it happens that some time later, that over engineering becomes the exact thing you need for the different requirements you have at that time, that's a win.
If you overengineer and you never make use of it, that's a loss.
I'm kinda sure you can just develop this kind of coding, rather than trying to shy away from it as you suggest. Then again, as you said, this is really difficult, so in many cases it might be worthwhile just to go with the shorter/hackier road for the time being...