I had a cheap drill press that never drilled straight holes and often found myself drilling freehand instead. Like many people, I tried the cheaper option first and was constantly frustrated.
Others say it's not cheap—and they're right. But I prefer quality over cheap. During a recent project that required a lot of precise, repeatable holes for threaded inserts, I finally decided I'd had enough of mistakes and wasted time.
It may not be the most-used tool in my shop, but it will see far more use than any drill press ever did.
Save up and get the right tool the first time. Seriously.