Here's what I'd change about my decisions:
Don't take on bad clients and customers. Go with your gut feeling from time to time. If you get bad vibes from a new customer, they'll likely end up costing you more in support and stress. ING Direct did this -- every quarter they would ask 1% of customers to take their business elsewhere (not in those exact words). These were the customers that used a majority of the support resources.
Validate your idea in under a week. I've spent too much time of my life building things without validation only to find out no one wants it. Here's my process now.
Create a team of advisors. This is especially important if you're a solo-founder. Your friend Steve from college doesn't count -- find people who have done the startup grind several times before.
Exercise. One of my biggest regrets. I spent far too many years eating crap until I started reading about how the human body works and eventually settled on a Paleo diet -- and my energy level went through the roof. I'd suggest reading Dave Asprey's Bullet Proof Executive blog.
Don't try to please every customer. It's always better to have a few who are fanatic about what you do best.
Do things that don't scale. Too often I had spent time building procedures and systems which were never used as planned. Just do some things manually even if they don't scale.
Hire an accountant from day one. It's easy to try to avoid this expense in the beginning, and then regret it once you have to raise money.
In a startup that I was an early employee in, things fell apart because we waited too long to get a public facing website up and running. We didn't even get started on it until the product was ready for prime time and it took months, during which time, we ran out of funding and the whole thing fell apart.
Looking back at it, I have three regrets.