1. Don't rent a full dedicated office space.
A cheaper alternative is co-working spaces or shared desks.
2. Try to use as many good, free alternatives to paid tools.
E.g. Wave for accounting.
3. When you need to travel, everyone on the team should be flying economy.
This should be a given.
4. Apple's own certified used store.
They're as good as new and if you have to buy a lot of Macbook Pros, you'll save a ton.
The 2 largest expenses a software startup can have is physical office and employees, thus saving on those 2 is a priority. I am not addressing companies that have to have physical presence - manufacturing, retail, research, etc. Chasing cheap products or services can cost you more in time long-term.
1. Consider a distributed team and avoid getting an office space, even a co-working, for as long as you can. You shouldn't need an office anytime pre product-market fit.
2. Don't hire employees if you don't have to - payroll for employees is an expensive item. Hire independent contractors and increase their hours to maintain a long-term relationship. Just make sure you follow IRS guidelines for classifying contractors and employees.
After product-market fit the growing expense line item will be sales and marketing. My new favorite quote is "Advertising is a tax you pay for being unremarkable." - Source. Thus, building some virality into your product helps lower the cost of acquisition.