I keep reading about how search engines like google keep changing and optimizing their algorithms so I wonder how effective SEO is beyond the basics of having the right keywords, robots.txt, etc and if instead its more valuable to focus on building a business and getting into various blogs, press etc. to naturally boost search ranking.
Do reputable SEO firms generally deliver positive ROI results?
In the past 10 years I have worked with several people who claim to be experts on SEO or with companies who say the same. I always found most of the strategies are not worth paying for. You can easily find these yourself with Dr. Google.
But recently I was hired by a company who need SEO for their business model. They are earning immense lots of money with a perfect SEO for their websites in combination with ads. I was working on a different area, but when I saw what they did, I was simply impressed. That company actually does not offer SEO consultation.
They told me that I was actually right: 95% (if not more) of the SEO people are always saying the same; this "knowledge" can be found on various blogs. But there are a few people who really create "strategies" which they apply. I think its more than "SEO". One of these recommendations was Karl Kratz. Unfortunately he does only have a german blog.
That said I think you should do the basic seo stuff you can find on the net. Then try to get into press and blogs. This is - for me - the most efficient way to be ranked.
I would only hire a seo company/consultant if I really trust him and he has proven some expert knowledge to me which I didn't read on the blogs before.
To answer you question, reputation does not necessary mean they can help you and your product. I might mean that they had some luck, because they had the right customers before.
With tools like those offered by seomoz, many of the evaluation / monitoring tasks that were hard to do have become simpler. But monitoring your competition, tweaking content, etc. still takes time (both ongoing) to get good results.
Sure, there are lots of people would will attempt to sell (as you say) snake oil solutions that promise immediate results - but that isn't unique to SEO optimization.
In addition, driving traffic to a site doesn't automatically mean more revenue - if the offer isn't valued by the visitor, then everything was a wasted effort. Understanding what your conversion rate is, and optimizing it first before traffic is worthwhile. This infographic is pretty good in explaining the CRO process:
The entire CRO guide is worth a read.
If you want to grow your business with internet leads I feel you or someone at your organization needs to get reasonably versed in the general SEO concepts.
Yes, good SEO companies are worth it. Yes, you can often do lots of these things yourself. It depends where you are in your business - if cash is a major concern and you have plenty of time then you are in a good position to learn a lot of take action.
Most of the work good SEO companies do are not on your website but working on strategies for link building, writing great content to get published on other websites to get links and quality traffic back to your website. Setting keywords, content density, title tags, etc. are just the minimum, must do basics. That's entry level. Most people use SEO to mean SEM, and encompassing a lot of different things. But just on your site SEO work does't cut it any more. It's required - but just the start.