How can a non-engineer founder know if the developer they hired is any good?

I'm a non-technical founder who is hiring my first employee, a Ruby on Rails engineer. While I'm great at execution on the business end, I'm clueless when it comes to the actual tech that my product is built on.

The first iteration of my startup's MVP was built by a contractor I hired on UpWork (formerly ODesk). They did an okay job for the low price point.

How that the idea has proven itself to be successful, I want to build out the technology in a proper manner and lay the right foundation.

How can I judge a programmer's skills when I myself don't know how to code?

Do you have any programmer friends? Maybe you can invite one of them over to do a technical interview with the candidates for you. Good programmers can quickly spot bad ones.

Aside from that, I'd suggest you have them actually do some programming in the interview. Give them some sort of thing to do or make (or optionally let them choose their own) and have them actually produce something during an hour or two interview. If you want, you can have them show you what they're doing as they do it. Them explaining what they're doing as they do it will help shed some light on their thought process, help you start to understand the technical side of things a bit better, and based on their work process and the produced results, you ought to be able to compare several candidates against each other.

You may not be able to distinguish the amazing programmers from the decent ones, but the bad ones will stand out like a sore thumb. (Note that an interview is never enough to distinguish amazing from decent anyway.)

You pretty much got two options: Like rbwhitaker says - get someone you know to be a good programmer to validate the skills (best option)

Or

See some of the past projects handled by this developer, and from there try your best to analyze if those projects seems to be well build.

You have a few options.

Firstly, know what you want them to do or the product they need to build. Be able to explain this in a concise manner. Most of the time a dev can't show his skills because he/she doesn't understand what the person wants them to create.

Secondly, ask about their experience. What they have built, their exact role on the project. Keep referring back to their CV and if they are confident you know you have a solid dev.

Third on the list is looking at their online profiles. Github, Stackoverflow and linked provide a wealth of information and any committed developer will be working on side projects. If their only profiles are blank or don't exist, code is a job to them and not their passion.

There are lots of ways to Interview Developers Without Technical Knowledge if you know what you want. Don't forget that they are interviewing you as a non-techie also so put your best foot forward!