Check with a qualified law professional. There could be a contract issue here. For example many contracts that I sign have a no direct competition clause that prevent me from working in the same sector for a year after I leave. One contract had a clause that stopped me from working for anyone else (even myself) at all.
On the other hand if your project is complimentary to what they are doing... maybe you can even build it and sell the company to your employer. Cue the sounds of manager positions and fat raises if done right.
It is possible but you must consult a professional to make sure you are not going to just loose your jobs and then get sued. But if you are safe from that...
Consider talking to the CEO and seeing if (a) he'd back you or (b) he'd give his blessing. With the existing company's leaders backing you there is little chance of failure and a big chance of an early exit with big cash. Consider it the ultimate Dragon's Den / Shark Tank.
It really depends on what is in your employment contract.
Copyright laws state that any written or creative work you produce is automatically copyright to yourself, however work to hire rules override this in some states/countries, by saying that if you are paid to produce this work, then ownership passes to the party who paid you. This creates a slight grey area as to who owns the work. As a result almost all employers will add a clause in your contract to explicitly state that your employer owns the copyright to any work you produce.
Now, depending on how this clause is worded in your contract, you may have issues. Some employers will be clear and the clause will apply only to work produced during work hours, or for the companies line of business/clients. Other companies may explicitly cover everything you produce, whether it's during work time or your own time. Others may not be so clear and the clause may be a little vague as to whether it covers personal work or not*.
Additionally, you may have a non-compete clause in your contract that prevents you from joining a rival company, or starting a company that competes with your existing employer.
I would read through your contract very carefully, preferably with a lawyer, and see exactly what is in there.