Is it legal to reverse-engineer a software in the US?

I'm creating a mac application that supports a file format from a closed-source application. We tried to contact the company but they refused to make their proprietary file format available.

Would it be legal for me to reverse-enginner their software and figure out myself how to read and write files in their file format?

I'm based in Australia and the company whose file format I want to reverse engineer is in the United States.

This question is asking about what is legal, and I am not a lawyer. A definitive answer probably requires talking to one who specializes in this sort of stuff and can look at all of the documents that are important.

Before I started doing some research for this question, I thought that the answer to this was a definite "no you can't", but that's not true.

What I'm seeing is that US law doesn't specifically prohibit reverse engineering, but that they've ruled that if an End-User License Agreement (EULA) prohibits it, that overrides the default law. (Reference.) I know the majority of EULAs explicitly state this, and I would go as far as to say the US Supreme Court case I referenced would push all "smart" companies to adding that to their EULAs. You'd have to check, specifically, on the EULA that comes with the product the file format is associated with.

In the European Union, it appears that reverse engineering of something is legal, as long as it isn't to make a competing product, and as long as you don't distribute what was learned publicly. In other words, you can do it for your own learning/benefit, but not in a way that would allow you or anyone else to compete against them. I suspect that if a person reverse engineered the format for fun it wouldn't cause any problems, even with a USA-based license. But since you want to create a product that actually consumes the file format, that probably wouldn't fly.

Sometimes, when a company really wants to protect their file format, they'll also provide a code library that allows you to load the file format in. Autodesk does this with their FBX format for 3D models. FBX is heavily protected (and to my knowledge, they're pretty active about enforcing it) but they have a DLL that will load FBX files and give them to you as in-memory objects. I think that DLL is free, but they may charge. You may discover that the file format in question has a similar thing available.

Your last option, if none of the above works out, is that you may be able to work out a deal to pay a small amount in royalties for every piece of software that you sell to be able to read their files, in exchange for them allowing you to reverse engineering the format (or perhaps they'll give you the documentation on it, or they'll give you code that will read it). This, of course, requires signing a legal contract with them that solidifies the details, so you'd have to work through them to get it.