Is it legal to display album art and artist images on my website?

I'm working on a music review website and I was wondering what the legality was of displaying album art and artist images on the site. Do I need permission from the labels/artists?

Unfortunately in US Copyright law there is nothing that allows you to do this. There is a "fair use" right and it is very possible that what you want to do could be OK under fair use but since "fair use" is so vague there is no way you can be sure you'll be ok.

With an issue like this it depends what your risk tolerance is. If you're not afraid of being sued and you're willing to defend your rights then you go ahead. If you don't want the headaches then you may want to go without.

Since "fair use" does provide for the use of limited portions of materials for commentary purposes you might be able to build a review website and use some copyrighted materials but in the end only a judge can decide whether what you are doing is "fair use" or not.

The things that will work against you are:

1. I'm assuming the site will be for-profit (you'll run advertising for example)
2. The site will probably be reviewing the music, not the album cover so the material being taken is not really what is being commented on
3. The effect upon the potential market for the work

There are things you can do to to minimize these issues:
2. allow and encourage people to review the album art itself
3. only display the art at low resolution so no one would want to print it therefore not replacing the value of the original.

Unfortunately this issue is complicated enough that you'll want to get a lawyer involved to really examine what you want to do and figure out how to minimize (as you can't eliminate) your risk.

EDIT: I found an article that explains some of the reasons this questions is never clear-cut at: http://www.digitalphotopro.com/business/is-fair-use-really-fair.html

Is there a site that already has the permissions and the images and also has an API and/or affiliate program that would allow you to also display the images on your site?

Seems like Amazon.com might be a good place to start. You might also want to check out http://remix.bestbuy.com as it looks like the top link in their sample gallery is a site that is displaying album artwork.