Niche News / Forum Aggregation Service Legality / ToS Violation?
I'm operating in a nice profitable niche right now. There are a lot of news and blog sites within the industry and a few very popular forums. My question is would either of the following services be legal / within ToS or would you need explicit permission from the site owners:
- A site that datamines all the news and blogs, provides a blurb w/ up voting / commenting and direct link to the news site to read the full article (like google news).
- The same sort of thing but the site datamines forums and posts trending threads and topics, exerpts and summaries w/ direct links to the actual forum posts.
-
It really is a grey zone in most cases. There's an [interesting discussion on Quora about this](http://www.quora.com/What-is-the-legality-of-web-scraping). – Phil Go20 13 years ago
-
If you were to always link back to the original article you should be ok. It would help if you didn't include the entire article and required the user to click on the original source to read the full article. – Ryan Doom 13 years ago
3 Answers
(1) should be legal based on "fair use" copyright provisions. It's similar to Google News, Reddit, Digg, and several others.
(2) might be OK, depending on the ToS. Most sites would be happy for the referrals, but the lawyers are in charge at some companies and you might have to exclude those sites.
It does not only depend on the terms of services of the website, it even depends on the law regulations of the different countries. For example Google News got big trouble in germany because they are aggregating the news of different newspapers. Usually one could say this is not a problem, because they are linking to the original articles. But it is.
Even when not disallowed by the terms of services (you are citing) you probably need to know about cite rights of the country.
I don't know what Google finally did, you might find that out with a bit of googling.
But you could try to build a service, were people are including their blogs themself. Usually this is called "planet" like this page: http://planet.apache.org/committers/ There is free software available to aggregate that content: http://intertwingly.net/code/venus/ You just need to make sure you have the allowance of the different bloggers who want their blog to be added.
I think you really have to read the terms of the sites you were interested in.
Also, bear in mind that if Google does something, it doesn't necessarily mean you can too. They may have licensed the data, or have some other agreement with the source.
