I'm developing a service that would collect subscriber's money for the webmaster, and we'd pay the website owner at the end of the month.
So for example, 10 subscribers pay "us" (the service) $5/month, and we pay the "website owner" their share ($50 minus fees) at the end of the month.
So, the question is: do we (as a US-based company) have to pay tax for the amount collected on behalf of the website owner?
Your income is taxable. It is your income, and you're a US-based company, so you're liable for taxes in the US. Of course you can deduct your business expenses to calculate the net taxable income, your accountant will help you with that.
The fact that your subscribers are non-US has no relevance, they're not the ones taxed. If you're remitting money to non-residents (for example, if the "website owner" is in India), then you also have to withhold 30% (unless the treaty says otherwise) from their distribution and remit it to the IRS on their behalf. If they should pay less - they should claim refunds from the IRS on their own.
You should talk to your accountant/tax adviser about the reporting rules and obligations of businesses.