Should I be concerned that my direct competitor just signed up for a paid account on my SaaS product?
IANAL, but unless your signup agreement specifically limits competitors from signing up, there is nothing illegal here. Competitors buy, disassemble and evaluate each others products for inspiration all the time.
If you don't want to have them in your system, then revoke / refund their account. First check the IP address range and then put some rules that block them. Of course, they will be able to circumvent this easily, but it may help you sleep at night.
Or - take another tact. Closely monitor their usage / clickstream. See what they are looking at. Then sign up for their account & compare their functionality to yours. At least it gives you a glimpse into what they are researching.
But - in the end, its your customer base and their actions that is most important to monitor - not your competitor.