I am familiar with Gumroad as someone who has used it to buy things. From what I recall, the UX was smooth but left me wondering about the ease of cancellations / ways to handle refunds. I recall not feeling like the product lacked something critical - perhaps it was Paypal integration, but I don't remember. Something about it didn't sit right with me as a customer.
From a UX perspective, I feel that some of the commenters here are onto something. The website does not make it clear what the product does. Looking a bit into it, it feels like there are two products in one - payments and audience building - something that I would expect to be separate. It's like the product is trying to do 2 things well instead of 1 thing really well. Of course, this is my personal opinion which could be harsh and unfair seeing how I am not really qualified to give advice. In all, you know your customers and their needs best and it's great to hear you havent given up on gumroad. I hope things go your way and you accomplish everything you set out to do.