Slack has raised our charges by $195k per year

Slack's business model has always been that you give them all your most critical data and they sell you access to it. This is basically the business model of the traditional kind of ransomware, before people got better at making backups.

You probably should expect large bill increases over time from ransomware-as-a-service companies like Slack. Not all of them—people are capable of behaving decently—but probably the nature of the category is such that you should expect it of most of them.

When switching providers is impossible, the pricing of maximum profit for the provider is the pricing where the buyer is exactly zero. Slack presumably doesn't have quite enough information about their clients' businesses to calibrate this exactly, but if they can approach it approximately, they'll make a lot of money; even though they drive some of their customers out of business, those losses are compensated for by the higher revenues from their surviving customers.

I was cancelling my annual slack premium last month and had to click to acknowledge that some of my members are using the AI features and they will lose access to them.

They then offered me a discount and if I refused there was another checkbox where I accepted that I was about to cause disruption for other staff.

I was tempted to take the deal until that point, but I'm the only member of the organisation and I absolutely do not use their AI

That sounds quite a bit like fraud.

Pretty sure it's perfectly legal marketing at least in the US.

They are verifiably false statements made for the purpose of monetary gain. I guess the question would hinge on intent: did they just forget to check if anyone is using those features and if there is anyone who would be disrupted, or are they intentionally deceiving users by purposefully not checking?

By the spirit of the law, yes, it likely is fraud. I doubt you could argue it is by the letter of it, though. Normally fraud involves lying to someone to get them to enter into a business relationship with you, not to keep one. Besides, regardless of how many people were using specific features of it, the service is what it is. This wouldn't be unlike you calling your ISP to cancel your subscription and they asking you if you're sure you want to cancel such a great service. If the service factually sucks ass compared to other providers wouldn't make it fraud. All that matters is that it meets the specifications that were sold to you.