Long story short, it's completely over-engineered by a bunch of intellectual engineers with no focus, no discipline, and no oversight. It ended up not delivering on any promises it made, and there were a lot of them.
I was warned left and right before presentations and meetings, "this customer hates your product because of ...." I started off every meeting with saying, "we're rearchitecting the product, this is how we're doing it, this is the tech we are using." Immediately there was a sense of relief from customers, followed by questions like, "why can't <current product> deliver <feature> that was promised?" I'm completely honest with bad decisions that were made and how it impacted the feature. Sure, there is skepticism on what we are doing, and I tell them they should absolutely be skeptical based on our track record. The result has been customers who have hated my product now offering to work with us on development.
I've also been completely forthcoming on configuration, security, resources, and setup issues I am finding, many of them are absolutely freakin' insane. I've flat out told customers it's frankly embarrassing and never let us do something like this in the future. The best feedback on this was, "At least you're telling us something. We usually get silence from this team."
God, this is the most depressing job ever.
I remember my first few days on the job just being ripped to shreds by our customers who (understandably) were slighted. Don't miss those days at all.
I'm curious what you'd like to do next. You could probably have a great career doing these sorts of turnarounds repeatedly across companies, maybe even as a consultant, but would you want to?
As a future representation of past me, I can tell you:
1. Everything it’s making you feel is valid.
1b. If you’re feeling burnt out, please listen to it. It gets worse if you let it.
2. While I can’t hire you now, I can already tell you’re eminently hireable. If you have any cautious inclination to move, you will probably be better served by greener pastures.
3. Just take care of yourself.
4. When 3 contradicts 2, favor 3.
It's hinted by the C-level that if I can pull this off, it would be nothing short of a miracle. I'm pretty sure I can negotiate salary, education, bonus, and what not if I can pull this off.
As far as next, I've thought about that. It would be funny to call myself a turnaround specialist. This would be quite a remarkable feat, but I really don't know if I would have taken this job if I knew what a mess this was...
First off, it helps I've been 15+ years as an engineer, 5 as an engineering manager, and throughout have the community contributions in the field on my resume. I instantly spotted the problems when I was given an architecture diagram on day 1 and discussed what I would do differently. All that gives credibility.
If it's internal audience, I am brutally honest. The organization needs to know this wasn't happenstance and bad luck that put us where we are now. It was a deliberate series of bad decisions based on a poor engineering and product culture. Now, for better or for worse, we are tasked with paying the debt.
There's a certain class of customers that are sister companies under the same parent. I'm honest with them, too, but go on the offensive. They have abused my team and our company in the past, and unfortunately, we have let them. I am more than happy to fire back and go toe to toe with bad behavior, and at the same time working to fix critical support issues.
For external customers, I've had remarkably good response in listening to their complaints. I am honest in discussing, in deep engineering detail, how the new product will address their problems, where issues might still be, and development timeline. I like to think the credibility portion comes into play here. In the past, customers were just told, "We'll look at it" and "We'll fix it" but nothing was ever planned.
My boss is supportive, but he's also under heavy fire. Like I mentioned, my peers are rightfully skeptical. My team are a bunch of sharp, good guys, but haven't had any good guidance of mentorship in years, if not decades. They're all different, but what the have in common is that they've been screwed and judged unfairly thanks to past incompetence. That just pisses me off.
There's hope from those around me, but it's a pretty darn lonely job. You just gave me the fuel to not feel already beat up when I walk in the door tomorrow.
i do enjoy programming, working with other devs, but as soon as i stepped into a product management role it's a hellishly different set of problems. you're in the middle of the tech, the developers, the problems, and the customers. lots of lessons in there. tiring, but worthwhile.
Do this up front. Do it as soon as you possibly can. You will lose a huge amount of negotiating leverage if you "wait until you show them". I cannot stress this enough.
Partially so I could learn from mistakes and partially since I’m a sucker for post-mortems :)
That's just to take a database log, put it into JSON, and zip it up.
That all for just one step in the data pipeline. The others are slightly less hairy, but 75% of this pipeline is literally just moving data around. It goes from JSON to MySQL to Postgres to Parquet. There is no data enrichment at all during these steps. It literally just unpacks from one format, packs to another and repeat.
The whole fucking thing is just one big masturbation circlejerk for a bunch of engineers that have thankfully been RIFed/forced out...
One trick you might try: write future press releases. This helps you look beyond the immediate problems and focus on the destination. For example:
“Q3 2023: ACME CO released version Z today, which dramatically simplifies our engine to focus on core user needs. ‘It does thing I want an doesn’t crash anymore’ says Key Buyer #1”
By writing this down, you can put the vision in front of everyone. Then check it against actual progress to see how you’re doing.
I am not sure how much you could negotiate but you can have something like that and being metric based. X% customers happy, x% rating change, x% customers retained when they were close to leave. Then you make the math of the revenue and profit and it’s hard to say no.