←back to thread

Open Riak – open, modern Riak fork

(github.com)
219 points amarsahinovic | 7 comments | 19 Nov 24 20:29 UTC | HN request time: 0.231s | source | bottom
Show context
tibbar ◴[19 Nov 24 21:09 UTC] No.42188120[source]▶
I've never met an engineering team that used Riak, but it is used heavily as an example technology in Kleppmann's 'Designing Data Intensive Applications'. (I would say, informally, it's usually the example of the "other way" as opposed to other more well-known databases.) This does make me wonder what became of it, why it didn't take off.
replies(8): >>42188162 #>>42188238 #>>42188584 #>>42188647 #>>42188675 #>>42188741 #>>42191594 #>>42191757 #
macintux ◴[19 Nov 24 21:24 UTC] No.42188238[source]▶
Speaking as a former tech evangelist/engineer at Basho, there were a few significant challenges.

Riak is horribly unfriendly as a database: no SQL, it exposes eventual consistency directly to the developer, it’s relatively slow, and Erlang is a fairly unusual language.

While you can run Riak on a single server, you’d have to really want to.

Its strength is the ability to scale massively, but not many projects need that scale, and by the time you do, you’re probably already using some friendlier database and you’d rather make that one work.

replies(5): >>42188533 #>>42188580 #>>42188614 #>>42188779 #>>42191772 #
binary132 ◴[19 Nov 24 21:56 UTC] No.42188533[source]▶
I wonder if some of these issues could be addressed sanely in an extension to the functionality
replies(1): >>42189569 #
1. macintux ◴[20 Nov 24 00:24 UTC] No.42189569[source]▶
We were working on ways of making it easier (such as CRDTs to reduce the amount of work developers had to do to leverage eventual consistency), but these were pretty challenging problems to solve.

One of our biggest disappointments: we had plans to add a way to enforce strong consistency leveraging (IIRC) something akin to multi-paxos, but couldn't get it to work.

replies(2): >>42190465 #>>42190678 #
2. binary132 ◴[20 Nov 24 02:59 UTC] No.42190465[source]▶
that sounds like a painful session
3. jtuple ◴[20 Nov 24 03:50 UTC] No.42190678[source]▶
TBH, we shipped fully working strong consistency in 2014. It just had a limited feature set, was disabled by default, and was never promoted/marketed since it didn't fit the direction the new CEO/CTO was pushing.

The engineering exodus around that time sorta killed the project though, and we never were able to do the big follow-up work to make it really shine.

(Disclaimer: Former Basho Principal Engineer, primary author of strong consistency work, lead riak_core dev from 2011-2015)

I think another 18 months would have been enough too. But it just wasn't the right environment after the hostile take-over / leadership transition.

replies(4): >>42190710 #>>42190924 #>>42191749 #>>42195459 #
4. macintux ◴[20 Nov 24 03:59 UTC] No.42190710[source]▶
My apologies for misremembering. I’m glad you chimed in to correct the record.
5. NickM ◴[20 Nov 24 04:54 UTC] No.42190924[source]▶
There were customers happily using strong consistency in production, but somehow the idea that it wasn’t “finished” kept getting repeated over and over by management. I was well on my way to solving the biggest rough edge (tombstone reaping in SC buckets) but then I got pulled off to work on the infamous “data platform” and never got to finish that work :-(
6. masleeds ◴[20 Nov 24 08:13 UTC] No.42191749[source]▶
I'm not sure though for how much longer it will continue to make sense for the project as-is to continue to roll riak_ensemble forward as part of future releases. As there are no contributors who have direct direct experience or knowledge of using it in production, so it is hard to claim it as being a supported part of the product in any real sense.

I apologise if we do eventually cut it. Having worked through the code when chasing unstable tests, I developed an appreciation for the quality of the work.

7. binary132 ◴[20 Nov 24 16:22 UTC] No.42195459[source]▶
If you don’t mind my asking, how did you find working with Erlang in such a context? I’ve always been curious about it, but these days it seems like most people are mostly interested in Elixir for REST APIs and I am content with my existing tools for that purpose. However, there are clearly places where OTP and other Erlang functionality would shine and I’ve always thought it might be fun to keep in my back pocket.