←back to thread

Show HN: Visual flow-based programming for Erlang, inspired by Node-RED

(github.com)
206 points Towaway69 | 4 comments | 16 May 25 14:54 UTC | HN request time: 0s | source

Hi There,

Erlang-RED has been my project for the last couple of months and I would love to get some feedback from the HN community.

The idea is to take advantage of Erlangs message passing and low overhead processes to have true concurrency in Node-RED flows. Plus also to bring low-code visual flow-based programming to Erlang.

Show context
oersted ◴[16 May 25 17:17 UTC] No.44007812[source]▶
> Node-RED is a amazing[*] tool for creating flows that describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use something that is multi-process from the ground up?

That's great! But now that we are doing this, it kind of makes me wish that it was not multi-processing and Erlang, but a more mainstream language with a better library ecosystem and focused on multi-threading instead, Rust comes to mind but there could be a better one.

Is there a visual flow programming language based on Rust that is relatively mature?

replies(6): >>44007924 #>>44007929 #>>44008197 #>>44008520 #>>44008981 #>>44011466 #
notpushkin ◴[16 May 25 17:59 UTC] No.44008197[source]▶
Note that this uses (I hope!) Erlang processes, which are not to be confused with OS processes.
replies(2): >>44008237 #>>44008362 #
macintux ◴[16 May 25 18:17 UTC] No.44008362[source]▶
Yep. I researched JVM and .NET thread sizes once, to compare/contrast with Erlang processes, and the difference is ludicrous.

Erlang: ~1k

64-bit .NET thread: 4MB

replies(3): >>44008595 #>>44009037 #>>44009315 #
SigmundA ◴[16 May 25 19:35 UTC] No.44009037[source]▶
.Net uses real OS threads and that 4mb is the stack not the heap which must be allocated on thread creation. The heap is shared in the process amongst all the threads whereas Erlang is separate lightweight green thread/processes with dynamic heap and stack per process.

.Net originates in the Windows world where new processes are expensive but threads are much cheaper however they still use native stacks for iterop and hardware protection, stack must be contiguous so it is fixed at thread creation this is typical for all native os threads.

Erlang is a VM and runs on top of of processes and threads but exposes its own process model to the language that does not map 1:1 and therefore does not have a fixed stack size allowing many light weight processes. It pays an indirection penalty in performance for this but allows some nice stuff.

Java started with green threads and abandoned them, both .Net and Java use asynchronous concepts instead to get high concurrency now days cooperatively releasing threads and rentering them. There was talk of trying green threads out in .Net and bringing them back in Java for ergonomics compared to async, these green threads would not have such large fixed stacks and many more could be made similar to Erlang processes.

replies(1): >>44009347 #
neonsunset ◴[16 May 25 20:07 UTC] No.44009347[source]▶
In the end, .NET is going hybrid route. Circa .NET 10/11 Roslyn will stop emitting state machines and something state-machine-like will be produced by the compiler itself iff the asynchronous execution suspends, and will be otherwise practically free.

https://github.com/dotnet/runtime/blob/main/docs/design/spec...

Mind you, short-lived operations are much cheaper with stackless co-routine model like in C#/F# or Rust than spawning an entire, even if virtual, thread/process.

replies(1): >>44009463 #
SigmundA ◴[16 May 25 20:20 UTC] No.44009463[source]▶
I get why they went the async route due to performance implications and such, just like the old cooperative multi-tasking days everything was much lighter weight but one misbehaved process not yielding can block the whole OS.

Nice to see further improvement coming along in .Net however I conceptually prefer the everything is a lightweight process model to this function coloring async virus that invades the codebase now days, wouldn't be so bad if it had been easy to do sync over async from the beginning without caveat in .Net so you didn't need to have dual sync and async api all the way down the stack.

replies(1): >>44009646 #
1. neonsunset ◴[16 May 25 20:42 UTC] No.44009646[source]▶
Usually dual APIs are offered when they do distinctly different things. Async calls on Socket go through epoll/kqueue via SocketAsyncEngine, which has observably different behavior to regular sockets. There is no such duality in high-level APIs like HttpClient's `.GetStringAsync`.

You are not free from having to worry about it with virtual threads either. If you block a carrier thread with such a low-level API call, the runtime will have a bad time. Or you will always end up paying for a context switch (if you maintain a separate pool of block-able threads).

Green threading UX is better for "regular" code that never bothers to take advantage of concurrency features, but if you do want them, you are quickly back to usually more ceremonious structured concurrency abstractions which, lo and behold, can reimplement the same task system with a worker pool but on top of virtual threads.

If you start to care about cancellation, suddenly you are back to coloring problem regardless. I agree with you that having to write many await's in quick succession is tiresome and unnecessary. It's partially a fault of library authors who name methods "GetSomeLongNameAsync" instead of "Get" or "Query". But other than this it's not as invasive as people make it out to be, and encourages writing highly concurrent code - reducing latency 2x can require moving one or two awaits around. Not so much in Java/Erlang/Go.

replies(1): >>44009776 #
2. SigmundA ◴[16 May 25 20:58 UTC] No.44009776[source]▶
I haven't found this is the case many times, many api will have two versions of the api with XxxAsync versions all the way down, maybe less so now days but it took a while to get async entry points added everywhere so you could use it properly, such as ActionResult.ExecuteResultAsync and IAsyncEnumerable as a couple examples.

HttpClient one prime example and you can see the back and forth here internally from MS: https://github.com/dotnet/runtime/issues/32125

This snippet of code or something like it has been used in countless libraries to try and give a workable sync over async which should be baked in: https://github.com/aspnet/AspNetIdentity/blob/main/src/Micro...

replies(1): >>44009801 #
3. neonsunset ◴[16 May 25 21:01 UTC] No.44009801[source]▶
I suppose the difference of opinion comes down to whether you consider hiding side effects a good or a bad thing. And whether you want to have easy composition and cancellation (Go struggles with passing context around a lot, in .NET it's much less of an issue if at all, depending on codebase).

If you don't want to join on some other virtual thread or goroutine completing immediately (and with Goroutines it's additionally painful since you need to communicate completion separately), it's more difficult and costlier than doing so with tasks/futures.

replies(1): >>44009870 #
4. SigmundA ◴[16 May 25 21:14 UTC] No.44009870{3}[source]▶
CancellationTokens can be passed in synchronous functions just fine, I have done cancellable sync apis before cancellation tokens existed usually its a .Cancel() on the thing doing the work just like Command.Cancel() in the Sql client or Connection.Cancel() in a network client. I agree CancellationTokens are better and always use them now and thats not the problem with async.

The problem is the need to color functions with async and if you have existing client code that call existing synchronous methods it not easy to move their internal implementations to async form so that you can remain backward compatible without copy/pasting the same layers of code with different function coloring to async.