Ask HN: Any insider takes on Yann LeCun's push against current architectures?

We are actually working on scaling energy-based models http://traceoid.ai

I hope you succeed. I've downloaded and at least skimmed hundreds of papers ML, many alternative architectures. A subset of them built prototypes that claimed good results on benchmarks. Of those, many didn't pass further scrutiny due to various failures. Those that did pass often failed on real-world tasks despite doing well on benchmarks. That we're so jaded by failures of published models makes us even more skeptical of unpublished methods.

So, each architectural advance needs published prototypes solving real-world problems. The smallest I've seen do useful stuff are in 100+M-3B range. There are also papers about testing advances with low, pretraining cost: BabyLM; GPT2 replications; MosaicBERT. Some do straight pre-training while others distill field-proven models. Alternative architectures would do well to crank out examples like this to prove themselves.

Please, do build at least one of the above using your method. Post it to your site. Link to demos of the actual prototype in use. This might get an ecosystem going that builds on your ideas.

How many of these were actually addressing scaling EBMs though? I'm guessing none.

Including yours. Your landing page has no architecture, model, or performance comparisons. It's non-existent. You need something more tangible for us to believe in.

Remember that scientific method requires us to reject everything by default. Only after rigorous review of a working theory or prototype do we treat it as truth. Build what you want us to believe in. Let us see it smoke the competing models of similar size in key metrics. That will do more for you than anything else.

Again, I hope you're right and I get to see energy-based models being highly competitive. I haven't.

> Remember that scientific method requires us to reject everything by default

You are nowhere near as smart as you think you are. You are a STEMlord who has never produced any new knowledge who just repeats some platitudes. People doing actual research do not talk like this.

You might benefit from watching this video https://x.com/styx_boatman/status/1811820327552315805

Our work is very much work in progress. I mentioned it because we have a very promising path to scaling EBMs and I wanted to have a convo about it.

If you were actually curious and you actually cared about my claims, you would have asked some concrete followup questions. You responded with the dumbest cliches, so I will ignore your comments.

I suggested that you put a description of your ideas on the link you use to promote your company. You responded with several insults. Your profile of me was even opposite of the truth. I caution you that, if you're the founder, speaking this way might block great talent who might worry you are similarly abusive to employees.

I can see the inner problem, though, since I was very arrogant. After seeing a miracle, I put my faith in Jesus Christ who died for our sins (even us) and rose again. He turned a cold heart of stone into a warm one of flesh. I no longer feel a need to beat or dominate people online. Even better, I won't burn alive in Hell for it. Even better, He's taught me to serve more humbly.

I believe Christ can help you, too. You can be like the first Adam who led us to sin by selfish choices or like the last Adam who saved us by His self sacrifice. The renewal of the Holy Spirit will cause inner change that permeates your social life, business, everything. You'll be amazed. I pray He also frees you from the slavery of sin, esp arrogance, that once drove my life.