Project Euler #912: Where are the Odds?

I've been thinking recently about how things like Project Euler, LeetCode, and to a bit less of an extent, Advent of Code, are so heavily focused on making clever use of math, data structures and algorithms, that it makes them suboptimal as a tools for getting familiar with a new programming language.

I know that that critique isn't new to anyone but it makes me think about how it would be cool if there were a code puzzler site that is specifically geared towards little self-contained tasks that are more to do with forcing you to get familiar with the common everyday tasks of software development.

Some example puzzlers could be like:

- open an http server on port 80

- open a file and write this data to it

- write temporary files to a location, deleting them when process exits

- query a database

- deal with such and such error scenario

- find a way to test this component

- bundle this code as an executable

- sanitize user input here

- make this behavior configurable

- take the config from environment variable variable and/or config file and/or arguments

- parse this data file

You do get a bit of parsing and file handling with Advent of Code but imagine a series of dozens of small problems that grill you on every corner of the python filesystem api. Would be a lot less dry than reading docs cover to cover.

Have a look at Rosetta Code

> https://rosettacode.org/wiki/Rosetta_Code

This might be a little bit nearer to what you look for.

---

> it would be cool if there were a code puzzler site that is specifically geared towards little self-contained tasks

In my opinion the tasks on Project Euler or LeetCode are much more self-contained than what you suggest as alternatives: many of your suggestions are deeply intertwined with libraries (and understanding them) for doing the respective task instead of being self-contained programming tasks.

to be fair, PE is not designed or meant for helping people learn a language. that isn't the project's intent.

people do like to say they use PE for learning new languages, but I doubt that is a useful exercise beyond maybe the first dozen problems or so. And even then, if the solution isn't obvious to you, you're doing two things at once - learning a language and solving a math puzzle. I don't see why people would sign up to get frustrated like that.

One that I like to suggest, which covers more terminal and Linux skills then programming, is https://overthewire.org/wargames/

That’s just a job. Might as well get paid while you do that kinda stuff

Oh yeah totally, it's not a criticism of PE, that's not what it's meant for. People just use PE and LC and AoC because that's the closest thing, but I think there is space in the market for a product I describe that really drills down on getting you familiar with the common tasks and stdlib of various languages.

I really, really like this list. I've been wondering for the last year what the "optimal" problem is to learn at least the syntax of a language. After learning to run something, how to print to the console, I like using heapsort to start learning syntax of a new language, then reading/writing to a file, then building a small TodoList server.

The best recommendation I have for anyone trying to learn a new programming language is to try and program a board game.

There will be clear rules (business logic), UI, etc.

It's a confined enough problem that you can implement it without too much effort but deep enough that you can get a feel for how that programming language, framework, whatever works.

Plus there's a near endless set to choose from and it's easily scalable to the level of complexity you want. If it works add AI players, network play, etc.

> Advent of Code, are so heavily focused on making clever use of math, data structures and algorithms

I've done a fair amount of Advent of Code and I wouldn't say it's at all "focused" on this. The vast majority of the questions use hash tables and graph traversal as the full extent of their use of math/DS/algos.

There's always one or two puzzles every year that require some particular math/CS insight but most of them just need you to know BFS and/or how to write a parser.

Your examples are also not bad, but they seem to be primarily concerned with "getting familiar with a new programming language" in the context of writing a web server, which is one of the parts of programming I try to stay away from. Most of your examples require less familiarity with the language's features and more with libraries you might use, which is less interesting to me personally (then again, I'm a PL fan and I write compilers for a living).

Meanwhile, I like AoC because I've used language features to take the fairly straightforward potential implementations and write them more elegantly in the language I choose. e.g. I use Monads in Haskell, or use Rust's easy threading to parallelize solutions, etc.

For me, learning a new programming language is largely uninteresting unless it changes the fundamental "shape" I think in, rather than what the exact names of the libraries I use change to. e.g. I already know Java so I'm not really going to bother "learning" C#. I already know Python so I don't really bother diving deep into Ruby, etc. However, I learn Haskell, Rust, Forth, Erlang, Scheme, etc.

I think we are a bit alike in our views, but I have a slightly different take on it. I consider coding something like a Chip-8 emulator to be more fun and optimal. It gives a holistic view of the language - you get to work with simple graphics, sound effects, and gain a feel for memory operations and data structures, as well as control structures like conditionals, looping, and exception handling. If that’s not all - for beginners, it provides an introduction to virtualizing CPUs with registers, stacks, opcode handling, memory units, arithmetic/bitwise operations, and more. You’ll even learn a bit about concurrency and synchronization, and by extension, threading. Also, performance optimization.

I suppose a decent game project could achieve these things too, but the real fun of Chip-8 is in throwing different ROMs at it and debugging the issues until it’s perfect enough to play all your favorite games!

Something I'd love to see is "AoC hard mode": the exact same problems but the input data set is ~10 GB, and/or similarly "scaled" such that naive solutions fail outright.

Other scaling-of-inputs could include: Text with line-lengths over 2 GB, numbers above 2^60, data designed such that naive nested-loop solutions (quadratic scaling) take over a year to compute the answer, etc...

Basically, force developers to solve the problem robustly with: streaming, parallelism, efficient algorithms with good big-O properties, correct data type choice (including intermediate accumulator values!), and so forth.

It could be a three-star challenge feature added to the current version. It wouldn't even require large downloads: a Python script or something similar could be used to generate arbitrarily large inputs. (Alternatively, a common CDN-cacheable prefix with a distinct suffix per competitor.)

That's exactly what Project Euler problems often do, especially once you get past the first hundred or two. Problems are scaled so a brute-force often means hours to days of compute time, or worse.

You get to recognize the effect - if I see a problem that's clearly number-theory related and with a limit of 10^12, I know they're looking for a sublinear algorithm, probably O(n^(2/3)) thanks to various multiplicative function ideas that appear over and over.

AoC is still is algorithms and data structures: there's minimal interaction with the outside world, just solving the problem for the input data. It's just about coming up with the algorithm yourself instead of applying fancy well-known ones.

There are a couple posters on 4chan's /g/ threads on AoC that create "Big Boy" inputs, which is what you're looking for. It's unofficial, though.

Most of these wouldn't qualify as "puzzles", would they?

I find it nice to learn new languages via data structure puzzles, because to me the data structures of a language feel like the grammar and once I have that down everything else falls into place

If you're trying to model those "puzzlers" on actual dev work, then doing any of those things without a library/framework is a wrong answer.

Or is that your point? That coding like a pro means gluing those things together?

> It's just about coming up with the algorithm yourself instead of applying fancy well-known ones.

Isn’t that the whole fun?

> just solving the problem for the input data

I think a lot of people here would be surprised how much of a step up this is from classic leetcode or DSA stuff. I have been involved in introducing people to AoC and helping them and the amount of people who have basic knowledge of algorithms but struggled to parse the input from a file was a little shocking to me at first. Of course, I do not blame anyone for not knowing something, classic academic courses can be misguided sometimes.

This doesn't negate the fact that there is somewhat of a lack in problems with more outside world interaction and it would be cool to see more of that.

Codewars can do this, they keep how problems was solved. If a problem was solved using some string by most answers than it is a good problem to learn that part of the language. With documentation you have really good way to cover all parts of a programming language.

> I don't see why people would sign up to get frustrated like that.

I actually use this as a learning trick. Pick two or three things to learn simultaneously, then when I get stuck on one aspect, switch to another. When I finally switch back, I often find the background time I gave my brain to process the problem means I'll now be much faster to get unstuck on the original issue.

There's definitely ways this can go sideways, and it's not for everybody, but I find it pretty effective.

I really think you'd like these two project pages, they were posted to HN some time ago but the original links have been slain by bitrot.

https://austinhenley.com/blog/challengingprojects.html https://austinhenley.com/blog/morechallengingprojects.html

A few different things:

1. Yes, solving these puzzles would mean often mean using a library.

2. However, for most of the things I listed above, in most languages, a competent software developer should be able to, and most likely would just use the standard library.

3. Why not both? I can imagine a catalog with thousands of problem sets. Some may challenge you to (re-)implement some existing functionality yourself (as a super basic example, re-implement the java Optional::flatMap method or something). Others could challenge you to make use of existing implementations. Learning to make use of stdlib and other libraries and tools in the ecosystem is part of one's growth, and also so is working through those tools, tinkering, trying to think how you would have implemented them, and getting a better understanding of their internals (or at least, an understanding of how their internals MIGHT work)

I disagree. Yes, you have to learn how to work with the basic data structures of a language, but 90% of programming, for most people, is not that. It's IO, error handling, db querying, logging, input parsing, parameterization, business logic, preserving backwards compatibility, persistence, state management, testing, mocking, benchmarking, build design (for lack of better term -- futzing around with Make/Gradle/Npm, Dockerfiles). All of that doesn't just fall out of learning DS/Alg's, it takes time to become familiar and fluent in how all these are done in your ecosystem.

When employers or team mates ask you if you "know" or "are competent in" Java they don't care if you know how to work with lists, arrays, loops, hashmaps and sets. Well, I mean, that's table stakes. They're asking if you're familiar with the idiosyncrasies of the language with respect to those above concerns.

I guess the trouble is that in the case of LeetCode/PE, people actually just want the one thing (programming language proficiency), and not the other (mathematics). Learning 2 things at once can be very useful, but a lot of people would probably choose something else for their second skill. Most relevant for developers might be, networking, OS's, version control, databases, testing, CI/CD, security, concurrency.

I think you overestimate the proportion of programmers who primarily work on corporate stuff.

Exercism.io does what you want? It has language tracks and each track has questions geared to seal your understanding of some language concept. It also has it gamified by building a community around it and folks comparing their solutions.

Exercism.io does what you want? It has language tracks and each track has questions geared to seal your understanding of some language concept. It also has it gamified by building a community around it and folks comparing their solutions.

Thanks, I'll take a look!

Imo DS/Algs to an extent are a good prerequisite. Once you know them in one language that knowledge is portable enough to get you up-to-speed in $lang. Then, and no earlier will I start worrying about doing real things in my programs and the "idiosyncracies"