Spaced repetition systems have gotten better

Spaced repetition has been all the rage for 20 years now.

Dozens of apps, thousands of lectures, and it turns out its not really a silver bullet.

There's nothing really wrong with it, it's just that people tend to fall off the same way they do on any other education pattern.

A couple years ago I was thinking "If Google and Apple really cared about kids they would make a spaced repetition unlock system", where by you have to make note cards every week and then have to answer correctly to get into your phone. (obviously requires some bypass system, other rules, etc)

You could probably jury rig it with a popup that comes up after you unlock, but people would never install it anyway.

If you're expecting it to be a silver bullet, then you're in for a bad time.

You still have to do the work.

It's a lever or a pulley, nothing more.

> There's nothing really wrong with it

And a gigantic amount right with it.

This is a strange comment because it shrugs off something that has been transformative and hugely useful to a lot of people because it doesn’t fix all conceivable problems.

> You still have to do the work

Spaced repetition is doing the work.

I think this is a brilliant idea! Surprised nobody has built it yet. I would definitely use it.

I would expect the flashcards produced under this regime to be utterly useless, like a flashcard with "A" and the answer is "B", or simple math problems. In other words, Goodhart's law.

Spaced repetition is time-optimised, but it isn't self-discipline optimised, nor motivation-optimised. If you're limited by time, it's very efficient, but it drains motivation. If you're anywhere close to being limited by motivation (or, failing that, self-discipline), it just causes burnout and failure.

I credit Anki to my success at GCSEs and A Levels despite having a head injury, and I also credit it to me burning out so hard I took a gap year!

And I'm enjoying the gap year, but Anki made it a near necessity.

The "work" is actively trying to recall the information on the card. Spaced repetition is just a more efficient way of doing this than (for example) cycling through every single card, every single day.

I don't think it's a strange comment. He's mostly right (and so are you, but I think you're talking past each other). There's nothing wrong with SRS, and I agree with you that it's basically like cheat codes for memorization, but there is a limit to what most people can do. i.e. most people do tend to drop off.

I remember reading some stats from WaniKani (Japanese SRS app) a while back...

WaniKani has 60 "levels" to learn 2000+ kanji. Each level takes about a week (there's no skipping ahead), so the material takes about a year of study to complete -- that's if you're going at breakneck pace, which most people aren't.

According to the numbers I saw on the WK forums, ~8% of users reach level 30 and less than 1% reach level 60... and that's just to learn as much kanji as a 9th grader. That's to say nothing of the grammar and the 20,000+ vocab words you'll need to SRS to truly learn the language, or the thousands of hours you'll have to spend speaking/listening/reading, immersing yourself in native content, etc.

People give up very easily. The language learning community often gives year estimates to reach "near-native level" in a language based on frequency of study. In reality, the process takes a lifetime. I don't know if people truly know what they're signing up for when install those apps and begin studying. It's a lifelong commitment. It's just something you do now, every day.

You can stop at any time of course, and most people do (more than 99% of them apparently).

There's some UX problems of SRS (that I'm working on) that makes it high friction 1) Time taken to create cards 2) Need for self marking 3) Creates a one to one mapping of prompt-answer 4) If you're an autodidact, you have to teach yourself first (alternatively called understanding, scaffolding, etc)

More fundamentally, SRS isn't a superpower because it's just very specific to creating a direct prompt retrieval. Generalization is poor. Even creating a graph of knowledge, is a chain of edges between bits of knowledge, isn't done very well here.

And I suspect there's a very deep, fundamental difference between recollection knowledge and logical-modeling knowledge. Recollection seems very similar to a dictionary access, and if you recorded the time to recall in humans I suspect they'd all be constant. But learning the knowledge of a logical model, like of a mathematical concept, appears to be vastly different and have very different time to compute.

Proponents of SRS will point out logical models need facts as well, like formulas, lemmas, etc. Which is true. But if you already grasped it before you'd grasp it faster the second time. So the practical use of SRS is a significant step above having a very well sorted and labeled notebook, but still way below becoming a genius.

This is a very big problem. Virtually all the results from research here comes from some form of simple word recall. Direct recall occupies some part of real world tasks, but IRL if you're stopped by doing something it's people not because you can't remember it (and you could look it up if you forgot).

There was an interesting post here awhile back about autonomy and motivation. The gist was people's motivation is proportional to their autonomy. This is quite intuitive, you can see people are really motivated when they have autonomy (think kids with Minecraft, musicians with instruments). One terrible thing about Anki is that it probably is horrible for autonomy. Quite possibly using anki actually has a negative effect on motivation.

> Dozens of apps, thousands of lectures, and it turns out its not really a silver bullet.

I mean, you say that, but I did mandarin for maybe 6 months, I did reviews for maybe a year or two on and off, I haven't done a review of mandarin for 8, 9 years now and I can still recall quite a bit of it. So for me it's worked quite well.

I don't understand, wouldn't it be worse for motivation to take longer to achieve the same results?

Learning a language as a hobby is tough. If you don't need the language to communicate and survive in your environment then you have essentially zero real motivation to learn it.

The problem with spaced repetition systems is that it doesn't supply that extra motivation. You're still just memorizing things in a vacuum. If you truly want to learn a language you need to use it to communicate. That means making friends, travelling, reading books, and consuming other media in that language.

I would say the implementations are time optimized, over the others. I’m building a language learning application, and have put in a lot of effort to make sure that the Spaced Repeition is motivation-optimized.

It’s centered around your performance and review times, to make sure you aren’t struggling to much; no due dates to avoid Anki slogs; gamified with some internal mechanics; dopaminergically influenced with aspects of randomness.

Spaced Repetition is just an equation (SM2 is laughable simple), but a lot of applications just slap a UI on it and call it a day, but that’s not the only way to use it!

Where do we find more about what you’re working on? :)

Motivation is some combination of real and perceived effort Vs expected reward. Shorter isn't always better. For eg. Counting every single calorie is the shorter way to lose weight, but for most people, eating approximately healthy is more optimal from an effort /motivation poi t of view.

Poor generalization (overtraining on prompts) and loss of context over time are the biggest issues I've found with them. Slow card creation workflows and needing to rate your own reviews are merely UX issues -- losing context and losing generalization make SRS actively harmful when used for some topics.

There's 2 solutions I've thought of but haven't tried implementing:

1. A free-recall based approach. Free recall allows you to operate at a higher level of organization and connect concepts at lower levels. However, how you would schedule SRS with free recall is not clear.

2. Have an LLM generate questions on-the-fly so that you don't overtrain on prompts. You might also instruct the LLM to create questions that connect multiple concepts together. The problem with this approach is that LLMs are still not so good at creating good test questions.

Spending half an hour mind-numbingly learning words through flashcards will teach you about as much vocabulary as an hour watching educational videos, but it'll be far less fun and you'll feel like it actually took two hours.

Keep that up every day and you'll burn out much faster with option 1 than option 2. Now, maybe you have enough motivation for that not to matter, or the self-discipline to keep going - as I did in my A levels - but don't be surprised if it kills your interest in the subject.

What's worse for motivation than taking longer?

Boredom?

Feeling like what you're doing is low-quality or superficial?

Doing something artificial for purely external reasons like grades or exams?

Can't speak for anyone else, but for me I would take slower progress over any of these... which makes spaced repetition a hard sell.

I‘m not super well versed in the literature but I know this has been researched, and—unless you are being hyperbolic—it completely fails the sniff test.

As OP points out, SRS is optimized for memory retention. You will almost certainly encounter many more words watching a two hour long video, but you certainly won‘t retain nearly as many words as half an hour of SRS.

Actually you can combine the two. Use the two hour long video to encounter new vocabulary in context, put the new vocabulary in your Anki deck, and review it with optimized SRS. You get the best of both worlds. As a bonus you often remember the source which will help you recall... This is actually common enough pattern that it has a name: Vocabulary mining.

You can also be motivated because you like to consume media in the language (relevant for English and Japanese), because you think it will be useful for a job (English) or for travel (French, Spanish, etc.), or simply because you like the language.

If you don’t have any of these reasons to motivate you, the question arises of why you’re bothering in the first place.

> Dozens of apps, thousands of lectures, and it turns out its not really a silver bullet.

Easy statement to make when you're not defining the silver bullet. Kind of like saying dieting turns out not to be a silver bullet.

I've used spaced repairing for over 6 years. It's been transformative for me.

You say that but it's completely revolutionized the way medical students study.

IIRC the effect was so profound they had to modify the structure of some tests or something to that effect.

And polyglots have been using SRS for years.

As always, the real problem when people fail to do something that works is psychological, not technical. I'd say anyone who made an Ozempic for motivation would make a killing, but I believe it's already a scheduled substance. Maybe one without potential for addiction or abuse. Or maybe an Ozempic for conscientiousness.

I think both have a place. When someone is starting for the first time, they're enthusiastic, but they haven't built faith in the process yet. It's easy for them to lose confidence if they're putting in work but the results are slow or ambiguous. I think it's best to take advantage of their beginner's enthusiasm and kick them off with something higher effort that is guaranteed to show them clear results. After they build confidence they can settle in to something lower effort (aka "more sustainable") where the benefit is longer-term and you don't see dramatic results every week.

I implemented free recall into FSRS pretty easily. Granted, it’s only for language learning, and I have it set up to work in a free recall friendly way (you don’t learn cards, you learn actual words and morphemes) but it’s been working for a few weeks now. I’m working on a product video atm, but once that’s done my next task (sometime this week) is to clean up the UI and merge it to master.

I almost never see someone talk about free recall so I was too excited to see it mentioned not to comment

IMO, the most limiting feature is that spaced repetition is a method for memorization, but memorization is only one part of learning and it's very often not the most prevalent part.

But when memorization applies, gamification is a really good way to avoiding burnout (as long as you don't overexpose yourself to it). There are many spaced repetition games for children, I don't know why people make so few of them for adults. (But then, fearing the duolingo owl is a popular meme nowadays.)

That sounds very interesting! Do you still have a link to that post?

> it turns out its not really a silver bullet

One thing that bothers me about SRS is that it doesn't get enough attention from people who understand the difference between memorization and language acquisition. It gets a ton of attention from people who are doing test prep or who get intrinsic reward from their memorization accomplishments.

Memorization is not my goal — I want to get better at reading Spanish and French — but I find that drilling on vocabulary and example sentences helps a lot. I compare it to using scaffolding in construction. Scaffolding is not a building. Scaffolding doesn't serve any of the purposes a building does. But if you need to build, expand, or refurbish a building, sometimes building scaffolding in and around it can speed things up a hell of a lot.

I wish there were better guidance for using memorization to assist in language learning, but the world seems to be split between people who are satisfied with memorization as their goal (for test prep or intrinsic satisfaction) and people who dismiss memorization entirely because it isn't their goal.

It's a silver bullet for learning facts. You still have to actually do the learning. Nobody was claiming it would download knowledge into your brain Matrix-style.

You'd probably say silver bullets aren't a silver bullet because you still have to load the gun and shoot it.

Yeah, and flashcards generally work better when you're reinforcing existing learning. Learning by flashcards is hellish for me.

I started learning Mandarin on Duolingo while dating a Chinese woman. After we broke up I continued with it just because I found it fun.

Now I have several Chinese friends and I'm learning Chinese cooking. I'm motivated to continue learning about Chinese food and Chinese culture, and the important role food plays within it.

> What's worse for motivation than taking longer?

Many things. I think HN is a bit of a bubble here, but you'll find a lot of people prefer something enjoyable but slower to something efficient and faster, even if they won't admit it.

See the popularity of Duolingo vs Anki as an example! Or Quizlet vs Anki. Or the scores of students who revise by half-watching dopamine-ified youtube videos rather than doing past papers and flashcards. If you ask people, they'll often say they care for efficiency, but their revealed preferences say otherwise.

Doing large amounts (hours) of Anki day in day out is truly miserable, particularly when the alternatives can be quite enjoyable. And if you burn out before you achieve your goal, is the "efficiency" really worth it vs going slower but eventually getting there?

Plus, a lot of people want to learn e.g. a language because they enjoy the process as well as the end result. Making the process miserable in order to get to the end result faster isn't always a good tradeoff.

Which is what it's about. It's a tradeoff. I'm a big proponent of flashcards, but I think it's important to recognise that you're trading enjoyment for speed in most cases.

>I also credit it to me burning out so hard I took a gap year!

Do you think targeting a sub-90% difficulty could help reduce burnout? My experience is that working to recall something I'm on the verge of forgetting can be very effortful.

Maybe an 80/20 approach where you only create flashcards for the 20% of knowledge that's most useful? E.g. for a language, you could create 1000 flashcards for the 1000 most common words which allow for basic real-world communication?

What info did you memorize?

Mastery by George Leonard touches on some of this; learners can fall into 3 categories during the learning process: Dabblers, Obsessives, and Hackers. Each one has strengths and weaknesses, but the core philosophy is that "mastery takes time". After 2-3 years of practice, you know all the moves in a school of martial arts, or all the chord progressions for an instrument. But its the "after" where you either continue to refine or move on to the next skill.

Anything advertising that you can learn X in Y days isn't addressing that "after" period. Once you've learned the skill, you need motivation toward applying it, which in turn refines your skills. Conversely, becoming hyper fixated can be detrimental to overall skill. "Jack of all trades, master of none" HOWEVER the rest of the quote goes "but often times better than a master of one"

Sometimes you gotta slog through the boring bits to progress.

I experimented with this, actually. The default was 0.85 iirc, and I tried pushing it up over time to 0.91, but I ended up reducing it to 0.83. It isn't that many more cards, and it makes it far less laborious.

My personal approach with A levels was to strictly learn content through classes, then organise things (which teaches you a lot on its own!) and make my own flashcards. Then I used the flashcards to keep it fresh till exam season. It was crazy, during revision lessons everyone else in the class would be going "uhhh I have no clue, that was two years ago" while I'd just know it.

I have tried different approaches, including using other people's flashcards (not as good - objectively they were high quality, but you gain a lot from writing your own + tailoring to your own way of looking at things) and learning from them (for my driving theory - terrible idea!). That hybrid approach is the best I've found, and the one I intend to use for my degree.

I'm not so sure. My brother's medical school had to tell students they had to study the actual material rather than Anki decks because so many students were failing tests.

That seems highly atypical.

Perhaps they were relying on LLMs to generate decks?

It's very likely that you're using Anki in situations that will burn you out and drain your motivation anyway.

The article specifically points out WaniKani as an example of a very bad implementation of spaced repetition (see the "FSRS in practice" heading, under the paragraph "for Japanese language learning specifically...").

> Learning by flashcards is hellish for me.

Wait, you're putting yourself in a situation where the first time you see a card, you have no idea what it is?

I've tried it in a few situations (e.g. my driving license theory test) and yeah it's absolutely awful and I quickly stopped. My modus operandi is obsidian notes->flashcards->revision to keep my knowledge up. However, a lot of people do actually do that!

People will genuinely download top x wordlists for a language and try to learn from them. Hideous, but they do it.

Nah, I love physics. It's just a lot!

Edit: It's worth noting I had a nasty head injury that was slowing me down. Optimising my learning was a necessity, and the injury meant I spent more time studying than my peers, in more optimised and less enjoyable ways, to get the same result.

> Many things. I think HN is a bit of a bubble here, but you'll find a lot of people prefer something enjoyable but slower to something efficient and faster, even if they won't admit it.

This is so well known that it was covered extensively in the book Make It Stick[1], that you might as well call it the "student fallacy." (And they might have; ironically, I've forgotten if they do or not!)

[1]: https://www.amazon.com/Make-Stick-Science-Successful-Learnin...

> People will genuinely download top x wordlists for a language and try to learn from them.

I’ve done that, and I’ve even created a whole SRS app to learn kanji which does that by default (https://shodoku.app).

I think this is common practice for the first 1000 words, and I don’t exactly recommend against it. Unless your target language is close to another language you already know, you are going to have to learn your first 1000 words somehow, and you will not learn them by comprehensible input in any reasonable time, unless you are actually living in the language area, and cannot use other languages.

I actually bought a vocabulary book which has 1000 basic words and example sentences and puts them in categories (e.g. work, travel, food, etc.). I then downloaded an Anki deck from the book and use it. To be fair though, I first read the word in the book, and practice it with a red-sheet (albeit in reverse, i.e. from english and try to recall it in japanese).

As for my kanji learning app. I made it so the first time you see a kanji, it does not hide any information, and it shows you the strokes in order as you write it on first encounter, after that you review it normally.

I appreciate you replying to my comments!

I confess I'm interested to hear your thoughts re: the usefulness of SRS from a more holistic perspective.

Gwern writes:

>...if, over your lifetime, you will spend more than 5 minutes looking something up or will lose more than 5 minutes as a result of not knowing something, then it’s worthwhile to memorize it with spaced repetition. 5 minutes is the line that divides trivia from useful data.

https://gwern.net/spaced-repetition

My sense is that there are very few facts I will spend more than 5 minutes of my life repeatedly looking up. And even then, many of those are facts that I will naturally end up memorizing regardless of SRS, since I'm using the info so often.

I understand the utility of SRS for test takers or language learners. When Google is impractical or unavailable, memorization makes sense. But for everything else -- why not just Google it?

> Spending half an hour mind-numbingly learning words through flashcards will teach you about as much vocabulary as an hour watching educational videos, but it'll be far less fun and you'll feel like it actually took two hours.

The first part is definitely untrue, you won't learn any vocabulary spending an hour watching an educational video, you'll be lucky if you remember one new word tomorrow. That half hour on Anki will be spread out over six months, and will teach you 20 words.

As for the second part, doing Anki is like doing through any sort of timeline that spits out random rewards and failures. I get a rush whenever I remember stuff, and I get bummed out when I forget; it's basically facebook.

I understand why one wouldn't think that with single-word vocabulary flashcards, because they are horrible to do and unhelpful. You should be running sentences, not words. Words rarely translate well, change form when they are in sentences, and often show up as part of seemingly ungrammatical set phrases.

Look up Deci/Ryan, self determination theory.

Doing anki cards with words two hours every day, and not doing any reading of material is very detrimental for me.

I do not study medicin, but for language I have to actually be able to use what I memorize. Knowing how to use words is harder than it sounds, you need context. A common mistake when constructing sentences is getting the tone wrong and choosing the wrong synonyms.

Honestly, I'm not sure I'm a particularly good person to ask about this. I've experimented with Anki for areas other than test taking and language learning, but it didn't stick.

I used it for some geography stuff, and that was fine I suppose, I like geography, but I stopped after a while.

I find it easy to remember things if I care about them. Ergo, if I care enough to put it in Anki, Anki is useless.

Particularly with LLMs being a thing, you don't even need to concretely know what you're looking for - just give chatgpt a vague description and let it list out suggestions until it jogs your memory.

When it comes to physics etc, you'll end up memorising everything relevant to the areas you actually use. In areas like "What is the star type of a star with a temperature of 6000K" - something that I memorised using Anki, and which took me a while - if I actually worked in that area of astrophysics I'd obviously learn that quite quickly.

I suppose it could be useful for maintaining knowledge. Had I not been burnt out I would have kept up my flashcards through my gap year, which would've presumably been quite useful - I'm currently going through the slog of relearning how to integrate so I'm not totally embarrassed at uni!

Yeah, 100%. I worded my comment a bit awkwardly, but that's exactly what I was getting at too :)

How are you handling scheduling with FSRS? The challenge that I quickly saw was that it was difficult to figure out when you should advance a segment of information. If you get 80% of the info right, should it be advanced? What happens to the 20% you missed? How do you prevent yourself from missing the same 20% every time it comes around?

Oh thanks, I didn't know there was a name for the thing I've been doing. Vocabulary mining is a nice term.

I agree with your general point too. People are correct to say SRS only helps with memorising and not with learning, but this is only a problem if you haven't developed functional learning techniques or you have to learn something you don't enjoy. Good learning essentially hinges on interest and excitement, and making the thing you're learning relatable or catchy.

If you have exams and deadlines, this can be hard. If you've no exams and no deadlines, just flashcard anything interesting that comes your way, include context and jokes, and focus on enjoying yourself. Delete flashcards with a smile if they annoy you a few repetitions down the line. Make all your own cards. Invent funny stuff, find quirky facts that stand out.

E.g., the area of Ireland is 84,421km^2 - all powers of 2. I never had a "yardstick" for big areas, now I do. Borneo is nearly nine Irelands in size.

Or another example, French Polynesia has 121 islands, 75 of which are inhabited. I found this fact shocking, so I thought I'd put it in to a flashcard. After some quick reflection, I'm sure you too could come up with ways to make those numbers stand out.

Another - the title of Wittgenstein's "Tractatus Logico Philosophicus" - where truth tables were popularised - ends in SOS. No more forgetting the title, or where truth tables are from!

In summary - learn things you like, and make it spicy for you.

Would love to hear about how it was transformative for you

...and more efficient than reading long-form text or course material

It's just logical that memorization is useful for broad areas like vocabulary and get progressively worse the more depth is involved, e.g. vocabulary>grammar>maths. The first one doesn't require generalization, the last one most certainly does. Even though I find that SRS leads to good generalization if it is used for relatively shallow conceptual knowledge.

Fellow longtime Anki user, I can say letting myself drop from 0.9 to 0.85 was like letting myself experience the taste of clover honey for the first time. Accepting a slightly worse hit rato in exchange for 2/3rds the workload made life considerably more pleasant.

..this is why you usually have whole sentences on Anki cards, not single words.

>I'd say anyone who made an Ozempic for motivation would make a killing

This is just ADHD meds right? That's why the uni students spend so much buying them.

Does not always help, but I am not picking. You are right it only fails in 1/100 cards.

If I weren't already busy enough using Anki to accomplish my own goals, I would almost certainly look into deeply integrating LLM calls into it so that we could e.g. practice writing and speaking with it in a way that is much closer to the kind of immediate feedback loop you get with a tutor.

For now, I make do by having Gemini Flash 2.5 open in a separate window and checking my short answer writing responses with that.