How does the SRS system work?

Hello-

It appears that the SRS system here is better than even Anki based on the adjustments it’s make. Here is the issue - I have yet to read an explanation that makes any sense as to how it works. I highly recommend the developers create a very simple breakdown of how it how works on the web page as it has made me want to switch to memrise more than once.

  1. Let’s assume were at 90% confidence level and you get a particular word right every time, what will the spaced repetition be for that word? Will the flow be 3 days, then 7, days, then 14, days, then 20 day intervals? If not then, what will the interval flow be?
  2. If a developer responds to this - is your SRS system roughly equivalent to a system such as the one used in memrise?
  3. It’s probably asking a lot to explain the differences in the confidence % but if we can at least get some questions explained above - it will go a long way.

Thank you

This old link says it’s based on Supermemo, and further links to more info:
https://legacy.skritter.com/forum/topic?id=53417724

Ah, here is another older link with details.
https://legacy.skritter.com/faq#scheduling

Thanks Pretty good explanation - this at least covers that the space repetition system is “normal”- the % settings are a little blurred however. 95% vs 90% retention. At the end of the day if your getting a card wrong, it’s still going to pop up…so who cares if its 95% or 90%? I hope the 95% doesn’t mean that it will give you one last review 2 months down the road (which will solidify it in your long term memory) whereas 90% will not. So what does say 95% retention setting do that 87% doesn’t? Thank you!

I am pretty equally baffled about how the system works, but re:" At the end of the day if your getting a card wrong, it’s still going to pop up…so who cares if its 95% or 90%?" my guess is that if you have a lower % setting for retention then you have a lower threshold of “proof” that you know the word. In other words, you will be required to get it correct a smaller number of times before it gets pushed off far into the future. Personally I have come to feel that less is more when it comes to retention. If you set the % requirement to high you will never get around to adding new words. At least not at the small amount of time that I commit each day to Skritter.

I think SRS is infinitely better than having no systematic review method at all, but unless somebody figures out a way to apply AI to really take fuller account of student habits there are always going to be limitations.

As for Memrise, I have used it a lot, but to my knowledge it doesn’t have anything similar to Skritter’s ability to let you practice writing by hand. Am I missing something?

1 Like

I meant to compare Memrise basic SRS system to that of Scritter’s. Basically, it must be a similar spaced repition.

If a developer (or someone who knows) on this site is able to explain the big difference between 87% vs 95% with Scritter it might go a long way. Is 95% just “front loaded” heavier as podster is assuming?

I would then, agree, 90% is better than 95% because EVEN if you get the 90% setting wrong a month down the line — it will resort back to the shorter term interval anyways.

Writing is good for initial acquisition of charachters but for long term memory I personally like the reading feature.

Yes, I think Skritter and Memrise are both ultimately descendants of SRS systems dating back to Supermemo, as Apomixis notes. This would include Anki as well. But I think the execution is sufficiently different that it would be misleading to say they are basically the same thing. From my (vague) recollection, Anki works on a daily cycle; once you finish reviews for the day you won’t see them again. Skritter, by comparison, might show them to you if you went beyond clearing your queue. Welcome anyone to let me know if I’m full of it . . . as I stated my recollection is vague. Also, Memrise has a lot of bells and whistles that might add some value. I have never been a premium user, and some of the features like “speed review” just seem counterproductive in that it would confuse your SRS record.

Hi guys, there’s some good discussion going on. I’ll try to address the points in the original question and clarify a couple details.

First, there is a a divide between the new mobile apps’ scheduling algorithm and the “classic” Skritter algorithm referenced in the legacy FAQ. The retention index (RI) is only used in our legacy/classic services. It isn’t used in the new mobile apps. I’ll get to why in a little bit. The RI affects scheduling–if you want to keep an index at 95% likely to remember vs. 87%, you’ll see items sooner and more often, like the FAQ says. How often exactly depends on a lot of other factors about the item and when it’s reviewed in relation to how due it is; it’s difficult to give a “one size fits all” graph. But the RI is a minor factor in the scheduling algorithm. Conceptually it is along the lines of the difference between interval * 0.87 and interval * 0.95.

My advice: don’t worry about it. Or if you are going to worry about it, keep it low (I personally have mine set to 87%). You’ll have fewer reviews and make progress faster. Most of the mathematical precision everyone praises about SRS falls apart if a user misses days of study. And even at its best, SRS is a guess at how a person’s memory will be and merely provides an efficient opportunity to re-encounter information they’ve seen before. In the long run, habit and consistency will reap more gains than tweaking a detail of an SRS algorithm. See https://www.hackingchinese.com/when-perfectionism-becomes-an-obstacle-to-progress/ for further philosophical reading.

All that being said, in the new app, we do not take the RI into account and I do have some numbers I can give you (with a caveat). As the RI was an overall minor detail but a complicated mechanic to explain, we decided to rely on other factors for our calculations. Instead, along with the normal SRS calculations, we do a lot of heuristics based on the larger context of the item–we don’t just blindly crunch the numbers. For instance, new items that don’t have many/any attempts and are answered correctly in consecutive reviews get an extra boost to their interval so they space out faster. If a new item’s first few initial attempts were all correct, it’s likely the user already knew the item–this is review after all! Or it’s a super easy item. Either way, we fast-track it so it gets out of the user’s way. Because of heuristics like these and the context around each item, it’s a bit hard to prescribe exact values for item spacing. But if a user reviewed a brand new item correctly at the exact second it was due each time, here’s generally what the interval spacing would look like for the first few attempts:

For the first row (answered only correctly [score of 3]), because of that heuristic I mentioned, the increase is a bit aggressive. On the other hand, an item with an incorrect attempt early on will increase its interval more slowly. The “initial score 1 interval” row shows some intervals for attempts on an item that was marked incorrectly on its first attempt and then correct for each subsequent attempt.

An item’s actual intervals will vary around +/-7.5% as we insert some minor randomness at the end of the calculations to prevent cards from appearing in the same order on different days. In the new algorithm, there is an upper bound on the interval of 1 year. If you don’t want to review something at least once a year, it probably doesn’t belong in an SRS queue. I don’t have the code open at the moment, but I believe the limit on the legacy system was 3 or 5 years.

In general, we’re trying to move away from exposing complicated settings for tweaking the SRS algorithm that, in the long run, adjusting will only have marginal, if any efficacy, and instead, provide better modes for a user to interact with their content. Test mode is a good example of this. Many users over the years have completely messed up their items’ intervals because they overstudied in order to reinforce some vocab for a test the next day. In any event, we’re always discussing how to improve things and welcome your feedback!

6 Likes

@SkritterMichael So, why does the Retention rate setting still show up in the latest iOS app settings pane, if it’s not used?

Or is a retention rate and retention index two separate things?

2 Likes

After digging into it, it just seems like an oversight we forgot to clean up from earlier R&D on the app. We’ll likely remove it from the UI in a future release. Good catch!

1 Like

ScritterMichael,

Thanks for the reply. It is good to hear that. I was a little concerned especially after reading reviews like the one below:

The most concerning part is in the summary:

"Skritter should be your choice if you are focused on handwriting. After using it for over 100 hours and learning over 1000 characters, my recommendation would be to stick to studying individual characters on Skritter (not words). Disable the other review types in Skritter. Use one of the other two tools for vocabulary building. Skritter is just too weak on meaning (lacking even parts of speech) and reviewing words with its SRS algorithm will force you to practice a lot of characters you already know (for example, my statistics show I’ve reviewed 然 126 times)"

-So first of all based on what you said, something like what the above poster said probably shouldn’t happen. Do you believe either 1) this is a secluded instance 2) perhaps the SRS system bugs in general have been fixed further (in the span of a year)…or some other reason? The reviewer appears pretty sober in his analysis. I think it would go a long way if you are able to provide a little context here.

Also it is definitely a good think that the Hacking Chinese blogger has had involvement to my knowledge here. That is definitely a good endorsement. Also, your point of “it doesn’t matter” for the exact science of an SRS with for example adding an “easy”…vs “hard” button because your SRS is “effective” and “normal”(it’s not some kind of voodoo SRS system) enough, was also a good one.

One thing I would recommend is posting this kind explanation in one of the main pages of Scritter. It’s simple and very easy to understand. Especially when there are reviewers that say “I don’t know how their system even works”. This kind of explanation is a perfect rebuttal. Otherwise people will make the easy decision to go to the obvious alternative for their SRS system such as PLECO. This is very well done device especially considering how the words interface with all the books you have. It’s a long term decision to stick with this one. Not to mention it is literally my job to learn this language. :slight_smile:

Thank you -

2 Likes

I have a lot of thoughts about this, but for now let me focus just on the “damning” example of having to “review” 然 so many times. First of all, this alleged redundancy might be a meaningful drawback if all you wanted to do was impress someone who asks “how many characters do you know?” But anybody who has been at this for a while knows why this is a meaningless question. (As opposed to how many words do you know, or how well can you read, etc.) 然 is a character that appears in many words / compounds. (既然、果然、然后、然而 。。。。。。). Now you might say that it is a wasted effort to have to hand write a “characters that you already know” each time you practice a new word that contains it, but I would argue that it is beneficial because it is reinforcing both the various meanings of said character, as well as making sure that you know that it is precisely THIS character and not another one with similar pronunciation, writing, etc.

That said, by all means do check out Pleco and Anki. I have very little experience with Anki and it has been a long time since I have used Pleco as a flashcard system, though I did do it a lot at one time. You could probably configure it to accept handwritten input to try to approximate the Skritter experience, but my guess is that you won’t find it very satisfactory compared to Skritter. For traditional flashcards though Pleco is very versatile and easy to use.

2 Likes

@apatino Thanks for finding that info. I think that review of Skritter is very surprising and misleading and agree with @podster regarding the review benefits of Skritter.

Studying characters only seems like a very poor method, unless you are already at a very high level of fluency.

Characters often have multiple meanings, some of them subtly differentiated, others widely different from each other.

It is only in the acquiring of bigger and bigger vocabulary that you get to see a character employed in some of its less common meanings. Practicing whole words thus enriches your understanding of all the different functions of that character because the character is occurring in different word contexts.

I often encounter new words using characters I already know whose meaning either confuses me, or I make the mistake of assuming I understand it when the word’s actual meaning is subtly or even grossly different than I guessed. Looking the word up and studying it in a dictionary is often eye-opening as to all the different roles the characters within it can play.

Having studied for some years now, I am at the stage where, as I come across them, I need to update all the old characters I already know with all of the various meanings I can find in various Pleco dictionaries. This is because I am often encountering these characters in situations (new words) where the old, memorised meaning no longer makes sense or else doesn’t quite do the job.

As for using Pleco’s flash cards, I find it tediously slow and nowhere near as useful as the system in Skritter.

But I love Pleco as a source of multiple dictionaries at once, I organise its cards into a personal categorised lexicon, and enjoy the text scanner abilities. I am delighted that Skritter integrates with it so easily.

Thanks for both your responses - Yes I think that reviewers issue has been resolved. He obviously was probably seeing that character is several separate words. My method is when I initially acquire the words, I’m using hand writing, then I use the reading for the SRS and I load up the mnemonics with the narratives from Zizzle (using that developer’s same system for monosyllables he hasn’t come up for yet, to create my own narrative mnemonic).

I SRS both the individual monosyllable words AND the Di-syllable words. (I’m also learning Traditional Characters in a separate account so it doesn’t overflow my SRS/I don’t learn a Traditional before learning a Simplified).

In it’s current state Scritter is pretty good. I find the TONES and DEFINITION tests redundant as you can just review READING (hiding the english definition) and mark it wrong if you get either of those wrong.

I like the fact they recently looked into some Cloze Deletion with the writing part. I think that has a lot of potential down the road.

Really awesome that this is out there. Because everything would be a little worse having to rely on some of the alternative apps out there for literally being the core of your language acquisition system (especially for beginner & intermediate learners) - I would say until I get to 3k words, this is by far the most important system. (and will always compliment future systems).

I don’t know how hard it is to implement, but if you can come up with a new test “cloze deletion” and roll it out on the reading side. And even improve upon Cloze Deletion pro. (understandably it would be a long term project-cost/benefit). This app would be amazing.

Maybe even like “hit” passive words you see in the example cloze deletion sentences to “add” to your queue. /tier the difficulty of the sentences. Possibly start out with a core of doing this with 600 or so words (fluent forever style). But otherwise, it is really great at what it does!

2 Likes

I don’t know what client(s) the Reddit user was using, so I can’t comment with certainty, but given the post’s timeframe and the user’s hours in with Skritter, it was likely one of the legacy clients, which did have more scheduling issues. In addition to some general scheduling edge cases, the older clients also counted writing card reviews on vocabs toward their contained characters. So as @podster mentioned, if a user studied the writing for words like 果然 or 然后, it would also count those as writing attempts for the underlying characters writing items (然, 果, and 后 in this example). This could explain the rather high number of attempts on 然.

We disabled this contained character review behavior in the new mobile app because it incorrectly conflates different types of results. A user could have gotten the character right the context of the word but still not be able to recognize/recall it on its own. Or they might have confused it with another character in the context of a word, but still have known how to write the character. In a best case scenario, this behavior shuffled around a users’ reviews and added a bunch of study attempts that the user doesn’t remember doing. In the worst case scenario, it messed up the scheduling entirely for individual character reviews so that the user never saw them.

As another angle to all this, it’s also possible this user might have had some underlying improvements that could be made to their studying. Any item with a high recall percentage, large interval, and decent double digit review attempts is a good candidate for being banned. Or if the user really did need to see 然 120+ times because they kept getting it wrong, the underlying memory probably needed some more intentional, focused attention–something about the information wasn’t sticking.

In any event, good points on why it’s important to study words and not just characters, @podster and @Therebackagain!

Cloze deletion prompts are a perennial discussion topic among the team, we’re fans. But prompt context becomes an issue rather quickly. If Skritter asked me which word in my queue means “very” in Chinese, there are probably ten different ways I could answer it, but unless I had a good example sentence or remembered which deck the vocab was coming from, it would be hard to narrow down which word I’m being quizzed on. We’ve got a few ideas of how we might be able to bring these types of prompts to Skritter, but have no definite or immediate plans at this time.

We’ve realized we’ve done a bad job publicizing and clarifying the SRS information, and this post has helped us realize that. We’re currently working on creating new articles and documentation to help de-mystify and centralize the knowledge. I’ll post an update here when we have them finished!

1 Like

This is why I continue to study deck by deck. I’ve finally reached a level where there are so many synonyms for “happy” or “lucky” or “rich” and so on, I get swamped and can’t remember which characters are required.

I cope when necessary by “cheating” and checking the pinyin before writing, and marking it correct if I would have known it in context.

Going back to Pleco and putting in all the possible English meanings also helps distinguish mostly swappable synonyms.

1 Like