Sicklied O’er

research

My grandfather used to tell me a story about a young boy who was stuck in traffic with his family for hours because an 18-wheeler had got itself pinned under an overpass bridge ahead of them. The huge truck was wedged in so strongly and strangely that a flock of engineers had descended on the scene. They argued back and forth about their favorite physical and mathematical models that would unpin the trapped vehicle and release the miles-long stream of cars idling behind it on the freeway. This bickering went on for hours—until the boy got out of his car, walked up to the group of engineers, and shouted, “Why don’t you just let the air out the tires!”

It’s a nice story, precisely because it’s so rare and noticeable. We don’t notice unbroken strings of solved problems from experts, because that’s what we expect of experts—and, for the most part, what we get from them. We notice when they fail. And, because these failures are more noticeable than the far more boring and numerable successes, we fall prey to availability bias, and assume that expert failure occurs with much more regularity than it actually does. (In turn, we start to think that it’s maybe a good idea to keep students naive and, therefore, creative and open-minded rather than have them study things that other people have already figured out.) As Tom Nichols writes in The Death of Expertise:

At the root of all this is an inability among laypeople to understand that experts being wrong on occasion about certain issues is not the same thing as experts being wrong consistently on everything. The fact of the matter is that experts are more often right than wrong, especially on essential matters of fact. And yet the public constantly searches for the loopholes in expert knowledge that will allow them to disregard all expert advice they don’t like.

A 2008 study which put this folk notion of expert inflexibility to the test compared chess experts and novices, and measured the famous Einstellung effect in both groups across three experiments.

In the first experiment, the experts were given the board on the left and were instructed to find the shortest solution. The board on the left is designed to activate a motif familiar to chess experts (and thus activate Einstellung)—the smothered mate motif—which can be carried out using 5 moves. A shorter solution (3 moves) also exists, however.

If the experts failed to find the three-move solution, they were then given the board on the right. This board can be solved by the shorter three-move solution but not by the Einstellung motif of the smothered mate. The group of novices in the experiment were all given this second board (the one on the right) featuring the three-move mate solution without the Einstellung motif as well.

Findings

If knowledge corrupts insight, as it were, then the experts would, by and large, be fixated by the smothered mate sequence and miss the three-move solution. And this is indeed what happened—sort of. What the researchers found was that level of expertise correlated strongly with the results. Grandmasters (those with the highest levels of chess expertise) were not taken in by the Einstellung motif at all. Every one of them found the optimal three-move solution. However, experts with lower ratings, such as International Masters, Masters, and Candidate Masters, all experienced the Einstellung effect, with 50%, 18%, and 0%, respectively, finding the shorter solution on the first board, even though all of them found the optimal solution when it was presented on the second board, in the absence of the smothered mate motif.

The novices’ performance showed a positive correlation with rating also. Sixty-three percent of the highest rated (Class A) players in the novices group found the optimal solution on the right board, while 13% of Class B players and 0% of Class C players found the three-move solution. Thus, the Einstellung effect made International Masters experts perform like Class A players, Master players perform like Class B players, and Candidate Masters perform like Class C players.

Experiment 2 replicated the above finding in a slightly more naturalistic setting, and Experiment 3 did so with strategic Einstellungs instead of tactical ones.

Knowledge Is Essential for Cognitive Flexibility

While this study shows that Einstellung effects are powerful and observable in expert performance, it also demonstrates that the notion that expertise causes cognitive inflexibility is probably wrong.

The failure of the ordinary experts to find a better solution when they had already found a good one supports the view that experts can be vulnerable to inflexible thought patterns. But the performance of the super experts shows that ‘experts are inflexible’ would be the wrong conclusion to draw from this failure. The Einstellung effect is very powerful—the problem solving capability of our ordinary experts was reduced by about three SDs when a well-known solution was apparent to them. But the super experts, at least with the range of difficulty of problems used here, were less susceptible to the effect. Greater expertise led to greater flexibility, not less.

Knowledge, and the expertise inevitably linked to it, were also responsible for both forms of expert flexibility demonstrated in the experiments. The optimal solution was more likely to be noticed immediately, even before the nominally more familiar solution, among some super experts. Hence, expertise helped super experts avoid an Einstellung situation in the first place because they immediately found the optimal solution. Even when experts did not find the optimal solution immediately, expertise and knowledge were positively associated with the probability of finding the optimal solution after the non-optimal solution had been generated first. Finally, when knowledge discrepancy was minimized, as in the third experiment, super experts had sufficient resources to outperform their slightly weaker colleagues. In all three instances, knowledge was inextricably and positively related to expert flexibility. . . .

The training required to produce experts should not be seen as a source of potential problems but as a way to acquire the skill to deal effectively and flexibly with all the situations that can arise in the domain. Creativity is a consequence of expertise rather than expertise being a hindrance to creativity. To produce something novel and useful it is necessary first to master the previous knowledge in the domain. More knowledge empowers creativity rather than hurting it (e.g., Kulkarni & Simon, 1988; Simonton, 1997; Weisberg, 1993, 1999).

Makin’ Copies

research

At the heart of many calls to improve education is the taken-for-granted notion that because the world is now changing so rapidly, it is better for schools to focus on producing innovative and critical thinkers and ‘not just’ knowledgable students. The common instructional approach deployed, at all scales, to produce this effect—whether it is inquiry learning or personalized learning—is to remove or dramatically lessen the influence of knowledgable others.

Copying the effective behaviors of knowledgable others was a much more effective learning strategy than learning directly from the environment.

But important research on learning strategies in the wild shows that, at the very least, different intuitions are possible here. Researchers discovered—much to their surprise—that, in a rapidly changing environment, copying the effective behaviors of knowledgable others (social learning) could be a much more effective learning strategy than learning directly from the environment (asocial learning). This result held even when social learning was “noisy” and asocial learning was noise free.

The team has gone on to further investigate and apply their findings to other animal studies, and a book, Darwin’s Unfinished Symphony, was released just last year, detailing their work.

Social Learning Strategies Tournament

The method used for this research was a tournament in which the researchers designed a computer simulation environment and entrants to the tournament (104 in all) designed ‘agents’ that competed to survive in the generated environment by learning behaviors and applying them to receive payoffs for those behaviors. Each agent had three possible moves it could play: Observe, Innovate, or Exploit. The first two of these moves—Observe and Innovate—were learning moves, which allowed the agent to acquire new behaviors (or not in some cases), and the third move, Exploit, allowed agents to apply their acquired behaviors to receive a payoff (or not, depending on the environment and the behavior). As was mentioned above, Observe moves were “noisy,” whereas Innovate moves were noise free:

Innovate represented asocial learning, that is, individual learning stemming solely through direct interaction with the
environment, for example, through trial and error. An Innovate move always returned accurate information about the payoff of a randomly selected behavior previously unknown to the agent. Observe represented any form of social learning or copying through which an agent could acquire a behavior performed by another individual, whether by observation of or interaction with that individual. An Observe move returned noisy information about the behavior and payoff currently being demonstrated in the population by one or more other agents playing Exploit. Playing Observe could return no behavior if none was demonstrated or if a behavior that was already in the agent’s repertoire is observed and always occurred with error, such that the wrong behavior or wrong payoff could be acquired. The probabilities of these errors occurring and the number of agents observed were parameters we varied.

Some Key Findings

When the winning agent, which learned primarily by copying, was modified to learn only through Innovate moves, it placed last.

It was not effective to play a lot of learning moves. But when learning moves were played, agents which relied almost exclusively on Observe outperformed the rest, and an increase in copying was strongly positively correlated with higher payoffs. When the winning agent (called DISCOUNTMACHINE) was modified to learn only through Innovate moves, it placed last.

Even when learning by copying was made noisier—the probability and size of copying errors increased—agents which relied on it heavily still did best.

Finally, agents who combined asocial and social learning in more balanced ways (winning agents used social learning at least 95% of the time) performed worse than those who opted for social learning most of the time.

Why Copying Is Effective

It must be underscored, again, that, in more naturalistic environments there is a cost to asocial learning that copying does not have. Learning by observation is safer than learning by interacting directly with the environment, alone. But in this simulation, that cost was erased. And social learning (copying) STILL outperformed innovation, even when social learning was noisy (Observe “failed to introduce new behavior into an agent’s repertoire in 53% of all the Observe moves in the first tournament phase, overwhelmingly because agents observed behaviors they already knew”).

So, why was copying effective? The researchers boiled it down to being surrounded by rational agents, which I choose to rephrase as “knowledgable adults”:

Social learning proved advantageous because other agents were rational in demonstrating the behavior in their repertoire with the highest payoff, thereby making adaptive information available for others to copy. This is confirmed by modified simulations wherein social learners could not benefit from this filtering process and in which social learning performed poorly. Under any random payoff distribution, if one observes an agent using the best of several behaviors that it knows about, then the expected payoff of this behavior is much higher than the average payoff of all behaviors, which is the expected return for innovating. Previous theory has proposed that individuals should critically evaluate which form of learning to adopt in order to ensure that social learning is only used adaptively, but a conclusion from our tournament is that this may not be necessary. Provided the copied individuals themselves have selected the best behavior to perform from at least two possible options, social learning will be adaptive.

Any takeaways for education from this will be stretches. The research was a computer simulation, after all. But, whatever. My takeaway from all this is that, as long as there are knowledgable adults around, we should encourage students to learn directly from them. A milder takeaway (or maybe stronger, depending on your point of view): regardless of how adept you feel yourself to be in your social world, social worlds are not intuitive. What seems to make sense to you as a strong connection between ideas A and B (in this case, changing world → promote innovation) will not necessarily be effective just because a lot of people believe it and it makes intuitive sense. The way to change that is not to stop making those arguments, because few people do. The way to change it is to stop forwarding those kinds of arguments along when they are made. That way, the behavior won’t be copied. : )

Coda

I should add, by way of the quote below from Darwin’s Unfinished Symphony, that, although copying was a more successful strategy than innovating, it was not, by itself, the reason for success. What made the difference was better, more efficient, more accurate copying behaviors:

The tournament teaches us that natural selection will tend to favor those individuals who exhibit more efficient, more strategic, and higher-fidelity (i.e., more accurate) copying over others who either display less efficient or exact copying, or are reliant on asocial learning.

Proprioceptive Knowledge

This paper ($) was a nice read, with some fresh (to me) insights about discovery, instruction, and practice. There are many points in it where I don’t see eye to eye with the author, but those parts of the text are, thankfully, brief. I took away some new thoughts, at any rate, the most robust of which was an analogy between learning and proprioception, as the title suggests.

Here are the two main ideas as I see them, involving a very healthy amount of paraphrasing and extrapolation on my part:

  1. An aspect of your learning over any topic that deserves attention from instruction is your subjective, first-person, thinking with the material taught.
  2. Good instruction not only manipulates you into knowing something, but enlists your cooperation in doing so.

Proprioception

Proprioception is the basic human sense of where your body parts are in space and the sense of your own movement in that space (i.e., you don’t use ‘touch’ to know where your left hand is in space; this is proprioception). For learning something abstract like adding fractions with unlike denominators, we might think of proprioceptive knowledge as what it is LIKE to add fractions with unlike denominators—physically, cognitively, etc. Certainly carrying out the computations procedurally is an important part of “what it is like” but there are many others, including “what it is like” to identify situations calling for working out common denominators.

The first paper uses as a candidate for proprioceptive knowledge (although they don’t call it that) an example of working long division to produce repeating decimals. Students are instructed, with an example, that for any number you write as an integer over an integer, the decimal digits will either be repeating zeros or a repeating pattern of some other kind. Students use practice, however, to gain access to the proprioceptive dimension of this instruction—the experience that this is indeed the case; a first-person view of the knowledge. It is not that they are not told why the digits repeat (there are a finite number of remainders that are linked together in what they produce) during the instruction with the example. They are told this. And it’s not necessarily the case that the students don’t understand what they have been told. It’s just that the first-person experience of this is an important node in the constellation of connections that constitute the schema of understanding rational numbers.

Indeed, I would argue that the explicit instruction is absolutely necessary in this example (and almost all other examples). It allows the student to connect his newly gained first-person proprioceptive conceptual knowledge about division and rational numbers with what he understands to be experts’ experience and see them as the same. In other words, the explicit instruction did not take away the student’s ability to discover something for himself, as the idiotic trope goes; on the contrary, it was necessary to facilitate the student’s discovery. Cristina says it well:

Cristina [the teacher in the example] did not consider it a crime to “tell secrets”—like the author, she believed that students have to work to figure them out anyway.

Making Connections

This perspective helps me make some intuitive sense of why, for example, retrieval practice may be so effective. Testing yourself gives you the first-person experience of—the proprioceptive sense of—what it is like to remember something successfully and consistently (and it ain’t as easy as you think it is before you do it). This knowing-what-it-is-like episodic knowledge is a knowing like any other, but it is one that can be easily neglected when dealing with cognitive skills and subject matter.

Or, rather than neglect proprioceptive knowledge, we tend to make it dramatically different from the conceptual, such that students don’t connect the two. It is not necessarily the case that our conceptual instruction is anemic (though it certainly is on occasion) but that our procedural instruction is so narrow, and when it is not narrow it is not procedural. As a result, students gain a strong sense of what it is like to find like denominators, for example, but little sense of what it is like to move around in “higher level” aspects of that topic. When it comes to those higher level aspects, we have done no better in education, really, than to just have students do things in order to learn things or just have students learn things in order to do things.

The episodic knowledge angle can also help make some sense out of the power of narrative, as this is a type of out-to-in instruction which contemporaneously solicits an in-to-out response. (Although, it seems to me to push too hard in the latter direction for an ideally holistic type of instruction.)

How to ‘Do’ Proprioceptive Knowledge

The above suggests some instructional moves. The paper itself suggests building this in-to-out proprioceptive knowledge during practice. Rather than relying on problems which hit only the procedural writing of fractions as terminating or repeating decimals, for example, practice can assist students in “discovering” the explicit conceptual instruction of the lesson from a brand-new, first-person perspective. So, if we want students to understand why rational numbers must have terminating or repeating decimal representations, we need to give them practice that explicitly allows them to ‘feel’ that connection and express it as they work.

For another example, I’m currently working on percents. I certainly want students to be able know what it’s like to determine a percent of a number, purely procedurally. But I also want them to understand a lot of other things: that p% of Q is greater than Q when p is greater than 100 (and why; because it’s multiplying Q by a number greater than 1) and less than Q when p is less than 100 (because it’s multiplying Q by a number less than 1); and so on. I should think about structuring my practice so that these patterns reveal themselves and explicitly point students to them.


Underlying, Deep, Critical?

Here’s a very reasonable statement, from this book, on techniques used by researchers to investigate conceptual knowledge of arithmetic:

The most commonly used method is to present children with arithmetic problems that are most easily and quickly solved if children have knowledge of the underlying concepts, principles, or relations. For example, if children understand that addition and subtraction are inversely related operations, even when presented with a problem, such as 354297 + 8638298 – 8638298, they should be able to quickly and accurately solve the problem by stating the first number. This approach is typically called the inversion shortcut.

Although, this borders on the problematic (for me at least). Why should ‘underlying’ be a prerequisite for calling something conceptual knowledge as opposed to plain old knowledge? Even the straightforward addition and subtraction here presumably requires knowing what to do with the numbers and symbols presented in this (likely) novel problem and thus involves conceptual knowledge of some kind.

Still, it makes some sense to distinguish between knowing how to add and subtract numbers and knowing that adding and then subtracting (or subtracting, then adding) the same number is the same as adding zero (or doing nothing). But the following just a few paragraphs later doesn’t make much sense to me:

The use of novel problems is important. Novel problems mean that children must spontaneously generate a new problem solving procedure or transfer a known procedure from a conceptually similar but superficially different problem. In this way, there is no possibility that children are using the rote application of a previously learned procedure. Application of such a rotely learned procedure would mean that children are not required to understand the concepts or principles being assessed in order to solve the problem successfully.

deep learning

The biggest problem is that the concept of ‘conceptual knowledge’ of arithmetic laid out here relies on the fact that the “inversion shortcut” is not typically taught as a procedure. But it seems easily possible to train a group of students on the inversion shortcut and then sneak them into a research lab somewhere. After the experiment, the researcher would likely decide that all of the students had ‘conceptual knowledge’ of arithmetic, even though the subjects would be using the “rote application of a previously learned procedure”—something which contradicts the researcher’s own definition of ‘conceptual knowledge’. On a larger scale, instead of sneaking a group of trained kids into a lab, we could emphasize the concept of inversion in beginning arithmetic instruction in schools. If researchers were not ready for this, it would have the same contradictory effect as the smaller group of trained students. If the researchers were ready for it, then the inversion test would have to be thrown out, as they would be aware that inversion would be more or less learned and, thus (for some reason) not qualify as conceptual knowledge anymore.

Second, why should adding and subtracting the numbers from left to right count as an application of a rote procedure (which does not evidence conceptual knowledge) rather than as a transfer of a known procedure from a conceptually similar but superficially different problem (which does show evidence of conceptual knowledge)? The problem is novel and students would be transferring their knowledge of addition and subtraction procedures to a situation also involving addition and subtraction (conceptually similar) but with different numbers (superficially different).

Clearly I Don’t Get It

I still see the value of knowing the concept of inversion, as described above. A person who notices the numbers above and can solve the problem without calculating (by just stating the first number given) is, most other things being equal, at an advantage compared to someone who can do nothing else but start number crunching (it’s also possible to not notice the equal numbers because you’re tired, not because you lack some as-yet undefined ‘critical thinking’ skill). What constantly perplexes me is why people insist on making something like knowing the inversion shortcut so damned mysterious and awe-inspiring.

You can know how to number crunch. That’s good to know. You can also know how to notice equal numbers and that adding and then subtracting the same value is the same as adding 0. That’s another good thing to know. The latter is probably rarer, but that fact alone doesn’t make it a fundamentally different kind of knowledge than the former. It is almost certainly rarer in instruction than calculation directions, so it should be no surprise that students are weaker on it generally. Let’s work to make it not as rare. A good place to start would be to acknowledge that inversion is not some deep or critical knowledge; it’s just ordinary knowledge that some people don’t know or apply well.

Coda

The section in question concludes:

Other concepts, such as commutativity, that is if a + b = c then b + c = a, have been investigated, but as they have not received as much research attention it is more difficult to draw strong conclusions from them compared to the concepts of inversion and equivalence. Also, concepts, such as commutativity are usually explicitly taught to children so, unlike novel problems, such as inversion and equivalence problems, it is not clear whether children are applying their conceptual knowledge when solving these problems or applying a procedure that they were taught and the conceptual basis of which they may not understand.

But how does it show that ‘conceptual knowledge’ is applied when we test students on something they haven’t been taught (do not know)? Where is the knowledge in conceptual knowledge supposed to be coming from? As long as it’s not from the teacher, it must be ‘conceptual’?


Transfer and Forgetting

I discovered a paper recently whose title is probably more interesting than its content: Unstable Memories Create a High-Level Representation that Enables Learning Transfer. Quite a thought—that the instability of memory could be advantageous for transfer.

Researchers conducted two experiments, asking participants in the first experiment to learn a word list and then a motor task and in the second experiment a motor task and then a word list. There were three conditions within each experiment: (1) the word recall and motor task had the same structure (see the supplemental material for how ‘same structure’ was operationalized here), (2) the two tasks had different structures, and (3) the tasks had no determined structure.

It’s Not “Transfer”, It’s Domain Similarities

transfer

In the first experiment, participants first learned the word list and then their skill at the motor task was measured over three practice blocks. When the word list and motor task were of the same structure, participants did significantly better across the three motor-task practice blocks. Similarly, in the second experiment, after the motor skill was learned, participants who then practiced the word list with a similar structure to the motor task improved significantly more than participants in the other conditions. This improvement on an unrelated though similarly structured task was measured as transfer, and it occurred in both directions.

Somewhat surprisingly, however, this transfer of learning between word and motor tasks (or motor and word tasks) was correlated with a stronger decrease in performance on the original task, when participants were tested 12 hours later. That is, subjects who learned the word list and then successfully transferred that learning to the motor task (because the tasks were of similar structure) showed a sharper decline in their word list recall than subjects in other conditions. The same results appeared in the experiment where subjects first learned the motor task and then the word list.

At first blush, this seems obvious. The subjects who actually transferred their learning saw their learning on the original task displaced by the similarly structured and thus interfering second task. But when researchers inserted a 2-hour interval between the original task and the practice blocks, this decline disappeared—and the transfer learning was no longer present. Thus, it seems that not only the similar structure of the two tasks but also the weakness of the memory for the first task were both responsible for the effective transfer learning. The authors put it this way:

By being unstable, a newly acquired memory is susceptible to interference, which can impair its subsequent retention. What function this instability might serve has remained poorly understood. Here we show that (1) a memory must be unstable for learning to transfer to another memory task and (2) the information transferred is of the high-level or abstract properties of a memory task. We find that transfer from a memory task is correlated with its instability and that transfer is prevented when a memory is stabilized. Thus, an unstable memory is in privileged state: only when unstable can a memory communicate with and transfer knowledge to affect the acquisition of a subsequent memory.

Forgetting, Spacing, and Transfer

This is intriguing. In some sense, this reinforces results related to the spacing effect. Spacing causes forgetting which causes “unstable memories.” When learning is revisited after a period of forgetting, it finds this unstable memory in a “privileged state”: a state which allows it to strengthen the connections of the original learning.

But the above also suggests that extending learning for transfer to other situations or to other concepts may be done optimally in concert with spaced practice. In other words, the best time for transfer teaching might be after a space allowing for forgetting.


Spacing and The Practice Meter

Without a doubt, students need to practice mathematics thoughtfully. Classroom instruction of any kind is not enough. Practicing not only helps to consolidate learning, but it can be a source of good extended instruction on a topic. And in recent years, research has uncovered—or rather re-uncovered—a very potent way to make that practice effective for long-term learning: spacing.

Dr. Robert Bjork here briefly describes the very long history and robustness of the research on the effectiveness of spacing practice:

It seems that not only is spaced practice more effective than so-called “massed” practice, but spaced learning is more effective than massed learning. A recent study by Chen, Castro-Alonso, Paas, and John Sweller, for example, provides some evidence that spaced learning is more effective than massed learning for long-term retention because spaced learning does not deplete working memory resources to the same extent as massed learning.

In one experiment, Sweller, et al. provided massed and spaced instruction on operations with negative numbers and solving equations with fractions to counterbalanced groups of 82 fourth grade students (from a primary school in Chengdu, China) in regular classroom settings. In both conditions, students were instructed using three worked example–problem-solving pairs. A worked example was studied and then a problem was attempted—for a total of three pairs (they were not presented together). In the massed condition, these pairs were given back to back, for 15 minutes. In the spaced condition, this same 15 minutes was spread out over 3 days.

In both conditions, a working memory test was administered immediately after the final worked example–problem-solving pair. And immediately following the working memory test, students were given a post-test on the material covered in the instruction. In the massed condition, this post-test occurred at the end of Day 1. In the spaced condition, the post-test occurred at the end of Day 4.

Students in the spaced condition scored significantly higher on the post-test than students in the massed condition. And there were some indications that working memory resource depletion had something to do with these results.

In the absence of…stored, previously acquired information, it was assumed that for any given individual, working memory capacity was essentially fixed. Based on the current data, that assumption is untenable. Working memory capacity can be variable depending not just on previous information stored via the information store, the borrowing and reorganizing, and the randomness as genesis principles, but also on working memory resource depletion due to cognitive effort.

Shorter, Smaller Chunks

spacing and practice meter

Taken together, the research on the spacing effect for both practice and instruction suggests that both instruction and practice should happen in shorter, smaller chunks over time rather than packed all together in one session.

As an example of this, here is a video of a module from the lesson app Add and Subtract Negatives. The user runs through this very quickly (and correctly), skipping the video and worked examples on the left side and the student Notes—and a lot of other things that accompany the instructional tool—to demonstrate how the work of this module flows from beginning to end. The Practice Meter is shown in the center of the modules (and instructor notes) on the homepage as a circle with the Guzinta Math logo. If you want to skip most of the video, just forward to the end (2:11) to see how the Practice Meter on the homepage changes after completing a module.

You can see that the Practice Meter fills up to represent the percent of the lesson app a student has worked through (approximately 55% in the video). Although not shown in the video above, hovering over the logo on the homepage reveals this percent. The green color represents a percent between 25 and 80. Under 25%, the color is red, and above or at 80%, the color is blue.

Whether or not the lesson is used in initial instruction, the Practice Meter fades over time. Specifically, the decay function \(\mathtt{M(t) = C \cdot 0.46^t}\) is used in the first week since either initial instruction or initial practice to calculate the Practice Meter level, where \(\mathtt{C}\) represents the current level and \(\mathtt{t}\) represents the time since the student last completed a module.

In our example above, during the first week after initial instruction or practice, the student’s Practice Meter level of 55 will drop into the red in about a day (about a point every 15 minutes). If she returns to the app in 15 minutes to see a Practice Meter level of 54 and then raises that up to an 80 by completing the same module again or a different module (100 is max score at any time), then her Practice Meter level will drop to below 25 in about a day and a half. If she raises it up to 100, then that will decay to below 25 in a little less than 2 days.

This fairly rapid decay rate applies only to the first week. After Day 7, and up until Day 28, the decay rate changes to \(\mathtt{M(t) = C \cdot 0.825^t}\), whether the student practiced during that time or not. This provides some incentive for spacing out practice a little more over time. Mapping this onto our example above, an initial Practice Meter level of 55 would decay to below 25 in a little over 4 days. A level of 80 would decay to below 25 in a little over 6 days, and a level of 100 would take about 7 and a half days to go red.

There are also decay rates for 28–90 days and after 90 days. For more information, see this Practice Meter Info page, which comes with the instructor notes in every lesson app.

(Lack of) Implementation Notes

The design of the Practice Meter is such that, if a student does not use a lesson for spaced practice, he or she will feel no interruption in their use of it. And it is important to implement it in a way that does not create extra responsibilities for the student if they aren’t required by their teacher. But if students and parents or students and their teachers do want to implement spaced practice, it can be easy to check in on the Practice Meter every so often, asking students to, say, keep their Practice Meter levels above 25 or above 80—perhaps differentiating for some students to start—at regular check-in intervals.

As always, though, implementing shorter and smaller in both instruction and practice is much more difficult than reading about it in research, especially when current practice or one’s institutional culture may be focused on “more” and more massed instruction and practice. But conclusions about spacing drawn from research are not regal edicts. We can keep them in mind as ideas for better practice and work to implement the ideas in the small ways we can—and then eventually in big ways.

Update: The Learning Scientists’ Podcast features a brief discussion of lagged homework, which definitely connects to what I discuss above. Henri wrote up something about it a few years ago.