Episode 5: Learning to Learn

Episode 5: Learning to Learn

“How to boost learning … Conversation with John Dunlosky … The effect of spacing … Conversation with Bob Bjork (Part 1) … Learn by doing … Conversation with Bob Bjork (Part 2) … Desirable difficulties … Learning in Think101x … Uncut conversations”
(Source URL)

Summaries

  • Episode 5 - Learning to Learn > How to boost learning > How to boost learning
  • Episode 5 - Learning to Learn > Conversation with John Dunlosky > Conversation with John Dunlosky
  • Episode 5 - Learning to Learn > Learning in Think101x > Learning in Think101x
  • Episode 5 - Learning to Learn > The effect of spacing
  • Episode 5 - Learning to Learn > Conversation with Bob Bjork (Part 1) > Conversation with Bob Bjork (Part 1)
  • Episode 5 - Learning to Learn > Conversation with Bob Bjork (Part 2) > Conversation with Bob Bjork (Part 2)
  • Episode 5 - Learning to Learn > Desirable Difficulties >Desirable Difficulties
  • Episode 5 - Learning to Learn > Learning in Think101x > Learning in Think101x
  • Episode 5 - Learning to Learn > Uncut conversations > Uncut conversations

Episode 5 – Learning to Learn > How to boost learning > How to boost learning

  • One of the things I think is important that people take away from the last episode is that it’s not really a matter of just slowing down.
  • As Danny Kahneman said during our interview, when we asked him how to improve everyday thinking, he suggested that we pick only a couple of areas and try to improve those, or recruit the help of a friend, someone who’s looking at the circumstance from the outside.
  • The reason he made this suggestion, I think, is that-you know, he’s been doing this a long time-50 years, I think, he’s been in this area-and he’s won a Nobel Prize.
  • If anyone is going to improve their everyday thinking by generally improving their problem-solving skills, it’s going to be Danny Kahneman, and he mentioned that his everyday thinking hasn’t improved over the last 50 years or so.
  • So where does that leave us? Well, just being more careful or being more deliberate really isn’t going to help necessarily.
  • How do they improve as an expert? You mentioned that we-going slow doesn’t necessarily help, being deliberative, and there’s this compounded issue of self-assessment that we often don’t know how good we’re actually doing at any given case.
  • I mean, what does that mean for improving practice? How do we get better? How do-the goal of this course that we’re taking is called The Science of Everyday Thinking.
  • So given your experience in the field of expertise and in medicine, how do we improve everyday thinking? You improve by knowing more.
  • This course wouldn’t exist if we didn’t think that there are some-that being explicit about everyday thinking and the traps in everyday thinking wouldn’t help people think better.
  • Very clearly, the single best predictor of how good you are is how much you know about the domain, not what problem-solving skills you bring to bear on it.
  • Things like-an obvious thing like mixing up examples from across multiple chapters so that you have to try and figure out which is which-it turns out to be an extremely powerful strategy for learning.
  • The idea of transfer, which is being able to take knowledge that you’ve learned in one context and apply it to another: one, it doesn’t happen at all as easily as we think it does; but, two, psychologists would devise strategies to make that happen better.
  • I think this is moving much more into the instructional educational psychology end of things.
  • I think it’s really clear now that just telling people to slow down, to be more careful, is not going to help people improve their everyday thinking.
  • As Geoff said, the key to improving thinking, to thinking better, is to know more.
  • That’s part of the story, I think, but the other half is, as Geoff said, yes, practice is important.
  • You need to spend the time and the effort actually doing the thing that it is that you want to improve on, but the other is actually taking the advice or learning from experts, so reading, right? You need to spend time looking at the things that experts say are important within that field in order to improve a lot more.
  • In academics, learning from experts or academics tell you what area or what things to read within that area so you’re not going down blind alleys all the time.
  • What is going to be successful is exactly as you mentioned: in order to improve everyday thinking, you need to know more.
  • That’s what we’re going to talk about today specifically: learning how to learn.
  • If your goal is to try to improve, is to try to know as much as you can about a particular domain, we can help.

Episode 5 – Learning to Learn > Conversation with John Dunlosky > Conversation with John Dunlosky

  • You’ve looked at a whole bunch of different types of learning techniques, things that students can use, or do use, to try to improve their learning.
  • What we didn’t look at were strategies that involved technology because we wanted to focus on just those things that any student could use.
  • Some of them we wanted to evaluate, we thought they probably did work, but why not check out the evidence? A couple of other strategies we knew students used a lot.
  • We wanted to know: are these really effective strategies, or should they be doing something else instead? Something that all students use-I still do it myself-is highlighter.
  • It turns out highlighting itself doesn’t really improve student learning.
  • I would never take a highlighter away from a student.
  • At least some of the things that students do like highlighting and re-reading really don’t have a big bang for the time buck, so to speak.
  • We’ll spend time re-reading and highlighting and really not learning a whole lot, when in fact they can replace those strategies with other ones that really do boost their learning, which is exciting.
  • Now we just have to retrain students, build a better student, use better strategies.
  • Students need to do things that are more-to engage them more actively.
  • I mean, a lot people, when they are studying or cramming for an exam, these are the things we do but they might not be the most effective.
  • What are the most effective? What are the things that actually work? Certainly cramming is not that effective, right? Students think it’s effective probably because they can squeak by potentially in the exam and then they’ll just forget everything.
  • I’m not suggesting students don’t study the night before a test.
  • If the student gets that answer right from memory, that has a really potent effect on subsequent performance.
  • Just retrieval practice is a really effective way to boost performance, especially if students use it distributed across time.
  • You know, most students at least say they use flash cards for simple, associate learning, things like foreign language vocabulary, but you can use flashcards for complex materials as well: concepts, or you write the key term on one side, definition on the other side, and then use that to basically test yourself, and then to restudy.
  • They can take notes in better ways to actually support this retrieval practice, which is an effective strategy that many students don’t use.
  • Retrieval practice-so then it’s, I suppose, really pretty effortful by my comparison to rereading or highlighting, so it probably takes a fair bit of desire on the part of the student to want to learn that.
  • The nice thing is, out of all these studies that are done to compare rereading to retrieval practice, the time on task is always equated, so the students who were just rereading spent the same amount of time that the students are who were practicing retrieval.
  • It’s like a little bit more painful to try to retrieve stuff from memory, but even the same amount of time used in one strategy versus the other, the students who were practicing retrieval, using that little bit of extra effort, are getting the major, basically, increase in their performance.
  • Let’s say you’re a student getting ready for an exam, and maybe you decide, “Okay, I can give this four hours of my time.” Most students, because they think cramming is good, they’ll spend that four hours the night before the exam-just study, study, study.
  • For stuff that students really need to know, this is an essential strategy, or else they just forget stuff.
  • I mean, you don’t have to tell a student that after they take a test that they crammed for, the next day they pretty much don’t remember anything that they studied.

Episode 5 – Learning to Learn > Learning in Think101x > Learning in Think101x

  • We just learned a fair bit within this episode about learning to learn, and desirable difficulties, and the best way to learn information and retain that information over a long period of time.
  • So what we do with this sort of information? How do we apply it in the course? Well, students-hopefully they can use this is as an exercise to see, for example, how the spacing effect is imposed in our course, or massed versus spaced learning, or interleaving content or retrieval practice.
  • There’s a quiz every single week instead of just a midterm and a final so students are having to retrieve that information week by week.
  • At the end of every episode, we have people provide an example from their everyday lives.
  • There’s a bunch of these examples-spaced learning, interleaving.
  • Every single episode, people might notice a shot in a different location.
  • There’s a bunch of these sort of context effects that we’re relying on throughout the course to try to make learning stick, to make these things stick in people’s memory for a longer period.
  • It’s a difficult sort of thing to do, as we said, because we have these desirable difficulties.
  • We have tens of thousands of students in this course, and we want to keep them around week to week.
  • If that’s the case, you can’t make it really effortful for people in the same sense that we can in the university.
  • We’re interviewing some of the best people on the planet.
  • It’d be an exercise for the students to watch each of these episodes as we do them to see what sort of learning principles that we’re applying and see whether they can recognize what sort of things that we’re doing.

Episode 5 – Learning to Learn > The effect of spacing

  • I mean, how is it that we can get it so wrong? That’s a really interesting problem, and it’s one that people are actually working on.
  • John Dunlosky, who we talked to, has done a fair bit of work.
  • Bob Bjork, who we’re going to hear from next, is also working in that field, and Nate Kornell, They’re trying to figure out exactly that: how do people get it so wrong? How is it that you can have a lifetime of experiences and learning; you’ve tried a lot of things that work and a lot of things that don’t, yet we continually make these sorts of mistakes like cramming.
  • Cramming is not effective for actually having things stick in your memory, so why do we continue to use it? I think from reading their work, there are at least a couple of reasons.
  • The second reason that people continually think that cramming is important, I think, is something called fluency.
  • They’re not the same thing, and so when you’re cramming, you’re massing all of this information together; you’re repeating each of these things, then you reread it, and you-all of this massed information feels like it’s going down and feels genuinely like you understand that material, and you’re mistaking that ease of processing with learning.
  • It feels good, and it seems to be one of those things again that our predictions about how things are actually working are completely disjointed with reality.
  • And I think that’s what we’re going to work on, particularly on this episode.
  • This one isn’t easy because what we think works for learning is almost-it’s not even that there’s no relationship between what we think and reality, but in fact it’s a negative relationship.
  • What we think works is the exact opposite of what actually works.
  • So we talked to Bob Bjork about this, and he’s been working in this field longer than anyone I know, and he had some pretty good advice about what works and what doesn’t.

Episode 5 – Learning to Learn > Conversation with Bob Bjork (Part 1) > Conversation with Bob Bjork (Part 1)

  • I once taught a course where the structure was in all the critical ways that we differ from anything like a tape recorder, and it really is just a whole array of things.
  • People’s general notion that they work something like a tape recorder or any kind of recorder leads them to do very non-adaptive things.
  • They take notes like a court stenographer in a course, trying to remember something, and a court stenographer can get every word down all day long and not be able to tell you what the whole trial was about.
  • It makes what you retrieve more retrievable in the future, things in competition with it less recallable.
  • One of the mysteries, one of the continuing interests in just how people learn and how they should learn, is why wouldn’t we-across our schooling years and stuff, just from the trials and errors of everyday living and learning-why wouldn’t we learn how the thing works? That really is a mystery.
  • About one component of that, I think, is we may not understand the engineering details of the computer memory, memory in the computer, or how a disc of some kind works, but we sort of understand the logic of it.
  • So given what we do know now-you’ve had a long career of studying how people remember and retrieve information-what’s the best way to try to maximize that? How do you-again, some of the people who are watching this course want to know how to study better, how to retain information better.
  • It makes a lot of difference what activities you engage in, trying to learn.
  • There are a couple of key things to remember that then lead to important-just things to have in your repertoire when you’re trying to learn.
  • The more knowledge you have in some domain, the more ways there are to link things up and hook things up.
  • It’s almost like a scaffolding structure, that the more it’s built up, the more places there are to put things.
  • You want to make an effort to link this to your everyday experience, to what you already know; think of an example, try to extend it-all of those things really help at the stage of trying to encode the information.
  • You can just stop and think: could you summarize this? You’re together with a friend; try to come up with a question and a structure you might ask.
  • Spend less time highlighting, copying, rereading-spending more time trying to come up with another example, trying to answer a possible question, trying to reproduce the outline of this chapter, all these things, try to sort of generate this material, rather than trying to have it write itself on you.
  • I ask them to bring their textbook, their notes, tell me how they’re trying to study.
  • Sometimes things are highlighted to the point the only thing that stands out is what’s not highlighted.
  • If you’re going to study something twice, don’t just read this chapter again right away to try and see what you missed.
  • How you manage yourself really matters, and there’s a whole series of things to learn about just managing your own study activities.
  • Then some sense of how easy it is to get misled-some of the poorest things that you can do from a learning standpoint, create an illusion that you’re learning rapidly.

Episode 5 – Learning to Learn > Conversation with Bob Bjork (Part 2) > Conversation with Bob Bjork (Part 2)

  • So therein the conditions of learning or practice examples, rather than keeping them constant and predictable, reducing using tests rather than presentations as learning, interleaving the separate things to be learned rather than blocking practice on each at a time-there’s a whole set of these that create this misimpression.
  • Really, another way to put all this is a very old distinction going back to the 1930s between performance and learning.
  • What happens is we can confuse performance during the process as evidence of learning.
  • The other thing that’s made these results so important is various things where you ask people, “What helped you learn better: this condition or that condition,” or, “Predict how well you’ll do on a test that comes in a week.” It shows that people are really fooled by their current performance.
  • It’s not only an unreliable indicator of whether learnings happen.
  • On one hand, it’s never been more important to know how to learn because more learnings happen outside of formal supervision: we’re on our own; we’re at a computer; we’re doing this; we want to learn some new tech feat for job purposes or whatever, across a whole lifetime.
  • We need to know how to learn, but it’s not easy.
  • It’s really a critical kind of juncture of: can people learn how to manage their-be effective stewards of their own learning? And that’s the ultimate sort of survival skill.

Episode 5 – Learning to Learn > Desirable Difficulties >Desirable Difficulties

  • I mean, we like to think we know how to learn, how we learn best, in what situations we learn best, and that we know when we’ve learned something.
  • It’s really difficult to predict what we’ll learn and whether we’ll be able to retrieve this stuff in the future.
  • Who better to provide an indication of how much we learn than ourselves, right? Again, as you said, we saw this in episode three.
  • We are not the best people to determine whether we understand something or not.
  • That’s exactly what’s happening here with learning.
  • If something is easy, if something goes down easily, we misinterpret that ease, that simplicity, with knowledge, with comprehension, with understanding, and that’s a real problem.
  • All the students were required to take this course, a fourth-year honors level course called Advanced Multivariate Statistics.
  • Students were understandably very worried, very anxious about taking this difficult topic, and it was difficult.
  • I really worked hard to try to do everything I could to make it easier for the students to understand this material.
  • She said, “Jason, you know, I completely understand. I get it. It feels so much better now that you’ve presented it to me. I understand everything about eigenvectors and eigenvalues.” I thought to myself, “Well, there’s no way. There’s no way that you could’ve-I struggled!” It took you weeks to put this stuff together.
  • Sure enough, when the exam came around, she didn’t do that well.
  • She clearly didn’t understand it nearly as well as she thought she did.
  • When I’m watching a beautiful BBC documentary by Brian Cox on the wonders of the universe, he’s talking about a complex topic like entropy, and he goes on and he’s using these beautiful examples with sand and talking about time and how it relates to basic physics and how we perceive time, it just-it looks fantastic.
  • I’m like, “Wow, I’ve never understood this so well.” I guarantee, if somebody paused that TV show and asked me one thing about entropy, I would not be able to tell them at all.
  • It felt really good, but I actually didn’t understand the material as well as I thought.
  • If you pause the film and said, “What is entropy,” and forced you to say, “Well, it’s, uh-yes, I really have no idea exactly what it is, but it went down easily,” then I think we’re getting closer.
  • You’re getting spacing in there as well, but it’s this idea interleaving, of doing one thing for while and switching to another task.
  • We tried really hard to make the online experience as best as it can possibly be.
  • Spend some time working, actively working, recalling this information and going through that struggle to make sure that you really understand it and that you’re really going to be able to retrieve this stuff accurately for months and years to come.
  • So in the next segment, we’re going to talk about exactly some of the things that we’ve done to take advantage of these desirable difficulties to try to help the learners in Think101 retain the content for a much longer period of time.

Episode 5 – Learning to Learn > Learning in Think101x > Learning in Think101x

  • We just learned a fair bit within this episode about learning to learn, and desirable difficulties, and the best way to learn information and retain that information over a long period of time.
  • So what we do with this sort of information? How do we apply it in the course? Well, students-hopefully they can use this is as an exercise to see, for example, how the spacing effect is imposed in our course, or massed versus spaced learning, or interleaving content or retrieval practice.
  • There’s a quiz every single week instead of just a midterm and a final so students are having to retrieve that information week by week.
  • At the end of every episode, we have people provide an example from their everyday lives.
  • There’s a bunch of these examples-spaced learning, interleaving.
  • Every single episode, people might notice a shot in a different location.
  • There’s a bunch of these sort of context effects that we’re relying on throughout the course to try to make learning stick, to make these things stick in people’s memory for a longer period.
  • It’s a difficult sort of thing to do, as we said, because we have these desirable difficulties.
  • We have tens of thousands of students in this course, and we want to keep them around week to week.
  • If that’s the case, you can’t make it really effortful for people in the same sense that we can in the university.
  • We’re interviewing some of the best people on the planet.
  • It’d be an exercise for the students to watch each of these episodes as we do them to see what sort of learning principles that we’re applying and see whether they can recognize what sort of things that we’re doing.

Episode 5 – Learning to Learn > Uncut conversations > Uncut conversations

  • How do people generally diagnose? I mean, it’s really kind of categorization problem, isn’t it? What is categorization, more generally, and how do people do it? Well, before we get there, let me back up a second and give you an idea as to how we got into this mess.
  • What emerged even back then in the late ’70s was that in fact the general skill really wasn’t very general.
  • At another level, it was too specific because success on one problem had very little to do with success on the next problem.
  • That led to thinking about, well, maybe this isn’t the way to characterize it at all.
  • I think, for me personally, the breakthrough was to work with people at McMaster who were really into what is called concept formation or categorization.
  • I think, has a long history in psychology.
  • I think it’s fair to say that, initially, the thinking was that there must be some implicit rules that governed each category, but a moment’s reflection on, say, a beanbag chair tells you that the implicit rules of chairs are unlikely to be necessary and sufficient.
  • So what makes a chair a chair is normally, someone would think, four legs, maybe wooden chair, but that sort of rule might not be all encompassing.
  • The two more prevalent views-one is that, essentially, we average our experiences into a prototype, and that prototypes are distinguished as having more of the features of the category and fewer than the features of other categories.
  • Glenn Regehr actually has a very nice example of that.
  • He says, “Suppose you grow up in the Yukon, and all you see is huskies. How many Chihuahuas do you have to see before you recognize a Chihuahua as a dog?” The answer is one.
  • I have a talk about this whole issue of medical diagnosis, and one of the key features and the one that grabs the audience inevitably is the video of my year-and-a-half-old daughter where we show her playing a Fisher-Price toy with dogs and cats, and B.F. Skinner would be happy as can be because what she’s learned is that if she puts that thing in that slot, then Mommy gives her popsicle and tells her how wonderful she is.
  • She can’t say, “Dog.” It is implausible that she would have a rule for “Dog.” Yes.
  • Obviously, the mystery would be what exactly is that similarity matching? How does that come about? I think then you can invoke more fundamental models of the nature of memory, which is associationist or connectionist, which basically says that we are somehow, in a very rapid and unconscious way, matched in individual attributes and seeking connections that way.
  • That’s far-too-basic science and psychology for me to play with, but that’s, I think, a pretty decent model.
  • What about your doctors? When we go from everyday classification or categories, like dogs and cats and tables and chairs, and you only need just a few exemplars to be able to get the job done, do docs work the same way? If you’re learning about skin lesions and mental disorders or more complex types of categories, is it just take a few exemplars there as well? Well, clearly, medical diagnosis is a bit more complex in part because the-yes, for skin lesions, we can imagine a picture that’s kind of like looking at a picture of a dog.
  • Clearly, what differentiates the expert from the novice is not how many hypotheses or how early, but what are the hypotheses, and expertise resides entirely in generating better hypotheses.
  • Exemplar models are very powerful way of thinking about that-that, essentially, prior experience is available to you.
  • One of the-a couple games I play with audiences quite often, one is to ask them, “How long after you graduated before you thought you were competent?” Now to the average person on the street, you think that would be ruled out of order from the outset-they’re competent when we graduated them, aren’t they? Right.
  • Yet there’s no textbook that has a chapter on how to diagnose “Sick”.
  • There are number of conditions that run the pages that are all considered sick conditions, but a very recent study actually just published this year showed that if you turn to the emergency doc and say, “Is this one sick or not,” and he makes a judgment in seconds, and that has about an 85-percent accuracy in terms of predicting whether ultimately, when they get to the wards, they end up with a very serious diagnosis.
  • I mean, that’s what they do in medical school, isn’t it, is learn the rules? They’re very careful and deliberative in coming up with these diagnoses.
  • I think in every clinical encounter, there’s really two fairly distinct phases: the initial phase of hypothesis generation-it’s so effortless; it just seems to be too easy-and then a second confirmation phase where they go back to the rules, the 29 causes of anemia, the signs and symptoms of pernicious anemia, and so forth, just to confirm that that’s what they’re dealing with.
  • People sort of think you’re not really doing your work; you’re not working hard enough.
  • He’s written a book that I think many people know-it was on “The New York Times” bestseller list for at least a year-called “Thinking, Fast and Slow.” His claim is just that: that essentially fast thinking, system-one thinking, leads to errors and that we should be cautioning people, doctors, and everyone else to be more slow, rational, thoughtful, and that that would reduce the errors that happen in human judgments.
  • I think we should wait just a second for that high pitch.
  • It seems-I agree, and in fact I think it’s safe to say that the majority view is that the slow process is going to be ultimately higher benefit in the long run, but that’s against the evidence.
  • It seems, at least given his preliminary work, that we’re not very good at it-that is, we don’t seem to be very good at assessing how well we can do various things.
  • Kevin should be here to speak to it, but I think I can pretty well paraphrase what Kevin would say.
  • If I think of all the people who are watching this video, I’m going to ask them all a simple question.
  • He said, “How can I know what I don’t know when I don’t know what I don’t know?” Right.
  • So how do they improve? As an expert, you mentioned that we-going slow doesn’t necessarily help, being deliberative, and there is this compounded issue of self-assessment, that we often don’t know how good we’re actually doing in any given case.
  • What does that mean for improving practice? How do we get better? How do we-the goal of this course that we’re taking is called “The Science of Everyday Thinking.” Given your experience in the field of expertise and in medicine, how do we improve everyday thinking? You improve by knowing more.
  • This course wouldn’t exist if we didn’t think that there are some-that being explicit about everyday thinking and the traps in everyday thinking wouldn’t help people think better.
  • At some level that’s true, but that’s generally locked in to what’s been called general problem-solving strategies, which are not very powerful.
  • Very clearly, the single best predictor of how good you are is how much you know about the domain, not what problem-solving skills you bring to bear on it.
  • So then would you suggest, in getting more experience, in gaining and kind of accumulating knowledge, is it just a matter of studying the domain more, of getting more experience? If I’m a novice diagnostician, in order to become an expert, in order to become an expert in the true sense, is it just a matter of working hard and studying the rules and getting as much exposure to a wide variety of examples within that domain? Yes, we can, but we can do better than that.
  • The idea of transfer, which is being able to take knowledge that you’ve learned in one context and apply it to another: one, it doesn’t happen at all as easily as we think it does; but, two, psychologists would devise strategies to make that happen better.
  • I think this is moving much more into the instructional educational psychology end of things.
  • There are things we can do to very much enhance the efficiency with which you acquire the knowledge you need to get the job done as a diagnostician or as a human.

Return to Summaries List.

(image source)

Leave a Reply

Your email address will not be published. Required fields are marked *