I recently completed three short papers based on Make it Stick (2013) by Brown, Roedinger, and McDaniel for a class I’m taking and thought I would share my work. I enjoyed the book and it confirmed many of the techniques that have worked well for me in the past as a student. Many of the activities are ones that UAF eCampus has been encouraging faculty to incorporate into online-asynchronous classes. The book further confirms these practices.
Make it Stick – Self Testing
The authors of Make it Stick (2013) Brown, Roedinger, and McDaniel bring up many techniques that students can adopt to learn material in a more efficient way than many students currently follow. One of those techniques is self-testing, “One of the best habits a learner can instill in herself is regular self-quizzing to recalibrate her understanding of what she does and does not know’ (p. 21). Instead of rereading material, the authors suggest you slow down your reading and periodically self-test to commit to memory what you’ve just read (p. 229).
One tool that the authors talk about that can help students with self-testing is a mobile app called, Osmosis. Developed by medical school students, the app contains a database of quiz questions from textbook publishers, a curated and vetted team from Osmosis, along with questions the individual student or team of classmates create (crowd source). As noted on the Osmosis.org website, “Osmosis was created to help med students break out of cram-forget cycles and maximize their learning efficiency.“ One of the features of Osmosis, is that you’re able to crowd source questions and add them to the Osmosis database. This is another technique, reflecting and summarizing content in your own words, that the authors of Make it Stick talk about (p. 89). The premise is that you select one of the categories of quizzes and answer multiple choice questions, many of which contain images. Beside keeping track of correct or incorrect answers, Osmosis allows you to judge your own confidence in answering the question. Select: “I’m sure,’ “I’m feelng lucky’, or “No clue.’
One aspect of Osmosis that makes it different from other self-test applications, is the ability to push question sets to the user on a user specified schedule. For example, you’re able to select the option to have 5 or 10 quiz questions automatically sent to your mobile device 4 or 5 times a day or once a week. This feature supports the spaced and interleave technique that the Make it Stick authors recommend throughout their book. The questions can also be scheduled around examination dates. Another feature is that on every question there is a chance to see an explanation so if you don’t get a question correct, you can quickly get instructional feedback.
There is also a feature where you can take a quiz with a friend, or a group of friends, in a game-like atmosphere. There is also an individual scoreboard to show how you are doing, both in accuracy and in confidence. You select a category and invite classmates to play with you. You have a set number of questions to answer is a set time. Your correct answers have increased points associated with them as you work through the questions (3 pts for “I’m sure’ correct answers, 2 pts of “I’m lucky’ correct answers, or 1 pt for “No clue’ correct answers. And in reverse, if your answer is incorrect, points are taken away from you.) At the end of the game there is a winner declared and individuals are given a scoreboard with information about their own performance.
The scoreboard is also available on an individual basis. It is also interactive so you can select those areas in which you were incorrect or less confidence. It also tells you how “fresh’ your learning is and how many questions you’ve answered with prior time periods.
Another feature makes a connection with your individual medical school and classmates of which I was not able to test or find too much information. It looks like instructors (or group administrators) can view when students are taking quizzes and using the Osmosis application. Osmosis app has also provided an interactive timeline where material for an entire curriculum can be uploaded (slides and documents) and indexed to specific dates so admins, teachers, and students can get a big picture view of content that is being covered and when. I’m not sure if future content is also being uploaded but that would give the learner a good idea of what they might want to brush up on, if the topics hasn’t been studied in the recent past. Another feature for teachers or group administrators is the capability to view analytics about when students are testing and other study habits (Video viewable within Osmosis app on the About page).
Currently, Osmosis is only for use for students studying medicine, but what about other disciplines? How could an instructor create their own “Osmosis’?
A simple and quick strategy would be for an instructor to incorporate a test bank of quiz questions into self-check quizzes that students could take on their own through a Learning Management System like Blackboard. Chapter questions could be added to a “Pool’ or collection of questions, and a weekly quiz could pull “x’ number of questions from that pool. The self-test could be set up so that students could take it multiple times. As the semester progresses, questions from the current chapter as well as a subset of questions from previous chapters, could be pulled into the self-tests, thus getting practice on previous learned material.
An even better activity might be for a student (or group of students) to be assigned to come up with the test questions for the content. Students could determine what the key points of the content is and come up with questions, with approval of the instructor..
An instructor could also use a tool like Twitter to push questions to students. You’re able to schedule tweets in advance to automatically be dispersed at a certain time. If your students were following your class hashtag, the questions could be pushed out to them. You might ask them to answer the question and include a reason why they chose that answer.
There are many more options that incorporate some of the features of the Osmsis app to help students with creating an atmosphere of self-testing. It would an interesting to work on a project that incorporated the features of Osmosis as an open source application so that more disciplines could take advantage of its architecture.
Brown, P.C., Roedinger, H.l., & McDaniel M.A. (2013). Make it Stick: The Science of Successful Learning. Belknap Press: Cambridge, MA.
Osmosis.org. (20140). Retrieved https://www.osmosis.org/.
Make it Stick – Reflection
The authors of Make it Stick (2013) Brown, Roedinger, and McDaniel talk a lot about the importance of student reflection while learning new material. ‘Reflection can involve several cognitive activities that lead to stronger learning: retrieving knowledge and earlier training from memory, connecting these to new experiences, and visualizing and mentally rehearsing what you might do differently next time’ (pg. 27). Too often students are fooled into thinking that they have a better understanding of the material then they actually do. Many student think that if they can repeat what the instructor has said or what the textbook says, that means they understand the material. Being able to do these two things doesn’t mean there is true understanding. When you’re able to use your own words to speak about the concept or to apply it to a similar or a different situation, that is when understanding happens (pg. 16).
Being able to transfer the knowledge and understanding from one instance to another is a true measurement of understanding. When you’re able to leave a situation and think critically about what went right, what went wrong, what could have been improved on, this is how you built your knowledge base and gain experience. When you’re able to apply prior knowledge and experience to a current situation, because you have previously allowed yourself to asked questions about your performance or concepts that you’re learning, this is where the strength of reflection will pay off (pg. 66).
The authors of Make it Stick say, “After a lecture or reading assignment, for example, you might ask yourself: What are the key ideas? What are some examples? How do these relate to what I already know? Following an experience where you are practicing new knowledge or skills, you might ask: What went well? What could have gone better? What might i need to learn for better master, or what strategies might I use the next time to get better results?’ (pg. 88). As an instructor you may have to prompt your students with some established questions in order to get them started. According to UAF eCampus’s website, iTeachU.uaf.edu, the unit encourages instructors to incorporate one of three types of contemplation into online courses: Reflection on learning and learning experience, Reflection on the real-world relevance and application of what they are learning or Meta-reflection on the learning materials, strategies, and structure employed in the course itself. Encouraging students to stop and consider what they are reading or studying has proven to be a productive and successful study habit.
Strategies that teachers can implement to help students with their learning is another topic written about by the authors of Make it Stick. Assigning “writing exercises that require students to reflect on past lesson material and relate it to other knowledge or other aspects of their lives; “ is an activity that can easier be incorporated into a class (pg. 227). Taking a few minutes after a class lecture to give students a chance to ruminate over the lecture or discussion can strengthen their understanding or at least point out where the weak spots exist. And taking a few minutes as the class begins to reflect in a retrieval exercise to recall what has happened previously can help students pull from the depths of their brains for what they have previously learned and have committed to memory (pg 222).
The authors mentions an instructor, Mary Pat Wenderoth, who has her students writing “learning paragraphs’ to reflect on the prior class’ topics. In an article called, Reflection to Deepen Learning and Self-Awareness’, Wenderoth says, “To maximize their learning, students need factual knowledge, which we give plenty of, but they also need conceptual frameworks to put the knowledge into.’ The basic principles that Wenderoth incorporates into her teaching strategy include:
- Make reflection part of the class routine.
- Ask questions that let students discuss what’s important to them while achieving learning goals.
- Motivate students through class credit, but keep evaluation simple.
- Give regular feedback.
- Collect student feedback on the exercise.
(University of Washington, pg. 8)
Writing reflections can be done simply with pen and paper but you could use technology to support reflection in a variety of ways. One advantage of using technology for reflections is the ability to share these reflections with a cohort of learners. Reflections shared with a class through a discussion board not only encourages the writer to be conscientious of his or her wording, but might just help another student make a connection that they didn’t realize was there. Students reflections can also be documented through media by using audio or video to capture students’ thoughts or through illustration or visualising concepts. Using different media can actually help students remember different aspects of their learning through their own creative process.
Brown, P.C., Roedinger, H.l., & McDaniel M.A. (2013). Make it Stick: The Science of Successful Learning. Belknap Press: Cambridge, MA
Moss, Jennifer. (2011). Reflection Mechanics. Retrieved from iTeachu.uaf.edu: https://iteachu.uaf.edu/online-training/develop-courses/constructing-a-course/reflection-mechanics/.
University of Washington. (2014). Reflection to Deepen Learning and Self-Awareness. Leading Change in Public Higher Education: A provost report series on trends and Issues. Retrieved from: https://www.washington.edu/provost/wp-content/uploads/sites/70/2014/05/edtrends_Innovators-Among-Us-Preparing-Students-for-Life-after-Graduation.pdf
Make it Stick – The Growth Mindset
There are many factors that can influence a student’s success in learning. Most of these factors can be controlled either by the community, institution, or the individual student. Access to materials or to experts, language disadvantages, learning disabilities, supportive home situation, these disadvantages and others can or could be controlled with the right resources and access to support. But what about student’s natural ability, talent, or intelligence? The authors of Make it Stick (2013) support the idea that although IQ may be semi-fixed, there are ways “to amp up the performance of the intelligence’ to allow students to learn more and obtain a deeper understanding (pg. 178). One way for students to succeed is through having a positive attitude and self-determination to know that they have control over their success, and that it isn’t “fixed’. The authors introduce Carol Dweck’s work on “Fixed Mindset’ and Growth Mindset.’ “… it’s discipline, grit, and a growth mindset that imbue a person with the sense of possibility and the creativity and persistence needed for higher learning and success…The active ingredient is the simple but nonetheless profound realization that the power to increase your abilities largely within your own control’ (pg. 183).
Students who apply themselves and make the extra effort can be very successful learners. The authors of Make it Stick say, “We make the effort because the effort itself extends the boundaries of our abilities’ (pg. 199). The choices students make about what they study, how much and to what depth, all these factors play into learning. Some people are naturally curious and passionate about the world and how it works and do not need encouragement to listen to their inner Growth Mindset voice. Some students might need to be encouraged and helped with changing their static Fixed Mindset voice to seeing the results that having a Growth Mindset attitude creates, until they see the results. I think one of the biggest messages that the authors of Make it Stick make throughout their book, is the idea that learning is hard work and that you need to stick with it to see the results. You may not get immediate gratification, but you have to trust that in the long run, you’ll have a better and deeper understanding.
The elements of a “Growth Mindset’ as describe by Carol Dweck on the Mindset website include:
- flourishing on challenges
- seeing failure as an opportunity to learn from one’s mistakes and trying again
- looking for feedback and applying the criticism to do better the next time
- realizing that blaming someone else won’t benefit your progress (Dweck, 2010).
One strategy for promoting a healthy Growth Mindset might be in applying game mechanics to curriculum. The concepts behind a gaming atmosphere are very closely related to those of a Growth Mindset. According to Rick Raymer in Gamification: Using Game Mechanics to Enhance eCampus, “game mechanics are the construct of rules that encourage users to explore and learn the properties of their possibility space through the use of feedback mechanisms.’ Raymer continues on and considers other elements that include setting goals and objectives so that a learner has an idea of where he or she is heading, providing regular and frequent feedback, creating some kind of scoreboard so a learner can track his or her progress, establishing a way to be recognized and achieving rewards for reaching competencies towards a higher level, being rewarded for effort, not solely on success, and some kind of peer or social motivation (Raymer, 2011).
The mechanics behind badging may also work towards establishing a Growth Mindset. The simple concept of acknowledging when a student achieves competency at a certain level and receiving a badge can help a student work towards becoming more competent at a higher level. This strategy gives affirmation to students for completing a level of understanding while encouraging students to continue to apply themselves to achieve either a higher level of competency or competency in another area. One of the big differences in badging and a growth mindset is that with badging, the creator of the badges is determining the criteria for what the badge means, as well as what the competencies and different levels are, it isn’t self-directed. At some point the student has to learn to take the initiative on his or her own self to achieve those increased levels of competency.
Matt Renwick writes about his student’s experience with playing Minecraft and correlating his observations of the students and a growth mindset. “For now, we are content with observing our students build not only complex worlds within Minecraft, but also develop key critical skills that can foster a growth mindset. The persistence, attitude, and effort observed in our students is all the evidence we need for now.’ His observations have found that students aren’t asking for help from the teacher when solving problems, but rather are turning to their peers for support. When you aren’t successful, you aren’t completely locked out of the game, but rather you begin again, armed with all your previous experience. And the feedback you receive isn’t simply a score, but you’re given critical feedback based on your effort (Renwick, 2014).
Using the gaming atmosphere in the challenge of learning seems to be a natural pathway to instilling a Growth Mindset in students and helping them to move from a static Fixed Mindset where the opportunity to explore and be passionate about learning seems to stall out.
Brown, P.C., Roedinger, H.l., & McDaniel M.A. (2013). Make it Stick: The Science of Successful Learning. Belknap Press: Cambridge, MA.
Dweck, C. (2010). How can you change from a fixed mindset to a growth mindset? Retrieved from https://mindsetonline.com/index.html.
Raymer, R. (2011). Gamification: Using Game Mechanics to Enhance eCampus. Retrieved from: https://elearnmag.acm.org/featured.cfm?aid=2031772.
Renwick, M. (2014). Passion-Based Learning Week 5: Can Minecraft Foster a Growth Mindset.? Retrieved from https://plpnetwork.com/2014/03/20/passion-based-learning-week-5-minecraft-foster-growth-mindset/.