Using Role-playing Games to Teach Astronomy: An Evaluation.

 

Paul Francis, Department of Physics, Faculty of Science, The Australian National University, Canberra 0200, Australia.

 

Joint appointment with the Research School of Astronomy and Astrophysics, Mt Stromlo Observatory, The Australian National University.

 

 

Abstract

Since 1998, Ióve been experimenting with the use of role-playing games to teach astronomy. Students play the role of competing teams of researchers, racing to solve some astrophysical mystery. In this paper, I review what has been learned from using these games around the world over the last eight years. The most common problem encountered is a tendency for students to become overly political. An unexpected benefit of these games is the boost they give to student self-confidence. Overall, they seem to work well with a wide range of students, ranging from 9^th grade to graduate school, and students exposed to this game comment repeatedly on how the games changed their attitudes towards the scientific process.

 

Keywords

Collaborative group activities ï High School ï College non-majors

 

1 INTRODUCTION.

 

The basic idea of classroom role-playing games is this: instead of telling the students about a given topic in the normal lecture mode, they work it out for themselves, playing the role of competing teams of researchers. The class is divided into teams, and each team is given a briefing paper containing some clues about the chosen topic (as well as several red herrings). The teams have to wander around the lecture theatre exchanging clues, until they can put together a complete theory. They then present this to the rest of the class, and if their peers like their theory, they win a small prize.

 

In the last seven years, more than 5,000 students have used these role-playing games at over 30 schools and universities around the world. In this paper, I analyse how they have worked, and show some surprising clues both on why they can be effective, and on how they can fail.

 

The motivation for using role-playing games came originally from a survey of first year Australian Physics students, carried out by Mario Zadnik from Curtin University. This showed that many of them perceived physics to consist of rote-learning, to be dull and anti-social. We all know that this is the very opposite of research: I wanted to see if I could expose even first-year students to some of the excitement of putting together theories. I was inspired by many previous examples of innovative teaching, in particular peer instruction (Mazur 1997), interactive quizzes (eg. Byrd & Coleman 2004, Zeilik & Morris 2004) and constructivist theories of learning (eg. Kavanagh, Agan & Sneider 2005, Sebastiˆ & Torregrosa 2005). Role-playing games are widely used in business education, and in university subjects such as environmental sciences, where they allow students to explore competing points of view. They are new, as far as I know, to the pure physical sciences.

 

In this paper, I briefly review the technique of classroom role-playing games (Section 2). My research methodology is described in Section 3. In Section 4 I describe the ways in which these games can fail, while in Section 5 I show a somewhat surprising factor that makes these games effective. In Section 6 I briefly comment on the sociology of how the use of these games has spread, before concluding in Section 7.

 

 

2 HOW THE ROLE-PLAYING GAMES WORK

 

I will illustrate how one of these role-playing games works using an example: a game designed to teach about the run-away greenhouse effect on Venus, which I use with first year students at the Australian National University (a class of around 60 students, roughly equivalent to an Astro 101 class in US terminology). A copy of this game can be found on my web site. The game is run in a normal lecture theatre, during a 50 minute lecture slot, and occurs mid-way through a three week segment on planetary sciences.

 

I start the lecture by telling the students that they are about to take part in a role-playing exercise.

 

ðIód like you to imagine that you are a group of internationally renowned scientists, brought here at great expense by NASA to help them solve a perplexing puzzle: why is Venus so much hotter than it should be? Iód like you to divide yourself into groups of around three people. Each group should send someone down to the front, to pick up a briefing paper from me.ñ

 

I hand out the briefing papers. There are typically around 8 such papers. In this case, one group plays the role of military researchers, working on how to spot enemy spacecraft, and hence experts in thermal balance and black body radiation. Another group work for an environmental charity monitoring pollution, and hence know about the infra-red absorption of various compounds. A third group builds spa baths and hence knows about evaporation rates and photo-dissociation of water exposed to UV. Thus none of the briefing papers says anything directly about Venus, but between them all they have all the facts needed to understand the run-away greenhouse effect.

 

ðIód now like each group to read over your briefing papers. You will find that each of you are experts in some bit of science. At first it may seem that your expertise has nothing whatsoever to do with Venus. But in every briefing paper are some crucial clues to working out why Venus is so hot, as well as several red herrings. Just as in the real world, no single expert could solve such a problem. You will need to wander around the room and swap clues with other groups. Only by putting lots of clues together, and working out which red herrings to ignore, will you be able to figure out this mystery.

 

Once you think youóve worked out a good explanation of why Venus is so hot, you should come down the front, and present your theory to the rest of the class. Just as in the real world, it doesnót matter whether you get the answer right ï what matters is getting an answer that everyone else will accept! If everyone thinks that your answer is pretty good, you will win a prize!ñ

 

I typically use glow-in-the-dark stars or small chocolates as prizes.

 

The normal reaction of students to this introduction is stunned silence. But slowly they begin reading their briefing papers, and discussing them amongst themselves. Students are often reluctant to start wandering around the room and swapping information, but once one or two groups start doing this, it snowballs and soon the room is abuzz with activity. I wander around the lecture theatre, listening in and giving occasional hints. This turned out to be one of the major benefits of these exercises: by listening in, I can often identify student misconceptions, which, had I been giving a traditional lecture, I would never have picked up until too late.

 

The atmosphere in class is wonderful ï excited students debating the science in an animated way, swapping clues and thinking deeply about possible alternative theories. It may look like anarchy (I once had a security guard come in and try to eject the students, thinking that a riot was in progress) but, somewhat to my surprise, students do stay focussed on the science, and typically make rapid progress.

 

Towards the end, I usually need to nudge one or two groups into making their presentations. The first group typically gets quite confused, and the other students shoot down their theories quite fast. But typically the second or third group to make a presentation comes up with something the class will accept.

 

This is only a brief account: for more details of how these games work, and some pitfalls to be aware of, see my original paper on the subject, Francis & Byrne 1999, available on my web page (listed in the resource section).

 

 

3 RESEARCH METHODOLOGY

 

I have used three main methods to gauge how and why these games work or fail: two deliberate and one accidental.

 

3.1 Feedback from Other Users.

 

I post copies of all my role-playing games on the web. Anyone is free to use and modify them, but I do ask request that they let me know how they worked. To date, I have received correspondence from over 30 users. Slightly over half are from various Australian and US universities, ranging from small liberal arts colleges to prestigious research universities. Eight are high-school teachers, in Australia, the US, UK and Seychelles. There are also users from universities in Slovakia, Canada and South Africa. The exercises have been used with age groups ranging from Year 9 (9^th grade in US terminology) through to graduate school, the bulk being senior high-school and first year university. Class sizes range from 8 up to 150.

 

3.2 Focus Groups.

 

In 2000, I carried out a research project into student perceptions of these role-playing exercises. As far as I could tell as lecturer, these exercises worked really well, and were highly popular with students. I was worried, however, that I might have been mislead by a voluble sub-set of students, and that there might be a significant number of quiet, sullen students who resented these games. To test this, I collaborated with staff at the Australian National Universityós Centre for Educational Development and Academic Methods (CEDAM).

 

Together, we carried out a focus group analysis of student perceptions of these games. We used students in an introductory no-maths astronomy class, very similar to an Astro 101 class in the USA. The class had an enrolment of 50, split equally between men and women. Most were science students, but primarily not in the physical sciences. The focus group sessions were held towards the end of semester, after the students had done six different role-playing exercises, spread over six weeks of lectures and tutorials. The focus group discussions were carried out in tutorials. No prior warning was given to students that they would take place, to avoid self-selection. Over 80% of students attending the class took part.

 

The focus group sessions were run by Dr Chris Trevitt from CEDAM: I was not present. He was instructed to try and prize out any hidden dissatisfaction with these exercises, but also to give the students the freedom to say what their perception of these exercises was. Notes were taken during the session and the conversation recorded, though this record was not made available to me until after the course was finished and all grades finalised, to minimise the chance of students keeping quiet about problems for fear of retribution.

 

3.3 Minute Papers

 

I regularly use the technique of minute papers to monitor how students are learning in my courses. Minute papers are a widely used technique across a many academic disciplines for assessing student understanding. At the end of the last lecture each week, I finish five minutes early, and ask the students to write down on a scrap of paper the answers to two questions:

 

1. What is the most important thing you learned this week?

2. What is the most important unanswered question this week left you with?

 

I collect and collate the information on all these scraps, and present a digest back to the students at the start of the first lecture of the next week. They were intended to monitor whether students understood the course materials. If students had basically understood the course materials, the ðunanswered questionñ responses would be diverse. But if Iód explained something badly, nearly all minute papers would mention this topic, allowing me to revisit it rapidly and fix the problem.

 

I did not use minute papers with the intention of gauging the effectiveness of role-playing exercises, but they proved to be an unexpectedly rich mine of information on how students perceived these exercises and what they were learning from them.

 

4. HOW THESE GAMES CAN GO WRONG

 

In this section, I discuss failure modes of these exercises. In particular, there was one problem I was expecting to happen which never did, and one problem I did not expect which seems to occur regularly.

 

When I first tried using role-playing games, my main worry was that the class would get out of control and start gossiping, throwing paper airplanes or chatting about the cricket. Somewhat to my surprise, Ióve never encountered this problem: students are, if anything, much more focussed while doing these exercises than in conventional lectures. I was worried, however, that other instructors, with different student populations, might find it a problem.

 

I found no evidence of this. Indeed, several users commented that it got students involved who were not normally engaged by lectures.  Here, for example, are comments from teachers at two US colleges:

 

ðOne hard-core cynical student who seldom does anything was sitting by himself looking through the book and I thought he was not participating but as it turns out, he was trying hard to find the answer to a question that the rotation group could not answer using his textbook, and we had a really nice discussion about the question (and it was the first time he showed any interest this semester).ñ

 

ðThe exercise went quite well. The students were much more willing to participate than I would have thought given that many of them are in the class only to satisfy a distribution requirement and so they don't always have a very positive attitude.ñ

 

The unexpected problem first surfaced when one of my games was being run with students as a prestigious US research university. The class became highly political, and instead of sharing information, they started deliberately misleading each other and withholding information, so as to gain a competitive advantage. I have since seen this same behaviour in one of my own classes (a group of year 9-10 high-school students), and have received two other accounts of the same problem. Here is an example:

 

ðThey didn't come to a full resolution because there was too much lying going on!  I mentioned the possibility of concealing information or misleading people and they took it to heart.  As a result they were struggling a bit to separate the facts.ñ

 

This problem has shown up in 3/5 reports from people using these games with grade 9-10 students, but only 1/30 reports with older students. Especially with younger students, it may thus be prudent not to play up the competitive nature of these games too much in the initial briefing.

 

Several other problems were also mentioned, though none as frequently as this politics problem. Most were anticipated and addressed in my original paper (Francis & Byrne 1999). Two rather interesting new problems were however reported, albeit by only one instructor each. The first problem was noted with a bunch of Australian high-school students, doing an extension course in cosmology:

 

ðMost interesting was that a majority of groups seemed insistent that they could work it out by themselves. I would stress the point that they didn't have all the information, and they should wander off and get some info from another group ... and then attempt to work that into THEIR knowledge base. They didn't readily catch the idea that the solution might come from GAINING knowledge and piecing that together. A few groups actively refused to get information from others. They worked on the problem and would approach me with theories based on ideas they'd heard or read about, things they had taught themselves in the past. The knowledge some of these kids had was amazing, but didn't come near to allowing them to get a full answer.ñ

 

The second problem occurred with students at a US liberal arts college:

 

ðThe only problem came when teams presented their theory. The first one had gaping holes and yet no-one in the audience raised any objections or asked for clarifications. At this college there is what has sometimes been termed ðthe culture of niceñ which means that the students will not readily express disagreement with each other or challenge each other.ñ

 

 

5. STUDENT PERCEPTIONS OF WHY THE GAMES WORK

 

There is a great deal of evidence suggesting that students enjoy these exercises and perceive them as valuable. Both formal questionnaire surveys and anecdotal evidence confirm this, as discussed in my original paper. The focus groups failed to identify any disenchanted students. But what specifically is it about the games that they enjoy?

 

One unexpected factor that showed up repeatedly, in the focus group discussions, the minute papers and even in comments from other instructors. Students seem to like these role-playing games because they make the students feel powerful and knowledgeable. Here are a couple of illustrative student quotes:

 

ðYou had your own sorts of breakthroughs when you found out another piece of information, and it was like Oh Wow! This fits in with this, and now we know this, andè You actually felt quite intelligentñ

 

ðWe enjoyed these tasks more because it gives us the sense that weóre the first ones to discover these things, and it gives us a sense of pride in what we were doing, whereas if we read it out of a book we wouldnót get the same sense of prideñ

 

Students noted that it was a wonderful feeling to be ðexpertñ in a given topic (by dint of having read the briefing paper), and to have others coming to them asking for information. They felt a sense of pride in working out puzzles for themselves, even partially. Feeling intelligent in class was clearly a novel and exciting thing for many students!

 

I was not expecting this at all. Ióm still not sure I fully understand it. It does seem that these exercises bolstered the self-esteem of a significant number of students. As all student feedback was anonymous, itós impossible to know precisely which students respond in this way: I guess it is the weaker students.

 

On reflection, I suspect that an astrophysics classroom is a pretty intimidating place for a mediocre student. Distant lecturers, who are clearly far more intelligent than you are, tell you stories of great researchers and the profound things that they discovered. The idea that there could be anything in common between them and you seems laughable. You struggle with assignments and get mediocre results, further confirming your lowly place in the intellectual class system.

 

I find it very heartening that these role-playing games can help break down this gap between the students and the researchers about whom they learn. The self-help literature is full of accounts of how role-playing yourself in a successful situation can help overcome negative thoughts. But this was a purely accidental side effect of these games, not foreseen when they were written. If we can really understand this effect, deliberately crafted pedagogies may be able to do much better.

 

Another surprise: I usually hand out small prizes (plastic glow-in-the dark stars, for example) to groups brave enough to propose a theory to the rest of the class at the end of the role-play. This was intended to be just for fun. But many students mentioned that they really valued it. They fully realized that it was ðjust for funñ. But still, they thought it was really important. They were not very articulate on why it was valuable, but this message was repeated enough that it must have been telling us something important. I do not understand the psychology behind this either.

 

One final trend showed up in the minute papers. I had expected these to concentrate on astronomical details. But during weeks in which I ran the role-playing games, over 40% of student responses to the question ðwhat was the most important thing you learned this weekñ were concerned not with particular astronomical facts, but with more general issues, as illustrated by the following student quotes:

 

ðThe way that as a group of people, we as humans try and answer the way things are using a collection of seemingly unrelated factsñ

 

ðThere are no right answers, only theories based on observations.ñ

 

ðAstronomy isnót all stiff attitudes and boring theoriesñ

 

ðThe best way to learn is to ask questions. Never accept an answer or a theory without thinking it through, and if you disagree, without argument.ñ

 

These had not been made explicit as learning goals of the course, so I was surprised that such a large fraction of students made these explicit as things they had learned. These general attitudes towards the scientific method are highlighted as important in recent discussions of what should be in introductory astronomy courses (eg. Partridge & Greenstein 2003).

 

6. SOCIOLOGICAL NOTES

 

Ióve found it very interesting watching how the use of these games has spread: in this section I briefly mention some interesting things Ióve observed about the process. This is not in any sense a serious study, but I hope you will find these tit-bits interesting.

 

What stops people from using these games? Two main themes came though in talking to users. One was fear:

 

ðBeing new to teaching, I wasn't brave enough to try it right away, but now I think I will, this week.ñ

 

ðI had tentatively planned to do it last winter, but chickened out as sometimes I get cold feet when trying to do something òdifferentó.ñ

 

Another obstacle was an over-stuffed curriculum. I know that this is a major problem with physics courses, but Ióm somewhat surprised that itós an issue with introductory astronomy courses, as they are not prerequisites for anything. Surely it makes sense to teach a few topics properly rather than race through all of astronomy as a shallow compendium of facts (a controversial issue)? But it does seem to be a problem in practice, as the following quotes show:

 

ðI am excited to try these exercises, and don't really see any downside to themè èexcept that my lecture time is reduced, and, for a 101 course that is everything-you-need-to-know-about-astronomy-in-sixteen-weeks, that could present a problem.  But, hey, I'm going to try it.  And, in the meantime, I am trying to talk my college into making Astronomy 101 into two semesters.ñ

 

ðI have had mixed success: I like a lot about them, but it takes a lot of class time.ñ

 

Another surprise: when I originally put these exercises on the web, Iód imagined that people would take the idea and write their own exercises, adapted to their particular courses. Instead, over 80% of users took my games and ran them unmodified. Fewer than  20% modified them, and I only heard of one case in which new exercises were written.

 

 

7. CONCLUSIONS

 

Overall, Ióve been very pleased by the results presented in this paper. These games seem useful, and the problems identified are not showstoppers. The effect these games have on student self-esteem is perhaps the most interesting and (to me) surprising result that came out, and one most worthy of further study.

 

 

Acknowledgements

 

I would like to thank the staff at CEDAM for their help in carrying out this research, the Department of Physics for being so supportive, the many users of these exercises around the world for sharing their experiences, and most of all the students, who have been such willing guinea-pigs in all this experimentation.

 

 

 

References

 

Byrd, G.G. & Coleman, S. 2004, ðExploring the Universe Together: Cooperative Quizzes With and Without a Classroom Performance System in Astronomy 101ñ, Astronomy Education Review, 3, 26

 

Francis, P.J. & Byrne, A.P. 1999, ðUse of Role-playing Exercises in Teaching Undergraduate Astronomy and Physicsñ, Publications of the Astronomical Society of Australia, 16, 206

 

Kavanagh, C., Agan, L. & Sneider, C. 2005, ðLearning about Phases of the Moon and Eclipses: A Guide for Teachers and Curriculum Developersñ, Astronomy Education Review, vol 4

 

Mazur, E. 1997, ðPeer Instruction, a Userós Manualñ, Upper Saddle River, NJ: Prentice Hall.

 

Partridge, B. & Greenstein, G. 2003, ðGoals for "Astro 101": Report on Workshops for Department Leadersñ, Astronomy Education Review, vol 2

 

Sebastiˆ, B.M. & Torregrosa, J.M. 2005, ðPreservice Elementary Teachers' Conceptions of the Sun-Earth Model: A Proposal of a Teaching-Learning Sequenceñ, Astronomy Education Review, 2005, Vol 4.

 

Zeilik, M. & Morris, V.J. 2004, ðThe Impact of Cooperative Quizzes in a Large Introductory Astronomy Course for NonïScience Majorsñ, Astronomy Education Review, 3, 51

 

 

Resources

 

Copies of all my games, and of my previous paper about them, are posted on this web page: http://www.mso.anu.edu.au/~pfrancis/roleplay.html.