» Learning Elsewhere: Tales from an Extra-curricular Game Development Competition

Learning Elsewhere: Tales from an Extra-curricular Game Development Competition

June 26, 2013 by Chris Johnson. Filed under public, writing.

Following is a draft of a talk I am to give at ITiCSE 2013. The work discussed was done several years ago at Iowa State University, during my first full year of full-time teaching. I wish I had more quantitative results, but its absence is sort of consistent with the premise that we cannot do everything as educators.

Let’s start with a little game. I will make three statements about me. One of them is not true, and you, the audience, must figure out which it is. Here they are:

One reason I married my wife is because she reminds me of a character from my favorite game.
I think every computer science program should offer game development classes.
I unknowingly recreated the depth-first search algorithm in 5th grade, when I was 11.

What do you think?

After this talk I’m happy to talk to you about my wife and my 5th grade exploits. For the time being, I’d like to talk about why I don’t think every computer science program should offer game development classes.

Game Development Education Fever

I can list a few reasons why adding gaming curriculum might not be a good idea. They all really have to do with maintaining our integrity as teachers.

Unqualified

Many of us are simply not game developers. The whole education system works best if we teach what we know. Irish game developer Tony Kelly writes this criticism in an essay:

Perhaps the single biggest criticism levelled at the current crop of courses in Ireland is the apparent re-badging of traditional media and computer science courses to include the word ‘games’ somewhere in the title, while failing to adapt the content and instruction offered to deal with real-world issues the games industry, and by extension the course graduates, face on a day-to-day basis.

His sentiment is echoed all over the Internet.

Interestingly, our not being game developers doesn’t mean we don’t recognize what a game developer needs to know. Monica McGill published a report in 2009 on the “expectation gap,” which is the difference between what academia values in their graduating seniors and what industry values in their new hires. She found through a survey posted to 15 game development programs and a number of game studios, that both parties have pretty much the same expectations. The only significant differences were that those 15 programs gave less attention to scripting languages and concurrency than industry would have liked, and those programs placed more emphasis on computer graphics than industry wanted. For most other skills, academia and industry tended to agree.

This sounds like a checkered flag: let’s roll out the gaming curriculum. However, a few things give me pause:

15 is not a large number. McGill says the candidate pool was only 40. Those 15 schools might have been top tier game development programs. It may be that our own departments don’t know what industry values.
Agreeing on paper is easier than examining real graduates. Let’s put together a study where we look at game development graduates from schools ranked like our own and see what industry thinks.

A fundamental problem stands in the way of our being qualified: a game developer develops games. Few of us have the time to be both an educator and a game developer. (Do you know how many hours game developers put in each week?) A number of institutions have dealt with the lack of qualified teachers by hiring real game developers as adjunct faculty, which is better than pretending to have the skills in house.

Our Motivation

Many of us snidely accuse programs that start offering game development classes as motivated by student tuition money. However, maybe we’re just mad we didn’t think of the idea first. McGill again chimes in on this with a real study on the motivation behind gaming curriculum in the US and the UK. She has this to say about American motivation for introducing gaming:

In the US, the interests of the students (both current and prospective) ranked as the highest motivating factor (41%) followed closely by the interests of the university or department (35%). The interests of faculty ranked at 18%. Of lowest consideration were the interests of industry (6%).

Industry push was only 6%. That seems dangerous to me. (The UK was a bit more balanced.) Think about what we’re doing: students are being given more leverage than industry on deciding our curriculum. Our governments will not be happy when they hear about this. We might lose some funding.

I heard Edward Tufte once encourage us not to pretend to know what motivates people. When we look at just our own motivations, we see that they are quite soupy. We can only expect that others’ are soupy too. Having said that, whatever the motivations are behind this video, we should not be fishing for students in this way:

http://www.youtube.com/watch?v=cjCo2I3ooK0

This approach is manipulative.

We Can’t Do That to our Students

A third reason we should think twice about adding gaming curriculum is that we may be paving our students way to an industry that may suck all the joy out of their lives, destroy their marriages, and make their children emotionally bankrupt. Consider these statistics from a recent survey of 1000 developers, conduct by Game Developer magazine:

24% work more than 50 hours a week.
10% work more than 80 hours a week during crunch mode.
22% regularly work on weekends.
52% do not feel adequately paid.
23% expect to be laid off after their current project.
62% would advise friends and family to get a job in the gaming industry.
58% have confidence in their managers. (That seems high, doesn’t it?)

The Alternative

My thesis today is not, “Turn away from gaming.” Games are a major part of our students lives. The Pew Internet & American Life Project reports that 97% of American children play video games. Nielsen Games says 50% of young adults in the UK play video games. We’d be silly not to take advantage of games. What I propose is that, instead of us lecturing on about something we don’t know about, let’s use the resources we do have to build an unstructured learning environment outside of any course. For the rest of our time today, I will share about one such environment that I helped put on: a year-long game development competition.

Outside the Classroom?

The first thought that should be jumping into your heads right now is “We are educators. We have to assess, and to assess, we have to have control and structure.” I don’t disagree, as long as we’re in the classroom. But working outside the curriculum means we have freedom. Our careers are not on the line, so we can step back and put students in charge of themselves, which has all sorts of advantages.

What are those advantages? Here are a few:

Autonomy leads to increased student motivation. I’ve heard education researchers say this isn’t open for debate any more, so we can stop reevaluating it.
When we plan a lesson or especially a whole semester, we reverse engineer our knowledge on the subject matter. We tell ourselves, “To teach D, I first have to teach C. And C relies on B and A. B doesn’t rely on A, so the the order of A and B doesn’t really matter.” In order to become the experts that we are, we’ve organized the content knowledge in our minds. However, can our predigestion of the course material rob students of organizing the knowledge themselves? The answer is yes. (See chapter 2 of How We Learn for some evidence of this.)
Students demand different things of themselves than we do. At Games + Learning + Society a couple of weeks ago, Edd Schneider and Tony Betrus shared about a study of task completion in the game Grand Theft Auto. When a teacher assigned tasks to students, starting with easy and progressing toward hard, more tasks got done. But when students picked their own tasks, they accomplished fewer tasks but they completed more difficult ones.
We can find different motivators to keep students involved—ones that don’t demand hours of our day like grades do. Competition is one such motivator. It doesn’t work for everyone, but some of the bloodthirstiness can be overshadowed by grouping students in teams. Those motivated by social cooperation are happy, and those motivated by competition are happy.

Our Competition

Setup

Okay, let’s talk about our competition, so that you can tell me what we did wrong. It started off because I sat in on a 3-D modeling class taught by a design professor. He said, “You know, we’ve got students in our two programs that love games. We should do something together.” Later, when a grant notice for student competitions came through my email, we applied and were awarded the grant. We were going to hold a game development competition.

The birthplace of our competition was a 3-D modeling class, where I made this model of my son’s pull toy.

Since our motivation was to build bridges between our two programs, we decided that students would team up and that each team that entered would have to have a member from a technology major, one from design, and one from business, because that seemed important. Others were invited to join too, but at least these three were needed.

Games are very diverse these days, so we figured we’d need some categories to help us distribute prizes in different directions. We created three: PC/console games, serious games, and mobile/web games. The teams would have the academic year to complete their games, and then at the end, we’d bring in judges to evaluate them at a public exhibition. The top three teams in each category would get a cash prize, provided by the grant money. The prize money was split evenly among team members. We can never give money away with state funds. Thankfully, this grant was from a private foundation.

We wanted teams to form early, but we knew there’d be personnel issues to sort out. We required teams to commit their rosters and category to us by the halfway point of the year.

Execution

We started the year off with a visit from a fledgling studio that had just published an Xbox game. People were excited. There was a buzz in the air. We decided to have monthly visits from developers. Our next visit was from two alumni who had gone into game development. Sadly, we found out the day they arrived that they had been laid off. In their talk, they shared horror stories of the industry: long hours, failed marriages, canceled projects, and incompetent managers. Some participants were sobered by their honesty.

Apart from these talks, the students really just worked on their own. They brought in skills from their classes and did a lot of reading and searching. By mid-year, however, we saw that more students might finish if we added a little bit of structure. A second design professor decided to offer an optional independent study studio course so that teams would have some dedicated time to work together. There were no lectures and everyone got As.

The categories proved to be disappointing. We had 9 PC/console entries, 3 serious, and 3 mobile. Bad planning on our part. The serious games category was especially frustrating, because we had gone out of our way to solicit “bounties” from non-profits across the state to commission games. We thought students would love to have real clients. No bounty was claimed and not all of the three entries in this category had a complete game deserving of a prize.

The major requirements were also disappointing. The design and technology majors approved of each other, but few saw the point of having a business major. After a while, I started to agree with them. We really needed an advocate from within the College of Business, which we had sought but failed to secure.

Even so, teams persevered. 120 students registered on a team at the midway point. At the end of the year, we held a public exhibition in our student union, with 80 of the 120 still in the game. The exhibition day was busy and great. A designer from Disney Interactive spoke, we had cake, and a dean gave out prizes. Our award sculpture was a 3-D-printed trefoil knot.

Assessment?

When all was said and done, we tried to figure out what happened to our students through a survey. One-third of the 120 registrants responded. I’ll be honest, the survey is really lacking in quantitative results. I really wish I had an sidekick that was good at experimental design.

We wondered if bridges had been built across the three programs. So, we asked, “Suppose in future competitions no major requirements were imposed. How likely would you be to recruit a student with a ______ major?” For technology, 34 said very likely, 1 said flip a coin, and 0 said very unlikely. For design, 25 said likely, 1 said flip a coin, 3 said very unlikely. For business, 1 said very likely, 8 said flip a coin, and 15 said very unlikely. It looks like art and design cozied up pretty well. Poor business.

Even though this was extracurricular, we were hopeful it had a positive academic impact. We asked students about what effect the competition had on their studies. Their choices were non-exclusive. 31 said, “I love my major and want to learn more.” 11 said, “It contributed to my classroom learning.” 15 said, “I’d like to pursue a career in the gaming industry.” 6 said, “I am unfazed.” Only 11 said it contributed to their classroom learning, but 31 were well-satisfied with their major. The way I interpret this, given the context of the quotations I’ll share next, is that students judge their major based on idealized notions rather than their professors, whom they often regard dismally. The competition helped them see the skills of their major outside of a professor’s command. Less than half wanted a career in the game industry, which is good for where I live.

The most enjoyable part of the survey were the open-ended questions. We asked, “What did you learn?” A few said:

“I’m a kinesiology major, but what I learned from this project can be applied to any future job I’ll have. Every job involves being part of a team in one way or another.”
“I learned many things, including basics for 3D modeling, complex 3 dimensional movement algorithms, and matrix based math. And in addition to those concepts, I learned a lot of about game programming, such as how to create particle systems, basic artificial intelligence, rail based movement systems, animations, collision systems and artificial intelligence directors.”
“I learned that I have a much better work ethic than I thought, I just needed to the chance to work on a project I really cared about.”

We asked, “What was it like to be your own bosses?” A few said:

“We would not have survived without an amazing PM [project manager]. He was the bomb.”
“Great opportunity to have a project that was completely unrelated to school become something of so much passion. This competition made me hopeful in starting a business.”
“I am a very self managed person by trait, but not knowing exactly what I needed to do was a challenge. There is a bit of security in having someone else set the direction. I definitely liked being my own boss though.”

We asked, “How did you feel about the lack of structure?”

“I learned SO much more than a regular class. The environment in game design is one where you have no idea what you are going to need to do, but you just find out and do it. I even had a very nice 3d modeling and animation internship last summer and I would say I learned much more than that as well, second to none learning experience.”
“The competitive element gives me much more drive than I would have in a university lecture…”
“It felt like what I imagine the ‘real world’ will be like. I had a lot more freedom in what I was doing and could make things how I thought they should be. In a traditional university course you have a strict schedule to follow but it wasn’t the same for the competition. Also for the competition I felt like I had to try harder because if I failed it would effect other people. In a classroom it isn’t quite the same.”
“I honestly feel like I learned so much more practical knowledge in this competition than in my traditional classes. I was working hands-on, with people from different majors, in languages I had never used, to create something I was passionate about. It was a fantastic experience.”
“There’s no comparison between sitting in lecture and chasing A’s by doing the bare minimum requirements in homeworks to simply coming up with your own idea, evaluating it and tweaking the game dynamics, then designing, implementing, testing testing testing the code, making mistakes along the way, all powered by your own passion. When you have total control over a project and are only participating because you want to, you become so much more creative and motivated. The project becomes your baby rather than ‘that stupid assignment that I just need to kind-of-finish.’”

Conclusion

In closing, I can’t tell you exactly what each student learned from this competition, but we got them excited, they reported they learned something, they built interesting games on their own, they worked with people of other majors, and we all had some cake. Our interest in gaming was a response to our students’ interest. One way we could have responded was by offering a new course, but we would have been faking the whole way through. However, we are educators and have more than one tool in our belts. We met our goals through an unstructured competition, without planning a single lecture or pretending to be something we are not.

teaching machines