Assessment in Games and Education

As the person behind Edubacon, I figured I should keep my word a post about the topic of the week, assessment. While I made another post for the topic ( http://blog.igenoukan.com/2009/06/designing-for-play-creativity-and.html ), this one is a recap of some related posts of mine.

Happy with Perfection?
http://blog.igenoukan.com/2008/09/happy-with-perfection.html

I see several problems with a 100% mentality to grading. This post covers some of them, and a possible solution or two. Part of the problem is parents, but part of it is that a higher difficulty can lead to lower grades, when there is more learning in the higher difficulty. Homework, testing and grade representation all have their parts in this. Expanding our assessments to include a variety of good qualities can give us a better result.

Iteration and Prototyping
http://blog.igenoukan.com/2008/10/iteration-and-prototyping.html

Perhaps not the first thing to come to mind for assessments, it is relevant. First is that each prototype is different, like a person's skills and approach differ after learning.

Pointless Obstacles are Bad
http://blog.igenoukan.com/2008/10/pointless-obstacles-are-bad.html

This is a little more directly related in that assessments shouldn't be limited by classes. Testing into and out of classes is an example, but not the limit. If a person has taken the course once, why should they be forced to take the course again just to retest on the curriculum. Perhaps this would make colleges and universities seem more like testing and accreditation institutions, but having to pay for the class and redo all of it just to prove capability doesn't make sense except as a business.

Testing Before Teaching, Why?
http://blog.igenoukan.com/2008/12/i-find-following-article-to-be.html

Defending pre-tests really wasn't my initial intent, but it was hard not to in response to a particular article. One example that even the author conceded was testing out of a class, but there is so much more to a well given and used pre-test. It can help teachers and students. One of the big points is it not counting towards the course grade. Are the results important? Yes, but a pre-test that counts for the grade isn't over material covered in the course. It has other uses.

Logical Content Breakdown
http://blog.igenoukan.com/2009/01/logical-content-breakdown.html

This one is about the content of the assessments, and the order. In the example covered in the post, splitting up the content differently and using a slightly altered explanation would make learning the curriculum very different from the current norms, for that topic. To further facilitate that kind of content breakdown and the learning that comes from it, assessments and self-assessments should be given. Yes, I just said that self-assessments should be given. It's a matter of getting students to see their mistakes, shortcomings and weaknesses. The content breakdown can make learning the material easier, but assessments can help learners see how well they have learned the different parts.

Metalearning: Learning How You Learn
http://blog.igenoukan.com/2009/04/metalearning-learning-how-you-learn.html

The big idea is that it's good to know how you learn, and so it is good to learn how you learn. To do that requires assessing the results of learning attempts, and comparing them; assessing how well and learning method works for you. Trial and error is the default method for this. Helping students understand themselves as a learner has many possible positive effects. One is giving students the knowledge to craft good learning environments and select good learning resources for themselves. If you don't know how you learn, experimenting while being graded can seem like a very bad idea.

Skill Based Grading
http://blog.igenoukan.com/2009/04/skill-based-grading.html

I'm a game designer, and this is one of the points I like in gameplay when there is to be improvement; skill based assessment. With skill based gameplay, difficulty is matched with skill levels to provide a suitable difficulty. Well, what if the debriefing of results and skill based results found in some games was used in assessment? I think it would be interesting to have achievements, multiple measurements and the ability to retry after learning from previous mistakes.

Automated Tests

No, I didn't miss a link. This isn't a post I've made before, but rather a record of my thoughts after going through those posts. I think one of the keys to improving assessments is to make them less costly to give and take. Automated tests are a possible solution, if done well. Randomly generated problems will always be able to be solved by the right software, but they can be used as non-graded self-assessments and graded assessments when other tools are not available.

I know that many proponents of improving education are for opening up the resources normally available to students, but the keys to automated tests are providing the resources and not allowing alternative resources. Basically it becomes a game. How well you do at the game shows what you can do, but if you have a tool that solves the problems for you, it's pointless to use. Yet it is also not logical to test a person in a way that removes information that would be available to them in the real deal merely because they don't remember it. So, if the assessment was on derivative calculus as a whole, it might start with picking the correct names of the rules, putting the equations together correctly, using them in problems and correctly identifying the abnormal cases. It would probably take a while, but everything needed is there and requires the learned skills and knowledge to succeed. I think most people would enjoy assessments more if it was less like being put on trial and more like exploratory play. Isn't education supposed to be a safe time to make mistakes before going out into the real world?

Have fun, spread the word and tell me what you think,
Igen Oukan
Read more!

Assessment in Games and Education

View Blog Reactions

Designing for Play, Creativity and Motivation

This week at Edubacon the theme, Weekly Special, is motivating learners. As a game designer and sole writer at my blog, Learning Science Meets Game Design, I'll be taking a look at this from the point of view of designing motivation into a user experience, which is as applicable to motivating learners as players. There are also good points for making a work environment with playfulness.

I look at how motivation is handled and I don't like it. When such views are brought up, it's common to hear that something being good for you should be enough, and I agree that it should. However, that doesn't mean it is. See, there's a problem with one good point, even a really good one, trying to outweigh a lot of bad points. Unless it is a stellar good point, the person isn't likely to think the action(s) is/are worth the effort.

Each of the following TED talks shows a part of the picture. I'll highlight the parts I think are most relevant after each. There are also a few web pages that I think are relevant, with commentary on each of those. So, let's get the show on the road.

NOTE: There's about eighty minuets of video in this post. If you don't have that kind of time, you might want to be selective of which ones you watch.

Sir Ken Robinson
Current style stifles play and creativity.

http://www.ted.com/talks/ken_robinson_says_schools_kill_creativity.html



Most of the current systems of education, training and instruction say there is a single right way. (I'm guessing this comes, in part, from education and religions interacting over the centuries.) That way is to be memorized and repeatable. That's pretty much it.

If you make a mistake, you are punished. This makes it so mistakes become taboo. Mistakes, things that could potentially cause pain, things that could potentially limit profits, actions without guaranteed positive results etc. are taboo and limit exploration of possibilities, innovation and creativity.

Recently this idea has come up in a Copyright Summit. The blog post about this occurrence was also picked up by Stephen Downes via OLDaily. Take a look to see what Stephen Downes says about this and also the post itself. It gives me hope just reading it.
http://www.downes.ca/cgi-bin/page.cgi?post=49235
http://copyrightandtechnology.com/2009/06/09/gridlock-at-the-world-copyright-summit/

Tim Brown
Linking play and creativity.

http://www.ted.com/talks/tim_brown_on_creativity_and_play.html



Having reminders of the intended playful atmosphere is a very good thing to have. Silly activities and symbols of play remind each person that while there is some seriousness to what's going on that there should be play and creativity too. With the fear of failure out of the way, more creativity can shine. In essence it is supporting the people to be free and playful, rather than just having the option.

Stuart Brown
More on the importance of play.

http://www.ted.com/talks/stuart_brown_says_play_is_more_than_fun_it_s_vital.html



More about play, but it focuses on the importance of play on being a balanced person and how it changes interactions. I know that personally I prefer to be around people who are willing to laugh at themselves and things they find funny. It's a way of being emotionally open to the world. This and the previous video are wonderful for sharing play as a concept. Simply put, it' important for your mental, emotional and physical well being to play, at least in some way, preferably many ways.

JJ. Abrams
This is playful, exploratory learning.

http://www.ted.com/talks/j_j_abrams_mystery_box.html



Not only is it important to deal with the fun and reminders of play, but also things like mystery. When should information be withheld to improve the method? What information should be withheld? Sharing everything at once would just cause a problem, covered in another TED talk called the Paradox of Choice (http://www.ted.com/talks/barry_schwartz_on_the_paradox_of_choice.html). The idea is that sometimes it is better to hold back the information, rather than giving all the options and information. Reaching the happy balance of participating to a significantly beneficial level without being overwhelmed is perhaps not easy to design, but we should strive to reach it for ourselves and others.

Dave Egger
There are ways for everybody to help.

http://www.ted.com/talks/dave_eggers_makes_his_ted_prize_wish_once_upon_a_school.html



I'd say this is the big one for showing how it can work. There is play and work, assessment and amusement. The other big thing is that participating is the important part on both the student's part as well as the tutors. Both sides need to participate, like in play where both sides submit to the rules of engagement. Everybody who "applies themselves" gets more out of it, because they are willing to play. It doesn't take much to help, but that little bit can mean a lot to those you help. It's also important to noticed the cooperation between those being helped and those giving the help. Just because you are helping doesn't mean you're the best choice to determine what should be done.

Influencing Others

http://www.gamasutra.com/view/feature/1573/becoming_a_stellar_games_industry_.php

While I do like this article, it seems a little aggressive in it's approach to what should be done with influence. "In a perfect world, you would be able to influence everyone around you – your team, your developer (or publisher), and management – to do things exactly your way." I think that the truth is in a perfect world everybody would agree to implement the best answer, but since we don't live in a perfect world we should try to help people, including ourselves, to explore different points of view to help determine the best answer. That, and we should all remember that the answer we choose isn't likely to be the absolute best answer, but it doesn't have to be.

Avoiding Meltdown

http://bbrathwaite.wordpress.com/2008/12/10/team-meltdown/

This blog post shows ways to avoid a team meltdown. The title says as much, but it's a good place to start. Preventing a meltdown is important to creating a playful environment.

Building Environments

http://www.thejonjones.com/2009/02/08/speccing-out-contracts-smartly-aka-automatically-building-awesome-teams/

This post is about preventing problems and making the way you do things create a positive work environment. I like it, and think it fits well.

All in all I really haven't said much in this post, but rather have strung together several pieces of the internet together to form probably a two hour presentation's worth of content that could be made into a multi-day event. I guess for once in my life I am being concise. It all works together to help motivate the people providing the service and those receiving the service.

Have fun, spread the word and tell me what you think,
Igen Oukan
Read more!

Designing for Play, Creativity and Motivation

View Blog Reactions

The Mystery of Motivation in Games

It has been noticed for a long time that people seem to be more motivated when playing games than in school. This motivation has been left as inherent to games, but the efforts to create educational games has for the most part shown that this is not true, maybe. I think the motivation is outside the game entirely, but that the game design principles for user experience design should be helpful in encouraging motivation in learners.

For the last week I have been participating in the Serious Games Jam and it came about that I posted about motivation and approach. This setting contributed to the efforts, but did not generate the motivation to figure this out. Rather it gave context, which was a serious buzz word during the Jam. That perhaps pushed me as something not quite right, but that's still doesn't explain the motivation.

No, the motivation was outside the entire situation and a part of the picture that I brought. It wasn't due to the context of the jam or the "game" of earning "Karma" points to win a Nintendo DS. There was something inside me that motivated me to try to find answers related to serious games, that motivated me to participate in the Jam, that motivated me to push through the hardships and get to the good parts, the rewards, but the rewards didn't generate motivation. I'd stopped expecting to "win" the "game" early on. So, what was it?

How I Figured it Out:

Due to a tweet from Jim Groom, if I remember correctly, I came across a parody article. That's where it started.

Parody: http://bionicteaching.com/?p=997

Real Article: http://chronicle.com/wiredcampus/article/3797/students-prefer-real-classroom-to-virtual-world

The real article was also mentioned in OLDaily, by Stephen Downes.

OLDaily Edition: http://www.downes.ca/archive/09/06_02_news_OLDaily.htm

Specifically: http://www.downes.ca/cgi-bin/page.cgi?post=49149

While I thought it was interesting, nothing triggered at first, but in the post by Stephen Downes there is mention of the comments made on the article, so I started reading them. In there, somewhere, the ideas started smashing together like atoms in a giant atom smasher giving me a glimpse of smaller parts. That's where I found the motivation from outside video games that educators have been looking to add to their curriculum.

From my SGJ forum post:
"What came to mind is that motivation, approach and goals all need to coincide for the learning to really happen well. In the normal school environment grades are the goal, not learning. When you have people together who want to learn, and have a decent way to learn, the learning is likely to happen. Why?

Students are told what they should and shouldn't do all the time it seems. Truth is most of us are taught from a young age to follow orders without thinking about them. In that case you are being told not to care about the the how or even the what. So, the question becomes, why should they care about learning? The answer heard most of the time boils down to, "I said so." That's not the kind of motivation found in games.

In a game you expect to fail sometimes. That failure gives you feedback that you can use to improve, in good games. Most of the time that feedback is in the form of watching what happens as it happens. Then you use the scientific method to improve. Hypothesis, testing, analysis and repeat as needed. That's an engaged approach that is looked for in education, however it isn't the games themselves that generate the motivation and approach.

Through experiences we see potential, we create dreams/goals and then we hope to reach them. The more effective the action is perceived to be in reaching the dreams/goals, the more hope the person has that it will work. That hope motivates the person to go through the hardships to reach their dreams and goals.

So, first a person needs the experiences that open up the potentials of their life to the person. That gives them things to dream about doing and becoming. Goals are born of the dreams and hope is born of the perceived possibility to reach those goals and dreams."

There's the motivation, "hope motivates the person to go through the hardships to reach their dreams and goals". Give a person hope to attain their dreams and aspirations and suddenly their willing to go through hardship to get there.


In another thread there was a discussion about what serious games are, since people didn't seem to be agreeing. Before a certain video was shared with the group the defining point of a serious game was intent, after it was shared the defining point was agreed to be serious application. In essence, any game that could be used in a serious application became a serious game in that application.

The Video: http://www.youtube.com/watch?v=rN0qRKjfX3s



Suddenly there were two elements that the group decided were at play, the game and the context. However, both had been in use for a while now, not very well most of the time, but they are used. They didn't answer the big questions for me.

Here's what I wrote in response to the pre-video discussion about the intent being the deciding factor.

"I believe the serious vs entertainment comparison is counterproductive. A serious game can be an entertainment game as well as a an entertainment game can be a serious game. It's like saying something is either tall or wide. First, it depends on the direction you turn it. Second it depends on comparison and scale. To be honest, I don't like the term "serious game" because it doesn't easily map to the real meaning, a game with a serious application. Any game that can be used for a serious application is a serious game in that application to me.

Fact is that game design is user experience design and communication, and that means all you are able to do is give the opportunity. It's the user's way of using what you designed that determines how serious the game is to me."

Games are activities, mental or physical, with rules and goals where the player takes on a role in an agreed upon fictional setting and state of play. Within that setting the person may be a paddle, a fighter, hero, villain, monster or anything else, including themselves. That's the identity they take on, the role they play, in that pretend scenario.

Earlier this year I countered Stephen Downes on the topic of Educational Games. In it I mentioned a video of James Gee, but I would like to add to that two more videos I've come across. The first is "The Urgency of Open Education" by Brian Lamb. The second is Dave Egger's TED talk "Once Upon a School".

See, when you go through those videos you can see the motivational hope to reach goals through participation. Games are by their nature interactive with goals to be attained with a possible promise of being entertained, engaged or something else. The player is immersed into an environment designed to teach the player how to succeed in the game, to make it more enjoyable to play. They have to teach well to prevent the player from deciding not to play, due to frustration.

The games and context require something from the player to really work, participation, as I noted in the forum discussions. "Context influences the perception of the game experience the user has, but does not determine it. Those using games for serious applications should design the context to support the serious application. The rest is up the the user's motivation and approach. While I agree the context is important, I say that because it is part of the game experience, just like elements of the game. If somebody isn't open to the serious application of the game, that's pretty much it."

So, where is the root motivation, the approach? Well, that's where I come back to the original forum post I made about Motivation and approach, "From that logic I see a need to help people see potential and that it IS possible to attain." We need to encourage people to dream and show them that it is possible to make those dreams come true. If they don't have dreams or goals, why would they care enough to put out effort? If they have dreams and goals that are very basic, why would they work really hard to excel? If they have dreams and goals that they don't believe can become their reality, why would they really apply themselves?

Be real. Go big, or go home. Live your dream. When there is no fantasy in the mind there is no fantasy in the life. My life is like a cartoon show because I live it like that. Nature, dreams, friends, games, fun, wonder, learning, improving ... When people tell me that what I'm about to do is impossible, I respond with, "Just watch." That's because if I do it while they are watching, they can see that it IS possible. Since I didn't tell myself it was impossible, I stood a chance of doing it if it is possible. I dream and live, because I want to DO.

Imagination and dreams are what propel us into the unknown waiting for us to reach for it. At this point I ask you to watch another TED talk, because it sums up several cultural issues to be faced to motivate people to learn.

http://www.ted.com/talks/ken_robinson_says_schools_kill_creativity.html



We need to stop limiting the creativity of others with our fears. Parents can't protect their children from everything, nor should they. I mean that very seriously, and not just for parents. As a toddler I bit my Mother, once. I only did it once because she thought it was a Texan giant bug of some sort and tried to smack it full force. Since there was no bug there, I was smacked instead. Pain is just pain, not damage. Failure isn't the problem, it's the results that can come from failure. Because she smacked me, I learned and didn't bite as a child, at all.

When we are wiling to take a little pain here and there with some failures along the way, we become open to exploring ideas, like when we play. When playing, which is the state that games are supposed to induce, we do not mind a little pain, boredom, failure and we let go of the tight laced straitjacket of conformist tradition. Why, because it's play, not serious. This is where the term "serious games" goes completely wrong. It still needs to be play, meaningful learning play. Yes, that's another TED talk. Watch it and you'll understand why it's there and what I meant just now.

http://www.ted.com/index.php/talks/stuart_brown_says_play_is_more_than_fun_it_s_vital.html



A play, is where people present role playing of a scene to the audience, and we call it art. It's not just art, it's make believe, pretend. It is encouraging people to think beyond their current lives and consider other possibilities that exist, showing them things that can be done. It's also entertainment.

This all comes together with an understanding of learning that says experience and knowledge together, not apart, enable learning. Work done to learn is a part of play. Play is a part of the motivation for doing some things, but even that does have a motivation sequence that matches what I have described. Seeing the enjoyment of those playing, one is likely to long to share such experiences. That's the exposure through experience creating dreams and goals. Since it is easy to participate and reach the goal, there is the hope to motivate the person to play.

This play motivation is the basis to really getting people motivated to do anything. If people refuse to play, it's nearly impossible to motivate them to do anything, because they have to at least be willing to entertain, play with, a thought to be motivated concerning that thought. People who open mindedly use logic play with it.

Have fun, spread the word and tell me what you think,
Igen Oukan
Read more!

The Mystery of Motivation in Games

View Blog Reactions

Creatively Getting Things Done

Regardless of the field, creative designing, and getting those paying jobs, requires a few things.

1) Keep coming up with ideas.
2) Record those ideas.
3) Find one you can do quickly an easily.
4) Do that one, just that one.
5) Have a real life.

While this isn't an exact set of rules, they are pretty good. This is because over time you will get some ideas that are possible with potential. If you can get a few made, you have work samples. That's what the fourth step is all about. Pick one, do it and do it well.

Number 1: Keep coming up with ideas.

If you are trying to be known for creativity, don't just be creative on command. Be creative all the time. Having a bunch of half formed ideas can be a life saver, inspiration or an extra bit of content for the on-command idea.

Yet, the big reason is that even for those who can be creative on command, being creative all the time helps. For those fortunate people, the quality of ideas will be better if you are already warmed up and well experienced. For the rest of us it is easier to just be creative most of the time than to be creative on command. Then you are just doing your normal level of creativity.

It also prevents the dreaded scenario of loosing your creativity due to being boring. Sometimes it is harder to be creative after a while having not been creative. If you are always being creative, you're not likely to go for a while without being creative. Yes, it's obvious, but some things just need to be reiterated.

Number 2: Record those ideas.

While it may be a pain to keep track of all those ideas, and worse yet to keep them organized on all those different kinds of paper, it's better than loosing them. That and when you need and idea, you have a bunch to go reading through. It helps.

Number 3: Find one you can do quickly an easily.

With all those ideas flowing and being recorded, you should look for an easy to do idea. Just keep going till you find one. It's important.

Number 4: Do that one, just that one.

See, that one idea is important, because you need something you can actually DO! With such ideas done one at a time, you can build a history, a portfolio, a resume etc... As you go through the ideas you've recorded you can find more. Plus, over time more will become possible and easy. More ideas and increased abilities means you will have more possible ideas you can do.

Number 5: Have a real life.

Look, you can't just be a creative [fill in the blank]. It doesn't work. It's like pouring water out of a jug, not a faucet. There's a limit to what you can do without input. During the time where you're looking for anything you can actually do, you were doing other things. They gave you input.

Read, play, socialize and live life, or you will run out of material.

http://www.ted.com/index.php/talks/elizabeth_gilbert_on_genius.html


Related Links:
http://www.chrisg.com/just-like-breathing/
http://www.dperry.com/archives/news/dp_blog/the_pirate_bay/
http://www.terrystarbucker.com/2009/05/11/it%E2%80%99s-like-breathing/
http://www.gameproducer.net/2009/05/12/game-production-lessons-from-a-half-marathon-trip/
http://www.gameproducer.net/2009/05/09/the-easiest-kind-of-way-to-prioritize-big-things/
http://www.gameproducer.net/2009/05/06/some-things-dead-wake-development-has-taught-me/
http://www.gamasutra.com/features/20051026/gabler_01.shtml

Have fun, spread the word and tell me what you think,
Igen Oukan
Read more!

Creatively Getting Things Done

View Blog Reactions

Efficient and Effective Everyday

In my post Keeping People Involved I made a point about sweating the small things, because that's what people do day-to-day. "When figuring out things to do for a community, think about the daily activities and supplying things to do to keep people involved and having fun." That also goes for teams, groups and individuals. Make the day-to-day a priority in designing how you work, it'll pay off in the long run.


Could I go on for a while about this? Yes, but I won't. The whole idea of the "Getting Things Done" theme is that you learn about ways to make yourself and your group more likely to get things done. It's not about me telling you how to do it. Take a look at these different ways and think about how they could be applied to how you work.

Virtuous Circle:
http://blog.broadbandmechanics.com/2009/05/29/5697/

Open Space:
http://www.openspaceworld.com/brief_history.htm

http://en.wikipedia.org/wiki/Barcamp
http://en.wikipedia.org/wiki/Coworking
http://en.wikipedia.org/wiki/Hot_desking
http://en.wikipedia.org/wiki/Crowdsourcing
http://en.wikipedia.org/wiki/Scrum_(development)
http://en.wikipedia.org/wiki/Extreme_Programming

Sorry about the delay, but the next couple post are already partially done. So, here they come.

Have fun, spread the word and tell me what you think,
Igen Oukan
Read more!

Efficient and Effective Everyday

View Blog Reactions