Measuring Impact

As I sit down to answer this week’s question- “How will you know if the users or participants in your project will be impacted in the way you hope?” I am rather depressed about the preliminary results of a SurveyMonkey survey that I just reviewed. This survey assessed student satisfaction with a Second Life patient interviewing experience that we just finished beta testing. Granted, the sample size is very small (only 2/5 students have responded so far) but these students did not appear to enjoy or appreciate the experience. This is an important survey as it will likely determine whether or not we proceed with Second Life at our institution at this time. To answer this question about impact you will first need to determine what type of impact you want to measure. Do you want to look at satisfaction as we did in our Second Life survey? Or, is it more appropriate to look at access or performance? This will help you to decide what tools to use to measure these parameters.

A survey delivered through a vehicle such as SurveyMonkey is just one of many tools that can be used to determine the impact of your project. It was fascinating to read the chapter by Loh in Games and Simulations in Online Learning that talked about designing online games assessment as “information trails”. I had just had a discussion with the director of our continuing education program regarding a push to track the time that doctors spent completing online programs. The accreditation agencies are concerned that doctors are not engaging with the continuing education materials and want to monitor time spent on task. Continuing education departments may be required to implement timing mechanisms in their online courses. Anyone who has shopped online has likely been the subject of a tracking technology. Have you noticed that Amazon.com has an amazing knowledge of your likes and dislikes? I also use tracking technology in my Moodle course in the form of participant logs to review the content accessed by students. We recently completed a study comparing access of online content as documented in the Moodle logs to performance on a related test question and found that access had a significant dependent association with scores. A bit of a no-brainer perhaps, but we found that students need to access the online content we provided in order for learning to occur.

Measuring performance is important if games are to be integrated into education due to the focus on measurable outcomes. This can make it challenging to build a game if one does not have access to a skilled team of designers. Commercial products are available that can integrate scores into a learning management system. For example, SoftChalk allows the novice to integrate activities such as crossword puzzles and drag/drop exercises into the LMS and SCORM technology allows for tracking of scores. A number of tracking technologies can be used to track activity in games and simulations involving avatars. Very useful reports can be generated but often this involves significant knowledge of coding and algorithms.

Gibson, D., Aldrich, C., and Prensky, M. (2007). Games and simulations in online learning: research and development frameworks. Hershey, PA: Information Science Publishing.

Beginning to Put the Game Together...

My current interest in gaming revolves around supporting chiropractic and health care education. The game/simulation I hope to develop will be based on the process of obtaining information from a history and exam to come to a correct diagnosis for a patient presentation. The proposed game described below supports my personal learning theory as it is based in cognitivism. Students need an underlying basis of facts in order to succeed. If they fail, they will be sent back to T1 in chiropractic school (i.e. a tutorial)- wonderful motivation! Far transfer will be supported as the elements will be derived from real-life situations and scenarios as much as possible using the typical forms used in an office and video clips of real a real patient. Hopefully the gaming elements will serve as motivating and reinforcing factors stimulating the students to want to complete the exercise. I am motivated to make an extremely important part of a student learning experience as fun and interesting as possible.

My primary concern still revolves around actually creating a game that I can use in my teaching. I do not currently have the resources or team players to create the ideal type of game that I visualize. Since I don’t want to simply present a game idea for my final project I will need to scale down my vision. I will try to create a simulation with embedded gaming elements and concepts rather than a full-fledged game. The student will start out by clicking to enter a chiropractic clinic. They would then a patient file folder to view information regarding their patient. Students will eventually have to progress through a series of cases to successfully complete the game. These cases will support the concepts emphasized in the Methods 3 technique course. The consequences of incorrectly diagnosing and treating a patient might range from the patient leaving your office, screaming in pain, demanding their money back, to the most serious- death or disability related to failure to identify a stroke in progress. Success results in an increase in patient files and money in the bank. I could eventually expand the game to include insurance coding elements and could have some fun with this- video clips of insurance adjustors, money entering and leaving the bank account, etc. Some of these elements may be theoretical at this point due to my limited skill set and lack of a game design team. However, the underlying simulation will still support the learning in my classroom. A simple game can be just as engaging as a multimedia, sound-blasting game with realistic bloodshed so that is what I am going to focus on. I think that using a branching concept might work. I could perhaps do this with the lesson function in Moodle or with a series of linked web pages created in Dreamweaver. I also have access to an interactive scenario builder software that could possibly be integrated into this process. The gaming elements would need to be embedded into these pages which could be challenging or impossible at this time. I will still describe the vision for these elements. Perhaps if the student answers a question correctly the branch takes them to a page with a flash animation adding money to their bank account and directing them forward in the process. If they answer incorrectly, they are sent back to chiropractic school via a tutorial and must answer the questions in this tutorial before they can see the patient again.

Evolution of Initial Thoughts on Game-Based Learning

As I proceed through the course, read, and review the work of my classmates I am getting a much deeper sense of how game-based learning can related to education. I’m still not much of a gamer myself but I have determined that the reason for this is primarily lack of time. It’s not that I don’t like to game, I simply don’t have the time to devote to become good at a game and it is much less enjoyable to play a game that you are not good at. This is a concept that I really need to address in the games that I create because my students are likely to be time-strapped adult learners as well. Previously I stated that I would like to develop the skills needed to integrate games into my current online teaching. I had suspected and now it has been confirmed that creating games is not an easy process and often requires a skilled team including programmers. Since I don’t have this team, I will need to focus on keeping any games quite simple at least in terms of technology. Realistically I will probably be adding gaming elements to my courses rather than developing full-fledged games. That said, I would really like to somehow develop a game similar Life or Death to use in chiropractic education.