The Entrepreneurship Serious Game Rubric – A Quantified Game Review System:

The next step towards a comprehensive review of good practices in available serious games for Entrepreneurship Education included the creation and implementation of a quantified game review system.

In order to achieve fair and valid quantifications, certain variables were selected that reflected best practices in serious games. The basis for the selection of these variables became the ISGEE Literature Review

Rubric Logic: Variables

The logic for the selection of variables rubric can be described as:

7 variables were chosen based on their relevance in the ISGEE literature review. These variables were then further divided between;

1. Evaluative variables: variables that can be scored
2. Categorical variables: characterization of variable in a “yes/no” format.

The variables are: a. Task sequencing (evaluative): Describing the relevance of the order in which the game presents its tasks. Good learning outcomes in serious games (and in other situations) have shown to depend on a relevant progression of tasks that build on each other, leading to better learning outcomes. A good task sequence can convey a lot of information in a short number of steps.

b. Fidelity of tasks (evaluative): Often time, tasks in classrooms are perceived as not relevant to real-life situations by students. Therefore, the fidelity (i.e. the degree of exactness) of tasks was also chosen as a relevant criteria for assessing serious games.

c. Quality of instructions (evaluative): a good learning experience has to be guided by good learning objectives. To achieve these objectives, the player of the serious game should be provided with clear and complete instructions, where the player knows the direct outcome of the tasks and a roadmap on how to get there.

d. Instructor involvement (feedback)(categorical): Naturally, an instructor’s involvement in any educational activity contributes to the effective delivery of educational content. In serious games, the involvement of instructors can help the student to stay focused and increase their productivity due to the available support and feedback from the instructor.

e. Entrepreneurial Learning outcomes (Outlined & Achieved): As with classroom environments, serious games also need learning outcomes defined by the creators of the game. These learning outcomes should not only be defined but also be achieved through the course of the game.

f. User choice (evaluative): Naturally, letting students/users have choices in the game allows for outcomes based on decision-making. User choices will create a distinction of the skilled users (depending on the task and the type of decision making).

g. Cooperation (team learning) (categorical): Research shows collaborative learning provides better learning effectiveness compared to individual learning . Serious games provide an easy and effective environment for collaborative exercises. However, the implementation can differ depending on the priorities of the serious game.

Rubric Logic: Ranking

It is a valid assumption that no two serious games are the same. Each serious games will perform well in some of the variables listed above, but not all of them. In order to evaluate which variable does a serious game perform better in, the evaluative variables, as outlined above, were further broken down into a 4-score ranking system. This ranking system reflects the different levels a serious game can be on. For example, a game can perform well in fidelity of tasks, but not so well in user choice. The ranking system allows for such a distinction.

Furthermore, the categorical variables do not have this 4-score ranking since these variables only indicate if a game fulfills the category in a simple “yes/no” system. This 4-score ranking system, along with the evaluative variables, is shown below in Table 2.

Table 2. All evaluative and categorical variables, broken down into their respective 4-score ranking system or “yes/no”. Red (1) represents lowest, while Green (4) represents the highest score.

Lastly, these rankings for serious games are visualized in the form of radar charts. Figure 1 below represents a sample radar chart for the rankings of a single serious game.

Figure 1: Sample Radar Chart for the rankings of a single serious game.

Rubric Logic: Cross-Variable Weights

Although the 4-score ranking system helps to rank serious games for individual evaluative variables, it fails to capture the cross-variable relevance of the individual variables in comparison to the others. As an added dimension, individual evaluative variables were assigned with “weights”. The weights are in the shape of points. An evaluative variable can either have a weight of 10 point or of 20 points. These weights were discussed and decided upon within the context of project ISGEE.

The variables were assigned their respective weights as shown below in Table 3. Table 3 also exemplifies the visualization of the weight points.

Table 3. All evaluative variables, broken down into their weights.

Rubric Logic: Total Points

The weights and the 4-score ranking system for the evaluative variables are further represented as combined points (total points), allowing to assign a holistic score to individual serious games, and to compare multiple games based on their total points.