It can be a bit 'strange' doing a test that quantifies your traits and behaviours, and maybe also seeing in your results that you haven't scored quite like you thought you would. We can totally understand this. That's why we have written this article, to explain a bit more about why, for example, you can trust that our games measure what they measure among other important points surrounding this topic. Hopefully, this article will clear things up a bit for you. I will talk about:
Validity - do the games actually measure what we say they measure?
When designing a new tool to measure a trait or ability that has no pre-existing valid means of measurement, there are some statistical methods that can be used to test the data produced by it. The goal of these tests is to detect patterns that can be indicative of the validity of the tool. When another valid measurement tool already exists, the validity of a new tool can be established by comparing the results of the new tool to those of the pre-existing one. If both instruments measure the same underlying trait, the response patterns of people using them both should be similar. This then confirms the validity of the new tool.
In our validation studies, we collected data from participants who played our games and took part in an assessment of the same underlying trait that our game intends to measure. If you would like to read the validation reports of each of our games, you can click on the articles down below:
What we measure is innate but...
Another point that is important to mention is that the traits which we measure are innate, which means that you instinctively prefer doing things or feeling things in a certain way. These traits have developed as you grew up, influenced by each turning point and environment in your life span. You can still develop these traits later in life as you continue to experience turning points and different environments. However, they will develop at a much slower rate and not drastically. What does this mean in the context of the games? It means that if you do the same test again after a year or so, there will be minimal to no change in your score.
This does not mean that you have not acquired certain techniques to help with something you might not be inherently good at. It just means that the games do not measure your use of these strategies and techniques because this makes the process biased. An assessment doesn't take into account your strategies, they aim to measure what you have in your genes.
Let's take prioritising as an example. We measure if you're good at prioritising through the Ferry game, but you can't use a famous prioritisation technique like Eat your frogs in the morning when playing this game. This means that you may score low in this game, despite having developed strategies to help you prioritise in daily life.
For any hiring managers and recruiters out there, this can be a really interesting topic to talk about during an interview. Asking a candidate about traits and their results gives them an opportunity to reveal any techniques they may use to overcome things they find more difficult, such as prioritising. Our hiring managers will have an interview with candidates they might have (some) doubts about because it can lead to an interesting conversation.
Quantifying something is always a bit strange
We know it can be a bit 'strange' to see something you think you're good at quantified through an assessment. Reflecting on the scores is an opportunity for growth, and our scores should not make people feel like they aren't good enough. But we simply can't be good at everything. That's why every candidate or team member will receive a link to an article with tips and tricks around strategies that you can apply to your day-to-day as well as your professional life.
As well as this, gamified assessments do predictive analysis of future job performance. We would love to be 100% sure, but that isn't statistically possible. This means that the scores tell you whether someone is 'more likely' to be, for example, good at adapting to different situations because they fall within this bucket in the game which measures cognitive flexibility.
Different work environments give you different insights into your strengths and weakness
Different work environments will challenge you in different ways, and it could be that you don't have issues in your current environment because the trait(s) in which you scored lower is not being impacted by the workflow. However, in an environment in which this trait is being challenged, you might have an issue and therefore be more aware that this is an area of weakness for you.
Why are self-assessments inaccurate?
Although we think we know ourselves inside and out, this is not actually the case when it comes to rating our abilities and traits. Like knowledge of one's abilities, knowledge of one's traits is imperfect. (Karpen, 2018).
A wide variety of psychological mechanisms underlie our flawed self-assessments. One of the most documented ones is the above-average effect. People lack the crucial information they need when comparing themselves to others; they also ignore valuable information that they actually possess or could seek out (Dunning et al., 2004). This leads people to believe that they perform much better than their peers.
You might also be surprised to read that there are weak correlations between self-assessment and performance which demonstrates that people misestimate their abilities (Karpen, 2018). Gamified assessments are much more accurate in assessing an individual's traits and abilities. Not only are our games more accurate than self-reports, but they also measure a candidate's General Cognitive Ability (GCA) which has a high correlation with future job performance. All in all, when it comes to accuracy gamified assessments beat self-assessments every time.
Hopefully, this cleared up any question marks you had surrounding not recognising yourself in the scores of the games and shed some light on why the scores can be trusted.
Are you interested in learning more about the science behind our games? Head over here to find out more.
Dunning, D., et al, (2004). Flawed Self-Assessment: Implications for Health, Education and the Workplace. Psychological Science in the Public Interest. 5(3): 69-106. doi: https://doi.org/10.1111/j.1529-1006.2004.00018.x
Karpen, S. C., (2018). The Social Psychology of Biased Self-Assessment. Am J Pharm Educ. 82(5): 62-99. doi: 10.5688/ajpe6299