ADAPTING TO DISRUPTION: BEWARE OF THE SHINY STUFF
Written by: Nicola Taylor, Director: Research at JvR Psyhometrics on 13 July 2018
The media is constantly telling us to be on the lookout for disruption. We are told that Industry 4.0 is here, machine learning and artificial intelligence are meant to be part of everybody’s skill set, and that innovations in technology and the digital field are set to completely change the world as we know it. Before succumbing to anxiety about whether you are ready to embrace this unfathomable phenomenon that will supposedly completely change the way we do things, let’s consider a few facts.
- In Deloitte’s 2018 Global Human Capital Trends survey, over 70% of respondents felt that implementing AI and robotics in their business was important, but only a third felt that they were ready to do so.
- While internet use is rapidly growing in Africa (http://www.africanews.com/2018/02/06/digital-in-2018-africa-s-internet-users-increase-by-20-percent//), the continent lags behind most other regions. Central African countries only report access to the internet for around 12% of the population, and southern African countries report an internet penetration rate of 53%.
This means that while there is a great desire to move to more digitally savvy mechanisms, the current uptake is low, and it will be a while before digitisation properly permeates other African countries. However, it does give us time to prepare ourselves, to try new things, and to get ready to catch the disruption wave at the right time.
Where are we now?
Technology already impacts the way we use assessments. There are already minor shifts away from the traditional Likert-type scale to game-based assessment, and in-person assessment centres to virtual reality simulations and interview processes. The future may introduce the use of MRIs and brain scans, as well as other biometric data collection.
For the most part, administration of assessments in the workplace has shifted from paper and pencil to online or computer based administration. There are other sources of data available that are not self-report based, and could include facial scans, biometrics, sociometric badges, natural language processing devices (like Alexa) or virtual reality simulations.
The way in which we do feedback is also changing. In the past, it was always conducted in a face-to-face setting, or possibly telephonically. Now there are video conferencing options, feedback and coaching via Whatsapp, Skype, Messenger, and SMS. The rise of telepsychology brings its own challenges (http://www.apa.org/practice/guidelines/telepsychology.aspx).
Engage your brain
One of the key skills we are meant to develop as psychology professionals is to be able to critically evaluate the evidence before us in order to arrive at a sound decision. We cannot just accept someone’s word that something does what it is supposed to without any evidence. Disruptive technologies can look very advanced but may not be based on sound theory or have any validity evidence whatsoever.
There are already fantastic innovations making their way into the assessment market. With regular advances in technology, there are so many new things out there that look good, and can completely change the way we do things in assessment. But, as responsible professionals, we need to ensure that we have the evidence to support putting a new technology into practice. In other words, we need to be aware of the “shiny stuff” – things that look good on the outside, but either don’t have substance or distract us from the real work to be done.
We have to rethink our approach to assessment. We will have to adapt our current guidelines and the way we evaluate tests. With the involvement of non-psychologists in test development, we may have to change our legislation in South Africa. We will have to look at ethics and best practice.
Mind your ethics
There are already companies who are disrupting the realm of psychological assessment, and not necessarily in a good way! It was recently announced that Facebook found over 200 apps that were blatantly misusing people’s private information in order to create “predictive algorithms” that could be used to sell or market to Facebook users, or to try and influence behaviour patterns. Many of these apps posed as “fun” personality tests (e.g., “Which Game of Thrones character are you?”) but the data collected was used for purposes other than stated in the original context. In the rush to be able to gain access to Big Data, these developers completely eschewed any ethical practices in the measurement of psychological constructs. There was no consideration given to best practice, only to profit.
Luckily (or unluckily for some), this means that there will be more of a focus on how psychological data is used for individuals, and may even result in legislation in other countries similar to what is found in South Africa. While these developments may somewhat hamper innovation, they do put the spotlight on the need to be ethical in the way innovative assessments are done and how people’s information is used.
Where to from here?
As a profession, and professionals, we will also have to be flexible, agile, and open to new ideas. Ignoring advances in technology won’t make them go away. Now we don’t have to simply accept every new innovation as “the next best thing”. We cannot remain closed to developments in the field, but we do have the responsibility to ensure that the research behind the innovations is solid.
There are a number of academic journals dedicated to the disruptions happening at the moment. These are a good source for checking the veracity of claims made by developers. We have to make sure that any new process or technology that we bring in to our practices has sufficient evidence to demonstrate that it is as good as or superior to existing practices.
When choosing any new assessment, you’ll have to make a number of considerations. These don’t really change based on the nature of the test. Reliability, validity, the lack of bias and being able to apply assessments fairly are still vitally important factors to consider when using assessments in South Africa, no matter how they are delivered.
Here are some do’s and don’ts to consider when evaluating new assessment technologies:
- Look at the research behind the assessment and make sure that there is a solid theory underlying the tool.
- Check that the mechanism of administration matches the goal of the assessment. Don’t overcomplicate things unnecessarily.
- Make sure that you keep the end user in mind. The new format may be terrifying for people with limited exposure to technology.
- Ensure that the assessment is relevant to the context.
- Don’t use technology or games just for the sake of it. Not all things can be “gamified” or should be.
- Apply your mind to these decisions and ensure that the psychometrics are in place. If you are not sure, ask an expert. Don’t necessarily rely only on the supplier.
A recent study showed that SA psychologists accepted a number of psychological myths as fact (http://journals.sagepub.com/doi/full/10.1177/0081246315581345). While some of the “myths” listed could brook argument in terms of how they are phrased, there are still many that have no evidence to support them that were endorsed by psychologists as fact. This is frightening when we are supposed to anchor our decisions and practices in science and empirical evidence. Be selective about what you accept as the truth, and interrogate the information you are presented with before making decisions.
These are exciting times, and they bring fantastic challenges and opportunities. Let’s be adventurous and try new things, but always keep the wellbeing of the individual you are assessing in mind.
Please comment on this blog at firstname.lastname@example.org or email@example.com