As organisations have access to ever-increasing volumes of data, they are faced with questions of how best to use it to improve individual performance in the workplace. By creating a field experiment studying elite football players,Henry Eyring (LSE), Patrick J. Ferguson (Harvard Business School), and Sebastian Koppers (WHU - Otto Beisheim School of Management and Laboratory for Innovation Science, Harvard) show how the design and quantity of performance data matter for improving player skills.
Why too much information can be a problem
Firms have generally been successful at using big data to improve business processes, from tracking customer purchases to automating supply-chains. However, Dr Eyring explains, organisations have generally lagged behind in using data science for internal evaluations:
“Very little has been done to take the vast amount of data that could be used to inform decision-makers inside of firms…. Generally what you’ll hear is that a firm has a data warehouse and the data gets stockpiled in there.”
In fields as varied as healthcare, aviation, and finance, for example, it is now possible to track and collect large amounts of information about employee activity. However, while it may be reported to senior management, there are many unutilised opportunities to: 1) sort the data into accessible formats that can be used by firms and their employees to understand their own performance; and 2) provide training and motivation to improve performance. In this scenario, more data and more information don’t necessarily lead to better outcomes.
Most firms routinely produce employee performance evaluations. But if they are not making good use of data to inform them, they are missing an opportunity to improve practices.
On the pitch: field experiments with elite footballers test how information helps performance
To test how different types and volumes of information can help people learn about their own performance and improve, Eyring and colleagues conducted a field experiment using a set of highly motivated people: professional European footballers (soccer players).
“Sport is on the cutting edge of data use because it’s brutally competitive”, says Eyring. Increasingly it measures player performance at a very granular level, because there are clear incentives to do so, and since even minor improvements can make all the difference.
The researchers designed a field experiment to test the causal relationship between the information provided to players and how much they improved during training. Using the training football simulator skills.lab, produced by Anton Paar SportsTec, the researchers manipulated the performance information provided to 117 professional and semi-professional footballers in Germany and Austria, who play for 11 different teams.
The drill tested the players’ ball-passing ability by speed and accuracy. The simulator fires a ball at the player from either left or right, and the player must pass it to a moving target – which simulates passing to a teammate. This is repeated a dozen times and the players have 11 seconds to complete each pass.
Players were randomly given different sets of information about their performance. This was either an absolute score, including for speed and percentage of passes on target, or relative information, giving them a ranking of how they compared to their peers. A third set of players were given both absolute and relative scores. The researchers also further varied the level of detail the players were given, with either an average score for all passes or a more detailed report disaggregating performance for type of pass (with a left or right turn). So there were six variations in total, assigned randomly to the players. After they were given the data, the players repeated the drill and their improvement was measured.
Comparison and more selective information motivate the biggest improvements
The results, published in the article, “Less Information, More Comparison, and Better Performance: Evidence from a Field Experiment”, in the Journal of Accounting Research, were clear: for all of measures of performance, there is a statistically significant performance benefit of providing relative rather than absolute information. The effects are substantial, and players given relative scores also performed better than those given both relative and absolute scores. However, it is not simply about avoiding information overload. As Eyring explains:
“When we broke it down by detail, that didn’t lead to any detriment of performance. The only reduction of performance came when we went from going from the relative-only information to the relative-plus-absolute. That means it’s not just that people are becoming overwhelmed with the quantity of information, or that it’s information overload, it’s actually that we’re changing their focus when we add a certain type of information. If you focus people on ranks, then their minds and their energy get focused on comparison and on social competitiveness.”
In follow-up surveys, players who showed higher levels of social comparison were associated with better performance. They also found the level of professional competitiveness each player faces – measured as the number of teammates who are competing for the same starting spot in the team as the player – affects the results. This suggests that career concerns are an important factor in explaining the results.
By using a randomised field experiment, which controlled for other variables, including the players’ initial performance, league level, and demographic factors, Eyring and colleagues could show the relationship between type of information given and performance improvements was causal.
“What we do show, and what our study is the first to establish, is that there is a possibility that adding some information will harm performance and you may want to focus people’s attention on the measures that are the most motivating,” explains Eyring.
This study suggests managers should “decide what they want to convey when they give people feedback, and maybe they should just pick a few measures. Especially if they feel like they’re going to drive performance, you can make those measures more salient and more relevant in people’s decision-making process if you don’t show them other information.”
The study speaks most directly to settings with highly skilled workers, but also has implications for the broader use of data in management accounting, and on the
marginal costs and benefits of providing information. More data is not necessarily better data, particularly if it distracts from helping people focus on what motivates them most.