Dunning-Kruger Effect in Software Development
In 1999 a paper by Justin Kruger and David Dunning of Cornell University appeared in the Journal of Personality and Social Psychology, this piece summarised research that they had been carried out into the difference between peoples perceived and actual ability in a subject.
Here is a particularly amusing account of stupidity that is mentioned within the paper:
In 1995, McArthur Wheeler walked into two Pittsburgh banks and robbed them in broad daylight, with no visible attempt at disguise. He was arrested later that night, less than an hour after videotapes of him taken from surveillance cameras were broadcast on the 11 o’clock news. When police later showed him the surveillance tapes, Mr. Wheeler stared in incredulity. “But I wore the juice,” he mumbled. Apparently, Mr. Wheeler was under the impression that rubbing one’s face with lemon juice rendered it invisible to videotape cameras
The third study of the paper focused upon recording perceived and actual test scores of students and it resulted in their famous graph.
This graph suggests that people who have less knowledge of a subject actual think they know more than some of those more informed. Quite simply put, they are too unskilled to be aware of it, and this is what is known as the Dunning-Kruger Effect.
So, how does this fit in with Software Development? I’ve seen the Dunning-Kruger Effect play out in following three ways:
1. Inexperienced Developers
We’ve all been there, looking back I feel sorry for the developers who had to work with me in my first few years in the industry!
This only becomes a problem for the developer in question if they decide that they know enough and stop learning. By learning I don’t mean the latest Javascript framework, I mean perfecting their art and looking to improve by learning from their mistakes. Such developers are often labeled 9 to 5 developers, but it’s not about working hours, it’s that they will do the bare minimum to continue getting paid.
It is also important that inexperienced developers should get to work day to day with more experienced colleagues, ideally through pairing. This will help rapidly improve their knowledge and appreciate what they don’t know.
2. Impostor syndrome
More experienced developers can suffer from Impostor Syndrome which gives them the impression that they know less than they do. Mixed with confident vocal inexperienced developers this may result in poor decisions being made.
Impostor Syndrome can also cause people to overwork, so that they avoid being detected as an impostor. This can lead to burn-out which is a serious problem in the IT industry.
3. Unknowledgeable Non-Technical Team Members
In Ben Goldacre’s excellent book Bad Science he claims that the media’s flawed coverage of the MMR vaccine scare was a result of journalists falling prey to the Dunning-Kruger Effect. They did not understand anything about medical research and statistics to recognise that the study that suggested the link between autism and the vaccine was laughably weak.
There is also a similar problem with non-technical team members, which can be particularly troublesome if they have client contact. In one case I have seen a Project Manager was discussing everything from divs to merging strategies with the client. They then took the results of the discussion and presented it to the development team as tasks. Unsurprisingly poor decisions had been made and the solution failed to meet the client goal.
As developers we can contribute to this by engaging in pseudo-technical conversations with non-technical team members. If you keep using branches and html elements in conversation people will pick up this terminology and use it. This will start to empower them to make ill-informed technical decisions which will lead to a culture of technical tasks rather than problem solving.
Dunning and Kuger’s full paper can be read here: