The Valuable Dev

The Expert Blind Spot In Software Development

learning involve the working memory

Once upon a time, in the Fabulous Land of Web Shops Where Rivers Are Made of Honey, there was me, fresh and young PHP developer with little experience, and my colleague and team leader, who was in the industry for 10 years.

It was the evening, at the office. I was pretty tired, but we had to develop a backend for an important client quickly, as always. I was in charge to develop a part of it.

My team leader was full of passion, agitating a weird diagram full of boxes and arrows:

“Here’s my idea. We should first create two interfaces, put two decorators here and maybe three factories above; we need the system to be flexible as much as we can, the client expect to keep his backend for 20 years!".

My knowledge in design pattern was pretty poor at that time; heck, I even had difficulty to understand the difference between inheritance and composition. I was looking at his schema like Champollion might have looked at the Rosetta Stone for the first time, bringing more questions than answer.

Speaking of questions, they were first flowing from my mouth: what’s a decorator? Why do we create interfaces? Why this factory, there? The explanations brought even more complex concepts I couldn’t really grasp either. The stress of the impostor syndrome was beginning to kick in heavily. I was a fraud, a newbie who wanted to play in the gods’ playground.

My brain was boiling. Sweat was pearling on my forehead. My confusion was growing proportionally to the amount of explanation given. The cleaning lady began to turn on her vacuum cleaner. I gave up, said yes to everything and went home.

As you might expect, the project didn’t go well. My team leader reproached me not to ask enough question for me to understand. Apparently, I wasn’t helping them enough to teach me correctly.

Why was that? I didn’t understand what was happening, but I swear to the Twelve Gods that I would remember this experience not to reproduce it. I wanted to be a better mentor when it would be my turn to be the Senior Ninja Developer of the Wizardry Crown explaining magic tricks to my apprentices.

At the same time, I knew that my team leader was not the only one responsible for my confusion. I was, too, and maybe even more than him. This lead me to another quest: learning how to learn.

Years passed. Then, the light appeared. A miracle. I stumbled upon the theory of the expert blind spot, finally explaining partly what happened on this cursed project.

Why do I bother you with my life, you might wonder? As developers, we inevitably work with other developers who don’t know everything. Therefore, we need to share our knowledge.

In our developer lifetime, we will be alternatively student and teacher.

Teaching is one of the best way of testing yourself that you understood what you’ve learned; teaching and learning are tightly linked. Doing it well will increase your understanding of whatever subject and, therefore, your level of expertise.

While we teach, we learn.

Seneca

However, to teach well, we need to understand and avoid the expert blind spot. This might begin to explain why, in my experience, good developers are not necessarily good teacher.

This article will specifically speak about teaching from a person to another. I believe that things are a bit different when you try to learn by yourself with a book, for example.

Here’s the menu for today:

  • First, I’ll explain what’s the expert blind spot.
  • Second, we will speak about the cognitive load theory in general, how understanding and learning work, and why the expert blind spot is a consequence of our learning process.
  • Finally, I’ll speak about principles as well as actionable solutions to avoid the filthy expert blind spot.

Now, it’s time to embark together in this fabulous journey!

What’s the Expert Blind Spot?

Definition

To answer what’s the expert blind spot, we shall first define what’s an expert is, to be sure we agree on the semantic of the key terms. From the Oxford Dictionary :

A person with special knowledge, skill, or training in something.

Beyond the obvious, I like to define expert as well as somebody who knows what he doesn’t know, as mentioned in the Dunning-Kruger effect .

According to this theory, a beginner has often a lot of confidence about his own knowledge and skills, because he’s not aware about what he doesn’t know. Thus, he thinks he knows a lot. This is called illusion of competence.

More you become skilled and knowledgeable, more you’ll know what you need to focus on to increase your expertise.

We live on an island surrounded by a sea of ignorance. As our island of knowledge grows, so does the shore of our ignorance.

John Archibald Wheeler

Even if an expert is more aware about what he doesn’t know, he’s not necessarily aware about what he knows in details. This sound pretty weird, but it’s exactly what the expert blind spot is.

Even if he forgot the details of what he knows, our expert can still apply his knowledge to perform tasks, but he will only use a higher level of abstractions of it.

The Problem

If an expert can still use the knowledge he’s not entirely aware of anymore, why is it a problem, you might ask, rightfully?

A developer, explaining to a beginner his knowledge, will brush away everything he’s not consciously aware of anymore. The learner, even if he needs this hidden knowledge to understand and learn the whole concept, won’t be able to access it. Thus, he won’t understand.

As a result, the expert won’t understand why the beginner doesn’t understand. After all, for the expert, the concept is obvious.

The learner, on the other side, won’t be able to ask the good questions, since he’s not aware about what he doesn’t know. This brings us back to the lengthy story I was narrating in the introduction.

Even worst: the beginner might misunderstand what the expert try to teach without even knowing it. Absence of knowledge is better than misconceptions, which are the basis of ignorance .

Why On Earth There Is Such a Thing as the Expert Blind Spots?

Researches on cognitive learning show us why there’s such a thing as the expert blind spot. First, we need to see how cognitive learning works.

Types of Memories

According to the cognitive load theory , we have two types of memory: the working memory and the long term memory.

The Working Memory

The working memory is very limited in space (we can only process 4 elements of information at the same time) and time (around 20s). These numbers can change from an individual to another.

You can use this memory to:

  • Learn a new resource.
  • Perform a task.

As you can imagine, problems pop up when you try to learn too much at the same time. Your working memory can’t process everything simultaneously, because of its limited space.

Similarly, when you perform a task, you need to recall what you have in your long term memory (what you learnt) in the working memory to do that task, with the same spatial constraints. That’s why simplicity is so important in software development.

Learning in the working memory
Attempt to learn the Linux pipe

The Long Term Memory

When you learn a new resource using effective learning methods, what’s in your working memory will go in your long term memory. To do so, you need to understand the knowledge you’re trying to process.

To understand, you need to have every piece of information required in your working memory. From there, you’ll create schemata (plural of schema, if you wonder) in your long term memory to organize this knowledge.

Later, you can use your new schemata to perform a task.

Understand in the working memory and long term memory
Understanding the Linux pipe thanks to new pieces of information and prior knowledge of the command line.

Pieces of information and Chunking

We didn’t define what a piece of information is. Each of them will occupy one spot in your working memory. The content itself will depend on what is learned and who’s learning it.

What’s in your long term memory are layers of schemata abstracting one another. When performing a task, you need to reduce the number of schemata you bring back to your working memory not to overload it.

Let’s take an example. Carl, your colleague developer, is running a shiny Ubuntu in his powerful laptop. He wants to know how many files, with names beginning with a “z”, are in a folder.

In my example, you are the famous command line expert of the whole company. Naturally, because of your extreme empathy and your out-of-this-world generosity, you come to help to solve his problem.

“Hey Carl!", you said, in the most friendly tone a human can produce. “It’s super easy! All you have to do is a good old ls | grep "^z" | wc -l. You see, you first run ls, you pipe with grep, wc and voilà! Your dreams come true!”

However, Carl doesn’t have your knowledge. Raised in the comfortable and boring GUI world, he doesn’t understand anything of your magical skills. He knows ls, but he doesn’t understand what a pipe is, and he has an artificial understanding of regular expressions. What about grep and wc? He has no clue whatsoever.

Do you think he would understand your explanation? I’m doubtful too. When you type your command line, you don’t have to remind yourself (bring from your long term memory to your limited working memory) what a pipe is and how it works. The whole concept is abstracted under the word pipe for you. You don’t need to even think about it. The concept is automatically applied to your task at hand.

Because of your expertise in the command line, you save space in your working memory. It reduces the mental energy you need to perform that task, too.

For Carl, things are different. To understand this command, he has to bring the concept of pipe into his working memory, what’s grep, what’s wc, how regular expressions work, and link all of that.

It’s a lot. Too much, even for a command which seems so trivial to you. Unfortunately, you have fallen into the expert blind spot trap. You assume that everybody has the knowledge (or schemata) you have in your long term memory, without even realizing it.

I know what you might think. “Carl has to ask questions! What’s a pipe? What’s wc? What’s grep? This is what Carl should say!".

You’re right, it can help. However, nobody likes to look stupid, especially when you are an insecure beginner. He might ask one or two questions, but for the pipes he might as well try to make some assumptions on its working; from there, misconceptions can quickly spread in his poor brain.

Another detail: you said to Carl that the task at hand was “super easy”. Super easy for you, but not for Carl. He might not want to look stupid by asking questions about something super easy .

I’m coming back to my little story in the article’s introduction: even if I was asking questions, the answers brought so many more foreign concepts that my working memory was totally overloaded. It was impossible to understand in that case.

Finally, as a beginner, Carl might not know that he doesn’t know. In that case, all the useful questions won’t even pop up in his brain.

What should we do to help the poor Carl?

Possible expert blind spot while expert explain the command line
In red: possible expert blind spots while an expert explains the command line.

How to Fight the Expert Blind Spot

What’s the first step we should all take to solve a problem? Clearly stating what the problem is. This is what we just did.

A problem well stated is a problem half solved.

Charles Kettering

Now, we need to keep the concept of the expert blind spot in mind when we mentor somebody.

Simplifying the Learning Material

The cognitive load theory advise to lower the number of pieces of information in the teaching material. This approach is good when you teach a beginner: since he doesn’t have so much prior knowledge, he needs first to create the foundations to build upon, in his long term memory.

This can be tricky to do. Simplification can bring you on the path of erroneous information, which again bring you to the terrible world of misconceptions.

To bring back my command line example from above, here’s how I would have explained it:

  1. I would have asked questions, to be sure that Carl has at least a basic understanding of the shell, during informal discussions if possible. More on that below.
  2. I would have explained the concept of pipe, and showed him some simple examples.
  3. I would have given him the solution.
  4. I would have asked him to do other tasks with pipes, for him to bring the concept in his long term memory.

The goal here is not for Carl to understand the whole every single part of the command line, but to teach the concept of the Linux pipe. This is a really important piece knowledge a developer should have. CLIs like grep or wc are important too, but it can wait. One step at a time. Carl doesn’t have to learn everything at once.

I would encourage him as well to use the command line as much as he can, for him to discover its power and to practice his new knowledge, if he wants. Don’t be too pushy, let’s the idea maturing in his mind. Maybe in some years he’ll really understand the power of it.

Let’s take another example. Some years ago, with a beloved colleague, we were managing a remote team. We understood very quickly that they didn’t have any experience with unit tests and TDD. We decided to teach them the basics.

We took this approach:

  1. We picked a task we had to do on an application
  2. We began to write functions and their unit tests, explaining our approach at the same time to the remote team.
  3. As the demonstration was progressing, we were asking questions: how would you test this method? What do we want to test, here? Is it a good idea to test this controller?
  4. We then reviewed thoroughly their unit tests during the next weeks. At the beginning, we didn’t try to correct everything, keeping in mind that multiple approaches (and not only ours) was possible. Being too picky, at least at the beginning, would have discouraged them, confused them, and overloaded their working memory.

Obviously, in order to learn, practice is important. It should be:

  • Not too easy, or the learner won’t learn anything.
  • Not too hard, or the learner will be quickly discouraged.

Accordingly, the learner needs to be rewarded with small successes and breakthrough as often as possible.

This approach is good for beginners. It might be better for experts (who want to increase their knowledge in something they already know well) to let them practice before giving any explanation. Feedback and explanations should follow after enough practice.

Humility and Respect as a Teacher

Let’s take 5 minutes to look at Carl, your colleague developer. Carl is a nice guy, but he’s very sure of himself. Even if his knowledge in programming could be improved significantly, he thinks he knows everything.

Again the Dunning Kruger effect kicking in.

In my experience, this is coupled with the beginner’s insecurity, who compare himself to experts. Add a pinch (or a tone) of impostor syndrome, and you’ll see Carl trying to be right in every circumstance, throwing his “knowledge” at your face to heal his ego’s problems.

Don’t get me wrong: experts can have these problems too. However, I think it’s important to show to beginners that “experts” don’t know everything. We are mere mortals, we do mistakes, and we should not be ashamed to recognize it. This includes the expert blind spot.

In fact, learning is accepting as well what we don’t know, to improve our weaknesses. Teaching can reveal them, if we keep in mind that some of our knowledge, as “experts”, is way too abstracted to be understood by beginners.

This will show to beginners that:

  1. Nobody knows everything, so they should not be ashamed of their blind spots, misconceptions, and mistakes.
  2. It’s necessary to accept what we don’t know, to know what to learn next.

Consequently, we should respect the colleagues we work with, even if they don’t know things we judge they should know. How many times I heard developers claiming, strongly and loudly to somebody asking a simple question: “but… you don’t know that??!". This “but-it’s-simple” attitude doesn’t bring anything good, for anybody.

Curiosity demands that we ask questions, that we try to put things together and try to understand this multitude of aspects as perhaps resulting from the action of a relatively small number of elemental things and forces acting in an infinite variety of combinations.

Richard Feynman

We have to accept our weaknesses as much as others’ weaknesses.

Keep an Interest for the Other

The best way to be aware of somebody’s level of knowledge in some precise areas is simply to speak with him. In my experience, informal, relaxed conversations, around a cup of coffee, a tea or whatever you like, is the best way to do so.

Let’s imagine the following scene: you have no clue about GraphQL and a GraphQL expert try to teach the concept to you. He’ll go through a demonstration first and try to assess your level of knowledge.

We can imagine our teacher saying to you:

  • “So, what do we do now? Yes! Indeed, very good!”
  • “And that, where do you put it?”
  • “And this? No, not at all. Think about it again.”
  • “This is not good either. You should do that and this. Let’s try again.”

In practice, many might feel these questions a tad arrogant.

Real life example time: I had a colleague teaching exactly this way. Unfortunately, he was giving the impression to the others (including the learner) that he was patronizing. This wasn’t the intent of the developer-teacher.

Instead, if the expert simply speak about GraphQL during a break, giving his experience and engaging a conversation, you might as well share yours. Doing so, he will break this strong relationship teacher / student.

If you have some experience with GraphQL, your teacher might goes into the details (“I love you can do this or that, it’s pretty powerful, isn’t it?"). He should then pay attention to your answers and your body language.

If you’re looking to the ground, not really knowing what to answer, or simply acquiescing, he might have found the limits of your knowledge. He should keep that in mind when it will be time for him to teach you. He’ll know that, on these precise points, he’ll need to avoid the expert blind spots, by giving enough details.

This method has multiple advantages:

  1. The teacher will develop a friendly relationship with the learner. It’s important to create a good relationship to learn from each other.
  2. The learner will feel considered, and not simply as somebody who doesn’t know.
  3. Others won’t judge the teacher as the guy-who-think-he-knows-everything. Difficult to teach when this image begins to stick to you.

It’s very important to assess the knowledge of your colleagues, without judging or blaming. Every developer has different learning backgrounds. Consequently, they have all different level of knowledge for different subjects.

In short, you need to adapt to your audience, to see where you have to be careful of the expert blind spot.

Retaining the Steps of your Knowledge Acquisition

Now that we know what our learners don’t know, how can we know what piece of information we might omit for the good understanding of our beloved colleagues?

After all, difficult to explain something you abstracted away in your long term memory. You need to retrieve it yourself first.

There are different ways to capture each time of your learning process in external resources, in a layer fashion. Then, when you fall in to the expert blind spot trap while teaching, you can come back to these resources to see what you might have forgotten to explain.

What are these resources? How to create them? What a suspense!

Mapping your Schemata

I love creating mindmaps of books or articles I read . You can capture in them what your learned, in layers. You can hide some of these layers to see the root of the knowledge, and expend the branches to see the different level of abstractions.

In short, mindmaps allows you to apply, in a very visual way, the principle of chunking we saw at the beginning of this article. It can, more or less, represent the schemata in your long term memory.

It means that you need to mindmap sources of knowledge which are explaining well and thoroughly the concepts it teaches. You need to know as well what prior knowledge you need to understand the material.

Then, when your learner look at you as you were an alien speaking Goa’uld, you can come back to your mindmaps to see what piece of information he doesn’t know and you’ve forgotten to explain.

Even if the answer is not in the mindmap but in your prior knowledge, only going through it can trigger what you forgot to explain.

Moreover, mindmaps are a fantastic way to store knowledge in your long term memory by actively going through them on a regular basis. They will help you to go through the Expert Path™.

Writing a Development Journal

I keep a journal where I write the bugs I encounter and how I solved them, the traps and inconsistencies programming languages throw to our wonderful faces, and other joyful details I want to log.

I use jrnl for that. You can tag your entries to be able to find them easily.

When a beginner encounter difficulties to understand my explanations how to solve a difficult bug or tasks, I can search in my journal using the tags to understand what was my thinking process at the time.

Then, I can try to remember the prior knowledge required to explain it to my learner, if he doesn’t know it already. Similarly to the mindmap solution, it can be used as a trigger to remember how I was thinking at the time, what difficulties I had, and how I solved them.

Writing a Blog

A big benefit of writting a blog is the appropriation (or reinforcement) of knowledge. Again, going through my blog can help me to remember what was the state of my knowledge at the time I wrote a precise article, and what I read to acquire this knowledge.

Then, I can have a mental model about what to explain to the learner, and have all the details I need to avoid the expert blind spot.

Thriving to be a Good Mentor

Being a good mentor helps you learning as well!

As I was saying in the introduction, I mainly speak, in this article, about teaching and learning from person to person.

When you learn from an external resource (video, article, book), things are a bit different. You can overload your working memory as much, but if the resource explain thoroughly every step you need to know to really understand the concepts (thus avoiding the expert blind spot), you can as well take your time and come back to the resource when you feel you need it.

It’s not easy; it took me some time to really appreciate detailed, in-depth resources. However, my experience shows me that they are the best to really understand deeply concepts and principles.

This is even more difficult today: we have a tendency to scan and reject whatever we don’t understand, because we think it’s too long, too complex, or both. We have difficulties to take the time we need to understand deeply.

I always try to keep that in mind to improve and expand my knowledge.

What did we learn together with this article?

  • The expert blind spots are pieces of knowledge an expert is not anymore conscious of, but he can still use to perform. Thus, he’ll have difficulties to explain it to another, who might needs it to understand a concept.
  • The working memory, used to understand, learn, and perform, is limited. Consequently, an expert needs to abstract his knowledge (“chunking”) and brush away its details. The benefit is to save place in the working memory to learn new concepts, or to accomplish tasks.
  • Informal communications and friendly relationships can give precious information about what a learner doesn’t know. It will help the teacher not falling into the expert blind spot.
  • Writing or representing the steps and layers of your knowledge (via mindmaps, blogs, or journals) can remind you the details you might have chunked, and consequently avoiding the expert blind spots when you need to explain these concepts.

In order not to overload your working memory, you should read this kind of article step by step, trying to practice the knowledge you learned. There is no magical recipes: avoiding the expert blind spot, as teaching or learning, is ultimately a personal quest.

In short: try, experiment, learn and practice!

Share Your Knowledge