Extra meaning: experts and dilemmas

Suppose you have an absolutely wonderful idea. You are afraid someone will steal it. You share the idea with people you trust the most. And they give you a cold shower. Probably this will have a strong effect on your confidence… But really, should it? Today’s reading can be found here, here, here, here, here, and here.

What the experts say

We all have narratives. This way we understand the meaning of our existence. The narratives can change, but then we change with them.

• Your spouse wants safety for you and your kids. She might be super supportive only if you do not take more risks than absolutely necessary. The smarter spouse will have a better understanding of the risks and a clear boundary.
• Business experts cannot say for sure what will work and what will fail. They can help you improve your odds. If you are excited, they will ask you to sleep on it. If you are calm and in control they allow you to make any decision you want.
• Coaches and psychologists want you to achieve your goals. Even if these goals are stupid, they cannot judge them. They will drive you to realize any goals you state and will try to keep you from pivoting. Any new idea you have will sound defocusing.
• Technical experts cannot be wrong. They will probably ask some time to do research and if they are good you will see a list of companies that do almost what you wanted to do. If they are very good, you will also see the effort estimate of what is required to be competitive.
• Only an insane investor will put money in something that has not been tested. If you meet one, you will probably be asked to bring an idea when it is more mature.

They all are good people interested in your success. The conversations will be polite and supportive and empathic. Then you will need to see how much you want the idea and are willing to fight for it.

Experts and confidence

When we can rely on a strong study, or when an expert can vouch that we have a solid idea we feel confident. Experts ask their friends for a second opinion to make sure they did not miss anything. Investors ask experts for evaluation and if the evaluation is good they present the idea to others. So when we want to say something we really want the experts to support us.

However, when someone approaches us with an expert opinion it is a very different story. We immediately need to check if there is a controversy. Maybe there is very different research with very different results. If the expert’s opinion has limitations we will try to find them. Then we feel that no expert can be truly trusted, but maybe we should find our own expert. Unless we accept the narrative as our own, we will try to fend against experts we do not know.

Knowledge tends to be distributed. It often takes several experts from different disciplines to evaluate any idea, product, company, or recommendation. Some evaluate the business, others the technology, and occasionally psychologists are also called in. The evaluation process itself is complex, tedious, and error-prone. No matter what the evaluation says, it is open for arbitration.

Why should we base our confidence on some expert opinion?

How do we validate ideas?

Unfortunately, there are very few ways to verify any idea:

• Experimentation is the most scientific way. Unfortunately, building and conducting experiments is a very long process. By the time the experimental data is analyzed, the hypothesis often becomes outdated. In technology, by the time QA finishes with one version, R&D often has a new version with new features.
• Simulation is the soundest way to test ideas. There are computer simulations, simulations of social processes, mockup review. Building a great simulation takes a significant effort, and then there is always the question: how close the simulation is to the real thing.
• Mathematical proofs and humanitarian argumentation are checked by peers. These peers tend to be busy and the evaluation superficial.
• Market and psychological ideas are verified by surveys. Surveys cost money and their results can be easily swayed by the formulation of questions and focus group selection.
• Precedents. Mostly for legal purposes but not only. If someone has already done something, there is a good chance it can be done again. When all the precedents show a certain direction any new case is likely to repeat that.

Most kinds of evaluation are flawed: either lengthy and costly or inconclusive, maybe both… It is much easier to bring experts and if they do not find a deadly flaw give it a try…

What do the experts know?

If we are experts ourselves, we understand the limitations of other experts in the same field. Otherwise, we are almost blind. Sure, we can see a resume with fancy degrees and professional achievements. We can interview other clients and peers. Then we need to trust the person.

In the middle of PhD students present their ideas to a panel of experts. One of the people in my panel happened to be a proponent of the paradigm I proved experimentally to be wrong. He tried to discredit me. The other four people in the panel loved my hypothesis. To make everybody happy, I was allowed to continue with my research, but I had to pivot to a different scientific direction. In fact, I lost 2 years of experimentation because I failed to predict that one of the experts will be hostile.

Experts should know the active theories and precedents, but they are people too. Sometimes they know things that later are proven wrong. They still happen to believe in them.

Not experts but the next best thing

When there are no cheap reliable experts in our surroundings, we tend to bring friends and family members who have a chance of understanding the relevant idea. These people often have a personal obligation to help, even when they do not have the relevant expertise. Generally, such “experts” tend to be unreliable. A confident and knowledgable person can convince them in any reasonable idea, and they cannot really check. A great idea represented by someone with a bit less confidence and rhetoric is likely to be killed. The best thing these “experts” can do is find someone who knows better.

Unfortunately, it is very easy for imposters to look like experts. It is enough to show flashy diplomas and say things that sound right but cannot be checked. I heard a story from a big company where a person with schizophrenia and zero education worked as a physicist with PhD from Stanford and 3 years of experience in JPL. His ideas sounded “revolutionary”. It took a year to find someone with enough expertise and credibility to discover the fraud and convince the management.

Not trusting experts

With so many questions and reservations, many managers do not trust their experts. Instead, they trust their own common sense and self-confidence. This is a huge issue, as the experts bring the best working methods and lower the risks. Not trusting the expert opinion increases the risks greatly and can cause several costly iterations.

Sometimes it is hard for an expert to explain the best methods. He might forget the reasoning behind them, not find the right words or simply use these methods without asking questions. This does not mean he is wrong. If he works on a certain project for a long time, he might have some specific intuition which is not shared by his peers. Expert opinion should be respected enough for detailed analysis.

On the other hand, some geniuses like Steve Jobs bet against all the experts of his time and won. These people are outliers, but then many managers are smart and creative people, outliers in their domain… Working with genius is complex. Even Einstein happened to be wrong on some occasions…

AI biases

How do the computers address such dilemmas? In computers, we may find out what caused en expert system to err.

• Data-driven bias. AI will tend to err in cases which are rare and not significantly represented in its training set.
• Interaction bias. AI learns from human inputs. If the people who created the set made some systematic error, AI will learn it.
• Confirmation bias. In some systems, several expert networks are trained in parallel. Then a generalist network is trained from the opinions of experts, but like in the movie “Minority report” the outliers are removed. As a result, the most unique insights tend to disappear.
• Similarity bias. When the system is trained on one task but then converted to another similar task, errors may follow.
• Conflicting goals bias. Quite often the target function of the AI includes conflicting goals and constraints. These constraints may confuse the AI.

Human experts tend to make similar mistakes…

Triviality of ideas

How does the situation look from the expert’s position? All the novices fear the experts will steal their ideas. Highly unlikely. Unlike novice that believes in the unique value of each new idea, an expert can generate a dozen of better ideas in an hour. In fact, the expert is usually slightly annoyed because he does not have enough resources for the greater ideas, and these resources are allocated for the confident novice. Moreover, other non-experts waste their time in conference rooms discussing the trivial aspects of these ideas, since it is harder to address the complex aspects.

So if the expert is not sufficiently enthusiastic to contribute to your wonderful idea, be patient with him…