The Economics of Technology Professor of Stanford Graduate School of Business, Susan Athey, says there is no more hot topic at the intersection of business and technology than the potential of artificial intelligence and machine learning.
She says, “You read about it in the newspapers how these sentient robotics will take over the world.” If you go to a conference, you’ll see that the technical experts are mainly interested in how computers can differentiate between images of dogs and cats.
They said that while popular misconceptions about the cutting-edge science behind artificial intelligence are to be expected, there are many misguided attempts at implementing it in practice. Athey, who spoke about the digitalization of business in a Stanford GSB conference titled “Boost Your SuccessOpen in New Window,” warned that decisions can be misled by misunderstandings of what machine learning models are capable of. She highlighted some risks of ignoring fundamentals and focusing on shiny new technologies.
She says, “I have seen companies lose hundreds of millions of dollars by letting engineers make decisions without consulting businesspeople.” “Machine learning solves simple but not intelligent problems. It struggles to solve many business problems.
Athey says machine learning models are best at making predictions, where performance can be evaluated quickly and accurately. She says deep neural nets find patterns within a stable environment. They are helpful when many features can describe a unit. For example, all the pixels of an image, the text of a review, or the history of a user’s interactions with a site.
Athey says that image classification is an example of a simple problem where computers analyze pixels to find patterns. Recent innovations have enabled algorithms to distinguish between dogs and cats with greater than 90% accuracy.
Thinking Long vs. Thinking Long
Machine learning models can be used to solve simple problems on a large scale. However, they are harder to use when measuring success takes time. If an algorithm is used for a 10-year loan or long-term investment, it can be challenging to determine whether or not it works. It is also hard to improve its performance iteratively. It could take a while to decide if the environment has changed in a manner that invalidates the model.
Athey describes “what-if scenarios” as a problem for most machine learning algorithms. What would happen if the company changed its prices or did not run a particular ad campaign? A misguided belief in machine learning accuracy can be problematic in practice.
Athey suggests that you consider an algorithm to predict the occupancy of hotel rooms based on prices observed. The algorithm would analyze historical prices and occupancy rates to determine that the hotel was complete when the prices were high. If this predictive model were used to optimize hotel prices, the result would be that you need to raise your costs to attract more guests. Athey says, “Which, of course, is wrong.” “Just because higher hotel prices correlate with full hotels doesn’t mean that if you lower your price, you will sell more rooms.”
Old-Fashioned Know-How
Any pricing specialist working for a hotel knows that demand curves do not work that way. It’s a problem when people start making decisions based on the belief that machine learning can solve all problems when used in a standard fashion. Athey argues that leaders should instead determine which specific aspects of a given situation can be delegated to algorithms and which parts need to be guided by traditional know-how. It can bechallengingt to make that distinction because most people do not understand the inner workings of black-box algorithms.
She says that if businesspeople aren’t confident and educated, they’re pushed out of the room because arguing against the technical people who create the algorithms is difficult.
The more we know and trust our basic ideas, the more it’s easier to say: “I don’t want your mumbo-jumbo. Let’s talk about basic principles.” Your words don’t make any sense. Certain [economic] fundamentals exist, and if the model you use tells us otherwise, it’s not working.
Bridging the Gap between Algorithms & Expertise
Athey is still excited about the impact machine learning and AI are having. Athey’s research is done mainly in collaboration with Stanford GSB Professors Guido Imbens and Stefan Wager. It focuses on building tailored machine learning models, which focus more on what-ifs than pure predictions. She has been encouraged by the early successes. But even with customized methods for what-if scenarios, it is essential to have business knowledge to apply them correctly. She wants executives to approach digital transformation in industries and business models holistically.
She says that technical experts cannot always see the bigger picture. They don’t realize that the metrics they are optimizing don’t consider all the business considerations.
Athey sees an excellent opportunity for new leaders to step up and bridge the gap between algorithmic insight and human expertise. “As it rolls out through the entire economy, every business will go through the journey.” They’ll all need to find new business systems and ways of thinking.
