There is no doubt that artificial intelligence and machine learning (ML) will play an increasingly vital role in advancing enterprise technology and underpinning a wide range of corporate initiatives over the next several years.
The global revenue of the artificial intelligence market, including software, hardware and services, is expected to reach $341.8 billion this year and grow at an annual rate of 18.8% to break the $500 billion mark by 2024, According to market researcher IDC. And by 2026, 30% of organizations will routinely rely on AI/ML-powered insights to drive actions that could lead to 60% increase in desired results (In fact, 30% may be a low estimate).
Despite the optimism, the dirty secret of deep neural network (DNN) models driving increased adoption of ML is that researchers don’t understand exactly how they work. If IT leaders use a technology without understanding the basis of its operation, we risk a number of bad outcomes. Systems can be unsafe in the sense that they can be biased, unpredictable, and/or produce results that cannot be easily understood by human operators. These systems can also have idiosyncrasies that enemies will exploit.
When ML is applied to mission-critical applications, CIOs and their engineering teams face a paradox, choosing between the best outcomes that machine learning can deliver versus the risks of poor outcomes. This could become an ethical dilemma. Let’s say a DNN used to process medical images can recognize certain forms of cancer better than the average practitioner. Are we morally obligated to use this technology, which can have positive, life-saving effects, even if we don’t know how to achieve its results?
The long-term goal of some machine learning researchers is to develop a more comprehensive understanding of DNNs, but what should practitioners do now and then, especially when poor outcomes can involve risks to life and/or property?
Create protection barriers for machine learning
Engineers have faced similar situations in the past. In the early days of aeronautics, for example, we did not have a complete understanding of basic physics or the ability to analyze aircraft design. To compensate for this lack of understanding, flight engineers and test pilots will determine operating envelope in which the aircraft can safely fly and then take steps – through flight control systems, pilot training, etc. – to ensure that the aircraft operates only within that safe envelope.
The same approach can be applied to developing a safe and predictable operating envelope on ML by creating guard bars that keep ML models on track and reduce the potential for unsafe and/or unexpected output. Here are some suggested approaches to creating ML systems with greater security and predictability:
1. Determine the range of model outputs that are considered safe. Once we have defined a safe output range, we can work our way back through the model to define a set of safe inputs whose outputs will always fall within the desired circumstance. The researchers showed that this analysis can be performed for certain types of DNN-based models.
2. Install guardrails “in front” of the model. Once you know the safe range of entries, you can install a software firewall in front of the form to ensure that entries that will move it to an unsafe location don’t appear. In fact, guard rails keep the ML system under control. Although we don’t know exactly how the model arrives at certain outputs, we will know that the outputs are always safe.
3. Focus on models that generate predictable results. In addition to keeping the output in a safe range, we also want to know that the models do not produce results that fluctuate dramatically from one part of the output space to another. For certain classes of DNNs, it is possible to ensure that if the input changes by only a small amount, the output will change proportionally and not jump unexpectedly to a completely different part of the output range.
4. Training models to be safe and predictable. Researchers are finding ways to subtly alter the training of DNNs so that they become amenable to the above analysis without compromising their pattern recognition capabilities.
5. Be agile. As this is a fast-moving space, the key is to build guard rails into the ML architecture, while retaining the agility to develop and improve them as new technologies become available.
The task before IT leaders is to ensure that the ML models they develop and deploy are under control. Establishing guardrails is an important interim step, as we develop a better understanding of how DNNs work.