Managing Operational Risk: Proven Strategies & Best Practices
Jun 30, 2025 in Industry Overview
Learn effective methods for managing operational risk. Discover key frameworks, controls, and culture tips to protect your business today.
Not a member? Sign up now
Smart Silence, Safer AI
Kelwin on Mar 16, 2024
When it comes to AI models, we often expect them always to provide an answer. But what if we could trust them more when they choose to remain silent? This concept, known as the ‘Reject option‘, allows AI models to abstain from answering when they are not confident, opening up many applications in your business and reducing risk.
Just like human experts, AI models can gain our trust when they know when to say, “I don’t know“. This capacity to abstain from answering, known as the ‘reject option’, can be a game-changer for many applications.
Consider a model that diagnoses whether a patient has cancer. Would you trust a model that always gives a definitive ‘yes‘ or ‘no‘, or one that sometimes says, “I don’t know, let’s get a second opinion”? The same applies to a model monitoring a manufacturing pipeline for quality assurance. A model that can say, “I don’t know, let’s get a technician to check this” can be more reliable than one that always guesses.
There are two main strategies for using the reject option in business: filtering and incremental deployment.
In the filtering strategy, the AI model automates most of the cases and asks for help with the rest. This help could come from a second AI model or a human expert. This strategy essentially adds an additional stage to your workflow, a fallback option for when the AI model cannot provide an answer.
In the incremental deployment strategy, you start by only covering the top cases where the model is most confident. As you gain trust in the model, you start giving it more and more cases, even if the AI is not that confident about them. This strategy allows for a safer deployment and is particularly useful for initial pilots.
To compute the expected impact of using the reject option, you need to understand the trade-off between the percentage of times the model abstains and its predictive performance.
You start by computing how much benefit you would get from the model predicting 100% of cases and how much you would lose from the errors. Then, for any given percentage of abstentions, you calculate how much you would gain from the cases the AI automates and how much you would spend on the cases it doesn’t.
Remember, there is always a fixed cost for both the AI (infrastructure) and the human component (salaries), and an elastic cost for each case, depending on how much the AI abstains or predicts.
The easiest way to have models that can abstain is by looking at the tails of your predictions and only predicting for those cases. This approach involves setting two thresholds, one for the lower bound and one for the upper bound, and choosing these thresholds to maximize your return.
However, this approach may not be enough. A more advanced approach involves allowing your model to predict which cases it wants to answer and what the answer should be for those cases. This can be achieved by having two outputs from your model: one indicating whether the model will abstain or not, and the other providing the answer if it’s not abstaining.
Allowing your AI models to abstain when they are not confident can open up many applications in your business and reduce risk. It’s a strategy worth considering as you plan your AI deployment. If you want to embrace smarter AI decisions, book a strategic meeting with our experts and transform risk into reliability today!
Like this story?
Special offers, latest news and quality content in your inbox.
Jun 30, 2025 in Industry Overview
Learn effective methods for managing operational risk. Discover key frameworks, controls, and culture tips to protect your business today.
Jun 30, 2025 in Industry Overview
Learn how bottleneck analysis can help you find and resolve constraints efficiently. Discover tips for effective bottleneck analysis today!
Jun 5, 2025 in Industry Overview
Master quality control automation with proven strategies that drive real results. Discover practical insights from industry leaders.
Cookie | Duration | Description |
---|---|---|
cookielawinfo-checkbox-analytics | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics". |
cookielawinfo-checkbox-functional | 11 months | The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional". |
cookielawinfo-checkbox-necessary | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary". |
cookielawinfo-checkbox-others | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other. |
cookielawinfo-checkbox-performance | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance". |
viewed_cookie_policy | 11 months | The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data. |