AI has been around since the 1960s but has only recently started significantly transforming business operations. Many organizations now embrace AI through targeted initiatives or broader readiness efforts. However, by some estimates, 80% of AI projects fail - twice the rate of technology projects not utilizing AI. This is frequently due to a misalignment between business goals and the technical team’s objectives. This misalignment becomes apparent when an AI model excels during testing but fails to deliver real value or impact. Below, we’ll discuss the importance of aligning technical decisions with broader business objectives and how to achieve that alignment.
At its core, AI models are data-processing systems that apply transformations to convert inputs into outputs. Unlike traditional programs, AI models can learn from data. During a pre-deployment training phase, AI models begin with random transformations and gradually refine their approach by comparing results to expected outcomes. The effectiveness of an AI model is typically assessed using an “error function,” which helps the model improve and serves as a quality measure.
In tasks like classification—such as identifying objects or predicting outcomes—accuracy is a common evaluation metric. However, accuracy may not be the best choice for measuring all business outcomes. Let’s explore how AI may be applied to investment finance. A model predicting stock movements should be evaluated based on its impact on profit and loss rather than just accuracy. For example, if a stock has small daily fluctuations but occasionally sees significant swings, a model that accurately predicts those large movements—even with lower overall accuracy—would be more valuable. The model should also consider real-world factors like transaction costs to provide actionable buy/sell/hold recommendations.
Some of these decisions can have life-or-death implications. Consider a situation where 1/1000 patients tested have a type of cancer. An AI model used in cancer diagnostics might achieve an impressive 99.9% accuracy by predicting "no cancer" in 100% of patients, but it is fundamentally ineffective at diagnosing cancer. Instead, metrics like precision, recall, and specificity provide a more comprehensive view of model performance and will ‘train’ the model to avoid some types of errors over others (e.g. false positives vs. false negatives). The decision over which metric to employ will depend on the problem context and objectives.
Regardless of the application, a technical team must work closely with business stakeholders to ensure a proper problem definition and a full understanding of the downstream effects of the decisions influenced by model output.
Aligning AI evaluations with business value is crucial. But how can we achieve this? AI development requires technical skills that may feel distant from business leaders, leading to communication silos.
For smaller projects, an experienced, technically knowledgeable program or project manager can ensure that technical execution reflects business objectives by bridging the gap between functional and technical stakeholders. Engaging business stakeholders beyond the project team to share insights about organizational needs is critical during data analysis and model development. Additionally, to ensure the effective use and adoption of the new tools, organizational change managers can guide the organization through the implementation process and train the business stakeholders to use the tools in a way that delivers on the expected business outcome.
For larger AI initiatives, organizational structure matters. If AI efforts are localized to specific departments, the AI team should report to the leader overseeing that area. For enterprise projects, a shared services model often works best, ensuring that the AI team serves the entire organization. Be cognizant that whichever group has a direct reporting chain to the technical team may introduce bias in priorities; sometimes, having the AI group report directly to the CEO can be beneficial as the organization learns to leverage this technology.
AI is reshaping many aspects of business, and while it may seem complex, it doesn’t have to be intimidating. Fostering AI literacy among leaders is vital for equipping technical teams with a clear understanding of the business context. I’d love to hear about your experiences as AI practitioners or business stakeholders. Have you encountered challenges with AI implementation? How do you ensure that business needs are prioritized during model development? What impact has your organization’s structure had on your AI efforts?
If you want to learn more about how AI can help you achieve your business goals, let's talk!
