Your AI Model is Only as Good as the Data It’s Trained On
Although AI and knowledge management are indeed complementary and synergistic, there’s sometimes an expectation that an LLM like ChatGPT can simply be trained on whatever is available in the pool of a department’s documents and records with good results.
While it’s true that the public version of ChatGPT gives surprisingly good answers to questions—and future GPT versions may further reduce response hallucination1 for instance—the answers of any corporate repository-trained AI will depend on the quality of that knowledge and organizations must remain in full control of those answers.
For example, in a scenario where an organization has lackluster engagement with knowledge management, the KMS may itself languish. Employees may allow information to become outdated, incomplete, inconsistent, or inaccurate.
The risk is that knowledge may not comply with internal or industry regulations and no longer align with the company’s goals and values.
APQC provides a useful way of looking at the issue with their Knowledge Management Maturity model2. For instance, an organization at Level 2 is driven by “localized and repeatable” processes, implying a lack of standardization of knowledge flows that are guided rather by specific business needs or individual managers.
Knowledge under such circumstances would be inconsistent across the organization and perhaps be duplicated, fragmented, and obsolete in some places, and a level of at least 3 in the maturity model should be achieved before considering deploying an additional AI-enhanced customer support channel.
A first step in leveraging AI in knowledge management is to assess the state of your knowledge processes, tools, strategy, and other enablers to ensure the quality, timeliness, and compliance of the information in your KMS.
By now it’s obvious that AI (artificial intelligence) in knowledge management is a game changer. LLM (large language model) systems like GPT (Generative Pre-trained Transformer) are designed to surface useful information and find patterns like never before.
For instance, integrating AI into chat platforms offers an opportunity to expand customer support channels, since ChatGPT-inspired models can respond with human-like and intelligent answers to customer queries.
Organizations that are currently exploring the integration of AI with knowledge management are still confronted with crucial questions, however. A few of these are:
- How can we assure the accuracy, relevance, and timeliness of AI answers?
- How can we ensure our AI systems align with the values and goals of the company, our customers, and society?
- Will AI answers also align with our brand voice and preferred communication style?
The data used for training and fine-tuning an LLM is key, which is why a well-managed KMS and a reliable governance framework are fundamental to the success of your AI strategy.
For companies operating in regulated industries, compliance is an additional hurdle and raises the stakes in terms of liability. For example, any wrong or non-compliant answer can pose significant risks to reputation and may even result in legal proceedings.
But leaving these risks aside, you’ll want your AI-KMS integration to continue yielding useful and high-quality answers while avoiding potential downsides that may undermine trust in your knowledge platform, such as generating text that is false or misleading.
Below we explore the benefits of a solid knowledge strategy as a prerequisite to introducing AI technology into a KMS. In particular we discuss:
- Principal risks of attaching an LLM to a knowledge repository that isn’t well managed
- The benefits of placing your KMS at the center of your AI strategy
- How a thorough governance framework can improve the quality of your AI responses
Discover how livepro knowledge management delivers better customer experiences with AI-powered search.
How a Well-Managed KMS and Governance Framework Paves the Way for Effective AI
“Effective” AI enhances knowledge discovery, retrieval, organization, analysis, and sharing in a way that positively impacts overall knowledge management practices. It’s truly a part of your KMS and is aligned with the specific needs and goals of your KM initiatives.
By contrast, ineffective AI can give inaccurate and unusable answers, fostering mistrust in the overall KMS and leading to lower engagement with the system.
Ineffective AI might also not be able to leverage user interactions or feedback to improve its performance, as the underlying knowledge management system has not properly integrated such mechanisms. The information remains static as the system fails to learn from its mistakes or evolve to meet user needs or other requirements.
5 Benefits of Harnessing an Efficient KMS for AI Success
A knowledge management system that follows best practices and is robust, comprehensive, and compliant is a requirement for effective AI integration.
Focusing on your KMS as the starting point in your AI journey has the following advantages:
1. Access to Structured and Relevant Information
Giving the AI access to a well-managed KMS with fully reviewed and approved knowledge enhances the reliability of answers specific to your organization rather than relying on generalized answers from an LLM’s pretraining. Ideally such information has also been preprocessed for consistent formatting and to provide simplified answers and outcomes. (livepro offers processes and tools for extracting work instructions and other structured content from corporate information—speak with an expert to learn more.)
2. Contextual Understanding and Informed Responses
Connecting the KMS directly to the AI significantly enhances its ability to provide the correct answer to the user. This ensures companies can be confident in the answer AI is providing the user.
3. Answers That Are Up-To-Date
A KMS whose information is well-maintained and current will enable AI to respond with the latest knowledge and insights. This is particularly important in dynamic and regulated industries where information rapidly changes or becomes outdated.
4. Personalization and Customization
KMS integration allows you to customize the AI’s responses based on your organization’s guidelines and policies. The answers can align with your brand voice and preferred communication style. This customization enhances the quality of responses and ensures consistency in messaging.
5. Scalability and Efficiency
By leveraging the information in your knowledge repository, AI can efficiently handle a wide range of queries without relying solely on general knowledge. This scalability allows the system to provide accurate and relevant answers even for complex or domain-specific questions, enhancing the overall quality of the responses.
How a Governance Framework Promotes Correctness of AI Responses
A robust governance framework for your knowledge management system will ensure your AI is providing the most up to date response every time. Part of your governance framework should include a content creation process.
The content creation process has specific stages the content goes through before going live, and each stage has clearly defined roles and responsibilities through which different people throughout the organization are accountable.
The review and approval stages are the most important pieces. Every single piece of content should go through review and approval before going live, thus ensuring the content is correct and has the business sign off to go live.
On top of the content creation process, each piece of content should have a knowledge owner who is responsible for its placement in an audit schedule for review every six to twelve months. This is to ensure that all content, even content that doesn’t change often, goes through a review and approval process at least once to twice a year depending on requirements.
A solid governance framework ensures that AI answers:
1. Are Accurate, Timely, and Compliant via User Feedback
A framework defines the specifics of how feedback is collected, who is responsible for reviewing it, and how changes are implemented based on that feedback.
2. Are Transparent and Explainable
AI answers should be traceable to their sources, methods, outputs, and outcomes, providing clear and accurate information to users (e.g., employees and customers) about identity, purpose, and limitations.
3. Build Trust and Confidence with Users
The quality of responses and transparency in how AI provides answers can enhance users’ trust and confidence. As well, clear accountability for the AI’s performance and responsive mechanisms for addressing user feedback and complaints can give a sense of reliability and dependability.
4. Align with the Values and Goals of the Company, Its Customers, and Society
AI should also contribute to business excellence by enhancing the well-being, dignity, and rights of customers and other stakeholders. A governance framework should establish clear principles and guidelines that align the use of AI with these values and goals. This could include policies on transparency, accountability, privacy, and data protection, as well as standards for quality and performance. The framework may also describe policies for monitoring and reviewing AI responses to ensure the system continues to align with these values and goals over time.
Final Thoughts
AI promises to reshape knowledge management and all that it encompasses, and certainly companies should examine ways to integrate this technology into their organizations to achieve more efficient knowledge discovery, better decision making, superior customer experience, and other benefits.
However the cornerstone of a successful AI strategy is the information you train it on, which in turn rests upon the usual components of effective knowledge management, namely: people, processes, technology, and KM strategy. These are building blocks upon which a reliable AI implementation can be established.
