Leveraging RAG and AI to Redefine IT Support at the USDA
Overview
In a step toward modernizing its internal IT support systems, the U.S. Department of Agriculture (USDA) partnered with The Cal Poly Digital Transformation Hub (DxHub), powered by Amazon Web Services (AWS) to implement an innovative generative AI chatbot solution designed to streamline IT assistance for its vast workforce of 100,000 employees across 4,500 locations.
This new AI-powered chatbot has the potential to serve as a first-line support system, capable of handling common IT inquiries, troubleshooting technical issues, and providing immediate assistance to USDA staff members. The chatbot not only helps modernize the USDA’s IT support framework but also ensures that employees across all 29 agencies and offices have access to consistent, reliable technical help, regardless of their location or time zone. This enhancement in IT support capabilities ultimately would ultimately enable USDA staff to better focus on their core mission of serving the American people through agriculture, rural development, and natural resource conservation programs.
Problem
The USDA’s IT help desk is continuously assessing new technologies to leverage that can aid in addressing the approximately 250,000 support tickets received annually. A large chunk of these requests pertains to well-documented issues that have known solutions, ranging from simple password resets to common software configurations. However, the traditional support model uses human intervention for nearly every ticket, leading to unnecessary bottlenecks and escalating operational costs.
This workflow not only drains valuable staff time but also slows down response times for more intricate technical problems that truly require human expertise. Recognizing the value AI technologies offer, the USDA is exploring AI solutions to integrate into their IT support processes. The goal is to minimize the number of routine support tickets handled by humans, expedite resolution times for common technical issues, and enhance the first-contact resolution rate without human intervention. The goal is to optimize help desk resources for tackling more complex technical challenges, reduce operational costs associated with basic support requests, and ensure 24/7 support availability for standard IT issues.
This scenario presented an opportunity to leverage modern technology to transform the USDA’s IT support model, making it more efficient and cost-effective while maintaining high-quality service for its employees.
Innovation In Action
To address the USDA’s IT support challenges, the team developed an advanced AI-powered chatbot solution that transforms how employees access IT support. The system leverages Retrieval Augmented Generation techniques that are based on trusted solutions from internal experts. Users’ questions are embedded and matched to find relevant articles from the USDA’s existing knowledge base articles and documentation. These results are then synthesized into technical content that is used in conversational, easy-to-follow responses offering the user step-by-step guided solutions. The system displays relevant snippets of each article to make it easier for the user to follow.
Conversations are in natural language with users about their IT issues and allow a user to follow-up with clarifying questions. The chatbot maintains context throughout the conversation for more effective problem-solving. The system has the ability to route complex issues to human agents when necessary. The system also has implemented guardrails to detect and block PII exposure as well as filter inappropriate content and language. When the interaction concludes the system collects and analyzes user feedback for continuous improvement and tracks resolution rates and satisfaction metrics. Lastly the system has administrative settings to enforce usage limits and cost thresholds.
Technical Solution
The USDA IT Proof of Concept is built on an architecture leveraging several AWS services and modern technologies. At the core of the application’s knowledge base is Amazon OpenSearch Serverless, which serves as a robust vector database. This technology efficiently stores and retrieves knowledge base articles, ensuring that users have quick access to relevant information.
Powering the application’s intelligent responses is Amazon Bedrock’s large language model, specifically the Anthropic Claude Sonnet 3.5. This model is able to provide sophisticated natural language processing capabilities, allowing the application to understand and generate human-like text responses.
To enhance the accuracy and relevance of the responses, the application employs Amazon Titan Text v2 for text embedding. This model converts text into vector representations, facilitating more precise matching and retrieval of information from the knowledge base.
Next Steps
USDA plans to begin using the pilot project internally, get user feedback, and explore how this might integrate with internal systems. Some of the features under consideration include image processing capabilities, expansion to multiple knowledge domains within USDA and automated detection and response to widespread system outages. The DxHub will continue to work on this application with other customers that have similar needs.
Supporting Documents
| Source Code | All of the code and assets developed during the course of creating the prototype. |
About the DxHub
The Cal Poly Digital Transformation Hub (DxHub) is a strategic relationship with Amazon Web Services (AWS) and is the world’s first cloud innovation center supported by AWS on a University campus. The primary goal of the DxHub is to provide students with real-world problem-solving experiences by immersing them in the application of proven innovation methods in combination with the latest technologies to solve important challenges in the public sector. The challenges being addressed cover a wide variety of topics including homelessness, evidence-based policing, digital literacy, virtual cybersecurity laboratories and many others. The DxHub leverages the deep subject matter expertise of government, education, and non-profit organizations to clearly understand the customers affected by public sector challenges and develop solutions that meet the customer needs.