SPARK: Transforming Public Works Project Management with AI-Powered Knowledge Retention
Overview
The Digital Transformation Hub (DxHub) at California Polytechnic State University (Cal Poly)—powered by Amazon Web Services (AWS) and part of the AWS Cloud Innovation Centers (CIC) program—partnered with San Mateo County’s Department of Public Works to create SPARK (Smart Project Assistant for Retained Knowledge), an AI-powered project management system that captures, organizes, and delivers institutional knowledge to engineers working on civil infrastructure projects. The Department of Public Works manages a diverse portfolio including road reconstruction, pavement preservation, drainage improvements, and utility projects across multiple special districts. SPARK leverages Amazon Bedrock and retrieval-augmented generation (RAG) technology to provide engineers with instant access to lessons learned, best practices, and standardized checklists—transforming how the county preserves and shares critical project knowledge across its engineering teams.
Problem
Public works departments face a persistent challenge: valuable project knowledge accumulated over decades risks being lost when experienced staff retire or move on, while new engineers face steep learning curves with limited access to institutional wisdom. At San Mateo County’s Department of Public Works, lessons learned, technical checklists, and proven practices were scattered across Word documents, Excel spreadsheets, SharePoint folders, and project closeout files—with no centralized, searchable system to retrieve them.
“We have a lot of different documents for projects… either it’s like an Excel spreadsheet or just a Word document where we try to capture lessons learned, best practices. But we don’t really have a good way of keeping all that information that’s readily available,” explained Krzysztof Lisaj, Deputy Director of Engineering & Resource Protection. The department’s existing approach relied heavily on informal handoffs, periodic meetings, and tribal knowledge. “Typically, the way we do it now is each engineer will work with their manager to talk about the project, depending on their level of experience… and depending on your level of experience, you sometimes you just don’t even know where to start.”
Lessons learned meetings occurred only twice yearly, with insights often distributed via email and subsequently lost in inboxes. A 17-page project checklist—the department’s “holy grail” for project execution—was frequently printed, placed in binders, and initialed by hand. When engineers needed reference documents like utility notification letters, they would pull templates from past projects rather than a single source of truth, leading to version control issues and inconsistent practices. This fragmented approach resulted in variable project execution, repeated mistakes, and significant onboarding time for new staff joining a department that manages sewer districts, water districts, drainage infrastructure, and annual pavement preservation programs.
Innovation In Action
SPARK reimagines how public works engineers access and contribute to organizational knowledge through three integrated capabilities. The AI Knowledge Assistant allows engineers to describe upcoming projects in plain language and instantly receive relevant guidance. An engineer starting a new slurry seal project can simply ask, “I’m starting a new Slurry Seal project, is there anything I should keep in mind?” and receive synthesized insights from past projects, relevant technical standards, and links to source documentation—all with citations back to original materials.
The Interactive Project Roadmap transforms the department’s comprehensive project checklist into a living, digital tool with smart task management and progress tracking. Rather than printing 17-page documents and manually initialing completed items, engineers now work through organized phases with real-time visibility for leadership. Tasks can be edited at both the project and global level, ensuring that improvements benefit all future projects.
The Lessons Learned Knowledge Base addresses the department’s long-standing challenge of capturing insights “in the moment.” As Lisaj noted, “Part of this is we want to be able to capture those kind of lessons learned in the moment… if something happens in the field today, it’d be great if I can go in this afternoon and go update this with this new information.” SPARK enables engineers to quickly document lessons through a simple interface or upload documents for automatic extraction—with all insights synchronized to the searchable knowledge base for immediate availability to colleagues.
The system supports project type-specific workflows for reconstruction, resurfacing, slurry seal, drainage, utilities, and custom infrastructure projects, ensuring that engineers receive contextually relevant guidance. Project creation walks users through 5-7 targeted questions about project name, type, location, area size, and special conditions, automatically generating appropriate task lists and surfacing related historical projects.
Technical Solution
SPARK’s architecture leverages a comprehensive suite of AWS managed services designed for security, scalability, and intelligent information retrieval. At its core, Amazon Bedrock powers the AI capabilities, utilizing Anthropic’s Claude foundation model for natural language understanding, lessons extraction, and conflict detection when multiple projects contribute overlapping insights.
The system implements a retrieval-augmented generation (RAG) pattern through Amazon Bedrock Knowledge Bases, which provides vector search capabilities across all project documentation and lessons learned. When engineers submit queries, their input is converted into mathematical representations and compared against chunked documentation to identify semantically similar content. The top relevant passages are then fed to the large language model along with carefully engineered prompts to generate comprehensive, cited responses.
AWS Lambda functions handle serverless compute for API operations including project management CRUD operations, lessons processing with extraction and synchronization, and search queries against the knowledge base. Amazon API Gateway provides secure REST API endpoints, while Amazon DynamoDB stores project metadata, status tracking, and checklist configurations. Amazon S3 serves as the document repository for project files, lessons learned, and deployment artifacts.
User authentication is managed through Amazon Cognito, ensuring secure access control, while the Next.js frontend is delivered via Amazon CloudFront for low-latency global access. The entire infrastructure is defined and deployed using AWS Cloud Development Kit (CDK), enabling consistent, repeatable deployments and rapid iteration during the prototyping phase.
The system’s configuration—including AI model selection, prompt engineering, and supported project types—is centralized in a configuration file that can be modified without code changes, allowing the department to evolve the tool’s behavior as their needs develop.
Next Steps
San Mateo County’s Department of Public Works is advancing SPARK through a phased implementation, with initial design and prototyping completed, pilot deployment targeted within six months, and full implementation planned within twelve months. The department has already incorporated user feedback to refine the system, including making work area size fields optional with support for multiple units, correcting project status logic to properly transition completed projects, and resolving issues with saving lessons learned edits.
Success will be measured through reduction in new staff onboarding time, increased reuse of checklists and documented best practices, higher project quality with fewer repeated mistakes, positive user satisfaction scores, and growth in lessons captured and retrieved over time. The department plans to focus initial deployment on pavement preservation projects—their annual slurry seal and microsurfacing programs—before expanding to reconstruction, drainage, and utility projects.
“This is kind of a common use case that we see in different departments. We have a lot of projects going on where knowledge is inconsistently captured after projects,” noted Julie Goebel, Innovation Program Manager for San Mateo County. “Seeing a tool that can help us with that would be exciting for a lot of departments.”
The solution’s open-source design enables replication by other public works departments facing similar knowledge management challenges. Organizations interested in implementing SPARK or engaging with the Cal Poly Digital Transformation Hub can reach out to Nick Osterbur (nosterb@amazon.com) or visit the DxHub website. The complete codebase and deployment instructions are available on GitHub.
Supporting Documents
| Source Code | All of the code and assets developed during the course of creating the prototype. |
About the DxHub
The Cal Poly Digital Transformation Hub (DxHub) is a strategic relationship with Amazon Web Services (AWS) and is the world’s first cloud innovation center supported by AWS on a University campus. The primary goal of the DxHub is to provide students with real-world problem-solving experiences by immersing them in the application of proven innovation methods in combination with the latest technologies to solve important challenges in the public sector. The challenges being addressed cover a wide variety of topics including homelessness, evidence-based policing, digital literacy, virtual cybersecurity laboratories and many others. The DxHub leverages the deep subject matter expertise of government, education, and non-profit organizations to clearly understand the customers affected by public sector challenges and develop solutions that meet the customer needs.