AI-Powered OCR System to Automate Package Routing for UC San Diego
Overview
The Cal Poly Digital Transformation Hub (DxHub), powered by Amazon Web Services (AWS), partnered with UC San Diego’s Residential, Retail, and Supply Chain Services team to develop an AI-driven OCR system for their new automated conveyor belt. The goal was to overcome the limitations of the vendor’s existing OCR software, which achieved only 64% accuracy in reading shipping labels. The DxHub solution integrates computer vision, intelligent document processing, and generative AI to extract critical package information in real time, achieving over 90% accuracy and enabling full automation of package routing across the UCSD campus.
Problem
UCSD’s new automated conveyor system was installed to manage growing package volume across campus, over 1,000 packages per day, but could not be launched due to low accuracy (64%) from the OCR software provided by the conveyor vendor. Labels vary widely across suppliers, and the system struggled to consistently extract critical information like the Purchase Order Number, Recipient First Name, and Last Name. Key issues included short recipient names (≤3 characters), inconsistent label layouts, and low-confidence OCR under variable lighting conditions. As a result, package routing still required manual intervention, delaying same-day delivery and adding significant operational load on UCSD staff.
Innovation In Action
To solve these challenges, the DxHub team developed a custom OCR pipeline optimized for UC San Diego’s diverse label formats and real-world operating conditions.
The solution uses computer vision techniques to detect and extract shipping labels from grayscale conveyor images. Each label is then processed with Amazon Textract to extract raw text, which is parsed using a combination of regular expressions and Nova Lite (via Amazon Bedrock) to identify the PO number and recipient name. The pipeline intelligently filters out buildings, halls, and departments that are often misclassified as people.
The system includes robust fallback logic:
- If a PO number is missing, the extracted name is matched against UCSD’s daily shipment manifest containing approximately 195,000 records.
- If no name is confidently extracted, the pipeline uses a secondary list of known generic recipients, such as “Seventh Market.”
- Partial matches are also retained, allowing UCSD to enrich its recipient database over time.
Each image is processed in under 4–6 seconds, producing a structured CSV output that includes extracted data, confidence scores, and per-label cost estimates for both Textract and Bedrock. During pilot testing, the system achieved over 90% extraction accuracy and demonstrated reliable batch processing performance with predictable operational costs.
Technical Solution
The OCR system is implemented in Python and integrates AWS services for scalability and low latency.
OpenCV is used to detect and segment labels using techniques such as contour detection, adaptive thresholding, and structural similarity indexing (SSIM). Once labels are isolated, the processing pipeline performs the following steps:
- Amazon Textract extracts raw text from each label.
- Regex-based filters detect PO numbers matching the format PUR########.
- Nova Lite (via Amazon Bedrock) performs semantic parsing to extract first and last names and remove false positives.
- Fuzzy string matching compares the extracted data against the shipment manifest to identify correct matches.
Each label’s output includes:
Supporting Documents
| Source Code | All of the code and assets developed during the course of creating the prototype. |
| Architecture Diagram | A diagram that describes the technical components needed to implement the solution. |
About the DxHub
The Cal Poly Digital Transformation Hub (DxHub) is a strategic relationship with Amazon Web Services (AWS) and is the world’s first cloud innovation center supported by AWS on a University campus. The primary goal of the DxHub is to provide students with real-world problem-solving experiences by immersing them in the application of proven innovation methods in combination with the latest technologies to solve important challenges in the public sector. The challenges being addressed cover a wide variety of topics including homelessness, evidence-based policing, digital literacy, virtual cybersecurity laboratories and many others. The DxHub leverages the deep subject matter expertise of government, education, and non-profit organizations to clearly understand the customers affected by public sector challenges and develop solutions that meet the customer needs.