Accelerated fMRI Research & Processing Using the Cloud

Overview

Professional portrait of Laura.

Laura Cacciamani is a professor in the Psychology department at California Polytechnic State University and conducts research to determine how the five senses (vision, touch, hearing, etc.) work separately and together to produce an understanding of the world around us. Her research focuses on both how we perceive and remember the objects in our environment, as well as their spatial relationships with one another. Her research confirms previous findings, which state that vision is often the sense we rely on most for a majority of cognitive functions, as it allows us to process an object or scene more efficiently and remember more details. To conduct her research, Laura places subjects into a functional Magnetic Response Imaging (fMRI) machine and instructs them to focus on different objects. She then measures which parts of the brain are most active during perception. The result of her research is raw fMRI files, which need to be processed to build a three-dimensional map of the brain to illustrate this activity.

Problem

Laura needs to utilize compute resources to process the fMRI files she collected during her research. She is able to use open source software to manually process this data on a high-powered iMac desktop machine. Her current process has the following constraints:

  • Each subject can take anywhere between 12-90 hours to analyze depending on available data
  • She uses the same compute machine to process fMRI files that she uses for work. As a result, the files are processed serially as to not impose too much memory and compute burden on her device
  • This process is prone to errors, which can waste additional time if a subject fails and must be reprocessed

Laura processed 30 subjects in six months due to the amount of time needed to process each participant in combination with other professional obligations.

Innovation in Action

Laura engaged the Cloud Computing Research Committee at California Polytechnic State University’s Digital Transformation Hub (DxHub) to see if her research could benefit from cloud technology. Her research became an ideal use case for cloud compute. She worked with Cal Poly technology engineers process the compute in the Amazon Web Services (AWS) cloud to parallelize the problem. Building off the work done by Paul Wighton of Harvard University, the DxHub team was able to use AWS Batch (a customized Docker container) and Amazon’s Elastic Container Registry to install open source software on a Docker container. Developers then configured the container, pulled the raw data from an S3 bucket and stored the post-processed files back on the S3. AWS Batch excels at running jobs like this on the right sized instance.  As the job queue began to increase AWS determined a larger instance was needed to process the Docker jobs accordingly.  AWS Batch automatically scaled the resources down after the jobs completed.

Results

Since the DxHub ran all jobs in parallel, developers were able to process all 30 subjects within the time it took for Laura to run the longest subject. After 90 hours and $284.56 from Amazon Web Services (AWS), the team completed a task that took Laura 6 months in just 90 hours. 

The FreeSurfer program that processed Laura’s raw fMRI data runs in a single thread.  A newer version of the program exists; this new version supports the utilization of Graphics Processing Unit (GPU) cores and using multiple threads simultaneously. In Laura’s solution, the team of developers at the DxHub was unsuccessful in their first attempt to get either of these options to work. Future iterations will focus on debugging the failure of those runs.The framework used in Laura’s solution could easily be adapted to any workload that is compute-intensive and will run inside a Docker container. DxHub developers would need to configure the Docker container to accommodate individual use cases and push the AWS customized container to improve the Enhanced Correction Result (ECR).

About the DxHub

The Cal Poly Digital Transformation Hub (DxHub) is a strategic relationship with Amazon Web Services (AWS) and is the world’s first cloud innovation center supported by AWS on a University campus. The primary goal of the DxHub is to provide real-world problem-solving experiences to students by immersing them in the application of proven innovation methods in combination with the latest technologies to solve important challenges in the public sector. The challenges being addressed cover a wide variety of topics including homelessness, evidence-based policing, digital literacy, virtual cybersecurity laboratories and many others. The DxHub leverages the deep subject matter expertise of government, education and non-profit organizations to clearly understand the customers affected by public sector challenges and develops solutions that meet the customer needs.