Off-campus UMass Amherst users: To download campus access dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.

Non-UMass Amherst users: Please talk to your librarian about requesting this dissertation through interlibrary loan.

Dissertations that have an embargo placed on them will not be available to anyone until the embargo expires.

Author ORCID Identifier

https://orcid.org/0000-0003-3880-8075

AccessType

Open Access Dissertation

Document Type

dissertation

Degree Name

Doctor of Philosophy (PhD)

Degree Program

Computer Science

Year Degree Awarded

2024

Month Degree Awarded

February

First Advisor

Subhransu Maji

Second Advisor

Daniel Sheldon

Third Advisor

Daniela Calzetti

Fourth Advisor

Erik Learned-Miller

Subject Categories

Computer Sciences | Data Science | Other Computer Sciences

Abstract

AI has the potential to accelerate scientific discovery by enabling scientists to analyze vast datasets more efficiently than traditional methods. For example, this thesis considers the detection of star clusters in high-resolution images of galaxies taken from space telescopes, as well as studying bird migration from RADAR images. In these applications, the goal is to make measurements to answer scientific questions, such as how the star formation rate is affected by mass, or how the phenology of bird migration is influenced by climate change. However, current computer vision systems are far from perfect for conducting these measurements directly. They may perform poorly when training data is limited, might introduce bias, and do not offer the statistical guarantees that scientists desire. This thesis addresses these challenges in three ways. First, we consider transfer learning to hyperspectral domains. The shape of the data, i.e., having more than three channels, restricts the use of pre-trained networks trained on color images. We design and investigate lightweight adapters that can be plugged into a pre-trained network to make it compatible with hyperspectral domains. Adapters allow for better generalization when training data is limited in various image classification tasks. Second, we explore how unlabeled data in a domain can be used to bootstrap a pre-trained network. We investigate the role of self-supervised learning in training networks for star cluster classification in astronomical images. Third, we address the scenario when a model is available but unreliable. This may be due to the task's difficulty or the model being deployed on out-of-domain data where performance cannot be guaranteed. We develop human-in-the-loop techniques that incorporate human vetting of model outputs to produce estimates with statistical guarantees. We ground these approaches in applications in astronomy, ecology, and climate where data is heterogeneous and has different measurement needs. Manual measurements pose challenges due to the required domain expertise and the scale of the data being analyzed. We apply ideas from this thesis to develop StarcNet, a deep learning model capable of classifying star clusters in Hubble images. It achieves a level of human agreement comparable to existing catalogs and produces similar scientific conclusions, such as age/mass or frequency/mass distributions in galaxies with existing catalogs. In collaboration with others, we use the model to automatically analyze sources from the M101 galaxy and conduct preliminary studies on the near-infrared bands of the NGC4449 galaxy. In ecology, we study the behavior of roosting birds using weather radars. Weather radars around the globe continuously scan the airspace and are sensitive enough to detect flying animals. However, the sheer volume of data makes manual analysis impractical. We have designed an AI-assisted system capable of extracting research-grade roost annotations from radar data. This system combines ideas from adapter design to develop an accurate spatio-temporal roost detector with a human-in-the-loop vetting system that produces estimates with statistical guarantees. In collaboration with others, we use this framework to quantify long-term phenological patterns of aerial insectivores such as swallow and martin roosts. These analyses represent one of the most comprehensive long-term, broad-scale examinations of avian aerial insectivore species responding to environmental change. Lastly, we consider the estimation of damaged buildings from satellite imagery on regions struck by a natural disaster. During disaster response, aid organizations aim to quickly count damaged buildings in satellite images to plan relief missions, but pre-trained building and damage detectors often perform poorly due to domain shifts. In such cases, there is a need for human-in-the-loop approaches that can accurately count with minimal human effort. We propose techniques for counting over multiple spatial or temporal regions using a small amount of screening. We conclude by discussing how AI and humans can collaborate to tackle various measurement tasks and outlining the future challenges associated with deploying AI in scientific research.

DOI

https://doi.org/10.7275/36423331

Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.

Download

Share

COinS