DECISION-ANALYTIC MODELS USING REINFORCEMENT LEARNING TO INFORM  DYNAMIC SEQUENTIAL DECISIONS IN PUBLIC POLICY

KHATAMI, SEYEDEH NAZANIN

Publication:
DECISION-ANALYTIC MODELS USING REINFORCEMENT LEARNING TO INFORM DYNAMIC SEQUENTIAL DECISIONS IN PUBLIC POLICY

dc.contributor.advisor	Chaitra Gopalappa
dc.contributor.author	KHATAMI, SEYEDEH NAZANIN
dc.contributor.department	University of Massachusetts Amherst
dc.date	2024-03-28T16:35:35.000
dc.date.accessioned	2024-04-26T15:51:01Z
dc.date.available	2024-04-26T15:51:01Z
dc.date.submitted	February
dc.date.submitted	2022
dc.description.abstract	We developed decision-analytic models specifically suited for long-term sequential decision-making in the context of large-scale dynamic stochastic systems, focusing on public policy investment decisions. We found that while machine learning and artificial intelligence algorithms provide the most suitable frameworks for such analyses, multiple challenges arise in its successful adaptation. We address three specific challenges in two public sectors, public health and climate policy, through the following three essays. In Essay I, we developed a reinforcement learning (RL) model to identify optimal sequence of testing and retention-in-care interventions to inform the national strategic plan “Ending the HIV Epidemic in the US”. The large dimensions of the solution space along with the computational complexity of the simulations over long analytic horizons create compounding computational challenges not suitable for solution algorithms. We show that reformulation of the problem by solving for proxy decision-metrics significantly reduces the solution space and ensures convergence to optimality. In Essay II, we developed a deep RL decision-analytic model for effective early control of infectious disease outbreaks, focusing on new or emerging outbreaks that do not yet have pharmaceutical intervention options. Using the COVID-19 pandemic as a test case, we evaluated the question of whether a lockdown is necessary, and if so, when it should be initiated, to what level (proportion lockdown), and how this should change over time, such that it minimizes both epidemic and economic burdens. A key component of this problem is decisions are jurisdictional, i.e., limited in geographical authority, but occurring in interacting environments, i.e., actions of one jurisdiction can influence the epidemic in other jurisdictions. We evaluated the above question in the context of two-geographical jurisdictions which make autonomous, independent decisions, cooperatively or non-cooperatively, but interact in the same environment through travel. In Essay III, focusing on the climate policy sector, we defined a cost-effectiveness metric called Levelized Cost of Carbon (LCC) that carefully accounts for the time-value of money and the time-value of emissions reduction. This metric is a simple tool that local government agencies can use to evaluate climate change projects alongside other issues they may face, such as safety, congestion, pollution, and political considerations. We also investigated the theoretical and practical implications and limitations of using a cost-effectiveness metric as an approach to rank projects.
dc.description.degree	Doctor of Philosophy (PhD)
dc.description.department	Industrial Engineering & Operations Research
dc.identifier.doi	https://doi.org/10.7275/26878620
dc.identifier.orcid	https://orcid.org/0000-0003-2449-2604
dc.identifier.uri	https://hdl.handle.net/20.500.14394/18760
dc.relation.url	https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=3506&context=dissertations_2&unstamped=1
dc.rights.uri	http://creativecommons.org/licenses/by/4.0/
dc.source.status	published
dc.subject	Sequential Decision-making
dc.subject	Reinforcement Learning
dc.subject	COVID-19
dc.subject	HIV
dc.subject	Pandemic Control
dc.subject	Markov Decision Process
dc.subject	Artificial Intelligence and Robotics
dc.subject	Disease Modeling
dc.subject	Dynamic Systems
dc.subject	Industrial Engineering
dc.subject	Operational Research
dc.subject	Other Public Health
dc.subject	Systems Engineering
dc.title	DECISION-ANALYTIC MODELS USING REINFORCEMENT LEARNING TO INFORM DYNAMIC SEQUENTIAL DECISIONS IN PUBLIC POLICY
dc.type	openaccess
dc.type	article
dc.type	dissertation
digcom.contributor.author	isAuthorOfPublication\|email:naz.khatami@gmail.com\|institution:University of Massachusetts Amherst\|KHATAMI, SEYEDEH NAZANIN
digcom.identifier	dissertations_2/2431
digcom.identifier.contextkey	26878620
digcom.identifier.submissionpath	dissertations_2/2431
dspace.entity.type	Publication