Papers »

Data-driven estimation of transition matrices in Markov Decision Processes for sustainable infrastructure maintenance

Given the poor conditions of the US infrastructure systems and the budget limits constraining ambitious repair plans, it is critical to optimize the maintenance and rehabilitation of these systems and identify the most effective policy. In doing so, it is equally critical to consider the life cycle implications of different policies. To identify the best policy given these conditions, a computational package can be used with two modules. First module, called dynamics module, describes the dynamics of the system of interest, and the second module, called optimization module, searches for the policy with the most desirable implication. The credibility of these two modules hinges on how accurately the uncertainties relevant to dynamics and implications are accounted. Markov Decision Processes (MDP) is one such framework, where the dynamics module is a Markov chain. The standard approach in using MDPs in sustainability studies is to estimate the Markov Chain parameters, i.e. transition matrices, in a deterministic and non-adaptive way. In this presentation, a new approach will be proposed where the transition matrix for the system of interest is considered to be random and its probabilistic model is updated using streaming data. The data to be used in this framework is the video feed from the pavement of roadways, which yields damage or health scores. It will be shown how incoming data drives the probabilistic models of these matrices and how it can even fundamentally inform the modelers about the dimensionality of the dynamics module. As a result, the framework will become more credible, due to the use of observation feed, and more computationally efficient, thanks to the model reduction task.

Author(s):

Hadi Meidani    
University of Illinois at Urbana-Champaign
United States

Negin Alemazkoor    
University of Illinois at Urbana-Champaign
United States