TY - JOUR

T1 - Dynamic programming subject to total variation distance ambiguity

AU - Tzortzis, Ioannis

AU - Charalambous, Charalambos D.

AU - Charalambous, Themistoklis

PY - 2015

Y1 - 2015

N2 - The aim of this paper is to address optimality of stochastic control strategies via dynamic programming subject to total variation distance ambiguity on the conditional distribution of the controlled process. We formulate the stochastic control problem using minimax theory, in which the control minimizes the payoff while the conditional distribution, from the total variation distance set, maximizes it. First, we investigate the maximization of a linear functional on the space of probability measures on abstract spaces, among those probability measures which are within a total variation distance from a nominal probability measure, and then we give the maximizing probability measure in closed form. Second, we utilize the solution of the maximization to solve minimax stochastic control with deterministic control strategies, under a Markovian and a non-Markovian assumption, on the conditional distributions of the controlled process. The results of this part include (1) minimax optimization subject to total variation distance ambiguity constraint; (2) new dynamic programming recursions, which involve the oscillator seminorm of the value function, in addition to the standard terms; and (3) a new infinite horizon discounted dynamic programming equation, the associated contractive property, and a new policy iteration algorithm. Finally, we provide illustrative examples for both the finite and infinite horizon cases. For the infinite horizon case, we invoke the new policy iteration algorithm to compute the optimal strategies.

AB - The aim of this paper is to address optimality of stochastic control strategies via dynamic programming subject to total variation distance ambiguity on the conditional distribution of the controlled process. We formulate the stochastic control problem using minimax theory, in which the control minimizes the payoff while the conditional distribution, from the total variation distance set, maximizes it. First, we investigate the maximization of a linear functional on the space of probability measures on abstract spaces, among those probability measures which are within a total variation distance from a nominal probability measure, and then we give the maximizing probability measure in closed form. Second, we utilize the solution of the maximization to solve minimax stochastic control with deterministic control strategies, under a Markovian and a non-Markovian assumption, on the conditional distributions of the controlled process. The results of this part include (1) minimax optimization subject to total variation distance ambiguity constraint; (2) new dynamic programming recursions, which involve the oscillator seminorm of the value function, in addition to the standard terms; and (3) a new infinite horizon discounted dynamic programming equation, the associated contractive property, and a new policy iteration algorithm. Finally, we provide illustrative examples for both the finite and infinite horizon cases. For the infinite horizon case, we invoke the new policy iteration algorithm to compute the optimal strategies.

KW - Dynamic programming

KW - Minimax

KW - Stochastic control

KW - Total variational distance

UR - http://www.scopus.com/inward/record.url?scp=84940707638&partnerID=8YFLogxK

U2 - 10.1137/140955707

DO - 10.1137/140955707

M3 - Article

AN - SCOPUS:84940707638

VL - 53

SP - 2040

EP - 2075

JO - SIAM Journal on Control and Optimization

JF - SIAM Journal on Control and Optimization

SN - 0363-0129

IS - 4

ER -