We consider a classical finite horizon optimal control problem for continuous-time pure jump Markov processes described by means of a rate transition measure depending on a control parameter and controlled by a feedback law. For this class of problems the value function can often be described as the unique solution to the corresponding Hamilton–Jacobi-Bellman equation. We prove a probabilistic representation for the value function, known as nonlinear Feynman–Kac formula. It relates the value function with a backward stochastic differential equation (BSDE) driven by a random measure and with a sign constraint on its martingale part. We also prove existence and uniqueness results for this class of constrained BSDEs. The connection of the control problem with the constrained BSDE uses a control randomization method recently developed by several authors. This approach also allows to prove that the value function of the original non-dominated control problem coincides with the value function of an auxiliary dominated control problem, expressed in terms of equivalent changes of probability measures.

Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes / E. Bandini, M. Fuhrman. - In: STOCHASTIC PROCESSES AND THEIR APPLICATIONS. - ISSN 0304-4149. - 127:5(2017 May), pp. 1441-1474. [10.1016/j.spa.2016.08.005]

Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

M. Fuhrman
2017

Abstract

We consider a classical finite horizon optimal control problem for continuous-time pure jump Markov processes described by means of a rate transition measure depending on a control parameter and controlled by a feedback law. For this class of problems the value function can often be described as the unique solution to the corresponding Hamilton–Jacobi-Bellman equation. We prove a probabilistic representation for the value function, known as nonlinear Feynman–Kac formula. It relates the value function with a backward stochastic differential equation (BSDE) driven by a random measure and with a sign constraint on its martingale part. We also prove existence and uniqueness results for this class of constrained BSDEs. The connection of the control problem with the constrained BSDE uses a control randomization method recently developed by several authors. This approach also allows to prove that the value function of the original non-dominated control problem coincides with the value function of an auxiliary dominated control problem, expressed in terms of equivalent changes of probability measures.
backward stochastic differential equations; marked point processes; optimal control problems; pure jump markov processes; randomization; statistics and probability; modeling and simulation; applied mathematics
Settore MAT/06 - Probabilita' e Statistica Matematica
mag-2017
Article (author)
File in questo prodotto:
File Dimensione Formato  
SPA articolo pubblicato.pdf

accesso riservato

Tipologia: Publisher's version/PDF
Dimensione 438.41 kB
Formato Adobe PDF
438.41 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
FNL_Markov_REVISED.pdf

accesso riservato

Tipologia: Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Dimensione 396.5 kB
Formato Adobe PDF
396.5 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/544097
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 14
  • ???jsp.display-item.citation.isi??? 14
social impact