Abstract

I present a large class of constrained non-cooperative stochasticMarkov games with countable state spaces and discounted cost criteria. In one-player case,i.e., constrained discounted Markov decision models, it is possible to formulate a static opttimisationproblem whose solution determines a stationary optimal strategy (alias controlor policy) in the dynamical infinite horizon model. This solution lies in the compact convexset of all occupation measures induced by strategies, defined on the set of state-action pairs.In case of $n$-person discounted games the occupation measures are induced by strategies ofall players. Therefore, it is difficult to generalise the approach for constrained discountedMarkov decision processes directly. It is not clear how to define the domain for the bestresponse correspondence whose fixed point induces a stationary equilibrium in the Markovgame. This domain should be the Cartesian product of compact convex setsin locally convex topological vector spaces. One of our main results shows how to overcome this difficultyand define a constrained non-cooperative static game.\ This is done for games with bounded costfunctions and positive initial state distribution. An extension to a class of Markov gameswith unbounded costs and arbitrary initial state distribution relies on approximation of theunbounded game by bounded ones with positive initial state distributions.In the case with a countably generated state space, we prove existenceof approximate stationary equilibria and stationary weak correlated equilibria.