On approximating the stationary distribution of time-reversible Markov chains

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require O(t/p(v)) operations to approximate the probability p(v) of a state v in a chain with mixing time t, and even the best available techniques still have complexity O(t1.5/p(v)0.5); and since these complexities depend inversely on p(v), they can grow beyond any bound in the size of the chain or in its mixing time. In this paper we show that, for time-reversible Markov chains, there exists a simple randomized approximation algorithm that breaks this “small-p(v) barrier”.