Definitions for "Markov chain"
A random process (Markov process) in which the probabilities of discrete states in a series depend only on the properties of the immediately preceding state or the next preceeding state, independent of the path by which the preceding state was reached. It differs from the more general Markov process in that the states of a Markov chain are discrete rather than continuous. Certain physical processes, such as diffusion of a molecule in a fluid, are modelled as a Markov chain. See also random walk.
A finite state machine with probabilities for each transition, that is, a probability that the next state is sj given that the current state is si.
Any multivariate probability density whose independence diagram is a chain.The variables are ordered, and each variable "depends" only on its neighbors in the sense of being conditionally independent of the others. Markov chains are an integral component of hidden Markov models.
a scan if the vertices of G are visited in an order specified by some permutation which must remain constant during each sweep
a unique kind of eigenvalue problem, because the largest eigenvalue is already known