Discrete-time Markov chains
A discrete-time Markov chain (DTMC) is observed at fixed steps \(n=0,1,2,\ldots\) and satisfies the Markov property.
Transition probabilities
\[p_{ij}=P(X_{n+1}=j\mid X_n=i).\]For each current state \(i\), the probabilities of all possible next states sum to one.
Evolution of a distribution
With a row probability vector \(\boldsymbol\pi_n\) and transition matrix \(P\),
\[\boldsymbol\pi_{n+1}=\boldsymbol\pi_nP.\]Trajectory versus distribution
A random draw selects one next state and generates one trajectory. Matrix propagation instead gives probabilities across all states.
Key idea. A DTMC combines fixed observation steps with probabilistic transitions determined by the current state.