Continuous-time Markov chains
A continuous-time Markov chain (CTMC) evolves in continuous time but usually has a discrete state space. Events occur at random times.
Transition rates
If the rate from state \(i\) to state \(j\) is \(q_{ij}\), then for a sufficiently small interval \(\Delta t\),
\[P(i\to j\text{ in }\Delta t)=q_{ij}\Delta t+o(\Delta t),\qquad i\ne j.\]Waiting for an event
If the total rate of leaving state \(i\) is \(a_i=\sum_{j\ne i}q_{ij}\), the waiting time to the next event is exponential with rate \(a_i\).
Key idea. In a CTMC the model specifies event rates; these determine both random event times and which transition occurs.