Discrete-Time Markov Chain
A Discrete-Time Markov Chain (DTMC) is a mathematical model that describes a system which transitions between a finite or countable number of states at discrete time steps. The key feature of a DTMC is that the future state depends only on the current state and not on the sequence of events that preceded it, a property known as the Markov property.
In a DTMC, each state has a set of probabilities that dictate the likelihood of moving to other states in the next time step. These probabilities are typically represented in a transition matrix, where each entry indicates the probability of transitioning from one state to another. This framework is widely used in various fields, including economics, computer science, and biology.