Abstract: Stochastic processes (SPs) are widely used in many real-world fields, especially AI algorithms and models. A discrete-time Markov chain (DTMC) is a fundamental SP where the probability of ...