One where the Future is only dependent on the Present

What are Markov Chains and Steady-State Probabilities

All about Markov chains and steady-state probabilities.

In addition to proving the central limit theorem, Andrey Andreyevich Markov played a pivotal role in developing Markov chains. Markov chains are used to model discrete-time, discrete space random processes with applications across multiple domains including Finance, Advertising, NLP, SEO, Physics, etc. In this blog post, we shall discuss the in and outs of Markov Chains with a specific focus on computing the steady-state probabilities for a given Markov chain.

Recap:

Before we jump into the nitty-gritty of the Markov chain, let us take a moment to define the fundamental concepts of probability theory.

1. Random Variable:

• A Random variable is a variable whose outcome is dependent on a random phenomenon. For example, a coin flip or temperatures throughout the year.
• A continuous random variable is one that can take an infinite number of possible values. Temperatures throughout are year can take infinite values within a range and can be considered as a continuous random variable.
• A discrete random…