Consider the following 3-state Markov chain denoted p:
0.1000 0.4000 0.5000
0.3000 0.6000 0.1000
0.7000 0.2000 0.1000
The limiting probabilities (stationary vector) denoted by π satisfies the equation π = π*p. While the steps aren't shown here, π is calculated to be:
0.3269 0.4423 0.2308
In the long-run, 32.69% of the time will be spent in state 1, 44.23% in state 2, and 23.08% in the rest. In the subsequent codes, the stationary vector is denoted by s, since MATLAB cannot work with the symbol π as a variable name.
s*p
0.3269 0.4423 0.2308
This demonstrates that π*p indeed equals the original π stationary vector.
s*p(:,1)
0.3269
s*p(:,2)
0.4423
s*p(:,3)
0.2308
Each of the operations s*p(:,i) represents multiply the 1x3 π stationary vector by the 3x1 i-th column vector of p. Each of the operations yielded the i-th entry of π. This demonstrates the property that π_j = Σ π_i * p_ij, summed over i, where j represents the column number of the matrix p. That also translates to π * (j-th column of p). The physical interpretation of s*p(:,1) is the long-run proportion of transitions into state 1. That number equals s(1), which is the proportion of times spent in state 1. It makes sense that the proportion of transitions into a certain state is also the proportion of times spent in that same state.
