A New Class of Learning Automata: The Probabilistically-Switch-Action-on-Failure Automaton Part 2 – Simulations
In this part of the two-part paper, simulations of Probabilistically-Switch-Action-on-Failure learning automaton (PSAFA) are presented in various stationary and non-stationary environments. The PSAFA is a novel fixed structure stochastic automaton (FSSA) framework, and its analytical model is presented in detail in Part 1 of this paper set. The key differentiating feature of this automaton is that it allows action switching in every state. We anticipate that this feature attributes PSAFA dynamic properties that make certain aspects of its performance superior to other FSSA that do not possess this property.
In this paper, simulations of the PSAFA in comparison with other FSSA are considered in two types of environments: a stationary environment (with fixed penalty probabilities) and a non-stationary environment, where the penalty probabilities are changing in time periodically as a sinusoidal function. In both cases the simulation demonstrates a dramatic difference in performance for these types of learning automata. The PSAFA shows its huge advantage in adaptability that leads to a better performance for the length of the simulation up to 30,000-150,000 steps. Only for very long stationary conditions Tsetlin automata outperforms PSAFA. In the case of sinusoidal modulations, the PSAFA tremendously outperforms other types of FSSA for all modulation frequencies and for all depths D>3. The performance of PSAFA does not deteriorate with increasing modulation frequency, while other FSSA are not resilient to that increase.