A New Class of Learning Automata: The Probabilistically-Switch-Action-on-Failure Automaton Part 1 – Analytical Models

We present the novel concept of Probabilistically-Switch-Action-on-Failure learning automaton (PSAFA). The PSAFA is a fixed structure stochastic automaton (FSSA), characterized by a fan-shaped state transition diagram where each branch of the state space is a chain of D states, and is associated with a particular action. The first states of all chains form a circle of initial states. The PSAFA can switch from a present state in any chain to the initial state of the next chain in the circle, on each failure, with some finite probability. This probability, which plays the role of an action- switching probability, is a function of the distance of the present state from initial state of its branch. The learning behavior of PSAFA is determined by the dependence of the action switching probability on the distance from the initial state.

The probabilistic action-switching capability distinguishes PSAFA from conventional FSSA that have deterministic action selection at each state, and only some states transit to states with a different Probabilistically-switch-action-on-failure Automaton action. This action-switching capability at any time is also typical for conventional variable structure stochastic automata (VSSA) but it comes with added computational complexity. VSSA are more adaptive than FSSA in non-stationary environments because of this action-switching capability. We believe that the addition of this capability should also make the PSAFA more adaptive in non-stationary environments than classical FSSA while preserving the simplified computational complexity of FSSA.

The effectiveness of the proposed framework is demonstrated through the theoretical analysis of optimality of the PSAF learning automaton in stationary environments in part 1 of this 2-part paper.