Abstract
This paper proposes a new method for interpreting and simplifying a
black box model of a deep random forest (RF) using a proposed rule
elimination. In deep RF, a large number of decision trees are connected
to multiple layers, thereby making an analysis difficult. It has a high
performance similar to that of a deep neural network (DNN), but achieves
a better generalizability. Therefore, in this study, we consider
quantifying the feature contributions and frequency of the fully trained
deep RF in the form of a decision rule set. The feature contributions
provide a basis for determining how features affect the decision process
in a rule set. Model simplification is achieved by eliminating
unnecessary rules by measuring the feature contributions. Consequently,
the simplified model has fewer parameters and rules than before.
Experiment results have shown that a feature contribution analysis
allows a black box model to be decomposed for quantitatively
interpreting a rule set. The proposed method was successfully applied to
various deep RF models and benchmark datasets while maintaining a robust
performance despite the elimination of a large number of rules.