Hierarchical self-triggered strategy for unknown dynamics based on transfer learning and application to motor systems

Abstract

This paper introduces a novel hierarchical self-triggered control strategy for unknown dynamics based on transfer learning. The introduced methodology stratifies the self-triggered control strategy into upper and lower hierarchical layers which the upper-layer policy exercises oversight over the decision-making process of the lower-layer policy. Consequently, this hierarchical structure diminishes the search range for the lower-tier policy and enhances the efficacy of the learning procedure. Additionally, a parameter transfer fine-tuning method is developed to address initial value sensitivity in the policy network parameters of the learning process. Based on this setting, the shallow parameters are first frozen to achieve efficient reuse of prior knowledge of the pre-trained hierarchical network. Subsequently, the unfrozen deep parameters are fine-tuned to avoid policy failure caused by system parameter changes during the self-triggered control strategy learning for a new scenario. This proposed method eliminates the need to train the hierarchical Actor-Critic network from scratch, further reducing the time and computational resources required. Applied the developed method to a motor system demonstrates that 30% network training efficiency is improved.