tsta

Home / سی و دومین کنفرانس ملی و دهمین کنفرانس بین المللی مهندسی زیست پزشکی ایران

DDQN-Learning of Hill-Type Musculoskeletal Arm Model for Elbow Motor Control

Authors :

Mohammad-Reza Sayyed Noorani¹ Abbas Jafarpour Mahalleh² Kimiya Khojand³

1- University of Tabriz 2- University of Tabriz 3- University of Tabriz

Keywords :

Double Deep Q-learning،Elbow Flexion Control،Hill-Type Musculoskeletal Model،Muscle Activation،OpenSim

Abstract :

This study aimed to develop a model-based reinforcement learning (RL) framework designed to partially emulate central nervous system (CNS) learning processes for goal-directed motor control. The RL model, implemented using a double deep Q-learning (DDQN) algorithm, interacting with a biomechanical arm model served as the simulated environment. The environment comprised a Hill-type musculoskeletal representation of the biceps brachii and triceps brachii muscles, enabling elbow flexion–extension over a range of 10–135°. Within this setup, the RL agent received state information, including elbow joint angle and velocity, from the environment and generated muscle activation signals as control outputs. These signals acted on the Hill-based biomechanical model, allowing the agent to learn reaching toward specified target through iterative episodes. To validate biomechanical realism, forward dynamics simulations were performed in OpenSim using a customized arm model driven by the RL-generated excitations. Results demonstrated that the agent successfully acquired stable and biologically plausible motor strategies.