RESEARCH INSIGHT

Conventional HRC treats humans as reactive environment components via predefined scripts, limiting coordination to finite interaction patterns. Such single-agent formulations or imitation learning fail to generalize to non-stationary human behaviors. Therefore, the transition to co-adaptation is imperative for handling latent human intentions. This work circumvents this by replacing scripts with learning-capable humanoid agents, and using MARL to force the robot to internalize a broader distribution of coordination patterns.

We design a stability-aware control law that rectifies decentralized gradients to satisfy a convergence certificate. The learning dynamics are governed by the interaction between local agent intentions and the global team objective. We formalize this interaction via two competing vector fields: the independent rationality field (uind) and the team rationality field (uteam). The rationality gap V(θ) is defined as the discrepancy between these two fields. Our control objective is to design an update d enforcing (∇θV, d) ≤ −σV(θ), ensuring exponential decay of the rationality gap and stabilizing decentralized learning.

HALyPO transforms a potentially oscillatory decentralized learning process into a dissipative dynamical system. Under regularity and smoothness assumptions, let {θk}k=0 be the sequence of parameters generated by the HALyPO update law. If the learning rate η satisfies the stability bound, then the rationality gap V(θ) is monotonically non-increasing. Beyond local stability, we establish that HALyPO drives the multi-agent system toward a state of rationality agreement, with the sequence of disagreement energies converging to zero asymptotically.

We demonstrate the performance superiority of HALyPO across physical coupling tasks including Orientation-sensitive pushing (OSP), Spatially-confined transport (SCT), and Super-long object handling (SLH). The training is performed in the Isaac Lab, and physical experiments are conducted on a Unitree G1 robot cooperating with a human partner with reliance on a motion-capture system. HALyPO is evaluated against state-of-the-art heterogeneous MARL methods: HAPPO, HATRPO, and PCGrad.

HALyPO addresses inherent structural instabilities in decentralized human-robot collaboration by establishing MARL as a unified paradigm for exploring expansive interaction manifolds. It defines RG as a variational mismatch between decentralized best-response dynamics and a centralized cooperative ascent direction, reformulating the learning process as a dissipative dynamical system. HALyPO introduces a formal stability certificate within the policy-parameter manifold, utilizing an optimal quadratic projection to rectify decentralized gradients and ensure the monotonic contraction of coordination disagreement.