Towards A Safe And Responsive Control Framework For Human-Centered Robots