Towards Computational And Sample Efficiency In Stochastic Optimization