Core-Collapse Supernova Simulations With Spectral Two-Moment Neutrino Transport