Accelerating AI-driven scientific discovery with end-to-end learning and random projection