Reduced-Complexity Modeling Of Multi-Scale Problems In The Low Data Regime