The flowers and foliage determine when to best harvest your crop. Harvest baby potatoes (new potatoes) two to three weeks after they've finished flowering, and harvest potatoes for storing (mature potatoes) two to three weeks after the plant's foliage has died back.