Down jacket with gathered detail