Not sure which Patagonia down vest you actually purchased - I don't see an image in your post, and they make a few different ones. But you're correct that if the vest is too roomy, it won't be as efficient. Vests are great for moderate weather, or as a an additional layer in tandem with a jacket. I often carry a Patagonia Nano vest on trips - it's lightweight and easily packable, but it doesn't offer the warmth of a jacket.
I just picked up a
Mtn. Hardwear 'Ghost Whisperer' down jacket and I'm really impressed. 800-fill hydrophobic down and it's insanely light and packable for the amount of warmth it offers. It will be my 'go-to' layer this fall/winter for lots of adventures.