HEAD: No matter how cold it is, I always go with ear warmers over a full hat. This is personal preference, but I always overheat if my head is fully covered. The ear warmers I use are Nike brand. You can't really go wrong with ear warmers- go with ones that have a fleece lining and are a little wider in the middle to be sure they cover your ears. I go with ear warmers if the temp is at or below 25F- but they are easy to take off and stuff in a pocket if you get too warm. When running at night, I go with a Sigma headlamp. It's one of the cheaper ones, and is quite bulky but safety first! If you have the $ to splurge I would get a light-weight headlamp from Black Diamond.
HANDS: With temps at or below 25F, I go with my Brooks mittens. There is a fleece-lined mitten inside of the weather-resistant outer mitten lining. My hands stay warm even in temps as low as 10F in these. If temps are between 25-40F, I go with Saucony gloves. They are light but do the trick and hold up well.
FEET: For cold temps, I go with a crew sock for a bit more coverage. My favorite winter sock are Balega socks with a blend of merino wool. If you have unsure footing on snow/ice, there are a number of additions you can put on your shoe to increase traction. If you go to your local running store, they can help you find the best kind for you. I go with ones that have mostly rubber traction with 6 spikes. The type and brand can vary greatly depending on personal preference and where you live (i.e.- type or terrain you run on).
BASE LAYERS: I have accumulated quite the collection of base layers over the years. Two of my favorites are a Craft base layer and a 1/2 zip fleece-lined North Face base layer. If temps are upper 30s-50F, I will go with only a base layer (no jacket). The key for choosing a base layer is to make sure the material is moisture wicking. If temps are colder, fleece-lining can give you a bit more added warmth and insulation.
JACKET: I have only 1 running jacket. I run anywhere from 1-4x/week and my Craft jacket has been with me since 2008! It's still in good shape. I can't say enough good things about Craft products (and not just because I'm Swedish). They are pricey but well worth it because of the return on quality of material and longevity of the product. As far as jackets, look for something that is relatively water-resistant (if its snowing, you don't want the jacket to soak up the moisture). My jacket is good at keeping out wind and water, and has lots of good pockets. If you have a good base layer on, you don't need the jacket to be super warm. I wear a base layer + jacket in temps from 10F-35F.
LEGS: If temps are 35-50F I go with capris (favorite brands include adidas, saucony, lululemon, athleta). In temps below 35, I go with a full leg tight. Look for tights with a fleece lining, moisture-wicking outer material, small zip pockets, and most importantly- a drawstring! My favorite full-length running tight are a pair of Mizunos. They have lasted 5+ years, and stand really well in the wash.
Collecting good running gear can be pricey, but once you have a 1-2 outfits they last for years!