I'd been considering changing our dryer vent to a better quality model that would prevent cold air from entering for a while now but kept putting it off. With the cold entering like that I couldn't put off any longer so I made a trip to my local Home Hardware store.
I'd seen energy efficient vents before at Home Hardware and had a particular model in mind. Strangely I couldn't find any energy efficient models at either Canadian Tire or Walmart.
There were only two models at the store but I picked a model by Broan. The other vent was not only more expensive but came across as a gimmicky "as seen on TV" product. Broan is a well known brand and it looked more like a regular vent only larger.
Basically instead of vent flaps that open and close it has a styrofoam ball that rolls forward and up when hot air is exiting. It rolls back and seals the vent when the dryer stops. The vent cover is also well insulated with the walls made of 2 inch thick foam.
By replacing the vent I was also able to see what was inside my exterior walls: straw. My house was built in the 1930 and straw was a common insulator back then but unfortunately it doesn't insulate very well. If we weren't planning on moving in less than 5 years I'd remove the interior drywall and put in spray insulation with a high R-Value but it probably isn't worth the effort.