I think some varieties of Rosemary are touted for hardiness, only to about freezing though so it's surprising they make it in Ottawa!
I've kept mine in the greenhouse as late as February one year, but then rescued it inside as temperatures were dropping lower. The mass that stores summer heat for the longer term is only enough for a mild winter after a hot summer, but we don't seem to be getting that combo much!
Other woody mediterranean herbs overwinter here very well with some dieback depending on how harsh a winter. I have thyme and sage plants that have kept going for 20+ years. Hyssop (which is the closest thing to an alternate for rosemary) is a bit less hardy and also seems intolerant of weeds or other competition so needs a bit more care... I had to replant once in the same time period.