Ah... there is no such thing as "work visa". There is a temp resident visa AND there is a work permit. Visa is stamped on passport (although the stamp also shows whether your status in canada is visitor, student or worker) , work permit is a separate letter which people OFTEN staple with their passports BUT it's not a travel document. It states conditions under which you can work in Canada.
I am not questioning your situation. May be there are additional details involved of some sort.
But the above is absolutely the documentary procedure. Please be very clear about it. I don't want others to read and "assume they can travel on work permits"