It's not worth to take the risk if it's not necessary for you to leave Canada and I don't think the government/CIC will provide you any document to "promise" that you'll be allowed to come back
Besides the visitor record, do you have another small one called "Canada Visa" ( you might not have it if you don't need to apply visa for visiting Canada). If you have it, please check the left up where showed "number of entries" if it showed "multiple" then it means you can come back again "without" applying a new visa but it doesn't mean you'll be allowed to entry Canada when you arrive.
Actually I had called CIC about this question at the end of 2007 when I held a student visa and the lady asked me if I had a multiple entries visa, I said yes. Then she said "she didn't see there is any reason why I can't come back, as long as I have a good reason." I asked her about the "not allowed re-entry", she said it only means that people "may not allow to come in Canada again for some reasons, even though people have a valid visa."
And then I found out it all depends on who your custom officer will be when you arrive Canada. I mean, I had student visa for 3 years and it also showed "not allowed re-entry.....", the same thing as your visitor record but I did go home several times ( at least once a year). Every time I came back Canada, the officer asked me the same question...."where are you from" me- "Taiwan" then he/she asked "why do you go there?" me-"Go home for vacation/New Year"..... and that's it. Sometimes they asked me "what school are you attending now?" or "what are you studying now?"
So, since you mentioned you have "working permit", I assume you have a job now, then I think you can go home if you really want. Because when you come back, you just need to tell the officer that you come back to work. It will be a better reason for you to re-entry. Please prepare some supporting documents with you to prove you're working in Canada just in case.
However, please understand, I just want to share my personal experience with you but I can't promise what will happen to you. Good luck