Read through the information about common-law qualification in Sections 5.34 - 5.36 (especially 5.36) and of the
OP2 Processing Manual. It states that once the qualification period has been reached, short periods of separation are permissible - as long as the intention is to resume the cohabitation as soon as possible. Being separated during the entire process, though, even because of him being refused re-entry to the States, might throw a monkey wrench into things for you. The longer you are apart, the harder it is for officers to be sure you actually intend to continue the relationship once you arrive in Canada. So I don't really agree with missmini when she says, "Don't worry".
I had an opposite problem - even though we are married, I knew that if I left Canada to visit my family at home in the States, I would be refused re-entry to Canada. In our case it wasn't about how that would ultimately affect the processing of our application - because our marriage was what made me eligible to be sponsored - but I still made the very hard decision NOT to leave because we didn't want to risk being separated. You have that - but you also have the added concern that a separation could affect your qualification. My advice: he should stay put and stay under the radar. It is very likely that he won't be allowed back into the States if he has been "visiting" frequently and for long duration - in fact, living with you - and they have already harassed him about it. He could even end up being banned from visiting the US for a period of 10 years! Also, conjugal partner applications do not usually "fly" between US/Canada couples. Even if he was banned from the USA, as long as you were not prevented from entering Canada to live with him, you'd be expected to meet and maintain the co-habitation requirement, or get married, in order to qualify to immigrate. Conjugal sponsorships are intended to waive the cohabitation requirement when a couple cannot marry or cohabitate due to fear of persecution because of their relationship, or because they are prevented from marrying (like a same-sex couple who cannot marry in the foreign national's country because it's not legal, and the FN cannot get a visa to enter Canada to get married here). There are no immigration barriers (to speak of) between the US and Canada, and there is nothing prohibiting you from marrying - so you'd be expected to do that if you could no longer meet the cohabitation requirement.