The webpage
http://www.cic.gc.ca/english/information/applications/guides/5445EA.aspclearly indicates that you can count the time you two spent together.
In my experience, I found that consulates, and consulate staff, don't always know everything (rules change, people don't get informed, or people just give you the standard answer, etc.). I suggest you contact the consulate with a copy of this webpage, just to make sure.