Since the instructions say not to add the address, who cares what the canada post lady says?
However, I don't think it's an issue if you do. But I prefer following the cic instructions.
Because it's a passport in there, I think CIC needs to make sure there is no error in the address, that's all...