I am worried, if I do that and even if I dont send the letter, there might be questions either by CHC or by immigrations authorities at PoE and they may want to know, that once I am gone, how will I be supporting my family for their basic needs and other things e.g. kids schooling fee ??
And they may also want to know whether my spouse have any problem me leaving country or not ??
Don't worry, such matters will not be asked by CHC or at the POE. It is your personal problem.
As your agent suggested, send the COPRs and passports for cancellation of PR visa by CHC New Delhi. Because you need to apply again for their visas later.
If the officer at the POE ask about your dependents, tell your intention.