How will you travel with the passport if you've lost it?
If you lose your passport before you are able to use your PR Visa, you will then have to get a new passport, contact your VO advising you have lost your passport but now have a new one, likely fill out some forms, send in your new passport to get a new visa stamped, and if you're lucky, you'll get all this done before your Visa was to expire.
And obviously before you travel take photocopies of relevant passport pages (visas, information pages etc) either paper copies (leave copy in home country) or preferably digital so you can print off later should passport be lost.
End of day travel and life is full of 'what ifs' everyone worries too much about, go enjoy the travelling and as long as you have copies stashed somewhere separate you should be fine. Probably will never happen.