Yes it's a concern for many.
Just to understand, how much you are showing for POF?
Usually, visa officer will check the amount shown against the exchange rate and validate your eligibility at the time of your submission.
This eligibility test is the first step of your processing, where you have to show minimum balance required for CIC.
When you pass that stage, it won't matter if your minimum balance decrease at later stages.
There are negligible instances where the visa is rejected citing POF. Visa officer may ask your updated bank statement at the time of processing your file.
If possible, try to deposit amount to balance minimum threshold but never withdraw amount frequently as it will raise red flags.