If you have enough points without it, you should be fine, you can see at
http://www.cic.gc.ca/english/information/applications/guides/EG74.asp that you would need to have score of 4.0 or higher on IELTS to get some points. If you have a score of 4.0 to 4.9 on only 1 category, you could get 1 point. If you get between 4.0 and 4.9 in every category, you could get 4 points.