Leave it on until it annoys you too much
Wikipedia explains it like this
In computing, phishing is a criminal activity using social engineering techniques. Phishers attempt to fraudulently acquire sensitive information, such as passwords and credit card details, by masquerading as a trustworthy person or business in an electronic communication. Phishing is typically carried out using email or an instant message, although phone contact has been used as well[1]. Attempts to deal with the growing number of reported phishing incidents include legislation, user training, and technical measures.