“Email service providers pay attention to your bounces, too, and having a high bounce rate reflects poorly on your business. It’s good practice to comb through your list on a regular basis and remove any problem email addresses.”
Having no list is better than keeping a bad one. Email service providers keep track of things like bounce rates and spam reports. Get too many of either of these things and you could be denied service. Keep your list clean and in good working order.