Sometimes you have to take these losses on the chin and not take it too personally, everyone loses customers for various reasons, its just the nature of the job, dealing with the public.
I would say to look after your customers, as some of them will pass you onto other friends and family, the more you have the more that tends to happen.
I would also add that you should be getting the most out of the customers you have, what I mean by that is that you dont just do the windows, you clean the conservatory roof, fascia boards, gutters, clean the gutters and let them know what you can offer, ask for referrals to any friends and family, offer a free clean to those that pass you on etc etc.
You have to offer more than just window cleaning these days to keep the customers loyal to you I feel...
Only my opinion but your losing money (and possibly customers - unless you already have a really well established round) by not offering these services.
Steve