Dried plaster will come off with just damp micro fibre cloth. And a finger in between cloth and window.
Dried morter is a different story.......
Make sure the area to be cleaned is wet with water and fairy, this is important. Scrape gently with a sharp scraper and you will get no scratches on the glass.
As for masonry/Gloss paint on the frames use a little thinners, just a little mind.As it could discolur the frames.
Hope this helps.
Marcus