It sounds like the rubber has perished, and best left well alone, if when you detail, does your cloth get covered in black?
you can spend hours on them and you will keep getting covered in black rubber.
You should be able to wipe it easily from the frame when wet, if it has now dried on, try a little TFR, once the frame is clean, strear clear of the rubber.
Someone might come back with another answer, but this is what I do when I come across perished rubber.