Using canned poly. I noticed after I successfully rubbed the previous coat & washed it. Later what looked like a spec of dust, but it wouldn't come off at all. I had to rub it out with more coarser sand paper, the surface seemed brittle. So after all this why do this surface problems happen?