Any automobile paint supply shop will sell you some primer for plastic parts in a spray can.  You don't want to sand it too fine or the paint will not grab as well.
When paint peels in big chucks its because they didn't use plastic primer.