BFuss had a similar problem with his swinger. I had some pieces of polyester rod that I machined into a bushing to replace the bearings. But I think his needed a little more machining for it to fit properly. But just in case you are wondering, I am using a similar set to adapt my Banshee swingarm to my Zilla frame. Seems to be working good so far.

Dez i changed everything, now when you guys say replace it with a plastic bushing are you talking about completely replacing the bushing and bearing with a plastic one or making a plastic sleeve to make up the difference then reinstall the steel bushings and bearings?  And if i replace the steel OEM setup with a plastic one will it hold up ok? There seems to be a lot of torque at that point in the swinger.

Same here Dez. I have nylon a-arm bushings, but I'm not sure they would hold up to the torque of the motor.

If the eyes are out of shape you can always have them cut out and new ones welded in place. I have had it done on an oem swinger in the past and it's still holding up fine.