You can try shimming, I had to shim a set of bearings by 1mm, tried using loctite, silicone, etc but nothing held, so tried some $2 super glue, and presto, it held. I'd try cutting some coke cans , and using that. Even if you have to use a few, bearings have to be a pretty firm fit.