I have found when installing rubber parts (i.e. grips etc.), I use gasoline on the interior of the rubber part. The rubber parts slip right on and when the gas evaporates the rubber part adheres to the intended other part.
You might also try heat shrink tubing. let it extend an inch or so past the end of the metal, then trim it after you shrink it. you can cut it off with a razor knife when you want to remove or replace it.