Ok, a little update...
I managed to get the bearing to the right spot on the shaft (or at least very close).
I have some old fence posts at the back of my property that are about the right diameter to fit onto the bearing. Ok, so property my be exaggerating a little, they are at the back of my yard..lol.
I took the shaft out there, initially to try and wack the bearing off. Of course the cross pin prevented that so I decided to flip the shaft over and give the bearing a wack and see if I could drive it down a little. Success!! It moved quite a bit.
So I tried it again...thunk...wait what was that....yep the top portion of the shaft dropped out of the shaft. Ah crap.
Anyway I managed to drive the bearing down to where it needs to be. The weird part is that the bearing really freed up and now turns with very little resistance. So I think I am going to leave it and go with it.
Then I cut down the fence post to retrieve the other half of the shaft.
No damage there. So now I can fix the shafts properly, so that's good.
So all in all, a good night.