Unless the bearing is bad don't mess with it. You would need a bearing puller which will damage the bearing getting it off.
I would start with just looking at the brushes, if the brushes and commutator look good then I doubt the bearings are worn at all.
Think about how much time the starter is actually spinning, it runs 1 or 2 seconds for each normal start. At 2 seconds per start, you get 30 starts to a minute of run time, or 1800 starts to get an hour of run time on the starter motor. That is a lot of starts to get one hour of wear, The bearings should be good for many hundreds of hours. The only thing that might take them out would be water in the motor.
by changing out good bearings you have a better chance of damaging the armature than you do of adding life to the motor.