Scope causing hop rise?

Just wondering, has anyone else out their experienced their scope causing an illusion in which the BB rises, but actually isn't? I'll take a shot while looking through my scope, and the BB will rise out of the FoV a little bit. The next shot I will level out the rifle and shoot. The BB then appears to float strait for a solid hundred to a hundred and twenty five feet or so. The reason I think it may be causing this phenomenom is due to the small objective lense of 32mm and the 4x magnificationf the scope amplifying the effects. Any input or am I crazy?