Here's a quick test video with a laser sight at point blank to show the difference between where the barrel of the gun is aiming and where the ADS snaps to.
It seems like the ADS snaps to the movement of the character and not where you're actually aiming... This to me is what is causing my issues since the ADS is literally random at any time. At point blank, it's not a big deal but if you're trying to shoot at someone mid-range this difference can be very drastic.