Are you playing docked or portable? Have you upgraded the game from version 1.0 to the latest?

There were several updates that have significantly improved the performance in terms of frame rates. I have not updated mine since they haven't bothered me enough to care. I'm not sure I've noticed when swapping weapons, but it can definitely happen when using the slate zoom, or in areas with lots of grass, water, or other effects. It's a lot less noticeable in portable mode, but still happens.