Significant frame drop after alt-tabbing out and tabbing back in

Often after alt-tabbing out of and tabbing back into the game, my FPS suddenly drops below 10, essentially making the game unplayable. Restarting the game obviously fixes it, but having to do this multiple times in one hour is quite frustrating.