Surges are removed on weaponswap

Surges tooltip says "surges are specific to your current weapon". But even if I have a single weapon bound to both weapon sets, swapping a set still removes the surges. So I can't use same weapon with different trees, even though it should be possible logically.