ancestral cry losing buff when swaping weapon set

if you swap your weapon set you instantly lose your ancestral cry buff. I assume that it shouldn't happen if there are no statements that it should work like this.
Why this is an issue? You can't make a build even using same weapon for 2 weapon sets which utilizes something else without losing this buff. I am not sure if it should work like this. Does anybody have same thoughts about this?
I think weapon buffs should translate to another weapon set if the weapon type used is the same or it's the same weapon type which is required for that buff.