energy gain and weapon sets

why can I not gain energy (cast on X/ invocation) with both weapon sets and cast the spell with just 1 weapon set? is this intended?

as an invoker spells cost a lot of many but deal no damage, or am I doing something wrong?

if i could gain energy with a quarterstaff and cast the spell with a magic-staff it would actually be good