Temporal Rift + Support Gems for skills "you use"

Temporal Rift is clearly a skill that I use, since you have to activate it to teleport to your oldest image.
You are unable to socket it with support gems that modify skills "you use" such as Unbreakable to provide more stun threshold while casting this unmodifiable 0.50 second cast time.