Steam Deck Unplayable

Ordinarily I play on PS5 but since I was going to be with family for a couple of weeks, I bought the steam deck hoping to play POE 2. Big mistake.

I thought just lowering all the settings would at least let it be playable. And while I was in Matlan Waterways that did work. It didn't look good but it was playable. But when I got to the Apex of filth, the game just became a slide show. It would take 5-40 seconds for the pause menu to be displayed after a single button press. Switching between the direct x version and vulcan didn't seem to make a difference. Restarting the steam deck also didn't work.

When I was standing at the entrance to Apex of filth from the Drowned City, paused and looking at the Graphs, I can see the metric for cpu steadily rise to 80 ms. Inside the Apex of filth, just near the waystone, I can often see it at 180ms.

For reference I have every setting as low as I can get it. Playing with the resolution doesn't work. Even the lowest resolution isn't playable.