Impressive engineering, but it comes with a curse.
Where it gets messy is you’d need to supply power on the PCIe rail as well as any extra plugs the GPU needs without powering on the system itself. And then even messier if someone powers on that system. Mad scientist shit.
The important bit is to power one on first before the swap, then you’ll have one setup where the pi was recently powered on and another setup where the connected devices were recently powered on. You might see the issue on only one of the devices, at which point you can say if it’s the pi being off for a while or the devices that triggers the issue.