Hi jasl,
This looks very similar to an issue I had. Are you using SWAG from LSIO by any chance?
My thread is here - the solution sped it up quite a lot but still not as fast as when I access n8n directly via the IP (bypassing SSL).
Let me know if you find a better solution!