Problem with OAuth on Outlook

LukasFrey · July 12, 2023, 7:27am

When trying to “activate” an OAuth credential, I always get an error 500 without any further details.

I activated some logging to identify the issue, here what I found so far:

the request to “Connect the account” seems to work as expected, I am seeing the corresponding request in the access logs
GET /rest/oauth2-credential/auth?id=6&clientId=bc2db3e1-11ca-4941-aeb4-fcaf54897a20&clientSecret=__n8n_BLANK_VALUE_e5362baf-c777-4d57-a609-6eaf1f9e87f6&csrfSecret=12ah81g1bPj5diGFHtAyFZ4e&grantType=authorizationCode&authUrl=https:%2F%2Flogin.microsoftonline.com%2Fcommon%2Foauth2%2Fv2.0%2Fauthorize&accessTokenUrl=https:%2F%2Flogin.microsoftonline.com%2Fcommon%2Foauth2%2Fv2.0%2Ftoken&scope=openid+offline_access+Mail.ReadWrite+Mail.ReadWrite.Shared+Mail.Send+Mail.Send.Shared+MailboxSettings.Read&authQueryParameters=response_mode%3Dquery&authentication=body&useShared=false&userPrincipalName=
the pop-up on login.microsoft.com is openend, and I get asked to grant the requested permissions.
the callback URL is then requested on my n8n instance, but then I only see the error above.
"GET /rest/oauth2-credential/callback?code=0.ATEAEnNYbTPEaE6VaN16NF6UzeGzLbzKEUFJrrT8r1SJeiAxAAA.AgABAAIAAAD--DLA3VO7QrddgJg7WevrAgDs_wUA9P9aMOOg5KH2dDuyKlygOBnJYvQuC8gD0ttnuOEgOCrGO_CJNql7xo_jydQhocziXx0HmZWQwIq-Ek6VStMWoExLIb22LBnxBuLv3UpSDTarHFFfx0otYREfQhij89Vk2GHLUrx8H9CTvjnAAh5xEIPLaMSYzfeEw2EGqyGVmv4QQ1q7Pias3PSnnRabJjzIb1W2e2fDPtJ-_P7RsVTp4Mdt-ijwT65I3cDMY7YwSDMtjCOuGye5Wqf3RPJbiHPHKShrvAumMlRwmWywTISmmWVW6nOIP34u2HJ-i2_YQYTwrDSzwRb9WJkyjS5wRErEio2GFuoLqzNSoD57k_XrxFI2r7043ex0s61m7xBX1JBoAGaV7tbhpgji3qi0kXjBGdiIEaQ8UGOXnMuAfc5NS8qXz921xxXXOtUtuPoHO7MFFiEhDx7HiDGmd6hW_90Xt1BKW1ryxvHzw1teiHPQHeA3JauLam44p0kxwcWsbA63d44Y4K3Uwbhf3zGlnRJIuM1npi6vSDKgD-6XUpcfjjsRJnbp9OhS1niEOWxubxBchUDyjUYuUtnPgllsTakY1hWVUpGnkxXFlrAtTmVMvYmavwxD7HGR0Ak4icYGjRVg_XFluKTT3R5PowXwR1UPVR7l9o-K5VQhOTlF1wI-NRHK-wJRE9YbtbqPhCmkO7gq9gY5EfT8Z1iZGtKNyRLnB4tq-cl1XwfA1z0VCKeNLJZsy1c8A_ufKREQWS9AtFLt4iui_tAqv7DfnpFlwYDHrIF6c7Vue0TUXCLTi0NjiJJ_piskIO_4kTptAAomzjMDx6wp1_g&state=eyJ0b2tlbiI6Ik5yeG1YTWlPLXNNYVo3U1FjSEVjandjMjZrMnplZlRDUVNqYyIsImNpZCI6IjYifQ%3D%3D&session_state=8e40232b-8a60-4867-ae07-a2b279986dec

I am using the following setup:

n8n version: 0.234.1
Database: postgres version 11
n8n EXECUTIONS_PROCESS setting: default
Running n8n via: Docker
Operating system: Linux (TurnKeyLinux, Debian derivate)

Is there a way to get more detailed logs, especially on the OAuth part.

I have the impression that the callback is ok, but that it is somehow not properly processed within n8n. The Microsoft account shows a newly bound client, so that side seems to be “happy” with the whole process.

MutedJam · July 13, 2023, 10:48am

Hi @LukasFrey, I am very sorry you are having trouble. When testing the OAuth auth flow on my own n8n instance against MS365, I was able to see the full error with the default log level settings:

This was using n8n v0.236.0. Perhaps you can try upgrading your current version of n8n and verify whether the issue still persists? If so, you could try increasing the log level and take a peek at the server logs, though OAuth logging in n8n is unfortunately rather limited.

Also, can you confirm which credential type exactly you are using? Are these n8n’s pre-configured OAuth2 credentials for Microsoft Outlook or different credentials such as generic OAuth2 credentials (like here)?

LukasFrey · July 17, 2023, 9:53am

Hi @MutedJam
Thank you very much for you reply.

I updated the instance to 0.236.0, and the behaviour is still the same.
We are using the preconfigured OAuth2 Tokens for Microsoft (Outlook) Service.

After setting the log level to DEBUG, and additionally setting DEBUG=* in the environment, I can see a lot of log entries from the express library.

It seems to process the callback URL, and I was also able to find an entry that looks like if n8n is executing an outgoing request to Microsoft to fetch the oauth tokens.

n8n-n8n-1       | 2023-07-17T09:31:59.241Z follow-redirects options {
n8n-n8n-1       |   maxRedirects: 21,
n8n-n8n-1       |   maxBodyLength: 10485760,
n8n-n8n-1       |   protocol: 'http:',
n8n-n8n-1       |   path: 'https://login.microsoftonline.com/common/oauth2/v2.0/token',
n8n-n8n-1       |   method: 'POST',
n8n-n8n-1       |   headers: {
n8n-n8n-1       |     Accept: 'application/json, application/x-www-form-urlencoded',
n8n-n8n-1       |     'Content-Type': 'application/x-www-form-urlencoded',
n8n-n8n-1       |     'User-Agent': 'axios/0.21.4',
n8n-n8n-1       |     'Content-Length': 1138,
n8n-n8n-1       |     host: 'login.microsoftonline.com',
n8n-n8n-1       |     'Proxy-Authorization': 'Basic xxxxxxxxxxxxx'
n8n-n8n-1       |   },
n8n-n8n-1       |   agent: undefined,
n8n-n8n-1       |   agents: { http: undefined, https: undefined },
n8n-n8n-1       |   auth: undefined,
n8n-n8n-1       |   hostname: 'proxy.szh.loc',
n8n-n8n-1       |   port: '8080',
n8n-n8n-1       |   host: 'proxy.szh.loc',
n8n-n8n-1       |   beforeRedirect: [Function: beforeRedirect],
n8n-n8n-1       |   nativeProtocols: {
n8n-n8n-1       |     'http:': {
n8n-n8n-1       |       _connectionListener: [Function: connectionListener],
n8n-n8n-1       |       METHODS: [Array],
n8n-n8n-1       |       STATUS_CODES: [Object],
n8n-n8n-1       |       Agent: [Function],
n8n-n8n-1       |       ClientRequest: [Function: ClientRequest],
n8n-n8n-1       |       IncomingMessage: [Function: IncomingMessage],
n8n-n8n-1       |       OutgoingMessage: [Function: OutgoingMessage],
n8n-n8n-1       |       Server: [Function: Server],
n8n-n8n-1       |       ServerResponse: [Function: ServerResponse],
n8n-n8n-1       |       createServer: [Function: createServer],
n8n-n8n-1       |       validateHeaderName: [Function: __node_internal_],
n8n-n8n-1       |       validateHeaderValue: [Function: __node_internal_],
n8n-n8n-1       |       get: [Function: get],
n8n-n8n-1       |       request: [Function: request],
n8n-n8n-1       |       setMaxIdleHTTPParsers: [Function: setMaxIdleHTTPParsers],
n8n-n8n-1       |       maxHeaderSize: [Getter],
n8n-n8n-1       |       globalAgent: [Getter/Setter]
n8n-n8n-1       |     },
n8n-n8n-1       |     'https:': {
n8n-n8n-1       |       Agent: [Function: Agent],
n8n-n8n-1       |       globalAgent: [Agent],
n8n-n8n-1       |       Server: [Function: Server],
n8n-n8n-1       |       createServer: [Function: createServer],
n8n-n8n-1       |       get: [Function: get],
n8n-n8n-1       |       request: [Function: request]
n8n-n8n-1       |     }
n8n-n8n-1       |   }
n8n-n8n-1       | }
n8n-n8n-1       | Mon, 17 Jul 2023 09:31:59 GMT express:view lookup "oauth-error-callback.handlebars"
n8n-n8n-1       | Mon, 17 Jul 2023 09:31:59 GMT express:view stat "/usr/local/lib/node_modules/n8n/templates/oauth-error-callback.handlebars"
n8n-n8n-1       | Mon, 17 Jul 2023 09:31:59 GMT express:view render "/usr/local/lib/node_modules/n8n/templates/oauth-error-callback.handlebars"
n8n-n8n-1       | Mon, 17 Jul 2023 09:31:59 GMT compression no compression: size below threshold

In our environment, I am required to use an outgoing proxy to access the Internet. From my earlier tests, I still use the setting N8N_USE_DEPRECATED_REQUEST_LIB=true as I read that this would be the only way to get proper proxy support.

Is it possible that this setup is currently not/no longer working?
Is the information about proxy support still valid?

MutedJam · July 17, 2023, 10:07am

Hi @LukasFrey, any chance you can test this against an n8n instance running outside of your current environment without a proxy? This is to verify whether it is indeed your specific setup using N8N_USE_DEPRECATED_REQUEST_LIB=true and your proxy that is causing the problem here.

In general, the current library used by n8n for HTTP requests (Axios) should support proxies. @Jon tested this recently over here, so it might be worth a shot removing this option (it will also become unavailable with the next update). Just make sure you have set http_proxy and https_proxy instead.

If this still fails for you, can you confirm which proxy exactly you are using and how its configured, so I can verify whether this breaks OAuth2 for me as well?

LukasFrey · July 17, 2023, 2:21pm

Hi @MutedJam

I did some more testing.

Removing the N8N_USER_DEPRECATED_REQUEST_LIB setting does not change anything, I have the same behaviour with and without the setting. This also means that proxy support seems to be there (although unclear whether it is working).
I did some tests on my local installation that does not require the use of a proxy. This setup works as expected. The only difference remaining seems the proxy stuff.

When working on my setup, I am using a free outlook.com account. The other setup is using a Azure AD integrated account. Maybe this makes some difference. I have scheduled a meeting for tomorrow to test with another guy as I do not have access to the required AAD credentials myself.

The request to Microsoft on my installation looks like this:

n8n-n8n-1       | 2023-07-17T12:58:43.612Z follow-redirects options {
n8n-n8n-1       |   maxRedirects: 21,
n8n-n8n-1       |   maxBodyLength: 10485760,
n8n-n8n-1       |   protocol: 'https:',
n8n-n8n-1       |   path: '/common/oauth2/v2.0/token',
n8n-n8n-1       |   method: 'POST',
n8n-n8n-1       |   headers: {
n8n-n8n-1       |     Accept: 'application/json, application/x-www-form-urlencoded',
n8n-n8n-1       |     'Content-Type': 'application/x-www-form-urlencoded',
n8n-n8n-1       |     'User-Agent': 'axios/0.21.4',
n8n-n8n-1       |     'Content-Length': 278
n8n-n8n-1       |   },
n8n-n8n-1       |   agent: undefined,
n8n-n8n-1       |   agents: { http: undefined, https: undefined },
n8n-n8n-1       |   auth: undefined,
n8n-n8n-1       |   hostname: 'login.microsoftonline.com',
n8n-n8n-1       |   port: null,
n8n-n8n-1       |   nativeProtocols: {
n8n-n8n-1       |     'http:': {
n8n-n8n-1       |       _connectionListener: [Function: connectionListener],
n8n-n8n-1       |       METHODS: [Array],
n8n-n8n-1       |       STATUS_CODES: [Object],
n8n-n8n-1       |       Agent: [Function],
n8n-n8n-1       |       ClientRequest: [Function: ClientRequest],
n8n-n8n-1       |       IncomingMessage: [Function: IncomingMessage],
n8n-n8n-1       |       OutgoingMessage: [Function: OutgoingMessage],
n8n-n8n-1       |       Server: [Function: Server],
n8n-n8n-1       |       ServerResponse: [Function: ServerResponse],
n8n-n8n-1       |       createServer: [Function: createServer],
n8n-n8n-1       |       validateHeaderName: [Function: __node_internal_],
n8n-n8n-1       |       validateHeaderValue: [Function: __node_internal_],
n8n-n8n-1       |       get: [Function: get],
n8n-n8n-1       |       request: [Function: request],
n8n-n8n-1       |       setMaxIdleHTTPParsers: [Function: setMaxIdleHTTPParsers],
n8n-n8n-1       |       maxHeaderSize: [Getter],
n8n-n8n-1       |       globalAgent: [Getter/Setter]
n8n-n8n-1       |     },
n8n-n8n-1       |     'https:': {
n8n-n8n-1       |       Agent: [Function: Agent],
n8n-n8n-1       |       globalAgent: [Agent],
n8n-n8n-1       |       Server: [Function: Server],
n8n-n8n-1       |       createServer: [Function: createServer],
n8n-n8n-1       |       get: [Function: get],
n8n-n8n-1       |       request: [Function: request]
n8n-n8n-1       |     }
n8n-n8n-1       |   }
n8n-n8n-1       | }

The request on the proxied instance looks like this:

n8n-n8n-1       | 2023-07-17T13:49:59.363Z follow-redirects options {
n8n-n8n-1       |   maxRedirects: 21,
n8n-n8n-1       |   maxBodyLength: 10485760,
n8n-n8n-1       |   protocol: 'http:',
n8n-n8n-1       |   path: 'https://login.microsoftonline.com/common/oauth2/v2.0/token',
n8n-n8n-1       |   method: 'POST',
n8n-n8n-1       |   headers: {
n8n-n8n-1       |     Accept: 'application/json, application/x-www-form-urlencoded',
n8n-n8n-1       |     'Content-Type': 'application/x-www-form-urlencoded',
n8n-n8n-1       |     'User-Agent': 'axios/0.21.4',
n8n-n8n-1       |     'Content-Length': 1134,
n8n-n8n-1       |     host: 'login.microsoftonline.com',
n8n-n8n-1       |     'Proxy-Authorization': 'Basic xxxxxxxxxxx'
n8n-n8n-1       |   },
n8n-n8n-1       |   agent: undefined,
n8n-n8n-1       |   agents: { http: undefined, https: undefined },
n8n-n8n-1       |   auth: undefined,
n8n-n8n-1       |   hostname: 'proxy.szh.loc',
n8n-n8n-1       |   port: '8080',
n8n-n8n-1       |   host: 'proxy.szh.loc',
n8n-n8n-1       |   beforeRedirect: [Function: beforeRedirect],
n8n-n8n-1       |   nativeProtocols: {
n8n-n8n-1       |     'http:': {
n8n-n8n-1       |       _connectionListener: [Function: connectionListener],
n8n-n8n-1       |       METHODS: [Array],
n8n-n8n-1       |       STATUS_CODES: [Object],
n8n-n8n-1       |       Agent: [Function],
n8n-n8n-1       |       ClientRequest: [Function: ClientRequest],
n8n-n8n-1       |       IncomingMessage: [Function: IncomingMessage],
n8n-n8n-1       |       OutgoingMessage: [Function: OutgoingMessage],
n8n-n8n-1       |       Server: [Function: Server],
n8n-n8n-1       |       ServerResponse: [Function: ServerResponse],
n8n-n8n-1       |       createServer: [Function: createServer],
n8n-n8n-1       |       validateHeaderName: [Function: __node_internal_],
n8n-n8n-1       |       validateHeaderValue: [Function: __node_internal_],
n8n-n8n-1       |       get: [Function: get],
n8n-n8n-1       |       request: [Function: request],
n8n-n8n-1       |       setMaxIdleHTTPParsers: [Function: setMaxIdleHTTPParsers],
n8n-n8n-1       |       maxHeaderSize: [Getter],
n8n-n8n-1       |       globalAgent: [Getter/Setter]
n8n-n8n-1       |     },
n8n-n8n-1       |     'https:': {
n8n-n8n-1       |       Agent: [Function: Agent],
n8n-n8n-1       |       globalAgent: [Agent],
n8n-n8n-1       |       Server: [Function: Server],
n8n-n8n-1       |       createServer: [Function: createServer],
n8n-n8n-1       |       get: [Function: get],
n8n-n8n-1       |       request: [Function: request]
n8n-n8n-1       |     }
n8n-n8n-1       |   }
n8n-n8n-1       | }

Not perfectly equal, but I guess the differences make sense…

I noticed that the callback parameters on my local instance are a lot shorter than on the proxied instance. As this stuff is encoded (probably even encrypted), it is hard to spot specific details.

Local callback:

/rest/oauth2-credential/callback?code=M.C107_BL2.2.76072fed-68f9-b746-e1d0-2f9daf68720b&state=eyJ0b2tlbiI6Ik5UaGxwUERLLUFsSGwzWTFIRkQxaXFQTmlVbDAyZzJ5bGJFcyIsImNpZCI6IjQifQ%3d%3d

Remote callback:

/rest/oauth2-credential/callback?code=0.ATEAEnNYbTPEaE6VaN16NF6UzeGzLbzKEUFJrrT8r1SJeiAxAAA.AgABAAIAAAD--DLA3VO7QrddgJg7WevrAgDs_wUA9P9b9TmLmNBInjaF06n-e7Sspeol-y0QkJTbVIKLAVHhtHlECZyOORCJkf981DdBECwgN3zoyt6OJdoBqppzLjkvZRQfnAuhnQ2lZaP9ZziUFG35qOpgXQ0hgCQDIR0a6PISuF41MHWc9zVDkzlhmBx7R5oR47IIxxBl5J3GtOdm2u0je3GfwrkeYQ_-SyG_WkVBiecCUJHVVCuH8OFiKkjT8R-OEPfSr0B8HfsfJP4hiSgo-ToeadEZe0VK3mz0sx95nOqRK1TBd4VwMk5Wvg1qEQfgftdE23ttWK-MC5iEGZmGUxwYiGjhf2jUXMQY3xsqQNxoOjhHopHZGQ2piykA5PvSAwhpUXQmBJQxqG4JvOsz5P_LvwiLyXGJs6hxWfMs593Cqj6dPRT8W5vnq1JiwQm_K6hZHbSlY4mivLU1vW1N9A265RVLIch_m8CtbI5kFUz7wGMMpv1e7Mrmj2qGopC6c1EwCrdeQjZFiJrZvbVXNga7xKWIxKR_rkAt8yN9c8zp3lK4UmwyJbIIZMTTXDGD5CUSBoptQVqBU9WMz4BAPNbCaNMyeRoiRLEm8V7NjGovQ8NpUOm3ja7DseU0HoAyQXZ-XrVAUb5pb3N03pMaGXTDsKMsTRhh4COyarIvdHYYg9IEppOpUcVjXBxcx01lpLSAHlzsH9eIEepbR7M-58vg3YCgaf7MDnN8-yEgcc2BtcEYWe5g9AeghxcO2g7QGqzXkiAC5qet8AOLoO-rSDUYFxEQ0A07XmpBGPgzZHkrUbnqvCjSKDHB0s7joCmqiHspro3hXx_bCbYPO-FtZoLd4bY&state=eyJ0b2tlbiI6IkJkam5XbkViLXdtQ1VSUTFndlVFUDhWT2hBVFJ4ZGRmQk5LNCIsImNpZCI6IjYifQ%3d%3d&session_state=f8c1005b-22ef-4599-8f0c-337db7db927e

How important is the correct account type when setting up the application on the Microsoft account?

MutedJam · July 17, 2023, 2:24pm

Hi @LukasFrey, thank you so much for providing these additional details

I’d like to give the proxy vs. non-proxy setup a go on my side to see if I can either find out more here or provide a reproducible description to add to our bug tracker. Can you confirm which proxy server exactly you were using and how it was configured?

LukasFrey · July 17, 2023, 2:36pm

Hi @MutedJam

You are incredibly fast responding, thank you very much.

It is a proxy that requires authentication whenever requesting “outside” stuff.
The product would support “interception” (man-in-the-middle with on-the-fly generation of new certificates), although this feature has been disabled for my servers, i.e. my requests pass the proxy unaltered (at least this is what I hope).

Update:

Just got the information that the product in use is “McAfee SkyHigh Web Gateway”.

Regards,
Lukas

LukasFrey · July 18, 2023, 7:55am

Hi @MutedJam

I was able to do the test with the same account on my installation (i.e. without proxy support).
The account binding worked, the callback URL is also “large”.

The length/content of the callback parameters probably depend on the chosen account type, but even if we chose a differerent type, this seems not to make a difference.

It really seams that there is some issue when communicating over a (or at least our) proxy.

Note:

I got informed that there is a bypass rule for login.microsoft.com, so the requests should be forwarded unaltered.
After asking the “proxy guy”, they observed that the 500 error is returned when the outgoing request happens. I was not able to find out whether the 500 error is returned from microsoft, or whether the proxy returns this http status.

More digging is in progress…

Regards
Lukas

LukasFrey · July 18, 2023, 8:28am

Next / last update

We got the permission to directly access login.microsoftonline.com, which allowed us to successfully activate the credential.

My issue is therefore resolved, but others might run into the same problem (if there really is/were a bug in the proxy support (not yet fully proven).

@MutedJam, thank you very much for your support!

MutedJam · July 18, 2023, 8:29am

Glad to hear you got this one working @LukasFrey, thanks so much for confirming! I’ll also play around with a bunch of proxy setups when I have a bit of time to see if I can reproduce the problem.

LukasFrey · July 18, 2023, 9:11am

Thank you.

Now that my Oauth token is valid, I hit the next road block with the requests to the graph api.

I definitely think that proxy support needs some additional “love”, because if I perform the corresponding requests with curl (via proxy), the requests succeed.

system · July 25, 2023, 9:11am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.