Binary files: Download and Upload through HTTP Request

Entrepositive · August 15, 2022, 6:16pm

It worked… sort of. I have no idea what happened, but it finally rendered the graphic from the View button in binary dialog. Now I’m working to get Drive to read it in as file upload…

This is the link I used https://trello.com/1/cards/62f6c4e029a762343527a444/attachments/62fa7a42473aea78be885281/download/image.png

EDIT: I’m now realizing I was using a URL without the “api.” subdomain. I ran it with the api. and got the same result. Maybe it doesn’t make a difference? Or the n8n module catches and auto fixes?

Entrepositive · August 15, 2022, 9:01pm

Mission Success!

I was able to pull an attachment from a Card in Trello (in my case triggered by a new attachment upload), and upload it to Google Drive with the same file name.

Important to note:

HTTP Request Node - 1st Node - Trello GET request for attachment binary data

The link in the Expression for the Trello HTTP request to pull attachment is the “{attachmentlink}” variable from Trello.
n8n correctly passes the appropriate authentication to Trello

Note

A third/intermediary Write Binary File node is not required.

Google Drive Upload

Binary Data toggle must be TRUE, and the Binary Property name must match the same from the (Trello) HTTP Request node; using the Expression was not required.
In the Parents section of the node, add an Expression that receives data with the GDrive folder ID, to tell Drive where to place the file. (Or just paste it in, if static).
Add the Option Properties and add a new property with Key uploadType and Value media for graphics. Refer to Google for more info.

That should be good to go; at least it worked for what we need. If you find any problems with this, please tag me and I’ll help any way I can. Thanks for the support @MutedJam and @Jon!

EP

Entrepositive · August 17, 2022, 4:33pm

Man I was really happy this was a fixed issue - but a strange thing happened and it’s not fixed.

I had the above solution fixed and working, in Test mode (Webhook), and workflow not “active”.
I saved and changed to “active” and used the production URL, and this happened:
A. workflow executed as expected without issue, but
B. the HTTP request to Trello to pull the file data once again returns the binary with no viewable image, and
C. the file created in GDrive is an empty graphic that does not display/cannot be opened or viewed.

Otherwise the seems normal:

Webhook output:

[
  {
    "headers": {
      "host": "127.0.0.1:1234",
      "content-type": "application/json",
      "x-forwarded-for": "serverIP",
      "x-forwarded-host": "someserver.com",
      "x-forwarded-server": "someserver.com",
      "content-length": "256",
      "connection": "close"
    },
    "params": {},
    "query": {},
    "body": {
      "Card title": "Bake sale",
      "Card ID": "_someCardID_",
      "Attachment Name": "surprised-kitty.png",
      "Attachment URL": "https://trello.com/1/cards/_someCardID_/attachments/3rj_someFileID_ecim/download/5.png",
      "ContactID": "xxyy"
    }
  }
]

Trello (HTTP request) output:
(clicking “View” shows blank screen)

[
  {
    "headers": {
      "x-dns-prefetch-control": "off",
      "x-frame-options": "DENY",
      "x-download-options": "noopen",
      "x-permitted-cross-domain-policies": "none",
      "referrer-policy": "strict-origin-when-cross-origin",
      "surrogate-control": "no-store",
      "cache-control": "no-store, no-cache, must-revalidate, proxy-revalidate",
      "pragma": "no-cache",
      "expires": "0",
      "x-trello-version": "1.149157.0",
      "x-trello-environment": "Production",
      "set-cookie": [
        "dsc=someBigLongCodeOfNumbersAndLetters; Path=/; Expires=Sat, 20 Aug 2022 16:14:25 GMT; Secure",
        "mab=5940; Path=/; Expires=Fri, 16 Sep 2022 16:14:25 GMT",
        "gdpr-cookie-consent=accepted; Path=/; Expires=Thu, 17 Aug 2023 16:14:25 GMT"
      ],
      "link": "<https://trello.com/cs/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"cs\", <https://trello.com/de/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"de\", <https://trello.com/en/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"en\", <https://trello.com/en-AU/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"en-AU\", <https://trello.com/en-GB/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"en-GB\", <https://trello.com/en-US/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"en-US\", <https://trello.com/es/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"es\", <https://trello.com/fr/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"fr\", <https://trello.com/it/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"it\", <https://trello.com/hu/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"hu\", <https://trello.com/nl/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"nl\", <https://trello.com/nb/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"nb\", <https://trello.com/pl/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"pl\", <https://trello.com/pt-BR/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"pt-BR\", <https://trello.com/fi/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"fi\", <https://trello.com/sv/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"sv\", <https://trello.com/vi/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"vi\", <https://trello.com/tr/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"tr\", <https://trello.com/ru/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"ru\", <https://trello.com/uk/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"uk\", <https://trello.com/th/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"th\", <https://trello.com/zh-Hans/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"zh-Hans\", <https://trello.com/zh-Hant/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"zh-Hant\", <https://trello.com/ja/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"ja\", <https://trello.com/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"alternate\"; hreflang=\"x-default\", <https://trello.com/login?returnUrl=%2F1%2Fcards%2F_someCardID_%2Fattachments%2F3rj_someFileID_ecim%2Fdownload%2Fsurprised-kitty.png>; rel=\"canonical\"",
      "content-type": "text/html; charset=utf-8",
      "content-length": "212648",
      "date": "Wed, 17 Aug 2022 16:14:25 GMT",
      "x-envoy-upstream-service-time": "11",
      "expect-ct": "report-uri=\"https://web-security-reports.services.atlassian.com/expect-ct-report/trello-edge\", max-age=86400",
      "strict-transport-security": "max-age=63072000; preload",
      "x-content-type-options": "nosniff",
      "x-xss-protection": "1; mode=block",
      "atl-traceid": "c75e1635fe6d4899",
      "report-to": "{\"group\": \"endpoint-1\", \"max_age\": 600, \"endpoints\": [{\"url\": \"https://dzvv6s.cloudfront.net\"}], \"include_subdomains\": true}",
      "nel": "{\"report_to\": \"endpoint-1\", \"max_age\": 600, \"include_subdomains\": true, \"failure_fraction\": 0.001}",
      "vary": "Accept-Encoding",
      "server": "globaledge-envoy",
      "connection": "close"
    },
    "statusCode": 200,
    "statusMessage": "OK"
  }
]

GDrive node output:

[
  {
    "kind": "drive#file",
    "id": "myDriveFileID",
    "name": "surprised-kitty.png",
    "mimeType": "image/png"
  }
]

It seems from my end that activating the workflow has caused Trello HTTP request node to create an empty/corrupted graphic file and I just can’t believe that to be the cause. I don’t remember making any change in between the solution and activating the workflow.

Can anyone shed insight on this?

EP

Jon · August 17, 2022, 4:58pm

What is in the file if you download it? Like as an ugly test pop it open with a text editor and see if it looks like garbage text or if you can read the contents.

I wonder if maybe it isn’t authenticated which is why it is failing and mentions the login page in the link. I think if you open that file you will see HTML.

Entrepositive · August 17, 2022, 5:28pm

Here’s the file

Entrepositive · August 17, 2022, 5:31pm

Yup.

You’re spot on. It’s HTML for the trello login page…

So you think this is because it wasn’t properly authenticated?

Entrepositive · August 17, 2022, 5:42pm

I tried removing the n8n native credentials and adding my own ‘?key=blah&token=blahblah’ parameters to the URL in the GET request to trello.

It didn’t return the same thing although it’s equally jumbled. It still returns:

“content-type”: “text/html; charset=utf-8”,

And now in Drive the file icon is no longer that of an image file, but now a doc file type. Previously, it showed an image file type icon, whereas normally it would render a small preview of the image itself.

Entrepositive · August 17, 2022, 5:53pm

Eh, I have no idea what I’m doing. I tried the Trello API Key with the Token, didn’t work. I tried the API Key with the API Secret, same exact result.

I’m not an expert in this…I’m kinda out of options. It seriously baffles me that it worked correctly, and then it just didn’t work again.

Jon · August 17, 2022, 7:47pm

I will take a look in the morning and see if I can work it out.

Entrepositive · August 17, 2022, 8:18pm

Case A

Stack

NodeApiError: UNKNOWN ERROR - check the detailed error for more information
    at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/HttpRequest/HttpRequest.node.js:1138:27)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/src/Workflow.js:598:28)
    at async /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/src/WorkflowExecute.js:557:53

All ABOVE (Case A) is using the following Auth Type:

with Header Auth:

CASE B

All BELOW is using n8n native credentials for the Trello account (confirmed functional for previous GET, PUT, POST requests).

[
  {
    "headers": {
      "date": "Wed, 17 Aug 2022 19:08:34 GMT",
      "content-type": "application/json; charset=utf-8",
      "content-length": "2356",
      "x-dns-prefetch-control": "off",
      "x-frame-options": "DENY",
      "x-download-options": "noopen",
      "x-permitted-cross-domain-policies": "none",
      "referrer-policy": "strict-origin-when-cross-origin",
      "surrogate-control": "no-store",
      "cache-control": "max-age=0, must-revalidate, no-cache, no-store",
      "pragma": "no-cache",
      "expires": "Thu, 01 Jan 1970 00:00:00",
      "x-trello-version": "1.149355.0",
      "x-trello-environment": "Production (Micros)",
      "set-cookie": [
        "dsc=biglongnumber; Path=/; Expires=Sat, 20 Aug 2022 19:08:34 GMT; Secure",
        "preAuthProps=someCode; Path=/; HttpOnly"
      ],
      "access-control-allow-origin": "*",
      "access-control-allow-methods": "GET, PUT, POST, DELETE",
      "access-control-allow-headers": "Authorization, Accept, Content-Type",
      "access-control-expose-headers": "x-rate-limit-api-key-interval-ms, x-rate-limit-api-key-max, x-rate-limit-api-key-remaining, x-rate-limit-api-token-interval-ms, x-rate-limit-api-token-max, x-rate-limit-api-token-remaining",
      "x-rate-limit-api-token-interval-ms": "10000",
      "x-rate-limit-api-token-max": "100",
      "x-rate-limit-api-token-remaining": "98",
      "x-rate-limit-db-query-time-interval-ms": "600000",
      "x-rate-limit-db-query-time-max": "7200000",
      "x-rate-limit-db-query-time-remaining": "7199990",
      "x-rate-limit-api-key-interval-ms": "10000",
      "x-rate-limit-api-key-max": "300",
      "x-rate-limit-api-key-remaining": "298",
      "x-rate-limit-member-interval-ms": "10000",
      "x-rate-limit-member-max": "375",
      "x-rate-limit-member-remaining": "362",
      "x-server-time": "1660763314148",
      "x-envoy-upstream-service-time": "54",
      "server": "globaledge-envoy",
      "expect-ct": "report-uri=\"https://web-security-reports.services.atlassian.com/expect-ct-report/trello-edge\", max-age=86400",
      "strict-transport-security": "max-age=63072000; preload",
      "x-content-type-options": "nosniff",
      "x-xss-protection": "1; mode=block",
      "atl-traceid": "xyz",
      "report-to": "{\"group\": \"endpoint-1\", \"max_age\": 600, \"endpoints\": [{\"url\": \"https://dznkvv6s.cloudfront.net\"}], \"include_subdomains\": true}",
      "nel": "{\"report_to\": \"endpoint-1\", \"max_age\": 600, \"include_subdomains\": true, \"failure_fraction\": 0.001}",
      "connection": "close"
    },
    "statusCode": 200,
    "statusMessage": "OK"
  }
]

…and binary output:

{"id":"attachmentID","bytes":417954,"date":"2022-08-17T19:08:30.203Z","edgeColor":"#e0dedd","idMember":"memberID","isUpload":true,"mimeType":"image/png","name":"surprised-kitty.png","previews":[a whole lot of meta data from Trello],"url":"https://trello.com/1/cards/cardID/attachments/attachmentID/download/surprised-kitty.png","pos":16384,"fileName":"surprised-kitty.png"}
* names like "attachmentID" used to replace ID for this example

In both Cases, I set the following Header

This has me stumped today.

EP

Jon · August 18, 2022, 11:12am

Hey @Entrepositive,

I have had a play and managed to get it working, to make life easier I have opted to use a trigger so it can be downloaded and uploaded as it happens but the trick is with the authentication on the URL. Turns out you need to use an Authorization header and using the the other method won’t work.

The below workflow appears to be working fine in testing and when made live so should do the job for you, Instructions are in the note.

Entrepositive · October 4, 2022, 8:12pm

Finally got this sorted, and it’s working. Thank you Jon for the insight!

I had made a lot of adjustments and couldn’t get it to save the file into drive correctly (scrambled data, not image).

Finally, I caught a couple mistakes I was making.

I entered Header Auth Value you shared in your example, in “The Magic” section. But I missed the missing quote for a while. Eventually caught that.
I still had this header setting in the File download node:

Got those sorted and it worked like magic. Thanks again Jon - super helpful!

EP