Sent binary data example

jwillmer · April 29, 2020, 9:21am

Related feature: https://community.n8n.io/t/how-to-sent-binary-data-as-multipart-form-data

I receive a webhook with some audio file that I like to sent to a webservice for conversion. However I can’t find the right options to successfully post it to the service. Below is my node configuration and the API of the service. Could someone help me find the right configuration for this?

{
  "nodes": [
    {
      "parameters": {
        "httpMethod": "POST",
        "path": "ptt-input",
        "options": {
          "binaryData": true
        }
      },
      "name": "PTT-Webhook",
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 1,
      "position": [
        260,
        300
      ]
    },
    {
      "parameters": {
        "requestMethod": "POST",
        "url": "http://audioconverter/convert-to/wav",
        "responseFormat": "file",
        "jsonParameters": true,
        "options": {
          "bodyContentType": "multipart-form-data"
        },
        "sendBinaryData": true,
       "binaryPropertyName": "file:data"
      },
      "name": "AudioConverter",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        540,
        140
      ]
    }
  ],
  "connections": {
    "PTT-Webhook": {
      "main": [
        [
          {
            "node": "AudioConverter",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

jan · April 29, 2020, 9:34am

Looks on the first look ok.

When you send the data to the n8n Webhook does the data end up in the binary property named “data”?

jwillmer · April 29, 2020, 9:45am

That was what I thought. As I understand this option stores the data per default in data:

jwillmer · April 29, 2020, 9:49am

This is the error I get on the audio (post) node when executing the workflow:

ERROR: 400 - {"type":"Buffer","data":[123,34,116,121,...]}

StatusCodeError: 400 - {"type":"Buffer","data":[123,34,116,121,...]}
    at new StatusCodeError (/usr/local/lib/node_modules/n8n/node_modules/request-promise-core/lib/errors.js:32:15)
    at Request.plumbing.callback (/usr/local/lib/node_modules/n8n/node_modules/request-promise-core/lib/plumbing.js:104:33)
    at Request.RP$callback [as _callback] (/usr/local/lib/node_modules/n8n/node_modules/request-promise-core/lib/plumbing.js:46:31)
    at Request.self.callback (/usr/local/lib/node_modules/n8n/node_modules/request/request.js:185:22)
    at Request.emit (events.js:310:20)
    at Request.<anonymous> (/usr/local/lib/node_modules/n8n/node_modules/request/request.js:1154:10)
    at Request.emit (events.js:310:20)
    at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/n8n/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:416:28)
    at IncomingMessage.emit (events.js:322:22)

jan · April 29, 2020, 10:43am

With multi-part, multiple files can be send at the same time and they all get send under a specific name which also n8n uses.

Here an example.

Receive data workflow:

Send workflow (URL of HTTP Request2 has to be adjusted to point to receive workflow):

For me, that worked fine and I did not get the error. Can you please create an issue on Github with a workflow to reproduce the issue you are seeing that we can fix it. Thanks!

jwillmer · April 29, 2020, 11:49am

O.K., I tested it locally - like you did - with a file that I retrieve by WebApi GET. That works without a problem. So I think the multipart POST is not the problem. The problem is the webhook that is receiving the file. Maybe webhook has problems handling files?

Webhook.site data for comparison:

My webhook:

{
  "nodes": [
    {
      "parameters": {
        "httpMethod": "POST",
        "path": "ptt-input",
        "options": {
          "binaryData": true
        }
      },
      "name": "PTT-Webhook",
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 1,
      "position": [
        260,
        390
      ]
    }
  ],
  "connections": {}
}

I can reproduce the issue by using curl to trigger the webhook via:

curl --location --request POST 'https://n8n...../webhook-test/53/ptt-webhook/ptt-input' \
--header 'Content-Type: application/octet-stream' \
--data-binary '@/C:/test.opus'

RicardoE105 · April 29, 2020, 5:13pm

@jwillmer Just tested it as you suggested using a .opus file and the webhook receives the file.

curl --location --request POST '/webhook-test/86/ptt-webhook/ptt-input' \
--header 'Content-Type: application/octet-stream' \
--data-binary '@/Users/Ricardo/Desktop/2019-05-01_-_Undercover_Spy_Agent_-_David_Fesliyan_1.opus'

jwillmer · April 29, 2020, 8:17pm

I believe the server proxy is preventing the file from getting to the n8n webhook. If I test my setup locally it works perfect. Will investigate in this direction. Thank you for validating my actions

jwillmer · April 29, 2020, 8:34pm

I found the problem - had nothing to do with the proxy. I had EXECUTIONS_PROCESS: main enabled for n8n and after starting n8n without this flag i received the file. @jan I think there is an issue with this flag.

jan · April 29, 2020, 10:55pm

Ah yes there were problems with EXECUTIONS_PROCESS=main:

A fix just got released with [email protected] can you please check if it works fine in that version else we will have another look.

jwillmer · April 30, 2020, 7:38am

Does not work for me. Without the option the webhook gets executed and I see the file. With the option the webhook gets executed but the file is missing.

Edit:
There is also a difference in Webhook vs HTTP request:
Receiving the ogg-file via HTTP request works fine and my next HTTP request can use the file.

Receiving the same file via Webhook works (without the EXECUTIONS_PROCESS option) but the next step throws the error I posted previously: Sent binary data example - #4 by jwillmer.

Edit2:
When I click on the show data in the HTTP request it plays the ogg file. If I do the same on the webhook it shows me:

File preview comparison:

jan · April 30, 2020, 11:47pm

It seems to only be a display issue. The data gets send so fast to the Editor-UI that it is not aware yet what execution is active.
So if you have the workflow active (so with the production webhook) it should work without a problem. But testing will so currently sadly not be possible. I will work on a fix. But for now you can simply test without EXECUTIONS_PROCESS=main. You can, however, have it set in production.

jan · May 1, 2020, 8:03am

Did push a fix which will be released with the next version.

jan · May 1, 2020, 4:45pm

Btw. I could sadly not reproduce the problem with the invalid video file for webhooks. I used this example file:
http://techslides.com/demos/sample-videos/small.ogv

And not matter how I get it into n8n, if via HTTP Request or Webhook it works fine and plays just fine.

jan · May 4, 2020, 10:09am

Got released with [email protected]