How do I get the raw POST payload from a webhook into subsequent nodes?

[sdkjfhsjk]

I’m self-hosting n8n 1.47.1 in Debian. I’m trying to send an HTTP POST to n8n and act on the raw bytes of the POST body. In this instance, I have a security camera that sends an HTTP POST to a user-defined endpoint and the HTTP POST body are the actual bytes of a JPEG frame. It is not multipart form-data; rather, the raw POST body itself is the JPEG.

My objective is to take that image and send it into a second HTTP POST endpoint that responds with some custom JSON, and then subsequent actions from that JSON. That second endpoint also expects a raw POST where the body is the JPEG data.

My problem is I can’t find any way of linking those two nodes together and actually passing the raw JPEG image from the webhook to the subsequent HTTP endpoint.

My webhook looks like this:

…and when I send a test POST to it like this:

curl -v -X POST -H 'Content-Type: image/jpeg' --data-binary @test.jpg http://...:5678/webhook-test/a9765f66-6f32-4294-b57d-cc851cf71917 | jq

…I see the image show up properly in the webhook’s output: I can click “View” and see my JPEG.

However, I see no way of actually taking the binary POST data from the webhook and passing it to my HTTP request node. There is nothing I see to select as an input:

How can I get the raw POST bytes that the webhook received in its body and send them as a raw POST body to the second HTTP request node?

[n8n]

It looks like your topic is missing some important information. Could you provide the following if applicable.

• n8n version:
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):
• Operating system:

[sdkjfhsjk]

• n8n version: 1.47.1
• Database (default: SQLite): SQLite, presumably
• n8n EXECUTIONS_PROCESS setting (default: own, main): Whatever the default is; I just use n8n start
• Running n8n via (Docker, npm, n8n cloud, desktop app): npm
• Operating system: Debian 12 in a Proxmox LXC

[ihortom]

@sdkjfhsjk

My problem is I can’t find any way of linking those two nodes together and actually passing the raw JPEG image from the webhook to the subsequent HTTP endpoint.

You would need to convert the binary to text (base64 encoded string) with Extract from File node. That would be your raw JPEG image. However, if your other service expects some other encoding, you can process this further.

[sdkjfhsjk]

I always get “No output data returned” from the Extract from File node. What do I provide for “Input Binary Field”? And am I configuring the webhook properly in my original screenshot, especially when it comes to the “raw body” checkbox? When the latter is checked, all I get is a MIME type, and unchecked I get file size, name, type, etc., but neither has any obvious field that actually has a reference to the data itself.

The HTTP endpoint that will eventually receive the data relayed from this webhook expects raw binary and not base64, so if this process will give me base64 data, I’ll presumably need another node to make it binary again.

I’m confused why I can’t just reference the binary data from the webhook directly in the subsequent HTTP request node. It seems like a strange design decision for me to transform the data to and from an intermediate format when it’s simpler to just reference the data from the webhook directly.

[sdkjfhsjk]

I also tested the cURL command extensively, even sniffing it with Wireshark. The request headers are:

POST /webhook-test/a9765f66-6f32-4294-b57d-cc851cf71917 HTTP/1.1
Host: 192.168.1.66:5678
User-Agent: curl/8.6.0
Accept: */*
Content-Type: image/jpeg
Content-Length: 289208

And indeed, my test image is precisely that size:

$ stat -f%z test.jpg
289208

The webhook’s response headers:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 270
ETag: ...
Vary: Accept-Encoding
Date: Sat, 29 Jun 2024 20:49:17 GMT
Connection: keep-alive
Keep-Alive: timeout=5

[sdkjfhsjk]

I’m closer, but not there yet. I put a Code node between the webhook and HTTP POST nodes with this Javascript:

const data = items[0].binary.data.data;
const decoded = Buffer.from(data, 'base64').toString("binary");
return {
  jpeg: decoded
}

Then, in the HTTP POST, I specified {{ $json.jpeg }} as the payload. The encoding is screwed up though because the received bytes starts with:

00000000  c3 bf c3 98 c3 bf c3 a0  00 10 4a 46 49 46 00 01  |..........JFIF..|
00000010  01 00 00 01 00 01 00 00  c3 bf c3 9b 00 43 00 0a  |.............C..|
...

The first bytes are supposed to be FF D8 FF. I did try base64 instead of binary and the endpoint receives a base64-encoded string, unsurprisingly. When I manually decode it, I get a perfect byte-for-byte copy of the original payload, which is my goal. The problem is my endpoint only accepts the raw bytes of the file, not a base64-encoded version, so if we can fix the encoding, it should solve the problem. Not sure how, though.

[sdkjfhsjk]

Okay, to help others, here’s how I got it to work finally. Connect these nodes together in this sequence:

• Webhook:
  • HTTP Method: POST
  • Binary Property: data
  • Raw Body: checked
• Code:

    const data = items[0].binary.data.data;
    return {
      jpeg: data
    }

• Convert to File:
  • Operation: Move Base64 String to File
  • Base64 Input Field: jpeg
  • Put Output File in Field: data
  • File Name: image.jpeg
  • MIME Type: image/jpeg
• HTTP Request:
  • Method: POST
  • Send Headers: Using Fields Below and Content-Type referencing the webhook’s mimeType
  • Send Body: checked
  • Body Content Type: n8n Binary File
  • Input Data Field Name: data

I can finally hit the webhook and forward the payload to my HTTP endpoint. For future versions of n8n, I really hope I can just connect the webhook’s received POST data directly to the input of the HTTP request because this was a convoluted mess that took two days to figure out.