Get binary and itemized input into the same function?

Hi everyone!

I’ve been playing with this software since a few days but hit a bump in the road:

Specifically, I can’t get the FunctionItem to actually get binary data from Google Drive AND each item (one by one, of course).

Basically, I want to get a PDF from Google Drive, insert some customer data using pdf-lib, and save it again as PDF on Google Drive.

I tried 30 variations over the last few hours, I hope I can recall them, not in order:

with the itemfuntion being:

setBinaryData(getBinaryData());
return item;

The google drive Upload fails with

ERROR: No binary data exists on item!

even though all looks good in the previous step:

(image 2, new user and such, will post it as comment below)

I figured out that the merge append is the problem.

It works fine if I attach the google drive download directly to the function.
It also works fine if I attach the google drive AND the itemized output to the function, but only if I press the play button of google drive at the right time.
So it appears I need to synchronize it, so I tried pass-through merge, but now I don’t have the data.
Passing it along the side created another synchronization problem.

(image 3, new user and such, will post it as comment below)

with the function being:

setBinaryData($node["Google Drive"].binary["data"]);
return item;

"No data found for item-index: "1""

(weirdly, I never access item index 1)

I also tried without the merge function and just accessing the data with $node:

(image 4, new user and such, will post it as comment below)

Now the Google drive upload returns:

ERROR: No binary data property “data” does not exists on item!

Which is probably another sync problem.

Anyway, I’m out of ideas xD

Any help appreciated :slight_smile:

PS: I could, probably, save the binary data in a file, and just read that file from javascript, but that would be cheating.

img 2:

img 3:

img 4:

Welcome to the community @Veit_Nachtmann!

Here example code for FunctionItem:

// Get all currently existing binary data
const binaryData = getBinaryData();

// Change file-name of existing
binaryData.data.fileName = 'newName.jpg';

// Create a new binary data
binaryData.data2 = {
  data: Buffer.from('my test text').toString('base64'),
  fileName: 'example.txt',
  mimeType: 'text/plain',
}

// Set the binary data on the item
setBinaryData(binaryData);

return item;

And here a workflow which uses it:

{
  "nodes": [
    {
      "parameters": {
        "url": "https://www.outbrain.com/techblog/wp-content/uploads/2017/05/road-sign-361513_960_720.jpg",
        "responseFormat": "file",
        "options": {}
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        500,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "// Get all currently existing binary data\nconst binaryData = getBinaryData();\n\n// Change file-name of existing\nbinaryData.data.fileName = 'newName.jpg';\n\n// Create a new binary data\nbinaryData.data2 = {\n  data: Buffer.from('my test text').toString('base64'),\n  fileName: 'example.txt',\n  mimeType: 'text/plain',\n}\n\n// Set the binary data on the item\nsetBinaryData(binaryData);\n\nreturn item;\n"
      },
      "name": "FunctionItem",
      "type": "n8n-nodes-base.functionItem",
      "typeVersion": 1,
      "position": [
        700,
        300
      ]
    }
  ],
  "connections": {
    "HTTP Request": {
      "main": [
        [
          {
            "node": "FunctionItem",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Hope that is helpful!

Hey Jan,

thanks!

My ItemFunction was just the minimal version to test if it works, but yours is actually better for debugging:

ERROR: Cannot read property ‘data’ of undefined

TypeError: Cannot read property 'data' of undefined
    at /usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes:5:12

The Merge before the ItemFunction has all the data (Table/Json data AND a binary called “data”), so I don’t understand the problem.

The FunctionItem node executes the code for each item. So looking at the screenshot I expect that the problem is that some of your items (probably the ones coming from the Merge2 Node) do not have binary data.

The binary data is always coming in from Merge1, Input 2.

In the Merge1 use merge by index instead of append.

still the same problem,

console.log(getBinaryData().data);

throws

TypeError: Cannot read property 'data' of undefined

even though the merge1 has correct data:

ah, question: there’s only 1 binary in that view.

is that one binary added to each of the items?

maybe the first execution in ItemFunction works, but no forther one?

Yes, exactly that is the problem. You seem to have 45 items but only one of them has binary data. So it would work 1x and fail 44x.

So assuming you have 45 items in the first input and 1 in the second input the following would happen:

  • “append”: Appends the items from input 2 to the data of input 1. You would have 46 items afterward. 45 without binary data and 1 with binary data.
  • “Merge By Index”: It will add the data of the items of input 2 to the data of input 1. So the first item would get the binary data attached and the other 44 not.

Do you have on purpose 45 items? What exactly do you want to do?

yes that’s on purpose.

I want to generate PDF files for each customer in that run.

the file coming in from google drive is the PDF template.
in the ItemFunction, I open it with pdf-lib, add the text from the current item, and save the pdf to google drive.

I managed to get a little further using node-fetch now.

fetching from google drive with a publicly shared URL didn’t work due to 403 errors, even when I faked the user-agent.
I now uploaded it somewhere else and fiiinally pdf-lib can do it’s thing.

now neither “write binary file” nor google drive can save it :confused:

Getting
ERROR: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined

Am I doing something wrong?

const { degrees, PDFDocument, rgb, StandardFonts } = require('pdf-lib'); 
fontkit = require('@pdf-lib/fontkit');
fetch = require("node-fetch");

url = 'https://cdn.shopify.com/s/files/1/0268/6658/1581/files/DE.pdf?v=1613534654'
existingPdfBytes = await fetch(url).then(res => res.arrayBuffer())

pdfDoc = await PDFDocument.load(existingPdfBytes);
pdfDoc.registerFontkit(fontkit);

fontUrl = 'https://cdn.shopify.com/s/files/1/0268/6658/1581/files/Gotham-Light.otf?v=1613534575'
fontBytes = await fetch(fontUrl ).then(res => res.arrayBuffer());
gothamLight = await pdfDoc.embedFont(fontBytes );

pages = pdfDoc.getPages();
firstPage = pages[0];
let { width, height } = firstPage.getSize();
firstPage.drawText('This text was added with JavaScript!', {
    x: 5,
    y: height / 2 + 300,
    size: 50,
    font: gothamLight,
    color: rgb(0.95, 0.1, 0.1),
    rotate: degrees(-45),
  })

pdfBytes = await pdfDoc.save()

setBinaryData({pdf: pdfBytes});
return item;

it’s just copypasted and slightly modified example code right now.
Should I open a new topic for this?

My original problem still stands, though. Uploading the files to a webspace is just a workaround that will make it difficult later.

I tried console.log(pdfBytes) and am getting

Uint8Array(249565) [
   37,  80,  68,  70,  45,  49,  46,  55,  10,  37, 129, 129,
  129, 129,  10,  10,  50,  32,  48,  32, 111,  98, 106,  10,
   60,  60,  10,  47,  70, 105, 108, 116, 101, 114,  32,  47,
   70, 108,  97, 116, 101,  68, 101,  99, 111, 100, 101,  10,
   47,  76, 101, 110, 103, 116, 104,  32,  51,  54,  51,  53,
   10,  62,  62,  10, 115, 116, 114, 101,  97, 109,  10,  72,
  137, 220, 151, 219, 110,  28, 199,  17, 134, 239, 249,  20,
  147, 139,   0, 187, 129,  56, 234, 234, 234,  99,  96,  24,
  136,  37, 193,  72,
  ... 249465 more items
]

so I’m assuming I’m using setBinaryData wrong.

I think the binary name “pdf” is working at least, if I set the “save” part to something else (“pdfklfwjfklw” for example), I get a different error.

lol I missed the .toString(‘base64’).

Why would the binary data be base64 encoded O_o…

Because all data in n8n is JSON and to save binary data in it we have to encode it that way.

ok thank you!

Any idea how to get the PDF file into FunctionItem for every Item?

Assuming you mean “as buffer” and the binary property is called ‘pdf’:

const binaryData = getBinaryData();
const thePdfFile = Buffer.from(binaryData.pdf.data, 'base64');