N8n file size limit

I have a workflow in which I’m downloading files from a Google Drive Folder into n8n, splitting those files into chunks, generating vector embeddings based on those chunks and then storing them in Pinecone Vector Store.

The main problem I’m having is that I’m not able to download files of size greater than 10 MB. Has anyone encountered a similar issue? If yes, what did you do to resolve it?

1 Like

Hi,

Please define “I’m unable to download > 10MB”. Where do you receive an error?

The only limitation I can find which mentions 10MB is here:

reg,
J.


The workflow fails at the circled node.

Hi,

I haven’t seen it myself.

Which type of files are they?

Regards,
J.

Hi,

Just a few docx files. However some of them are >10MB. Any suggestions on how to get around this?

Regards,
Daniyal

Hi @Daniyal_Sabir

Are you using the Google File Conversion option in Google Drive node?

I’m not sure about this option it might be trigger the 10 MB export limit here:

Exports a Google Workspace document to the requested MIME type and returns exported byte content. Note that the exported content is limited to 10MB.

Try disabling this option, You should be able to download files of any size…

Hi

Are you talking about this @mohamed3nan ?

1 Like

Yes, Are these your configurations?


Also, I tested a 20MB file on my end, and it worked without any issues…

1 Like

I did what you have done but it shows that same error that’s been occurring for me. Although it says ‘perhaps check your credentials’, I know it’s really to do with size since if I do it for files size <10MB, they are downloaded without any issue.

1 Like

I am having this same issue. When we try to upload to Pinecone from JSON we get memory error. Maybe this is because of my account level. It would be helpful if n8n could list file upload limitations by account type. I am at the level below Enterprise.

are you on n8n cloud?

@Daniyal_Sabir same question? n8n cloud?

I have onprem setup so can try also if you want.

I’m wondering why this is happening, happy to try help troubleshoot further if you’d like :slight_smile:

please share ure workflow
n8n attach workflow

This will help with quick testing, please ensure to remove anything senative like creds etc.

Yes. I am in n8n cloud…sorry I am a newb I should have specified.

I am in the Pro Plan. I have a Google Drive trigger download a .json file. Extract from File has been stopping due to Memory…however it did work this last execution.

type or paste code here{
  "nodes": [
    {
      "parameters": {
        "pollTimes": {
          "item": [
            {
              "mode": "everyMinute"
            }
          ]
        },
        "triggerOn": "specificFolder",
        "folderToWatch": {
          "__rl": true,
          "value": "1H1cxYnor0LUNZww_bW67qsS-ankyHboQ",
          "mode": "list",
          "cachedResultName": "ai",
          "cachedResultUrl": "https://drive.google.com/drive/folders/1H1cxYnor0LUNZww_bW67qsS-ankyHboQ"
        },
        "event": "fileCreated",
        "options": {}
      },
      "type": "n8n-nodes-base.googleDriveTrigger",
      "typeVersion": 1,
      "position": [
        0,
        0
      ],
      "id": "ba331996-f925-49ac-b754-4fa339d6bf00",
      "name": "Google Drive Trigger",
      "credentials": {
        "googleDriveOAuth2Api": {
          "id": "sK2dNTezPPYI3arM",
          "name": "Google Drive account"
        }
      }
    },
    {
      "parameters": {
        "operation": "download",
        "fileId": {
          "__rl": true,
          "value": "={{ $json.id }}",
          "mode": "id"
        },
        "options": {}
      },
      "type": "n8n-nodes-base.googleDrive",
      "typeVersion": 3,
      "position": [
        220,
        0
      ],
      "id": "b75a259b-951e-421d-9af4-20a76feb45d6",
      "name": "Google Drive",
      "credentials": {
        "googleDriveOAuth2Api": {
          "id": "sK2dNTezPPYI3arM",
          "name": "Google Drive account"
        }
      }
    },
    {
      "parameters": {
        "mode": "insert",
        "pineconeIndex": {
          "__rl": true,
          "value": "knowsys-openai-small",
          "mode": "list",
          "cachedResultName": "knowsys-openai-small"
        },
        "options": {
          "pineconeNamespace": "handlers enhanced"
        }
      },
      "type": "@n8n/n8n-nodes-langchain.vectorStorePinecone",
      "typeVersion": 1.1,
      "position": [
        660,
        0
      ],
      "id": "08f8c554-6bcd-4083-ad3d-4b2ee456c8e6",
      "name": "Pinecone Vector Store",
      "credentials": {
        "pineconeApi": {
          "id": "fYUf6Ch140dZtTpf",
          "name": "PineconeApi account"
        }
      }
    },
    {
      "parameters": {
        "operation": "fromJson",
        "options": {}
      },
      "type": "n8n-nodes-base.extractFromFile",
      "typeVersion": 1,
      "position": [
        440,
        0
      ],
      "id": "caf51d0f-a304-45f5-af07-6943637c40f7",
      "name": "Extract from File"
    }
  ],
  "connections": {
    "Google Drive Trigger": {
      "main": [
        [
          {
            "node": "Google Drive",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Google Drive": {
      "main": [
        [
          {
            "node": "Extract from File",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract from File": {
      "main": [
        [
          {
            "node": "Pinecone Vector Store",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {},
  "meta": {
    "templateCredsSetupCompleted": true,
    "instanceId": "e7ce631bb95c0c51f0313a3fe4fac577d5c0f07c4ca48445d1f66d9affce09af"
  }
}

hmm, it could be limits on ure host, but 10mb doesn’t seem right, it could be mem or cpu too due to storing embeding, have u tried locally see if it runs okay?

I did not know this was something you could do locally until recently…I watched Udemy and everything was cloud… I wish n8n would share how much RAM we had for cloud computing and usage like Google Colab does.

We have pretty much pivoted to pushing to Pinecone via API in Colab because our files are GB of data…but I would still like to have some triggers like this for smaller files like for training a Help Bot

1 Like

they may enable /metrics endpoint soon or by request, I made grafana dashboard for it grafana but yeah am not sure about cloud used it a bit, but not the inside no aaha, but if ure embedding stuff to vectors too I think this uses a lot my local dev instance nearly crashed when doing 50mb file pdf, file was local too.