Hubspot node 'get all contacts': only subset of max 300 entries

Im using Hubspot node to maually get all contacts from hubspot but I receive only a subset of max 300 entries. Database on Hubspot side has 1K+ contact entries…?!

Are you within this limit?

1 Like

aha, thank you. This seems to be the reason…
Is there a workaround that allows me to get all contacts from hubspot?
Im trying to setup a synchronisation between hubspot and google contacts.
Initially I wanted to copy paste all contacts from hubspot via manual workflow to google and start sync afterwords… Dont know how else its possible. Any ideas?

1 Like

Ah. I’m sorry buddy. I’m not a hubspot user. May be somebody else can help you.

Yeah, you will need initial sync and after that, you can use the Hubspot trigger and the Google Contacts node to keep them in sync.

Are you sure you are hitting the rate limit?

@RicardoE105 Yes, pretty sure I reach the limit. Is it possible to “reduce” the execution speed of the node or is it possible to get the full dataset 100-blockwise with some seconds delay in between? Or is there any other chance to get whole contact data set >1000 from hubspot? Atm I see no way with the basic Hubspot node functionality…

You can use the HTTP node and use the options batch interval and batch size.

haha, okay. Thanks for this hint, @RicardoE105 - can you explain a bit more how it works or may you give me an example? Have no clue…

Just add your hapikey in the HTTP node and change the NoOp node for the Google Contacts node and it should work.

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        320,
        300
      ]
    },
    {
      "parameters": {
        "url": "={{$node[\"Function1\"].json[\"next\"]}}",
        "options": {},
        "queryParametersUi": {
          "parameter": [
            {
              "name": "hapikey"
            },
            {
              "name": "limit",
              "value": "100"
            }
          ]
        }
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        770,
        140
      ]
    },
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1330,
        140
      ]
    },
    {
      "parameters": {
        "functionCode": "const { results } = items[0].json\n\nconst response = []\n\nfor (const result of results) {\n  response.push({ json: result })\n}\n\nreturn response;\n\n\n\n"
      },
      "name": "Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1050,
        140
      ]
    },
    {
      "parameters": {
        "conditions": {
          "string": [],
          "boolean": [
            {
              "value1": "={{$node[\"HTTP Request\"].json[\"paging\"] ? true : false}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1320,
        360
      ]
    },
    {
      "parameters": {
        "functionCode": "console.log(items)\n\nlet next = 'https://api.hubapi.com/crm/v3/objects/contacts'\n\nif (items[0].json.next) {\n  next = items[0].json.next\n}\n\nreturn [\n  {\n    json: {\n      next : next\n    }\n  }\n]"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        580,
        300
      ]
    },
    {
      "parameters": {},
      "name": "NoOp1",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1630,
        380
      ]
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "string": [
            {
              "name": "next",
              "value": "={{$node[\"HTTP Request\"].json[\"paging\"][\"next\"][\"link\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        1160,
        610
      ],
      "executeOnce": true
    },
    {
      "parameters": {
        "functionCode": "return new Promise((resolve, reject) => {\n      setTimeout(() => { resolve(items) }, 5000);\n    })\n"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1630,
        140
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "HTTP Request": {
      "main": [
        [
          {
            "node": "Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "NoOp": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "NoOp1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}
2 Likes

@RicardoE105 you are the best! Saw your message before, jumped out of my bed, copy-pasted and it worked. Thank you so much. Made my day already at 8 in the morning :smiley:

Never would have been able to get it run by myself. Will later try to dive in deeper to understand how it works. Now I understand why people often say http-request is one of the most powerful and versatile node.

4 Likes

Need help again @RicardoE105: Thanks to your workflow I receive now all data. But in 4+ separate executions. How can I merge them to one table? Found this here How to retrieve all executions data of a node? but sadly seems not working in my case without adaption… Thank you!

Glad that it worked. What do you mean with merge them to one table? I thought you wanted to add them to Google Contacts? You can add them in batches. Am I missing something?

have several use cases i can realize with this solution now. would be great to be able to merge the executions to one single dataset. e.g. makes it easy to create a contact list in sheets and many more things… when I try to work further with the data e.g. with set node i only get the first execution.

Whatever you want to do with the data, you can solve it by adding nodes after the function node.

makes it easy to create a contact list in sheets and many more things

In my mind, it’s the same amount of work either way unless I’m missing something.

But to answer your question, as far as I know, this is yet not possible.

Hi Ricardo, thank you for your support. As far as I understood your workflow the IF-node “false” output gives me the final result, right?

I think you are right. I tried to work further with the data and it seems I can do everything as usual! Think it is more a barrier in my head than in n8n :smiley:

However, Im wondering if its possible to put all executions together before executing the rest of the workflow with the full dataset… The example workflow from the thread I posted above seems to do exactly that, doesn’t it?

Hi Ricardo, thank you for your support.

Of course.

As far as I understood your workflow the IF-node “false” output gives me the final result, right?

No, it just tells you the loop finished. Perhaps, what confused you it’s there was a bug in the workflow that was executing the NoOp1 node multiples times when it should have been just once. Just updated the workflow.

However, Im wondering if its possible to put all executions together before executing the rest of the workflow with the full dataset… The example workflow from the thread I posted above seems to do exactly that, doesn’t it?

Just had a better look and it’s possible. Did not know this was possible. Like you, I’m still learning things about n8n all the time.

{
  "nodes": [
    {
      "parameters": {
        "url": "={{$node[\"Function1\"].json[\"next\"]}}",
        "options": {},
        "queryParametersUi": {
          "parameter": [
            {
              "name": "hapikey"
            },
            {
              "name": "limit",
              "value": "100"
            }
          ]
        }
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        660,
        200
      ]
    },
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1160,
        200
      ]
    },
    {
      "parameters": {
        "conditions": {
          "string": [],
          "boolean": [
            {
              "value1": "={{$node[\"HTTP Request\"].json[\"paging\"] ? true : false}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1150,
        430
      ]
    },
    {
      "parameters": {
        "functionCode": "\nlet next = 'https://api.hubapi.com/crm/v3/objects/contacts'\n\nif (items[0].json.next) {\n  next = items[0].json.next\n}\n\nreturn [\n  {\n    json: {\n      next : next\n    }\n  }\n]"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        410,
        370
      ]
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "string": [
            {
              "name": "next",
              "value": "={{$node[\"HTTP Request\"].json[\"paging\"][\"next\"][\"link\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        990,
        680
      ],
      "executeOnce": true
    },
    {
      "parameters": {
        "functionCode": "return new Promise((resolve, reject) => {\n      setTimeout(() => { resolve([{ json: {} }]) }, 5000);\n    })\n"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1460,
        200
      ]
    },
    {
      "parameters": {
        "functionCode": "const allData = []\n\nlet counter = 0;\ndo {\n  try {\n    const items = $items(\"HTTP Request\", 0, counter).map(item => item.json.results);\n                    \n    const aja = items[0].map(item => {\n      return { json: item }\n    })    \n    \n    allData.push.apply(allData, aja);\n    //allData.push($items(\"Increment\", 0, counter));\n  } catch (error) {\n    return allData;  \n  }\n\n  counter++;\n} while(true);\n\n"
      },
      "name": "Function2",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1430,
        450
      ]
    }
  ],
  "connections": {
    "HTTP Request": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "NoOp": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Function2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

btw just added a small change, the wait should be 5000 instead of 1000.

Hahaa, thank you so much @RicardoE105 !!! You are my hero! Now it’s exactly what I’ve been dreaming of…

Glad that it worked. Have fun.

Hey @RicardoE105! Sorry, I need to get your help on this one more time. Is it also possible to use this workflow to get all contacts from google? Since there is no trigger node for google contacts i need to look up for changes also on google side… When trying to get all contacts from that side I ran into the same problem since I have more than 500 contacts in the list… :-/

“Google Contacts error response [429]: Quota exceeded for quota metric ‘Critical read requests’ and limit ‘Critical read requests per minute per user’ of service 'people.googleapis.com

Slowly I get tired from trying to set up a two way sync between google contacts and hubspot. System seems not to be made for this. If its not possible I will stop the project and will only do one way sync from hubspot to google :exploding_head: :face_vomiting: