Hubspot node 'get all contacts': only subset of max 300 entries

Need help again @RicardoE105: Thanks to your workflow I receive now all data. But in 4+ separate executions. How can I merge them to one table? Found this here How to retrieve all executions data of a node? but sadly seems not working in my case without adaption… Thank you!

Glad that it worked. What do you mean with merge them to one table? I thought you wanted to add them to Google Contacts? You can add them in batches. Am I missing something?

have several use cases i can realize with this solution now. would be great to be able to merge the executions to one single dataset. e.g. makes it easy to create a contact list in sheets and many more things… when I try to work further with the data e.g. with set node i only get the first execution.

Whatever you want to do with the data, you can solve it by adding nodes after the function node.

makes it easy to create a contact list in sheets and many more things

In my mind, it’s the same amount of work either way unless I’m missing something.

But to answer your question, as far as I know, this is yet not possible.

Hi Ricardo, thank you for your support. As far as I understood your workflow the IF-node “false” output gives me the final result, right?

I think you are right. I tried to work further with the data and it seems I can do everything as usual! Think it is more a barrier in my head than in n8n :smiley:

However, Im wondering if its possible to put all executions together before executing the rest of the workflow with the full dataset… The example workflow from the thread I posted above seems to do exactly that, doesn’t it?

Hi Ricardo, thank you for your support.

Of course.

As far as I understood your workflow the IF-node “false” output gives me the final result, right?

No, it just tells you the loop finished. Perhaps, what confused you it’s there was a bug in the workflow that was executing the NoOp1 node multiples times when it should have been just once. Just updated the workflow.

However, Im wondering if its possible to put all executions together before executing the rest of the workflow with the full dataset… The example workflow from the thread I posted above seems to do exactly that, doesn’t it?

Just had a better look and it’s possible. Did not know this was possible. Like you, I’m still learning things about n8n all the time.

{
  "nodes": [
    {
      "parameters": {
        "url": "={{$node[\"Function1\"].json[\"next\"]}}",
        "options": {},
        "queryParametersUi": {
          "parameter": [
            {
              "name": "hapikey"
            },
            {
              "name": "limit",
              "value": "100"
            }
          ]
        }
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        660,
        200
      ]
    },
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1160,
        200
      ]
    },
    {
      "parameters": {
        "conditions": {
          "string": [],
          "boolean": [
            {
              "value1": "={{$node[\"HTTP Request\"].json[\"paging\"] ? true : false}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1150,
        430
      ]
    },
    {
      "parameters": {
        "functionCode": "\nlet next = 'https://api.hubapi.com/crm/v3/objects/contacts'\n\nif (items[0].json.next) {\n  next = items[0].json.next\n}\n\nreturn [\n  {\n    json: {\n      next : next\n    }\n  }\n]"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        410,
        370
      ]
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "string": [
            {
              "name": "next",
              "value": "={{$node[\"HTTP Request\"].json[\"paging\"][\"next\"][\"link\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        990,
        680
      ],
      "executeOnce": true
    },
    {
      "parameters": {
        "functionCode": "return new Promise((resolve, reject) => {\n      setTimeout(() => { resolve([{ json: {} }]) }, 5000);\n    })\n"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1460,
        200
      ]
    },
    {
      "parameters": {
        "functionCode": "const allData = []\n\nlet counter = 0;\ndo {\n  try {\n    const items = $items(\"HTTP Request\", 0, counter).map(item => item.json.results);\n                    \n    const aja = items[0].map(item => {\n      return { json: item }\n    })    \n    \n    allData.push.apply(allData, aja);\n    //allData.push($items(\"Increment\", 0, counter));\n  } catch (error) {\n    return allData;  \n  }\n\n  counter++;\n} while(true);\n\n"
      },
      "name": "Function2",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1430,
        450
      ]
    }
  ],
  "connections": {
    "HTTP Request": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "NoOp": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Function2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

btw just added a small change, the wait should be 5000 instead of 1000.

Hahaa, thank you so much @RicardoE105 !!! You are my hero! Now it’s exactly what I’ve been dreaming of…

Glad that it worked. Have fun.

Hey @RicardoE105! Sorry, I need to get your help on this one more time. Is it also possible to use this workflow to get all contacts from google? Since there is no trigger node for google contacts i need to look up for changes also on google side… When trying to get all contacts from that side I ran into the same problem since I have more than 500 contacts in the list… :-/

“Google Contacts error response [429]: Quota exceeded for quota metric ‘Critical read requests’ and limit ‘Critical read requests per minute per user’ of service 'people.googleapis.com

Slowly I get tired from trying to set up a two way sync between google contacts and hubspot. System seems not to be made for this. If its not possible I will stop the project and will only do one way sync from hubspot to google :exploding_head: :face_vomiting:

I’m going crazy… should work to get 500+ contacts with google contact node. Google Contacts Integration [GOT CREATED]

Why i get an error? I don’t understand this…

ahhh ahh to make it work with Google Calendar you have to make some small changes as the pagination it’s not the same as hubspot. With the Google Contacts node, you are hitting the rate limit?

Yes, see error message above. Seems that I hit the limit. Or what else can cause these messages?

Calendar is unimportant for me atm…

It seems like the People API which is the API that the Google Contact uses has the following rate limit.

So after a minute it should returns results again. Did you try just using the Google Contacts node contact:getAll with returnAll set to true. For just 500+ results I do not it will hit the rate limit.

Yes, I did try using the Google Contacts node contact:getAll with returnAll set to true and result was the error message above… dont know whats wrong, but it does not work…

mm weird. let’s try something. Instead of setting returnAll to true. Set an expression in the limit field like this {{1000}}

@RicardoE105: Big sorry! My mistake. Had for test reasons several nodes in the workflow and started before a google sheets request. After throwing it out no more problems to execute!

But since Im happy working with the “http get hubspot contacts loop” Im wondering if its possible to add custom properties as known from the regular Hubspot node where I can call a lot of additional fields I want to work with in n8n. If not I think Im thrown back again to GO without getting what I need… damn

However, don’t want to give up my dream :slight_smile: Do you see any chance make this possible?

Maybe some of the requirements we discuss here can be implemented in the regular Hubspot node to make it out of the box more flexible?!

But since Im happy working with the “http get hubspot contacts loop” Im wondering if its possible to add custom properties as known from the regular Hubspot node where I can call a lot of additional fields I want to work with in n8n. If not I think Im thrown back again to GO without getting what I need… damn

Sorry, can you please elaborate not sure what you mean by custom properties? Are you talking about listing contacts or are you talking about creating a contact?

Maybe some of the requirements we discuss here can be implemented in the regular Hubspot node to make it out of the box more flexible?!

You mean somehow controlling the rate limit?

Hi @RicardoE105, I mean the get all contacts loop you set up for me. Is there a chance to add properties like address, city, phone, mobile and so on?

The Hubspot Node „get“ supports the function. But here I hit the border of max 100 contacts.

Its a dilemma: With the Hubspot node I get all contact details but limited to 100 max and with your http request loop I can get all contacts but without additional properties.

Hope this makes the problem more transparent and clear for you.

Next to a solution for this problem I also experienced that the Hubspot node „create/update“ supports only std properties and no custom properties. But I thought this is a limitation on hubspot side, isnt it? If not it would also be great to make this possible.

For example: I have a custom property called „Membership type“ in Hubspot. There is no std property for this contact detail in Hubspot. So I created one. In n8n I can get this additional property but as far as I can see I can not create/update it.

@RicardoE105 i must apologize! Due to my inexperience I firmly assumed I would hit the API limit when I tried to get more than 500 contacts with the Hubspot node “get all”.

Now I have found out that it is possible! However, not if I enable the “get all” checkbox. Then I get an identical subset of 100 contacts several times as result. But if I overwrite the limit manually with an expression of lets say " {{10000}} " then it works properly.

In conclusion I think that the reason for all irritations and the effort we have made here in the past days on this topic is based on a tiny bug/malfunction of the Hubspot node?! It seems to me that with the “get all” checkbox active, the limit of 100 is kept and the node makes multiple calls in background until the maximum number of contacts in Hubspot is reached…!!!