Hubspot node 'get all contacts': only subset of max 300 entries

It seems like the People API which is the API that the Google Contact uses has the following rate limit.

So after a minute it should returns results again. Did you try just using the Google Contacts node contact:getAll with returnAll set to true. For just 500+ results I do not it will hit the rate limit.

Yes, I did try using the Google Contacts node contact:getAll with returnAll set to true and result was the error message above… dont know whats wrong, but it does not work…

mm weird. let’s try something. Instead of setting returnAll to true. Set an expression in the limit field like this {{1000}}

@RicardoE105: Big sorry! My mistake. Had for test reasons several nodes in the workflow and started before a google sheets request. After throwing it out no more problems to execute!

But since Im happy working with the “http get hubspot contacts loop” Im wondering if its possible to add custom properties as known from the regular Hubspot node where I can call a lot of additional fields I want to work with in n8n. If not I think Im thrown back again to GO without getting what I need… damn

However, don’t want to give up my dream :slight_smile: Do you see any chance make this possible?

Maybe some of the requirements we discuss here can be implemented in the regular Hubspot node to make it out of the box more flexible?!

But since Im happy working with the “http get hubspot contacts loop” Im wondering if its possible to add custom properties as known from the regular Hubspot node where I can call a lot of additional fields I want to work with in n8n. If not I think Im thrown back again to GO without getting what I need… damn

Sorry, can you please elaborate not sure what you mean by custom properties? Are you talking about listing contacts or are you talking about creating a contact?

Maybe some of the requirements we discuss here can be implemented in the regular Hubspot node to make it out of the box more flexible?!

You mean somehow controlling the rate limit?

Hi @RicardoE105, I mean the get all contacts loop you set up for me. Is there a chance to add properties like address, city, phone, mobile and so on?

The Hubspot Node „get“ supports the function. But here I hit the border of max 100 contacts.

Its a dilemma: With the Hubspot node I get all contact details but limited to 100 max and with your http request loop I can get all contacts but without additional properties.

Hope this makes the problem more transparent and clear for you.

Next to a solution for this problem I also experienced that the Hubspot node „create/update“ supports only std properties and no custom properties. But I thought this is a limitation on hubspot side, isnt it? If not it would also be great to make this possible.

For example: I have a custom property called „Membership type“ in Hubspot. There is no std property for this contact detail in Hubspot. So I created one. In n8n I can get this additional property but as far as I can see I can not create/update it.

@RicardoE105 i must apologize! Due to my inexperience I firmly assumed I would hit the API limit when I tried to get more than 500 contacts with the Hubspot node “get all”.

Now I have found out that it is possible! However, not if I enable the “get all” checkbox. Then I get an identical subset of 100 contacts several times as result. But if I overwrite the limit manually with an expression of lets say " {{10000}} " then it works properly.

In conclusion I think that the reason for all irritations and the effort we have made here in the past days on this topic is based on a tiny bug/malfunction of the Hubspot node?! It seems to me that with the “get all” checkbox active, the limit of 100 is kept and the node makes multiple calls in background until the maximum number of contacts in Hubspot is reached…!!!

Dude sorry, I thought I had answered.

Now I have found out that it is possible! However, not if I enable the “get all” checkbox. Then I get an identical subset of 100 contacts several times as result. But if I overwrite the limit manually with an expression of lets say " {{10000}} " then it works properly.

Yes, each request can load up to 1000 items but I set it to 100. So it’s easier to hit the rate limit that way. I gonna update to set the default to 1000 so it can better avoid hitting the rate limit.

In conclusion I think that the reason for all irritations and the effort we have made here in the past days on this topic is based on a tiny bug/malfunction of the Hubspot node?! It seems to me that with the “get all” checkbox active, the limit of 100 is kept and the node makes multiple calls in background until the maximum number of contacts in Hubspot is reached…!!!

Well yes, you can call it a malfunction I will call it inefficiency. I Will fix it and hopefully will be available in the next release.

@RicardoE105 back again on this topic. Thank you for your answer. Today I used the regular Hubspot node to get all contacts from our company account where we have 13K contacts.

  1. When I tried to set the expression manually to {{13000}} I received 500 entries (2 times an identical subset of 250 contacts) and

  2. when I tried to get all contacts with active “Return all” button I received 300 entries (3 times an identical subset of 100 contacts).

  3. When I tried to get all contacts with the http request loop it worked well and I received all contact entries without duplicates → Perfect

BUT: When I use the http request loop I only receive some basic contact information and I have not the option to use “Add Field” → “Properties” to add eg. phone numbers and so on to the request.

QUESTION: 1) Is it possible to add custom properties to the http request and if possible how can I do that?) or as an alternative 2) Is it possible to use the regular Hubspot node in such a loop to get all contacts with custom properties?

Sorry to keep bothering you… Hope you can help me find a solution to get all contacts not only up to 500 but 10K+ with all individual properties from Hubspot I need. :nerd_face:

  1. When I tried to set the expression manually to {{13000}} I received 500 entries (2 times an identical subset of 250 contacts) and

Ok, this won’t work cuz the contact APIs max limit is 1000. How many input items does the node have? Sounds like it has two items. It will be the reason for the duplication.

  1. when I tried to get all contacts with active “Return all” button I received 300 entries (3 times an identical subset of 100 contacts).

I’m investigating right now. It seems like there is an issue with the pagination.

BUT: When I use the http request loop I only receive some basic contact information and I have not the option to use “Add Field” → “Properties” to add eg. phone numbers and so on to the request.

QUESTION: 1) Is it possible to add custom properties to the http request and if possible how can I do that?) or as an alternative 2) Is it possible to use the regular Hubspot node in such a loop to get all contacts with custom properties?

Yes, this is possible. Just add a query parameter with key properties and value a comma-separated list of the fields you want the API to retrieve.

Sorry to keep bothering you… Hope you can help me find a solution to get all contacts not only up to 500 but 10K+ with all individual properties from Hubspot I need. :nerd_face:

Ahh, you are not bothering me at all. This is literally my work and by letting us know about these issues you help us to make the product better.

Hey, so cool! Can you assist please… I add “properties” to query parameters in the http request node and add properties comma seperated as value:

But as result I only get the first value… or do I need to add a field for each property I want to add?

Check the example below:

{
  "nodes": [
    {
      "parameters": {
        "url": "https://api.hubapi.com/contacts/v1/lists/all/contacts/all",
        "options": {},
        "queryParametersUi": {
          "parameter": [
            {
              "name": "hapikey"
            },
            {
              "name": "property[]",
              "value": "firstname"
            },
            {
              "name": "property[]",
              "value": "lastname"
            }
          ]
        }
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        540,
        290
      ]
    }
  ],
  "connections": {}
}

Something seems to be wrong with the example. For me it does not work… First and lastname is in the regular result included and the additional property fields do not change that. When I try to add custom fields like phone number or address it does not work… :exploding_head: :woozy_face:

Found what is happening. The workflow below does it, but you gotta wait until the next release as I had to add a new field to the HTTP Node since Hubspot has a particular way of sending arrays in the query string. Hopefully, we can figure what is happening tomorrow and you do not have to use the HTTP node but the Hubspot Node.

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "url": "=https://api.hubapi.com/contacts/v1/lists/all/contacts/all",
        "jsonParameters": true,
        "options": {
          "useQueryString": true
        },
        "queryParametersJson": "{ \"property\": [\"mobilephone\", \"firstname\"], \"hapikey\": \"yourapikey\" }"
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        530,
        300
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Version [email protected] with support for “useQueryString” just got released.

Hi @RicardoE105, hi @jan, thank you very much!
I tried the new http node. Properties can now be defined but the loop does not longer work and I get only a subset of 20 records.

{
  "nodes": [
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        850,
        300
      ]
    },
    {
      "parameters": {
        "conditions": {
          "string": [],
          "boolean": [
            {
              "value1": "={{$node[\"HTTP Request\"].json[\"paging\"] ? true : false}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        450,
        610
      ]
    },
    {
      "parameters": {
        "functionCode": "\nlet next = 'https://api.hubapi.com/crm/v3/objects/contacts'\n\nif (items[0].json.next) {\n  next = items[0].json.next\n}\n\nreturn [\n  {\n    json: {\n      next : next\n    }\n  }\n]"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        450,
        300
      ]
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "string": [
            {
              "name": "next",
              "value": "={{$node[\"HTTP Request\"].json[\"paging\"][\"next\"][\"link\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        650,
        550
      ],
      "executeOnce": true
    },
    {
      "parameters": {
        "functionCode": "return new Promise((resolve, reject) => {\n      setTimeout(() => { resolve([{ json: {} }]) }, 1000);\n    })\n"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1050,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "const allData = []\n\nlet counter = 0;\ndo {\n  try {\n    const items = $items(\"HTTP Request\", 0, counter).map(item => item.json.contacts);\n                    \n    const aja = items[0].map(item => {\n      return { json: item }\n    })    \n    \n    allData.push.apply(allData, aja);\n    //allData.push($items(\"Increment\", 0, counter));\n  } catch (error) {\n    return allData;  \n  }\n\n  counter++;\n} while(true);\n\n"
      },
      "name": "Function2",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        650,
        700
      ]
    },
    {
      "parameters": {
        "url": "=https://api.hubapi.com/contacts/v1/lists/all/contacts/all",
        "jsonParameters": true,
        "options": {
          "useQueryString": true
        },
        "queryParametersJson": "={\"property\": [\"mobilephone\", \"firstname\", \"lastname\", \"address\"], \"hapikey\": \"key\"}"
      },
      "name": "HTTP Request1",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        650,
        300
      ]
    }
  ],
  "connections": {
    "NoOp": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Function2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "HTTP Request1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "HTTP Request1": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Hope you can help me once more.

@Jakob_Tiebel just add the api key. Once the fix to the get:all is released, this will not be needed anymore.

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        360,
        300
      ]
    },
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1060,
        290
      ]
    },
    {
      "parameters": {
        "conditions": {
          "string": [],
          "boolean": [
            {
              "value1": "={{$node[\"HTTP Request1\"].json[\"paging\"] ? true : false}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        660,
        600
      ]
    },
    {
      "parameters": {
        "functionCode": "\nlet next = 'https://api.hubapi.com/crm/v3/objects/contacts'\n\nif (items[0].json.next) {\n  next = items[0].json.next\n}\n\nreturn [\n  {\n    json: {\n      next : next\n    }\n  }\n]"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        660,
        290
      ]
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "string": [
            {
              "name": "next",
              "value": "={{$node[\"HTTP Request1\"].json[\"paging\"][\"next\"][\"link\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        860,
        540
      ],
      "executeOnce": true
    },
    {
      "parameters": {
        "functionCode": "return new Promise((resolve, reject) => {\n      setTimeout(() => { resolve([{ json: {} }]) }, 1000);\n    })\n"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1260,
        290
      ]
    },
    {
      "parameters": {
        "functionCode": "const allData = []\n\nlet counter = 0;\ndo {\n  try {\n    const items = $items(\"HTTP Request\", 0, counter).map(item => item.json.contacts);\n                    \n    const aja = items[0].map(item => {\n      return { json: item }\n    })    \n    \n    allData.push.apply(allData, aja);\n    //allData.push($items(\"Increment\", 0, counter));\n  } catch (error) {\n    return allData;  \n  }\n\n  counter++;\n} while(true);\n\n"
      },
      "name": "Function2",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        890,
        730
      ]
    },
    {
      "parameters": {
        "url": "={{$node[\"Function1\"].json[\"next\"]}}",
        "jsonParameters": true,
        "options": {},
        "queryParametersJson": "={\"property\": [\"mobilephone\", \"firstname\", \"lastname\", \"address\"], \"hapikey\": \"key\"}"
      },
      "name": "HTTP Request1",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        860,
        290
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "NoOp": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Function2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "HTTP Request1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "HTTP Request1": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Ok, understand. I have to wait for this fix, right? It‘s not fixed in the Version [email protected], I installed yesterday to try it…?

Hey @Jakob_Tiebel!

We added the Use Querystring option to the HTTP Request node in 0.100.0. We are still working on improving the Hubspot node. :slightly_smiling_face:

ok, cool! Thanks much, I’m looking forward… Please let me know when done.