Mailchimp Return All Members crashes

[xristos]

Hi,

I am using a Mailchimp node with Get Many members operation. The segment has around 50k members.
If I select Return All the execution runs endlessly and crashes.

Once I got an error message but cannot reproduce it. As far as I remember it was a time out error.

I am currently on starter plan. Could this be a memory issue that will be resolved by upgrading to pro plan?
Alternatively how can I get all the members? The split in batch as far as I understand cannot help in this case.
The only other option that comes to my mind is an HTTP request with split in batches to get the results.

• n8n version:1.3.1 also tested in 1.6.1
• Running n8n via n8n cloud

[EmeraldHerald]

Hi @xristos Sorry to hear you’re running into this - but this does unfortunately sound like a memory issue I believe we have a test account with quite a few “users”, let me give this a go

[EmeraldHerald]

Me again @xristos - While n8n hasn’t crashed for me, it’s taken quite a while and I still haven’t gotten all the results from the Mailchimp node for approximately 50,000 test users, so I’d believe your HTTP Request workaround would be the best suited, here

[xristos]

Thank you @EmeraldHerald. I tried with HTTP request with count 100k and it fetches it if I accept to show the data in the output warning message. In case I need to get the data in batches could you please describe how the workflow should look like?

Another question is how to access the values from the HTTPs note output in a Set Node? the JSON Schema is

[{"members": [{"id":"asasadsf", 
               "email":"acscf",
               ..... }
            ],
  "list_id": 1233,
  "total_items": 500000,
...
}
]

I thought I could use {{$json.members.id}} to access all id as I did in the Mailchimp node. This doesn’t work though.
If I use {{ $jmespath($json.members,"[].id" )}} the output is in an array so all ids end up in 1 item. I would like to have an item per member id. I must be missing some kind of JSON command. Or should I just use Split Items node?

[EmeraldHerald]

No problem, @xristos - could you try something like {{ $('HTTP Request').item.json.members.id}} as an expression?

[xristos]

@EmeraldHerald this also doesn’t work. Like {{$json.members.id}} the expression is correct only until members. I try to understand the Syntax and why it cannot access the JSON items. The output of Mailchimp node was equivalent to the “members” value and I could access all items Ids with {{$json.id}}.
Currently I am solving this by importing all fields as arrays and using a Split in Items node afterwards to seperate them.

[EmeraldHerald]

Can you post your workflow here along with some sample data (remove anything sensitive!) structured like what Mailchimp is returning? That would help us take a closer look

[xristos]

Sure @EmeraldHerald.

Mailchimp Node dummy output ({{$json.id}} returns ids in separate items)

[
  {
    "id": "8904e4325878d494cef710016d5e02ebe",
    "full_name": "xxx",
    "web_id": 69103010,
    "email_type": "html",
    "status": "unsubscribed",
    "unsubscribe_reason": "N/A (Unsubscribed by admin)",
    "consents_to_one_to_one_messaging": true,
    "last_changed": "2023-09-07T19:23:57+00:00",
    "language": "",
    "vip": false,
    "email_client": "iPhone",
  },
  {
    "id": "57b6823289354737d672519e32d70da427",
    "full_name": "sss",
    "web_id": 69077150,
    "email_type": "html",
    "status": "subscribed",
    "consents_to_one_to_one_messaging": true,
    "last_changed": "2023-09-07T19:23:57+00:00",
    "language": "",
    "vip": false,
    "email_client": "iPhone",
  }
]

HTTP Request output ( {{$json.members.id}} not working. Current solution {{ $jmespath($json.members, "[*].id")}} and Split in Items node)

[
  {
    "members": [
      {
        "id": "8904e432587d787494cef710016d5e02ebe",
        "full_name": "xxx",
        "web_id": 69103010,
        "email_type": "html",
        "status": "unsubscribed",
        "unsubscribe_reason": "N/A (Unsubscribed by admin)",
        "consents_to_one_to_one_messaging": true,
      },
      {
        "id": "57b7868232354737d672519e32d70da427",
        "full_name": "aaa",
        "web_id": 69077150,
        "email_type": "html",
        "status": "subscribed",
        "consents_to_one_to_one_messaging": true
      }
    ],
    "list_id": "0aec523fc5",
    "total_items": 30359
  }
]

Workflow is quite simple

I am using the workflow with HTTP Request because Return All in Mailchimp node crashes

[EmeraldHerald]

I’m not sure if this will be more memory efficient, @xristos , but you can give this a try without jmespath: