Workflow stops if website error or doesn't load; processes some leads, then stops, won’t proceed to next lead

Describe the problem/error/question

Workflow stops if website error or doesn’t load; processes some leads, then stops, won’t proceed to next lead. Right nodes set to continue on error.

What is the error message (if any)? - it wouldn’t produce error, just says workflow complete.

Workflow stops when…

  • Website has an error and doesn’t load, shows up on google as website unexpectedly closed the connection.

  • Website is live but it might be blocking being scrapped

  • Website is live, scrapped, filled in all rows on google sheets, then deletes the actual created copy and cold email, the rest of the rows are still there though. So the copy pops up for a second, then disappears.

  • Once to the last row without a lead, it either keeps trying to process and then wipes out all the copy created for previous leads. If it does make it to the bottom of the lead list, how do I stop this from happening.

Workflow does works if I refresh, log out and log back in - but only like 3-8 leads get processed at a time. But sometimes it will keep stopping on leads that will process correctly but I have to keep doing this. The loop is set to process 1 lead at a time, and all leads are gathered before to enter into the workflow and showing up on the canvas.
The trouble nodes are…

Extract Page Links - which I have set to continue on error and links as array.

Split extract links - links to split out

Filter pages - json code for links

I tried adding an IF node, for if their are no links for the website and continue on error, right after Extract page links (and then the other 2 nodes as well) and none of it is working.
I’ve added code nodes; done all the AI suggestions and Grok. Nothing seems to be working.

I’d like to process more than 3-8 leads at a time! :upside_down_face::grin:

Has anyone had this issue before - what did you do to solve it?

Please share your workflow

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

{
  "nodes": [
    {
      "parameters": {},
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        -176,
        1056
      ],
      "id": "828a3a80-5fb6-4aa6-8803-c3147fa47b6d",
      "name": "When clicking ‘Execute workflow’"
    },
    {
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "strict",
            "version": 2
          },
          "conditions": [
            {
              "id": "85a175e9-bd34-41bc-aeda-d768423466ac",
              "leftValue": "website_url",
              "rightValue": "",
              "operator": {
                "type": "string",
                "operation": "notEmpty",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      },
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.2,
      "position": [
        400,
        1280
      ],
      "id": "a89014bb-6cd3-4e60-91d7-cbb82f01b756",
      "name": "If"
    },
    {
      "parameters": {
        "documentId": {
          "__rl": true,
          "value": "1sDp9fA8E57rQvQRhLEXTrxwLpZuSYfaf07NkcLtl2kg",
          "mode": "list",
          "cachedResultName": "C&C - FL - 1-10 employees - 881 emails verified + websites  - Apollo DL",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1sDp9fA8E57rQvQRhLEXTrxwLpZuSYfaf07NkcLtl2kg/edit?usp=drivesdk"
        },
        "sheetName": {
          "__rl": true,
          "value": 1464883367,
          "mode": "list",
          "cachedResultName": "Test",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1sDp9fA8E57rQvQRhLEXTrxwLpZuSYfaf07NkcLtl2kg/edit#gid=1464883367"
        },
        "options": {}
      },
      "id": "d3073732-ef64-41b1-ad53-6298767443a8",
      "name": "Fetch Leads from Sheet1",
      "type": "n8n-nodes-base.googleSheets",
      "typeVersion": 4.5,
      "position": [
        80,
        1072
      ],
      "retryOnFail": true,
      "credentials": {
        "googleSheetsOAuth2Api": {
          "id": "tQjusqhporXcfxag",
          "name": "Google Sheets account"
        }
      },
      "onError": "continueErrorOutput"
    },
    {
      "parameters": {
        "options": {}
      },
      "id": "14a97926-52d4-4039-a83f-465dc93274af",
      "name": "Process Each Lead1",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 3,
      "position": [
        304,
        1072
      ],
      "retryOnFail": true,
      "onError": "continueErrorOutput"
    },
    {
      "parameters": {
        "url": "={{ $json.website_url }}",
        "options": {
          "allowUnauthorizedCerts": true
        }
      },
      "id": "cc2437bb-ee4f-443a-b09b-34e441412dbe",
      "name": "Fetch Home Page1",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        592,
        1200
      ],
      "retryOnFail": false,
      "onError": "continueRegularOutput"
    },
    {
      "parameters": {
        "operation": "extractHtmlContent",
        "extractionValues": {
          "values": [
            {
              "key": "links",
              "cssSelector": "a",
              "returnValue": "attribute",
              "attribute": "href",
              "returnArray": true
            }
          ]
        },
        "options": {
          "trimValues": true
        }
      },
      "id": "3770f0f3-108b-4b89-874b-9b9c6228c2aa",
      "name": "Extract Page Links1",
      "type": "n8n-nodes-base.html",
      "typeVersion": 1.2,
      "position": [
        752,
        1200
      ],
      "retryOnFail": false,
      "onError": "continueRegularOutput"
    },
    {
      "parameters": {
        "fieldToSplitOut": "links",
        "options": {}
      },
      "id": "3ec3aca4-cabe-4694-871f-5999bb365b83",
      "name": "Split Extracted Links1",
      "type": "n8n-nodes-base.splitOut",
      "typeVersion": 1,
      "position": [
        944,
        1184
      ],
      "onError": "continueRegularOutput"
    },
    {
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": false,
            "leftValue": "",
            "typeValidation": "strict",
            "version": 2
          },
          "conditions": [
            {
              "leftValue": "={{ $json.links }}",
              "rightValue": "/",
              "operator": {
                "type": "string",
                "operation": "startsWith"
              },
              "id": "3d1ae5de-b593-4745-a858-5129386d98c7"
            },
            {
              "leftValue": "={{ $json.links }}",
              "rightValue": "about|program|offer|service|blog|price|course|coach",
              "operator": {
                "type": "string",
                "operation": "regex"
              },
              "id": "bb8056cf-9471-47f4-8e4f-327293fec72f"
            }
          ],
          "combinator": "or"
        },
        "looseTypeValidation": "={{ false }}",
        "options": {
          "ignoreCase": "=\"options\": {\n  \"caseSensitive\": true,\n  \"leftValue\": \"\",\n  \"typeValidation\": \"strict\",\n  \"version\": 2\n}"
        }
      },
      "id": "011ffcf1-9896-40e4-8932-146d9c6df17a",
      "name": "Filter Key Pages1",
      "type": "n8n-nodes-base.filter",
      "typeVersion": 2.2,
      "position": [
        1232,
        1168
      ],
      "retryOnFail": true,
      "onError": "continueErrorOutput"
    },
    {
      "parameters": {},
      "id": "caca891c-d03c-4cec-94ef-75fbbdd4eea1",
      "name": "Deduplicate Links1",
      "type": "n8n-nodes-base.removeDuplicates",
      "typeVersion": 1.1,
      "position": [
        1408,
        1072
      ]
    },
    {
      "parameters": {
        "jsCode": "return $input.all().map(item => {\n  let links = item.json?.links; // Safely access links\n  let cleanedLink = '/'; // Default fallback\n\n  if (links) {\n    if (Array.isArray(links) && links.length > 0) {\n      links = links[0]; // Use first link if array\n    }\n    if (typeof links === 'string') {\n      if (!links.startsWith('/')) {\n        if (links.startsWith('http')) {\n          try {\n            // Check if URL is defined, use it if available\n            if (typeof URL === 'function') {\n              const u = new URL(links);\n              cleanedLink = u.pathname.endsWith('/') && u.pathname !== '/' ? u.pathname.slice(0, -1) : u.pathname;\n              if (cleanedLink === '') cleanedLink = '/';\n            } else {\n              // Fallback: Manual parsing for protocol and path\n              const pathStart = links.indexOf('/', links.indexOf('://') + 3);\n              if (pathStart !== -1) {\n                cleanedLink = links.substring(pathStart);\n                cleanedLink = cleanedLink.endsWith('/') && cleanedLink !== '/' ? cleanedLink.slice(0, -1) : cleanedLink;\n                if (cleanedLink === '') cleanedLink = '/';\n              }\n              console.log('URL not available, using manual parsing for:', links);\n            }\n          } catch (e) {\n            console.log('URL Parsing Error:', e.message);\n            cleanedLink = '/';\n          }\n        } else {\n          cleanedLink = '/'; // Non-HTTP, non-relative fallback\n        }\n      } else {\n        cleanedLink = links; // Keep relative URLs as-is\n      }\n    } else {\n      console.log('Invalid link type:', typeof links);\n    }\n  } else {\n    console.log('Links undefined or null');\n  }\n\n  item.json.links = cleanedLink;\n  return item;\n});"
      },
      "id": "ec334353-d845-4f85-8af8-d8376e5a4b83",
      "name": "Clean Link Paths1",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        1632,
        1072
      ]
    },
    {
      "parameters": {
        "url": "={{ $json.website_url + $json.links }}",
        "options": {}
      },
      "id": "314a8354-f28f-404c-872f-462a4daddf76",
      "name": "Fetch Sub Pages1",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        1984,
        880
      ]
    },
    {
      "parameters": {
        "html": "={{ $json.data || '<empty>' }}",
        "options": {}
      },
      "id": "fcb5f400-1b30-4bba-949b-3f09854851ae",
      "name": "Convert to MD1",
      "type": "n8n-nodes-base.markdown",
      "typeVersion": 1,
      "position": [
        2288,
        1072
      ]
    },
    {
      "parameters": {
        "modelId": "gpt-4o-mini",
        "messages": {
          "values": [
            {
              "content": "Expert at concise website summaries.",
              "role": "system"
            },
            {
              "content": "Summarize page in laconic style, 1-2 paragraphs max."
            },
            {
              "content": "={{ $json.data }}"
            }
          ]
        },
        "jsonOutput": true,
        "options": {}
      },
      "id": "72cc5b37-2e0d-4dc5-b439-ed9e966b5036",
      "name": "Page Summary AI1",
      "type": "@n8n/n8n-nodes-langchain.openAi",
      "typeVersion": 1.6,
      "position": [
        2496,
        1008
      ],
      "credentials": {
        "openAiApi": {
          "id": "4oIN3GWxQzEGLoXq",
          "name": "OpenAi account"
        }
      }
    },
    {
      "parameters": {
        "fieldsToAggregate": {
          "fieldToAggregate": [
            {
              "fieldToAggregate": "message.content"
            }
          ]
        },
        "options": {}
      },
      "id": "4c9b0f7b-745e-4bb1-b6ce-8a8c676a0a17",
      "name": "Combine Summaries1",
      "type": "n8n-nodes-base.aggregate",
      "typeVersion": 1,
      "position": [
        2800,
        1008
      ]
    },
    {
      "parameters": {
        "modelId": {
          "__rl": true,
          "value": "gpt-4o",
          "mode": "id"
        },
        "messages": {
          "values": [
            {
              "content": "We just scraped a series of web pages for a business called. Analyze the coaching business deeply.  Next, your task is to take their summaries and turn them into catchy, personalized openers for a cold email campaign to imply that the rest of the campaign is personalized.",
              "role": "system"
            },
            {
              "content": "=From summaries: 1. Business summary <3 paras, laconic. 2. List offerings with category (hourly, low-ticket <$2500, high-ticket >$2500), extract prices. 3. Infer growth/scale problems. JSON: {\"summary\": \"...\", \"offerings\": [{name, category, price}], \"problems\": [\"...\"]}"
            },
            {
              "content": "=={{ $('Combine Summaries1').item.json.content.map(item => item.summary.overview || item.summary.description || item.summary.introduction || '').join('\\n\\n') }}"
            }
          ]
        },
        "jsonOutput": true,
        "options": {}
      },
      "id": "adcf33cc-905c-4554-aef4-5accb6a7b029",
      "name": "Business Analysis AI1",
      "type": "@n8n/n8n-nodes-langchain.openAi",
      "typeVersion": 1.6,
      "position": [
        2976,
        1008
      ],
      "credentials": {
        "openAiApi": {
          "id": "4oIN3GWxQzEGLoXq",
          "name": "OpenAi account"
        }
      }
    },
    {
      "parameters": {
        "jsCode": "const analysis = $json.message.content;\nconst offs = analysis.offerings || [];\nlet cat = 'high-ticket';\nif (offs.some(o => o.category.includes('hourly'))) cat = 'hourly';\nelse if (offs.some(o => o.category.includes('low-ticket'))) cat = 'low-ticket';\nreturn [{json: {...$input.item.json, lead_category: cat, analysis}}];"
      },
      "id": "d48e321d-37ae-42b1-8f6f-c4c0bd03a2d6",
      "name": "Categorize Lead1",
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        3280,
        1008
      ]
    },
    {
      "parameters": {
        "modelId": "gpt-4o",
        "messages": {
          "values": [
            {
              "content": "Craft laconic, personalized icebreakers.",
              "role": "system"
            },
            {
              "content": "=Profile: {{ $('Process Each Lead1').item.json.first_name }},{{ $('Process Each Lead1').item.json.last_name }}, {{ $('Process Each Lead1').item.json.location }}\nSummary: {{ $json.analysis.summary }}\nOfferings: {{ JSON.stringify($json.analysis.offerings) }}\nProblems: {{ $json.analysis.problems.join(', ') }}\nCategory: {{ $json.lead_category }}\nCreate 1-2 sentence multi-line icebreaker, non-obvious details, shorten names/locs. If hourly: shift to premium sig offer. Low-ticket: high-ticket position. High-ticket: more qual leads. JSON: {\"multiline_icebreaker\": \"Hey {short_name}...\\nCreeped your site, noticed {unique}...\"}"
            }
          ]
        },
        "jsonOutput": true,
        "options": {
          "temperature": 0.6
        }
      },
      "id": "75aa352f-23e2-42a6-af16-2f4ad6a0d330",
      "name": "Generate Icebreaker1",
      "type": "@n8n/n8n-nodes-langchain.openAi",
      "typeVersion": 1.6,
      "position": [
        3472,
        1008
      ],
      "credentials": {
        "openAiApi": {
          "id": "4oIN3GWxQzEGLoXq",
          "name": "OpenAi account"
        }
      }
    },
    {
      "parameters": {
        "operation": "update",
        "documentId": {
          "__rl": true,
          "value": "1sDp9fA8E57rQvQRhLEXTrxwLpZuSYfaf07NkcLtl2kg",
          "mode": "list",
          "cachedResultName": "C&C - FL - 1-10 employees - 881 emails verified + websites  - Apollo DL",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1sDp9fA8E57rQvQRhLEXTrxwLpZuSYfaf07NkcLtl2kg/edit?usp=drivesdk"
        },
        "sheetName": {
          "__rl": true,
          "value": 1464883367,
          "mode": "list",
          "cachedResultName": "Test",
          "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1sDp9fA8E57rQvQRhLEXTrxwLpZuSYfaf07NkcLtl2kg/edit#gid=1464883367"
        },
        "columns": {
          "mappingMode": "defineBelow",
          "value": {
            "multiline_icebreaker": "={{ $json.message.content.multiline_icebreaker }}",
            "row_number": "={{ $('Process Each Lead1').item.json.row_number }}",
            "first_name": "={{ $('Process Each Lead1').item.json.first_name }}",
            "last_name": "={{ $('Process Each Lead1').item.json.last_name }}",
            "email": "={{ $('Process Each Lead1').item.json.email }}",
            "website_url": "={{ $('Process Each Lead1').item.json.website_url }}",
            "location": "={{ $('Process Each Lead1').item.json.location }}",
            "phone_number": "={{ $('Process Each Lead1').item.json.phone_number }}",
            "summary": "={{ $('Business Analysis AI1').item.json.message.content.summary }}",
            "offerings": "={{ $('Business Analysis AI1').item.json.message.content.offerings }}",
            "problems": "={{ $('Business Analysis AI1').item.json.message.content.problems }}"
          },
          "matchingColumns": [
            "row_number"
          ],
          "schema": [
            {
              "id": "first_name",
              "displayName": "first_name",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "last_name",
              "displayName": "last_name",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "email",
              "displayName": "email",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "website_url",
              "displayName": "website_url",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "location",
              "displayName": "location",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "phone_number",
              "displayName": "phone_number",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "summary",
              "displayName": "summary",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "offerings",
              "displayName": "offerings",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "problems",
              "displayName": "problems",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "category",
              "displayName": "category",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": true
            },
            {
              "id": "multiline_icebreaker",
              "displayName": "multiline_icebreaker",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true
            },
            {
              "id": "multiline_icebreaker 2",
              "displayName": "multiline_icebreaker 2",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "row_number",
              "displayName": "row_number",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "number",
              "canBeUsedToMatch": true,
              "readOnly": true,
              "removed": false
            }
          ],
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {}
      },
      "id": "638731e4-7284-4c0c-b8d2-e60e5ca427c9",
      "name": "Update Icebreaker in Sheet1",
      "type": "n8n-nodes-base.googleSheets",
      "typeVersion": 4.5,
      "position": [
        3840,
        1328
      ],
      "credentials": {
        "googleSheetsOAuth2Api": {
          "id": "tQjusqhporXcfxag",
          "name": "Google Sheets account"
        }
      }
    },
    {
      "parameters": {
        "maxItems": 10
      },
      "id": "ff4dc207-1813-44b5-8a28-6b012aa5c508",
      "name": "Cap at 10 Pages1",
      "type": "n8n-nodes-base.limit",
      "typeVersion": 1,
      "position": [
        1808,
        1072
      ]
    },
    {
      "parameters": {
        "url": "={{ $('Process Each Lead1').item.json.website_url }}\n",
        "options": {
          "allowUnauthorizedCerts": true
        }
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        2032,
        1072
      ],
      "id": "b78e8909-ed46-4571-94f0-427dfeb151f6",
      "name": "HTTP Request1"
    },
    {
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "strict",
            "version": 2
          },
          "conditions": [
            {
              "id": "80886ced-8c60-4afc-ad99-798dabb1f3e3",
              "leftValue": "={{ $json.links }}",
              "rightValue": "",
              "operator": {
                "type": "number",
                "operation": "notEmpty",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      },
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.2,
      "position": [
        976,
        1344
      ],
      "id": "c5878d51-c9c0-48fd-b534-12b89caba17c",
      "name": "If1",
      "onError": "continueRegularOutput"
    }
  ],
  "connections": {
    "When clicking ‘Execute workflow’": {
      "main": [
        [
          {
            "node": "Fetch Leads from Sheet1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "If": {
      "main": [
        [
          {
            "node": "Fetch Home Page1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Leads from Sheet1": {
      "main": [
        [
          {
            "node": "Process Each Lead1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Process Each Lead1": {
      "main": [
        [
          {
            "node": "Update Icebreaker in Sheet1",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "If",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Home Page1": {
      "main": [
        [
          {
            "node": "Extract Page Links1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract Page Links1": {
      "main": [
        [
          {
            "node": "Split Extracted Links1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Split Extracted Links1": {
      "main": [
        [
          {
            "node": "Filter Key Pages1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Filter Key Pages1": {
      "main": [
        [
          {
            "node": "Deduplicate Links1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Deduplicate Links1": {
      "main": [
        [
          {
            "node": "Clean Link Paths1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Clean Link Paths1": {
      "main": [
        [
          {
            "node": "Cap at 10 Pages1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert to MD1": {
      "main": [
        [
          {
            "node": "Page Summary AI1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Page Summary AI1": {
      "main": [
        [
          {
            "node": "Combine Summaries1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Combine Summaries1": {
      "main": [
        [
          {
            "node": "Business Analysis AI1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Business Analysis AI1": {
      "main": [
        [
          {
            "node": "Categorize Lead1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Categorize Lead1": {
      "main": [
        [
          {
            "node": "Generate Icebreaker1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Icebreaker1": {
      "main": [
        [
          {
            "node": "Update Icebreaker in Sheet1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Update Icebreaker in Sheet1": {
      "main": [
        [
          {
            "node": "Process Each Lead1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Cap at 10 Pages1": {
      "main": [
        [
          {
            "node": "HTTP Request1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "HTTP Request1": {
      "main": [
        [
          {
            "node": "Convert to MD1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "If1": {
      "main": [
        [],
        []
      ]
    }
  },
  "pinData": {
    "When clicking ‘Execute workflow’": [
      {}
    ]
  },
  "meta": {
    "templateCredsSetupCompleted": true,
    "instanceId": "58c83bc792d48487def27e8ded5c4e3ca6c50e300db7131642beca444c0254a6"
  }
}

Share the output returned by the last node

Information on your n8n setup

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

Hey there!

I think you should disconnect the Done output for the Loop node, as it will probably wipe the sheet at the very end. That should be connected to a (slack, email, etc.) notification or not at all.

Can you try that and let me know what remains of the issues?

Hi Kris, thank you for your prompt reply! I should have put in the notes I already removed the done as an previous try. That’s not working either. It will process leads until it gets to a website that doesn’t load or won’t let me scrap, that does load or a REALlY old style website. ;-?/

And if I put an IF node right after filter key pages, where IF links is not empty, continue on error with output, it still doesn’t work. I have 2 IF nodes prior as well to catch it before and that’s not helping. I set a 3rd up with continue on error and all outputs to go to next step or loop back to process each lead node to start the next lead, if it doesn’t work. Not working either.

I’m really not sure what to do at this point.

Any other ideas?

Hey, I did a quick example flow, which handles errors along the way and loops back before continuing to the next node, if that happens. I used Continue via Error output in the settings of the nodes that can fail.

I haven’t tested with a website you can reach but not scrape - feel free to provide one you know fails in that specific way, so I can test that too.

Ah great idea!

Here are the websites so far stopping the workflow and not working. Some don’t load at all, some are old and some load but probably can’t be scrapped.

Curious to see if this works for you!

If you wish to reduce the time before the page fetching fails, you can set the timeout option to a value less then a default 10 seconds. This is going to help with sites like

https://webandflow.net

which doesn’t fail right away when fetched.

Thank you Jabbson, I get what you’re saying now, deleted my previous comment as I thought I had done that already. I added that on the fetch home node to process 3 times w/ 1000 ms; but it goes through but stops the workflow still.

I’m trying to solve for 1) bad website that doesn’t load and workflow still goes to next lead, 2) good website but blocking scrapping and WF still goes to next lead, 3) old website that isn’t letting scrap or causing it to stop and WF goes to next lead.
Any other suggestions? Thank you so much for your help!

This works for all the sites you provided, give it a try. I only added a timeout to the HTTP node.

1 Like

Kris you’re the best - thank you!! That worked, it’s now skipping the bad or websites that can load but can’t scrap and moving to the next and completely WF. And then stopping at the end and nothing is erased. Thank you!

Awesome!! Would appreciate if you mark my last comment as Solution :slight_smile:

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.