Welcome @MTA, solid end-to-end pipeline - the Telegram control interface keeping the human just in the topic selection step is the right call for this kind of content. One thing to watch as you scale: the YouTube Data API v3 has a default quota of 10,000 units per day, and a single video upload costs 1,600 units. With 5 videos per run (1 main + 4 Shorts), you’re using 8,000 units in one go - leaving very little headroom for metadata updates, thumbnail sets, or retries. Worth adding a quota check at the start of the workflow and parking the upload if you’re close to the limit, rather than having it fail mid-batch.
Clever idea to create a content funnel from SEC filings with almost no manual intervention really plays into n8n’s strengths. How do you handle the error recovery if a step fails mid-run? Is there any way to have a retry/checkpoint type of logic or does it just die and need to be restarted manually?
Great question! Currently the workflow stops at the failed node and requires manual restart from Telegram (just type “yap” again).
The current_video.txt file on Google Drive acts as a basic checkpoint — if ElevenLabs or Shotstack fails, the script is already saved so you don’t lose the AI-generated content.
For proper error recovery, I’m planning to add:
n8n’s built-in “Retry on Fail” option on critical nodes (ElevenLabs, Shotstack)
An error trigger workflow that sends a Telegram alert with the specific failed step
A “retry” command in Telegram that re-runs from the last saved checkpoint
n8n’s error workflow feature makes this pretty clean to implement — you can catch any node failure and route it to a recovery path. Just haven’t prioritized it yet since the main flow is stable.
Would love to hear how others handle retry logic in long-running pipelines!
You can access the automation system via the provided link, or if there is an automation system that suits your needs, you can contact me about that as well.
For retry logic in long-running pipelines like this one, the approach I use is stage-based state tracking in Google Sheets (or a simple database). Each pipeline run writes its current stage (script_done, voice_done, video_done) back to a row as it completes. If a node fails, the Error Workflow triggers, reads the current stage from that row, and can resume from the last completed checkpoint instead of restarting everything.
For the specific nodes you mentioned (ElevenLabs, Shotstack), I’d also enable n8n’s built-in “Retry on Fail” (2-3 retries, 30-60s wait) since both are external APIs that can have transient failures. The Error Workflow handles only the cases that exhaust all retries.
The Telegram “retry” command approach you planned is solid - just have it read the current stage from the sheet and jump to the right sub-workflow from there.
This is exactly the kind of production-grade thinking I was missing — thank you!
The Google Sheets stage tracking approach is elegant. Writing script_done, voice_done, video_done as the pipeline progresses means you always know exactly where you left off. Much cleaner than my current “restart from scratch” approach.
I’ll implement this as my next iteration:
Create a “pipeline_runs” sheet with columns: run_id, topic, stage, timestamp, status
Add a “Write Stage” node after each major step (script, voice, images, video)
Set up an Error Workflow that reads the current stage and routes to the right sub-workflow
Enable “Retry on Fail” (3 retries, 45s wait) on ElevenLabs and Shotstack nodes
The combination of Retry on Fail for transient API errors + Error Workflow for persistent failures covers basically every failure scenario cleanly.
Going to update the Gumroad package with this pattern once I’ve tested it. Will share the updated workflow here when it’s ready.
Really appreciate you sharing this — this is why I love the n8n community
