Looking for a challenge? (filtering twitter webhook events)

Questions
1

Hello everybody,

So if you are reading this, I bet you are accepting the challenge!

Here is it: I have managed to create a n8n twitter account activity webhook (Overview | Docs | Twitter Developer Platform). Although Twitter has this weird thing that they send a token every day to which your webhook needs to respond with some HMAC code, I’ve managed to get that to work too (happy to share the workflow for those interested).

Now, in theory all these activities will be sent to the webhook

image
image799×378 10.6 KB

Yeah great, but that means A LOT of workflow triggers. I’m afraid my n8n cloud bill will ramp up quickly, and also it seems such a waste to not filter any of those activities. Twitter doesnt allow for any filtering, so I will have to find a way myself. In short: current setup will trigger n8n a lot, though I only need the DM activity.

Then I thought, perhaps I could find some webhook forwarder that filters the activity stream for me? Some do exist: hookdeck.com, webhookrelay.com… But then it doesn work with Twitter, because it is pinning the first time and every 24 hours with this token that you need to respond to within 3 seconds (the CRC token:

image
image786×130 8.59 KB

)
Those simple relays just give an instant 200 response.

So I am back to:
1- receive all the activities and risk to trigger many times the workflow (not scaleable)
2- find a tool that filters only specific activities (like DM) and can handle the Challenge response check

Who is the bright mind that will help me out with this one? :slight_smile:

Thanks!

2

Hey @bees8, this honestly sounds like n8n cloud might not be the best environment for the job. You could consider self-hosting n8n instead where the execution count wouldn’t matter (at least from a billing perspective, you’d still need to be careful when storing execution data to not have your database grow superfast).

3

hmmm, I like a challenge. But this is like @MutedJam said simply not possible with n8n cloud.
Also putting a tool in between your applications(in this case twitter) and n8n would be a bit of a work around not a real solution.

So selfhosting will be indeed the way to go. There are plenty of tutorials on how to do this on different types of hosting. There was also a talk from @Shirobachi I think on this at the last community meeting.
It would need some time of you to set it up but should be a fairly straight forward experience.

2 Likes
4

hey @BramKn and @MutedJam Thanks for your responses, really appreciate it :slight_smile:

I’ll see what I can do with self-hosting, perhaps just for the sake of this use case - I could filter all incoming requests and trigger the n8n cloud instance
Thanks!

1 Like
5

I’ll allow myself to link my repo what I was to talk about at last meetup like @BramKn said.

There is also tutorial step by step (link in repo) :wink:

2 Likes