Newbie needing help converting a developer set up automation to work in n8n

[Robm]

A developer wrote and hosted and executed scripts for me that queried a 3rd party data base, pulled out data, processed that data and then inserted it into Mautic, an open source marketing automation tool for which n8n has a node.

I have access to all the scripts and code that the developer wrote. They are JS functions from what I can see.

My goal is to move this all to n8n but it is all quite a bit above my current know how. I am new to n8n (done the beginners course, and passed ). I am at the no code to cut and past code level.

I have been able to access the 3rd part database via api in n8n and retrieve the data I need. I can also access the mautic instance. It’s the middle part of processing data that I am struggling with.

Unsure of how to proceed to get community help. Do I detail out all the rather laborious detail here? Do I rather hope to connect with a kind soul who can help me out in my learnings be a sort of mentor?

I have used the “connect with an expert” service but I fear my tiny budget will be laughed at. My goal is to get this working but also learn and understand how it works.

[David_Go]

Hi @Robm

So if I understand you correctly you have the following:

• Full access to the 3rd party database & knowledge of how to pull the data you want using n8n
• You connected Mautic to n8n

So if I understand this correctly it should be pretty simple to connect both together and process or manipulate any data along the way before pushing it over to Mautic.

Since you did not provide any insight on how you need to "process that data " it would be hard to give you insight on how to do so : )

Hope that makes sense. If you want to provide an example of what you want to do with the data that would be best if you are looking for some help on the matter. Still, sounds like you figured out the hard part or having the 2 data sources connected to n8n

[Robm]

Hi David,

Glad you said I have figured out the hard part!

For me the middle part of processing the data is the hard part. Extracting it from the JSON, writing (or in this case, rather adapting a pre written) function/s is a skill I don’t have.

But the landscape has shifted slightly since my original post. Let me re - adjust before commenting further.

[David_Go]

@Robm

Extracting it from the JSON, writing (or in this case, rather adapting a pre written) function/s is a skill I don’t have.

I can relate to that - Still, that being said, rewriting a function might not be as hard as you might think.
What I did was look at other functions that did something similar and adapt it to meet my needs.

If you can strip out any sensitive data and make a sample flow with your current function script that might be helpful. Personally, I found that the code nodes in n8n can do about 90% of what is needed and the last 10% needed a function code.

Feel free to tag me when you make an update to your post or need any help along the way and I will do my best in any way I can.

[Robm]

@David_Go if I can get your help that would be great.

My exact use case will differ, but I figure what I will learn from this will help me learn.

I getting data via HTTP request and API.

The data received is entry data for all paddlers that have entered a race.
K1 is means they are paddling a single canoe. So if there is data for K1 but K2,3,4 or all null then that person is in a single.

Obv is there is data for K1 and K2 that is a double canoe and those are a 2 people canoe

Sample data set below.

[
{
"ok": true,
"entries": [
{
"k1": {
"name": "JON  DOE",
"csaId": 6769
},
"k2": {
"name": "Dave Doe",
"csaId": 61277
},
"k3": null,
"k4": null,
"entryDate": "2019-08-09",
"entryTime": "21:29:03",
"raceName": "GARA DOLPHIN COAST 17&18 AUG 2019",
"orderNumber": "ORD00001197"
},
{
"k1": {
"name": "Jane Doe",
"csaId": 8977
},
"k2": null,
"k3": null,
"k4": null,
"entryDate": "2019-08-09",
"entryTime": "18:12:36",
"raceName": "GARA DOLPHIN COAST 17&18 AUG 2019",
"orderNumber": "ORD00001194"
},
{
"k1": {
"name": "Sharon Doe",
"csaId": 555
},
"k2": null,
"k3": null,
"k4": null,
"entryDate": "2019-08-09",
"entryTime": "17:28:27",
"raceName": "GARA DOLPHIN COAST 17&18 AUG 2019",
"orderNumber": "ORD00001190"
},
}
]
}
]

The csaId is a unique identifier.

I need to extract all the entries for a race, determine if they are paddling a k1, k2 or a k3 (3 person canoe).
If they are paddling in a k2 or k3, Create a team name for them that will have the format of “T Jon Doe and Dave Doe” If a K3 then the third name would be added.

I then to send the data to mautic for which n8n has a node.

In mautic
The mautic node requires an email as the unique ID so I would need to get the API updated so I can get that info from the entry data base.

I need to look to see if the casId (email aswell?) already exists, if it does I need to update the both record with the team name if appropriate. There is a field called “Team name 2022” waiting to receive that. Both members of the team would need their individual records updated with the same team name.

If the record does not exist I would need to create it and populate it. There is a second API call I can do that searches the 3rd party data base by csaId and returns all the info needed to populate the new record in Mautic. How do I bring in the second API call? Below is the data I get back.

{
"ok": true,
"memberFound": true,
"memberDetails": {
"csaId": 394,
"firstName": "TIM",
"lastName": "TESTER",
"dateOfBirth": "1994-04-16",
"clubName": "Freedom Paddlers Club",
"unionName": "Western Cape",
"licenceTypeName": "Senior",
"expiryDate": "2022-08-29",
"activationDate": "2021-08-30",
"gender": "MALE",
"singlesProficiency": "AR+",
"doublesProficiency": "AR+",
"skiProficiency": "Open Ocean"
}
}
]

I need to tag the record “2022 Surfski Double” or “2022 Surfski Single” or “2022 Surfski Triple” based on if they are k1 k2 or k3.

In production I would have a cron check via API for new entries and I would only want to bring across new entries since the last check.

Complicated and you may need to be a detective to follwo all that.

[David_Go]

@Robm - When said send "make a sample flow " I was referring to in n8n then you can copy the sample flow in your reply
There are MANY ways to make this flow, and it would depend on a few factors… but here is a very very simple way to do it (what I understood from ur post)

(is this in the right direction? )

Here are a few other points:

• Mautic may require an email, but if you don’t need one for your records or for your flow you can disable that requirement in mautic
  

  

• You can use your “csaId” as a unique identifier in Mautic, this way you will know if the recode is already in Mautic or not.

• • Checking if the record is already in mautic or not can be done then just make an IF node right after to see if Mautic found the record or not.

[Robm]

@David_Go - you are legend - thank you.

Your sample flow gives me the basic logic of how to create this in n8n. Thanks!

Question.

The 1st set node, where does it get its data from? Can a set node pull data in via API? I would have thought a HTTP node would have been needed before the set?

The Switch node is new to me so I will go check the docs on how that works.

Is the 2nd HTTP node pulling in the license info via API or the event info? I m assuming the license info as the vent info would be needed at the start of the work flow?

I know I am going to struggle configuring the set and switch nodes. But let me give that a try before being lazy and asking how.

[David_Go]

@Robm I am happy to help.

@Robm

The 1st set node, where does it get its data from? Can a set node pull data in via API? I would have thought a HTTP node would have been needed before the set?

My example just had a simple Set note, so you would swap it out for your data input API or whatever your data source is.

@Robm

The Switch node is new to me so I will go check the docs on how that works.

There is great documentation on how this works Switch | Docs

@Robm

Is the 2nd HTTP node pulling in the license info via API or the event info? I m assuming the license info as the vent info would be needed at the start of the work flow?

That is just a simple HTTP get, I didn’t add any info to it. You can put any data source you want there or skip adding an email altogether

@Robm

I know I am going to struggle configuring the set and switch nodes. But let me give that a try before being lazy and asking how.

There are LOTS of videos on youtube on the basics of n8n. I would start there to give you lots of the basic info needed to get started. ie the flow I provided as an example for you is just an outline and would take about 30 min to set up and test to make sure works as intended. : )

[Robm]

Systematically working through this and trying hard to find the solution before posting here.

@David_Go I have set up the workflow as you have outlined. The HTTP node is pulling the correct data via API. The data set has lots of “entries” but the following set node only pulls in the data from one “entry”, the 1st one. I suspect that is because in the HTTP node all the data for all the entries are being dumped into 1 table row? In the beginner’s course, the data from the API call was in a distinct row for each record. See the screengrab below.

Am I correct in assuming that is why the set node is returning only a single record when there are 20 entries. Or is the issue with how I am configuring the set node?

And how to split the data into one record per row?

[David_Go]

@Robm - Sure you can split the data into one record per row.
The Set Node is there to set the data as you see fit. It is not there to change the number of entries. if you have 20 entries you will see them pass by the set node (and other nodes) one by one…

You can use other nodes to change the data around like the “Item Lists” node and “split in batch” node to change how your flow works.

Still, all that being said, I can’t give you much insight into your flow with partial data like the screenshot provided.

[Robm]

I seem to have fixed this, perhaps clumsily, using a combination of item list and set nodes.