Merge node not behaving as expected

Questions
1

I have a merge node set to remove key matches.

I have set input 1 to be from my source data. - Padler 1 CSA

Input 2 comes from another branch of the automation that is checking to see if the record exists in Mautic, using the Mautic node.

The label for variable 2 is different (csa_no) from the variable for input 1 - but the possible out put would the same, in this case, a unique number.

But the merge node is not removing items that have the same result. They are still being passed through. I expect to be left only with the items from input 1 that do not have an number that matches input 1.

What am I doing wrong?

3

@Robm in the Merge node, you need to use the name of the JSON key instead of using an expression to resolve the key’s value.
So you have: {{$node["IF K"].json["Padler 1 CSA"]}}
Instead should just write: Padler 1 CSA

The “Property Input” parameters also support dot notation, so FYI you can write a nested key like “parentObject.myKeyName”.

We are working on adding hints underneath parameters like Merge node’s “Property Input 1”. If the hint below was written underneath “Property Input 1”, would it have helped you solve this issue? Would be very helpful to know!

Hint: “Enter the name of the property (JSON key). Supports dot-notation.”

1 Like
4

@maxT Thanks for the input.

I made the recommended changes, and both input’s resolve to the same output, but still all 4 items are passed through when it should only be 2 items.

Yes hints would help. Especially when one has very similar (identical most often) UI’s from node to node, all with gear icon that allow one to browse for expressions. There is no way to know what 1 node uses over another, for many nodes.

New Note
New Note804×432 42.3 KB

I am messing with the mock up data to insert an email address, which is why the last record does not have an email field. I am expecting the merge field to field match the 1st 2 records, and pass through the final 2 as the final 2 wont match with input, as input 2 is only bring in 2 records that both match with records 1 and 2.

5

Hi @Robm, could you share the JSON data returned by both your Mautic5 and your IF K1 node here? You can redact or remove everything but the Padler 1 CSA and csa_no fields.

My initial suspicion would be that the data returned isn’t 100% identical so it would be good to see the values you are working with here.

6

@Robm you’re in good hands with @MutedJam . Without having the output data of the Mautic node, I had to make some assumptions so I replaced it with a Function node.

Hoping this example helps unblock you - if not, please post the Mautic output as @MutedJam requested and we can take a closer look.

Also thanks for the feedback on parameter hints, totally agree that the UX of Merge node Property Input parameters arn’t ideal since it’s a different pattern than most params where you’re using an expression. The merge node is one of a handful of core nodes we’re planning to overhaul in the nearish future.

7

The mautic node returns a MOUNTAIN of data, so I have just pulled out the section that deals with the csa-no. for the 2 records.

Although I am using the csa no as the variable to compare, I will also want to be able to do the same with email. I have tried with email and had the same unexpected result.

"social": {
"csa_no": {
"id": "43",
"label": "CSA No",
"alias": "csa_no",
"type": "number",
"group": "social",
"object": "lead",
"is_fixed": "0",
"properties": "a:2:{s:9:"roundmode";s:1:"3";s:9:"precision";s:0:"";}",
"default_value": null,
"value": 12093,
"normalizedValue": 12093

For the other record

"social": {
"csa_no": {
"id": "43",
"label": "CSA No",
"alias": "csa_no",
"type": "number",
"group": "social",
"object": "lead",
"is_fixed": "0",
"properties": "a:2:{s:9:"roundmode";s:1:"3";s:9:"precision";s:0:"";}",
"default_value": null,
"value": 394,
"normalizedValue": 394

Data from the If K1 node

[
{
"Padler 1 CSA": 394,
"Padler 1 email": "redacted"
},
{
"Padler 1 CSA": 12093,
"Padler 1 email": "redacted"
},
{
"Padler 1 CSA": 3,
"Padler 1 email": "[email protected]"
},
{
"Padler 1 CSA": 5,
"Padler 2 CSA": ""
}
]
8

Hi @Robm, thanks for sharing the example data. It seems this doesn’t quite add up though.

Your Merge node returns a field Padler 1 CSA, but this field doesn’t show up in the example data you have provided. The Merge node would return items coming in on its Input 1 from the IF K1 node, so at least the four offending items should have this field.

For Input 2 (connected to your Mautic5 node), the example shared doesn’t seem to include a fields field which was part of your original expression. So I am not sure I have the full data structure here.

However, based on the sample data, your original expression and the screenshot you have shared, I would assume that the value you are currently using in the Property Input 2 field of the Merge node is the culprit.

As per @maxT’s comment above, nested fields can be referenced using the dot notation, so the correct values to use in your Merge node should probably be these ones:
image

The above example should work assuming:

  • Your items coming in on Input 1 have a top level field named Padler 1 CSA
  • Your items coming in on Input 2 have a field value nested inside csa_no nested inside social nested inside fields
1 Like
9

Super appreciate the help. I am waiting on the owners of the database I am querying to update their API so I pull the data I need. Been working with place holder data in the meantime to learn.

Can I press pause on this until I get the actual data and then pick it up again, after applying the insights supplied.

PS Blown away by the support provided by this community. Hoping one day I am skilled enough to do my part and help others.

3 Likes
10

I have the final data that I need from the API. I seem to have the merge node working and comparing email addresses.

Only 1 email address is not being removed by removed key matches and the only difference seems to be the input 1 is upper case and input 2 is lower case. Would the merge node be case sensitive?

If so how do I change case?

11

Yes, n8n is built on JavaScript which is a case sensitive language. In n8n you can, however, convert strings into all lowercase (or in any other way) using JavaScript inside expressions.

If, for example, the field Paddler 1 Email is the one containing an uppercase text you could simply add a Set node between your IF K1 Entry (k2 is empty) node and your Merge node. In the Set node, configure the Paddler 1 Email field to use an expression like {{$json["Paddler 1 Email"].toLowerCase()}}. The .toLowerCase() method would convert a value like [email protected] into [email protected].

Example Workflow
12

Thanks - super useful.

Since posting this I have realised that I have other data that is all in UPPER CASE that I need to have in Sentence Case, particularly First name and Last name.

What would the code for that - would it be .toSentenceCase? I just want to capitalize the 1st later of the name. A quick google shows it may not be that simple…?

13

You’re right, it’s not quite as simple (since there is no pre-built method for that), but still possible.

This would be a transformation where I’d personally use a Function Item node since the code is a bit more complex. The code could look like this (assuming the field I want to update is called myValue):

// First, split string into array of individual lowercase words
const words = item.myValue.toLowerCase().split(" ");

// Now convert the first character of each word in an UPPERCASE character and convert the array into a string again
const result = words.map(s => s.charAt(0).toUpperCase() + s.substring(1)).join(" ");

// Write the result into the myValue field
item.myValue = result;

// Return the updated item
return item;
Example Workflow

If your field name has spaces in it, you would need to use a different syntax than above. Instead of item.myValue, you’d need to reference it using item["my field name with spaces"]. Hope this helps, let me know if you run into any trouble here.

1 Like
14

Thanks. Struggling to get the function to work

My issue is identifying the variable I think?

{
"First Name": "LUKE",
"Last Name": "JOHNSON "
},

The function node would come after the set node that results in the above data. I assume “First Name” is the variable but I am pretty sure JS does not like spaces and wants camel case (just learning about camelCase and some very basic JS.)

15

Ah yes, I realized you might be using spaces and edited my post once I did. If your field name has spaces, you should be able to make this work by replacing item.First Name with item["First Name"].

16

Slow progress, but progress. I added in

item ["First Name"]

in the 1st line of the function as wells as the 3rd line of the function

item.["First Name"] = result;

But I can an error

ERROR: Unexpected token ‘[’

item.["First Name"] = result;
     ^

I have tried removing [ and ] but I get more errors

17

You’d need to remove the left over dot between item and ["First Name"].

I.e. item["First Name"] instead of item.["First Name"].

1 Like
18

Wow! I am almost have a complete working automation with a fair degree of complexity! STOKED!

It seems I have one final issue (for now…)

The source database allows for 2 records to have the same email address

The destination database (mautic) uses the email address as the unique identifier so duplicate emails would result on 1 record over writing the other - not good.

This is all data for the entry of a paddling race, 1 person, 2 person and 3 person craft are all entry options.

I have set an IF node to check if, in a 2 person boat, paddler 1 email address equals paddler 2’s email address. Then a SET node to add “+1” before the @ sign to make the email unique.

Question 1 - I am struggling to get an IF node to work to check if paddler 1’s email equals paddler 3’s email address, or if paddler 2’s email address equals paddlers 3’s email address, for the 3 man boat.

Question 2 - how to set mautic to use the modified email address, only when there is a duplicate?

19

After hrs of thinking, experimenting, sipping whiskey and trying again, I have this all fixed up.

2 Likes
20

Aww snap! That “aha” moment when the flow is working beautifully - happy to hear it!

If I may @Robm, if you had to pick 1 single thing… what was the most confusing part of this experience? To that end, if you had one magic wish to change anything in n8n, what would it be?

Thanks :pray:

21

Great question. There were, and continue to be, many frustrations, and aha moments. Not being a dev or a coder I had to really learn how to switch my thinking and get “into the flow” of n8n. That took time.

Perhaps the most confusing was not any particular node or expression. It was more understanding the logic of building out a flow and the thinking process it took to devise a solution. I would suggest more use case examples of different flows. I know there are quite a few but i would like to see more emphasis put on why a particular solution was put in place and the thinking that created the result, not just instructions on how to build out a flow. How does one transition from the "story: of what needs to be achieved, to the logical n8n steps that would be required to the physical building out and configuring of the flow.