New and need a custom function

Hey all! I’m Matthew and I’m a complete noob to N8N.
I have very little experience (like an absolute beginner) with js and JSON so I’m sure I’m in over my head with n8n. I spoke with Thomas and Harshil for my onboarding zoom, they mentioned trying out the community… so here I am. a little more background info. I have a basic understanding of the architecture needed to accomplish what I have in my head but I lack the experience or knowledge to see it through. i have managed to accomplish a slimmed-down version on another software but they were insane price-wise. I am hoping to accomplish the same or more here on n8n but to be honest the custom functions are intimidating for a novice like me.

Project scope:
on receipt of webhook call from Jotform parse, responses modify some data for conversion several layers, (I have this figured out, mostly) simultaneously authorize with HTTP request and pull job data model, done.
Please Keep in mind the JSON I receive from the Job Data Model is 1200+ things in 1 item response and is variable depending on the job I am pulling in. The software we pull the job data requires me to chunk the final payload in the same format & Structure and send the HTTP post chunked.

What I am needing:
using a custom function specifically replace specific items in the job data model JSON response with the form responses after modifications or from the original raw JSON from Jotform respectively.

Merge the data and place it back in place of the original Job Data Model in its full format and all the unmodified data from the original payload. stringify/chunk it and set it as output for the last step the Post HTTP request.

One important thing to note is the JSON seems to be a Multi-Tier Array Containing a lot of other arrays of key-value pairs.

[{“id”:“Project Name”,“value”:“Matt-30X50X16-Tulsa”};
{“id”:“Job Number”,“value”:“100562387”}]

etc etc there is upwards of 100-140 items I will need to change and merge into the Value side where the “id” is equal to a specific name.

I have hit a snag or two with trying to get a snippet of the JSON Structure. I will work through the issue and get a copy of it and post it slimmed down. I will either add a comment or update the original post.

I hope I explained it well enough thanks for taking the time to read and not stoning me for my ignorance.

UPDATE: here is the slimmed-down JSON sample of the jobdatamodel, when I copied it into VSCode to slim it down it lost formatting.

[
    {
    "answers": [
    {
    "Model": {
    "TemplateId": 50,
    "Doors": [
    ],
    "Windows": [
    ],
    "Sheds": [
    ],
    "WallOpens": [
    ],
    "Accessories": [
    ],
    "WallOffsets": null,
    "InteriorWalls": null,
    "AddedRecs": [
    {
    "addedrecid": null,
    "Category": "9",
    "Kind": "Catalog",
    "Usage": "Freight Truck--$4.00 Non-Taxable Freight-$3.75 Non-Taxable Per Loaded Mile",
    "Sku": "MISC_FREIGHT375NT",
    "FullSku": "MISC_FREIGHT375NT",
    "Material": "$3.75 Non Taxable per Loaded Mile",
    "Calculation": "if("{JOB-Delivery}"=0,{JOB-Milestosite}*0*{JOB-DeliveryTrips},If("{JOB-Delivery}"=1,{JOB-Milestosite}*1*{JOB-DeliveryTrips}))",
    "CalcBase": null,
    "Qty": 54,
    "CalcFactor": 1,
    "Length": 0,
    "ColorCode": null,
    "ColorName": null,
    "Cost": 4,
    "Price": 4.4,
    "Weight": 0,
    "PriceError": 0,
    "TaxClass": 0
    },
    {
    "addedrecid": null,
    "Category": "9",
    "Kind": "Catalog",
    "Usage": "Freight Donkey--$50.00 Donkey Charge-DONKEY OFF-LOAD",
    "Sku": "MISC_DONKEY",
    "FullSku": "MISC_DONKEY",
    "Material": "Donkey Off-Load Charge",
    "Calculation": "if("{JOB-Delivery}"=0,1*0*1*{JOB-DeliveryTrips},If("{JOB-Delivery}"=1,1*1*1*{JOB-DeliveryTrips}))",
    "CalcBase": null,
    "Qty": 1,
    "CalcFactor": 1,
    "Length": 0,
    "ColorCode": null,
    "ColorName": null,
    "Cost": 50,
    "Price": 55,
    "Weight": 0,
    "PriceError": 0,
    "TaxClass": 1
    }
    ],
    "Overrides": {
    },
    "ModeledOptions": [
    ]
    },
    "Answers": [
    {
    "id": "ProjectName",
    "value": "211982-Test-30X500X10.5-MUSKOGEE"
    },
    {
    "id": "JobNumber",
    "value": "299001"
    },
    {
    "id": "CustomerCompany",
    "value": null
    },
    {
    "id": "CustomerFirstName",
    "value": "TESTY"
    },
    {
    "id": "CustomerLastName",
    "value": "MCTESTERSON"
    },
    {
    "id": "CustomerAddress",
    "value": "10001 E DAVIS FIELD RD"
    },
    {
    "id": "CustomerCity",
    "value": "MUSKOGEE"
    },
    {
    "id": "CustomerState",
    "value": "OK"
    },
    {
    "id": "CustomerZip",
    "value": "74403"
    },
    {
    "id": "Zone",
    "value": "ZONE A"
    },
    {
    "id": "CrewTrips",
    "value": "1"
    },
    {
    "id": "Installation",
    "value": "1"
    },
    {
    "id": "Delivery",
    "value": "1"
    },
    {
    "id": "Milestosite",
    "value": "54"
    },
    {
    "id": "DeliveryTrips",
    "value": "1"
    },
    {
    "id": "Loading",
    "value": "20/5/1"
    },
    {
    "id": "WINDLOAD",
    "value": "115MPH"
    },
    {
    "id": "SnowLoad",
    "value": "20LBS"
    },
    {
    "id": "BCDL",
    "value": "1"
    },
    {
    "id": "LocationType",
    "value": "RURAL"
    },
    {
    "id": "ROCKPOTENTIAL",
    "value": "UNKNOWN"
    },
    {
    "id": "ContractValidUntil",
    "value": "10/5/2021"
    },
    {
    "id": "VALIDFOR",
    "value": "7 Days"
    },
    {
    "id": "WEEKSOUT",
    "value": "4 to 6"
    },
    {
    "id": "LINKTO3DMODEL",
    "value": null
    },
    {
    "id": "CUPOLAACTION",
    "value": "Not Applicable"
    },
    {
    "id": "Weathervane",
    "value": "Not Applicable"
    },
    {
    "id": "WallHeights",
    "value": "12.5'"
    },
    {
    "id": "CONTRACTTYPE",
    "value": "MATERIALS & LABOR"
    },
    {
    "id": "DocumentType",
    "value": "CONTRACT"
    },
    {
    "id": "ConstructionType",
    "value": "POST FRAME"
    },
    {
    "id": "ConstructionMethod",
    "value": "Direct Buried"
    },
    {
    "id": "ServiceType",
    "value": "BUILD FREELY"
    },
    {
    "id": "LevelSiteStatement",
    "value": "Delivered To And Built On"
    },
    {
    "id": "BagsperPost",
    "value": "2"
    },
    {
    "id": "QuikreteSize",
    "value": "80"
    },
    {
    "id": "BagCount",
    "value": "32"
    },
    {
    "id": "Trates",
    "value": "5.15"
    },
    {
    "id": "PropertyOwner",
    "value": "Yes"
    },
    {
    "id": "LivingQuarters",
    "value": "No"
    },
    {
    "id": "SprayFoam",
    "value": "No"
    },
    {
    "id": "Concrete",
    "value": "Yes 4In"
    },
    {
    "id": "PadStatus",
    "value": "Yes"
    },
    {
    "id": "Levelbigger",
    "value": "Yes"
    },
    {
    "id": "RockHardDigging",
    "value": "No"
    },
    {
    "id": "WARRANTYTERM",
    "value": "1 Year"
    },
    {
    "id": "CustomerCustID",
    "value": "LEGA01"
    },
    {
    "id": "SelectedFoundation",
    "value": "CUSTOM-S"
    },
    {
    "id": "MeasureFrom",
    "value": "SteelLine"
    },
    {
    "id": "Width",
    "value": "30'"
    },
    {
    "id": "Length",
    "value": "50'"
    },
    {
    "id": "CeilingHeight",
    "value": "9' 0 1/2""
    },
    {
    "id": "RoofStyle",
    "value": "Gable"
    },
    {
    "id": "Slope",
    "value": "3"
    },
    {
    "id": "SalesTaxRate",
    "value": "0.0"
    },
    {
    "id": "Package-20900",
    "value": "false"
    },
    {
    "id": "Package-20824",
    "value": "false"
    },
    {
    "id": "Package-20825",
    "value": "true"
    },
    {
    "id": "Package-20695",
    "value": "false"
    },
    {
    "id": "Package-51656",
    "value": "false"
    },
    {
    "id": "Package-51180",
    "value": "false"
    },
    {
    "id": "Package-73963",
    "value": "false"
    }
    ]
    }
    ]
    }
    ]
1 Like

Sounds like fun, It sounds like it will be possible which is handy. Once you have some sample data I am sure someone will be able to have a proper poke.

1 Like

Hi @LBCAPI2, glad to see you on the forum, welcome!

To make sure I understand your scenario correctly:

  1. Your workflow is triggered by a webhook receiving data from Jotform. Would this be the example dataset you have shared?
  2. Based on this data you would then send a HTTP request to an external API in order to get job data. Would this be one request for for each of the items received in step 1 or just one single request returning a large amount of data? Could you share an example for this dataset as well?
  3. You then want to replace data in the dataset received in step 2 based on the strings in the value field from Jotform described in step 1.

I think we can work this out but it would be super helpful if you could share examples for the additional dataset too.

here’s a drawing I put together, yes I know I’m the next Picasso!

  1. Your workflow is triggered by a webhook receiving data from Jotform. Would this be the example dataset you have shared?

no that’s the Job data model and structure that the end result must conform to and a lot of the data contained in it has to remain untouched

  1. Based on this data you would then send a HTTP request to an external API in order to get job data. Would this be one request for for each of the items received in step 1 or just one single request returning a large amount of data? Could you share an example for this dataset as well?

yes once I parse and modify all the response data from Jotform submission it will trigger an HTTP request to pull in a single large amount of data as a single JSON Response which is just one item containing 1200-1500 items in a multidimensional array each item in this JSON response is required to be resubmitted once modified & chunked. the jobdatamodel snippet is a cut-down version I can share the whole thing if you want?

  1. You then want to replace data in the dataset received in step 2 based on the strings in the value field from Jotform described in step 1.

Yes, replace specific items with data from the Parse & modified response data from the Jotform submission.

then chunk it together and push it through an HTTP Request as a raw JSON Body content and at the endpoint it will create a new job based on and containing the jot form data.

each nod show in the drawing output one or more items needed to be placed in the final JSON Payload to be sent, there will be likely 100-140 items passed into the final payload.

hope this helps sorry for the horrible drawing

Hey @LBCAPI2,

You’re a Picasso :wink:

Coming to your workflow, you will have to either break it down into smaller workflows or use the Merge node to merge the data from the different branches. These branches (different nodes connected to the JotFrom node), will not run in parallel, they will run sequentially. Hence, you will need to structure your workflow in a way that the steps you want to execute at the end are connected to the same series. With the parallel approach, you also won’t be able to refer to the data in the other branches correctly.

Since the Job Model returns a single item with ~1.2k records, you can use the Item Lists node with the Split Out Items operation. This will return an array of the 1.2k records. You can then use a Function node and write a custom function to find and replace the data, and again the Item Lists node to aggregate the data.

It would be helpful if you can share:

  1. Data that you receive from the Job Data model API
  2. Data you want to replace with (I am assuming this is coming from your database or JotFrom?)
  3. The final output you want to send.
1 Like