Request for feedback: Node Generator

Hello everyone,

I am a part of a summer fellowship that is contributing to n8n. After creating a few regular nodes, we decided to work on a tool to make it easier to make the nodes you need. We are looking for community suggestions and feedback!

Basically, our application is going to automate the creation of regular nodes (as they are generally just HTTP clients). Specifically, when you input the basics of your node (which resources, operations, and other input fields are needed), the nodemaker will return back the code for this node. This product will simplify node creation and helps PRs get merged faster because the nodemaker takes care of codebase conventions for you.

As long as the nodemaker directory is placed next to the n8n and n8n-docs folders, the nodemaker can generate and place the proper files to add in a new node to n8n.

These operations are currently supported:

Script Action
nodegen Generate *.node.ts, GenericFunctions.ts, *Description.ts (optional), and *.credentials.ts.
packgen Generate a package.json updated with node path and credential path insertions.
docsgen Generate a node functionality doc file and a node credential doc file in markdown.
icongen Generate five images as icon candidates.
resize Resize one of the generated icon candidates to a 60×60 px PNG file.
place Move files at /output to the n8n and n8n-docs repos.

A more detailed description of the nodemaker is here.

We would also love feedback on the UI mockups here.

Some of the main questions we are considering are:
Would you use this as a CLI tool, a desktop app, or a web app?

  • The CLI tool will be an easy extension to a desktop or web app.
  • We are leaning towards creating a desktop app because a web app will not have the file placement functionality.

What other functionality would you like to see in the nodemaker?

Feel free to message me with any questions, comments, or concerns about this project.

5 Likes

Hey @erin2722,

The node maker is one of the ideas that I had a while back which I was going to tackle when I had time. Thanks for jumping in on this!

To answer your questions:

For me personally, I would use it regardless of the UI. But, a few thoughts on the tool:

  1. The sense that I get is that this will be a tool used by people who deploy n8n. This may or may not be the end user and is likely someone who is more technical. Because of this (and like me), the UI may not matter much to the end user.
  2. Pros and Cons of each tool (from my perspective):
    • CLI
      • Pro
        • Easier for scripting and automating
        • UI very easy to develop
        • Common way of using tools for pretty much all applications
        • Smaller footprint and generally fewer dependencies
      • Con
        • Point and click users may be lost
        • Not as intuitive as a GUI interface
        • Might be a barrier to entry for those who never user CLI tools
    • Dekstop
      • Pro
        • Everyone is familiar with this type of an application
        • Usually very intuitive as long as standards are followed
      • Con
        • May run into challenges making it work cross-platform (though I do know there are ways to do this)
        • Often more difficult to troubleshoot, especially when the desktop environment is not managed and they could have anything running that might cause problems with the app
    • Web
      • Pro
        • Everyone knows how to use a web browser
        • The platform where services run (i.e. the web server) are consistent for all users, making troubleshooting a lot easier
      • Con
        • May run into issues with how different support different aspects of HTML/javascript
        • Some security conscious users will not allow scripts to run in their browser which makes it difficult to ask them to reduce their security levels to use your app

So, here is the big one for me. I would love to be able to point this tool at an API that follows the Swagger OpenAPI Specification, RAML or API Blueprint and have it crawl the API/documentation to automatically create the node and all of its functions.

Because each of these standards is well documented, there should be enough logic behind them to allow this tools to know how the API will behave and test that behavior before adding it into the node(s).

3 Likes

I would love some form of Swagger to Node converter.

1 Like

@ivov probably has something to say regarding it.

@JohnSmi05099416

Work in progress for OpenAPI here:

Any feedback is welcome.

5 Likes

I had actually created a simple npm script to generate the nodes for me for my own project, this is just using the start node as the template so no api functionality etc will be added into the generated node, I just run npm run add and it’ll give me prompts to input the ID, Name and description and it’ll then generate the folder with the files as well as updating the package.json file

1 Like

So I’m not an expert with this stuff. Followed the instructions and got this after placing this Swagger file Workiom Swagger:
(Use node --trace-warnings ... to show where the warning was created)
(node:444) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see Command-line API | Node.js v17.8.0 Documentation). (rejection id: 1)
(node:444) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Fixed, it will now output the node for Workiom.

This being a work in progress, application/x-www-form-urlencoded is the only content type supported for now, so the OpenAPI nodegen output for Workiom can only be used as a template to work on. Or if you are interested in a subset of the API, the YAML mapping option might help.

Hi @ivov, thanks for making the nodebuilder available via github.
I have just tried that on my mac with a yaml file from Brightcove and ran the commands but I am not seeing anything in the src/output directory.

Here is a segment from the log in terminal.

npm run generate

> [email protected] generate
> ttsc && node dist/scripts/generate.js

? Select the input file format. YAML
? Select the input YAML file. Brightcove_live_openapi_v1.yaml
/Users/pier-andre/nodebuilder/node_modules/typescript-is/index.js:89
        throw new TypeGuardError(errorObject, obj);
              ^

TypeGuardError: validation failed at object: expected 'metaParams' in object, found: {
  openapi: '3.0.0',
  info: {
    title: 'Live API Reference',
    description: 'Reference for the Brightcove Live API, used to create and manage live streaming events.\n' +
      '\n' +
      'For additional in-depth guides to features of the API, see the **[Support Site](/)**.\n' +
      '\n' +
      '**Base URL**: https://api.bcovlive.io/v1',
    version: '1.0.0'
  },
  servers: [ { url: 'https://api.bcovlive.io/v1', variables: {} } ],
  tags: [
    {
      name: 'Live Jobs',
      description: 'Operations for creating live jobs, listing live jobs, getting job details, and canceling live jobs. There are also operations for activating and deactivating SEP jobs, scheduling activation and deactivation of SEP jobs, and for inserting cuepoints or ID3 timed metadata.'
    },
    {
      name: 'Schedule SEP Job Start Stop',
      description: 'Operations for scheduling the activation and deactivation of an SEP job.'
    },
    {
      name: 'Redundant Groups',
      description: 'Operations for creating redundant groups of live jobs for failover in case one stream fails'
    },
    {
      name: 'RTMP Outputs',
      description: 'Operations for creating and managing RTMP outputs. Note that RTMP output hours will be billed against event hours.'
    },
    {
      name: 'Live Job Clip',
      description: 'Operations for creating and managing VOD clips, including scheduling clips for SEP jobs'
    },
    {
      name: 'Schedule Clip',
      description: 'Operations for scheduling the creation of a VOD clip for SEP jobs only.'
    },
    {
      name: 'SSAI',
      description: 'Operations for managing server-side ad insertion with live streams.'
    },
    {
      name: 'Credentials',
      description: 'Operations for creating and managing credentials for secure destinations.'
    }
  ],
  paths: {
    '/jobs': { post: [Object], get: [Object] },
    '/jobs/{job_id}': { get: [Object] },
    '/jobs/{job_id}/playback-token': { post: [Object] },
    '/jobs/{job_id}/activate': { put: [Object] },
    '/jobs/{job_id}/deactivate': { put: [Object] },
    '/jobs/{job_id}/cuepoint': { post: [Object] },
    '/jobs/{job_id}/cuepointdata': { post: [Object], delete: [Object] },
    '/jobs/{job_id}/id3tag': { post: [Object] },
    '/jobs/{job_id}/cancel': { put: [Object] },
    '/jobs/{job_id}/stop': { put: [Object] },
    '/jobs/{job_id}/rtmpouts': { get: [Object], post: [Object] },
    '/jobs/{job_id}/rtmpouts/{rtmp_out_id}/stop': { put: [Object] },
    '/redundantgroups': { post: [Object], get: [Object] },
    '/redundantgroups/{redundant_group_id}/jobs': { post: [Object] },
    '/redundantgroups/{redundant_group_id}/jobs/{job_id}': { delete: [Object] },
    '/redundantgroups/{redundant_group_id}': { get: [Object], delete: [Object] },
    '/redundantgroups/{redundant_group_id}/switch': { put: [Object] },
    '/redundantgroups/{redundant_group_id}/cuepoint': { post: [Object] },
    '/scheduler/jobstartstop': { post: [Object] },
    '/scheduler/jobstartstop/{workflow_id}': { get: [Object], put: [Object], delete: [Object] },
    '/vods': { post: [Object] },
    '/vods/{jvod_id}': { get: [Object] },
    '/vods/{jvod_id}/cancel': { put: [Object] },
    '/jobs/{job_id}/vods': { get: [Object], post: [Object] },
    '/scheduler/clip': { post: [Object], get: [Object] },
    '/sceduler/Clip/{workflow_id}': { get: [Object], put: [Object], delete: [Object] },
    '/ssai/applications': { post: [Object], get: [Object] },
    '/ssai/applications/account/{live_account_id}': { get: [Object] },
    '/ssai/applications/application/{application_id}': { get: [Object], put: [Object], delete: [Object] },
    '/ssai/beaconsets': { post: [Object], get: [Object] },
    '/ssai/beaconsets/beaconset/{beacon_set_id}': { put: [Object], delete: [Object] },
    '/ssai/beaconsets/account/{live_account_id}': { get: [Object] },
    '/ssai/slates': { post: [Object], get: [Object] },
    '/ssai/slates/slate{slate_msa_id}': { delete: [Object] },
    '/ssai/slates/account/{live_account_id}': { get: [Object] },
    '/credentials': { get: [Object], post: [Object] },
    '/credentials/{credential_id}': { put: [Object], delete: [Object] }
  },
  components: {
    schemas: {
      CreateRedundantGroup: [Object],
      CreateRedundantGroupResponse: [Object],
      GetRedundantGroupResponse: [Object],
      AddJobsRedundantGroupRequest: [Object],
      GetStatusRedundantGroupResponse: [Object],
      ForceRedundantStreamFailover: [Object],
      ActivateSepStreamResponse: [Object],
      playback_added_cdns: [Object],
      PlaybackToken: [Object],
      AddAdMetadatabody: [Object],
      Application: [Object],
      'Application.applicationAdConfiguration': [Object],
      'Application.applicationAdConfiguration.adConfigurationTransforms': [Object],
      BeaconSet: [Object],
      'BeaconSet.beaconUrls': [Object],
      BeaconSets: [Object],
      CancelLiveJob: [Object],
      StopLiveJob: [Object],
      CreateAdConfigurationbody: [Object],
      'CreateAdConfigurationbody.applicationAdConfiguration': [Object],
      'CreateAdConfigurationbody.applicationAdConfiguration.adConfigurationTransforms': [Object],
      CreateBeaconSetResponse: [Object],
      CreateBeaconSetbody: [Object],
      'CreateBeaconSetbody.beaconUrls': [Object],
      CreateCredential: [Object],
      CreateCredentialbody: [Object],
      'CreateCredentialbody.oauthSettings': [Object],
      CreateLiveJobRequestBody: [Object],
      'CreateLiveJobRequestBody.addCdns': [Object],
      'CreateLiveJobRequestBody.addCdns.tokenAuth': [Object],
      'CreateLiveJobRequestBody.addCdns.tokenAuth.media': [Object],
      'CreateLiveJobRequestBody.encryption': [Object],
      'CreateLiveJobRequestBody.outputs': [Object],
      'CreateLiveJobRequestBody.videocloud': [Object],
      'CreateLiveJobRequestBody.videocloud.video': [Object],
      'CreateLiveJobRequestBody.videocloud.video.cuePoint': [Object],
      'CreateLiveJobRequestBody.videocloud.video.geo': [Object],
      'CreateLiveJobRequestBody.videocloud.video.schedule': [Object],
      CreatePlaybackTokenRequestBody: [Object],
      CreateVodClipbody: [Object],
      'CreateVodClipbody-Alternate': [Object],
      'CreateLiveJob.outputs.videocloud': [Object],
      'CreateVodClipbody.outputs.videocloud.video.schedule': [Object],
      CreateVodJobResponse: [Object],
      CreateRTMPOutputRequestBody: [Object],
      CreateRTMPOutputResponse: [Object],
      Credentials: [Object],
      CuePoint: [Object],
      DeactivateSepStreamResponse: [Object],
      DeleteAdConfigurationResponse: [Object],
      DeleteBeaconSetResponse: [Object],
      DeleteSlateMediaSourceAssetResponse: [Object],
      GetSlateMediaSourceAssetsResponse: [Object],
      Id3Tag: [Object],
      IngestSlateMediaSourceAssetResponse: [Object],
      IngestSlateMediaSourceAssetbody: [Object],
      InsertId3TimedMetadataResponse: [Object],
      InsertId3TimedMetadatabody: [Object],
      'InsertId3TimedMetadatabody.id3Tag': [Object],
      Job: [Object],
      'Job.inputMediaFile': [Object],
      'Job.outputMediaFiles': [Object],
      'Job.stream': [Object],
      'Job.stream.destination': [Object],
      'Job.stream.location': [Object],
      ListCredentials: [Object],
      ListLiveJobsResponse: [Object],
      ListRTMPOutputs: [Object],
      RTMPout: [Object],
      ListVodJobs: [Object],
      GetVodJob: [Object],
      LiveJob: [Object],
      ManualAdCuePointInsertionResponse: [Object],
      ManualAdCuePointInsertionbody: [Object],
      RedundantGroupCuePointInsertionResponse: [Object],
      Notification: [Object],
      Outputs: [Object],
      StopRTMPOutputResponse: [Object],
      UpdateAdConfigurationbody: [Object],
      'UpdateAdConfigurationbody.applicationAdConfiguration': [Object],
      'UpdateAdConfigurationbody.applicationAdConfiguration.adConfigurationTransforms': [Object],
      UpdateBeaconSetbody: [Object],
      'UpdateBeaconSetbody.beaconUrls': [Object],
      UpdateCredential: [Object],
      UpdateCredentialbody: [Object],
      'UpdateCredentialbody.oauthSettings': [Object],
      VodJobs: [Object],
      VodJobs2: [Object],
      CreateJobStartStopRequestBody: [Object],
      'CreateJobStartStopRequestBody.TaskInfo': [Object],
      JobStartStop: [Object],
      'JobStartStop.TaskInfo': [Object],
      ListJobStartStopResponse: [Object],
      UpdateJobStartStopRequestBody: [Object],
      CreateClipRequestBody: [Object],
      Clip: [Object],
      'Clip.ClipInfo': [Object],
      ListClipResponse: [Object],
      UpdateClipRequestBody: [Object],
      'CreateClipRequestBody.ClipInfo': [Object],
      'CreateVodClipbody.outputs': [Object],
      'CreateVodClipbody.outputs.videocloud': [Object],
      'CreateVodClipbody.outputs.videocloud.ingest': [Object],
      'CreateVodClipbody.outputs.videocloud.video': [Object],
      'CreateVodClipbody.outputs.videocloud.video.cuePoint': [Object],
      'CreateVodClipbody.outputs.videocloud.video.geo': [Object]
    },
    parameters: {
      ContentType: [Object],
      XAPIKEY: [Object],
      job_id: [Object],
      live_account_id: [Object],
      redundant_group_id: [Object],
      JobId: [Object],
      jvod_id: [Object],
      rtmp_out_id: [Object],
      CredentialId: [Object],
      SlateMsaId: [Object],
      BeaconSetId: [Object],
      start_token: [Object],
      page_size: [Object],
      sort: [Object],
      sort_dir: [Object],
      user_id: [Object],
      account_id: [Object],
      created_at: [Object],
      force: [Object],
      modified_at: [Object],
      ad_insertion: [Object],
      static: [Object],
      RedundantGroupState: [Object],
      scheduler_account_id: [Object],
      state: [Object],
      ssai_state: [Object],
      region: [Object],
      jvod_state: [Object],
      jvod_type: [Object],
      label: [Object],
      application_id: [Object],
      WorkflowId: [Object],
      scheduler_state: [Object],
      start: [Object],
      end: [Object]
    }
  }
}
    at Object.assertType (/Users/pier-andre/nodebuilder/node_modules/typescript-is/index.js:89:15)
    at arePreTraversalParams (/Users/pier-andre/nodebuilder/dist/services/YamlParser.js:45:21)
    at YamlParser.run (/Users/pier-andre/nodebuilder/dist/services/YamlParser.js:33:13)
    at /Users/pier-andre/nodebuilder/dist/scripts/generate.js:62:73
    at step (/Users/pier-andre/nodebuilder/dist/scripts/generate.js:33:23)
    at Object.next (/Users/pier-andre/nodebuilder/dist/scripts/generate.js:14:53)
    at fulfilled (/Users/pier-andre/nodebuilder/dist/scripts/generate.js:5:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  path: [ 'object' ],
  reason: { type: 'missing-property', property: 'metaParams' },
  input: {
    openapi: '3.0.0',
    info: {
      title: 'Live API Reference',
      description: 'Reference for the Brightcove Live API, used to create and manage live streaming events.\n' +
        '\n' +
        'For additional in-depth guides to features of the API, see the **[Support Site](/)**.\n' +
        '\n' +
        '**Base URL**: https://api.bcovlive.io/v1',
      version: '1.0.0'
    },
    servers: [ { url: 'https://api.bcovlive.io/v1', variables: {} } ],
    tags: [
      {
        name: 'Live Jobs',
        description: 'Operations for creating live jobs, listing live jobs, getting job details, and canceling live jobs. There are also operations for activating and deactivating SEP jobs, scheduling activation and deactivation of SEP jobs, and for inserting cuepoints or ID3 timed metadata.'
      },
      {
        name: 'Schedule SEP Job Start Stop',
        description: 'Operations for scheduling the activation and deactivation of an SEP job.'
      },
      {
        name: 'Redundant Groups',
        description: 'Operations for creating redundant groups of live jobs for failover in case one stream fails'
  },
.......

Thanks!

@Pier-Andre_Maynard Thanks for the report.

Nodebuilder accepts either an OpenAPI spec in JSON, or a custom (user-written) API mapping in YAML. Yours is an OpenAPI spec in YAML, so it needs to be converted first.

I just pushed an update to handle this conversion automatically, so your OpenAPI YAML spec should generate output now. Docs updated as well.

https://github.com/ivov/nodebuilder/commit/eef5aeafe13738a081a0784562ff0bb83a9d0ca3

2 Likes

Thanks @ivov, I have just updated locally and used it to generate the files.

I added my .yaml file in the OpenApi folder, ran the commands and see files in the output now.

At this point I just need to move these files elsewhere for n8n to pick them up as nodes right?

Is it in the same location as in the Nodemaker repo? GitHub - ivov/nodemaker: Desktop app and CLI utility for auto-generating n8n nodes

Thank you!

@ivov , thanks again for the update with nodemaker.

Is there a specific folder location where I can put the generated files in order to the generated node(s) to be available within n8n?

I have done a quick search locally on my system (Mac) but couldn’t really figure out if there is a specific folder outside the base nodes. I have checked the documentation / directory structure on github and couldn’t figure it out.

Sorry for the beginner question.

@Pier-Andre_Maynard

The project is early stage so the nodegen output is not yet perfect for all OpenAPI specs, but it should be close for most. Brightcove Policy API is small - I will check what needs to be adjusted for it as soon as I have the time.

Automating the placement of the output files is a good idea. Will add this over the weekend. For general reference, you can check this guide:

1 Like

Thanks @ivov, I’m following that document closely at this point but have an initial question. If I installed n8n via npm/pnpm, does it mean I have to remove and clone from git instead? I like the convenience of having an updated package via npm/pnpm.

I don’t mind rebuilding from source, the only concern is to have an outdated install eventually.

Thank you again for the great work!

No need to remove and clone. n8n, whether installed via git or npm, can read custom nodes from /Users/<user>/.n8n/custom.

The placement feature will offer the option to place the output in the custom dir (for private use) or in a git-installed clone (to make it easier to contribute back through a PR).

1 Like

Hello, Thanks for this idea.
I run into a problem and open an issue Cannot use 'in' operator to search for 'oneOf' in undefined · Issue #7 · ivov/nodebuilder · GitHub

Feel free to ask more detail. I am not sure if I can help you to solve this but I can try. :slight_smile: