Automatic tests for CI

I was wondering about running a new version of n8n, but it broke my workflow. After looking at the code of that specific node it was clear: it got refactored and something was broken. But a cool project, needs cool CI (bad joke, I guess) - but it really needs it. So I found this. It already runs on every commit, but there aren’t any tests.

Are there any plans to get those nodes tested? So we can be sure, that a code change might not break a node?

Greets, Michael

Very sorry that you had problems with upgrading!

Yes you are totally right we need definitely more tests! It is on our to-do list for a while now and I hope we can get to it soon. In this case we can sadly not do much and have to trust the community as we simply do not have a self-hosted Jira-Version running. At some point we would have to mock all APIs and then run the tests against them. Sadly do we however have currently not the resources to do that.

Normally do we try to avoid any breaking changes in general. Sometimes it is however needed and then they get added here:

So before upgrading this file should always be checked first, to see if anything has to be done.

Hi @jan,

thanks for answering! I read the changelog before upgrading. The issue linked above doesn’t seem to be related to renaming, instead it changed code in how to retrieve credentials and thus, broke. Anyway, I’ll fix that later.

What I’m asking for is a guideline on how n8n’s plan look like for integrating tests? This project is wonderful, but looses trust on every update if anything broke, that could be easily avoided! I know adding tests afterwards is a hard challenge. But I think it could be easily done by:

  1. adding a test-suite, writing tests “as example” (dear dev, look here for an example on how to test your node).
  2. encouraging developers to write tests for bugfixes (“do you please mind, to add some tests?”).
  3. requiring tests for new nodes - prohibiting merge requests for new nodes without a single test.

That said, do you have an example on how to test a node? As soon as I can spend some time I’ll add tests for jira (on-premise version). I’m eager to learn new things, so please guide me (and other developers) in the correct direction. Please share your plans so this can happen :slight_smile:

Ah yes. The breaking-changes only contain breaking-changes we are aware of, do knowingly and on purpose. The issue you are experiencing was not supposed to be there so is more a bug.

About testing. That “plan” or “framework” did sadly not get created yet. Once we have it we will also create some examples, documentation and helper-functions to make it easy for other people to do the same and to create tests.

Thanks a lot for your suggestions, they are very helpful and appreciated! Also very great that you offer to help, so also thanks for that! Great to have people like you being part of the community!

I will update here once we have something ready.