UTF-8 and NocoDB row creation

Hi!

Describe the issue/error/question

When I have a workflow posting rows to NocoDB, having unicode characters in the strings causes NocoDB to reject with error 400, with an error Incorrect String Value from MariaDB. I initially thought this was a NocoDB issue, but I’m not sure anymore, because using curl I can insert unicode characters without problem in NocoDB. For example:

 curl -X 'POST' \
                       'http://pi4:4567/nc/random_2fmg/api/v1/table2' \
                       -H 'accept: application/json' \
                       -H 'xc-auth: replaced \
                       -H 'Content-Type: application/json' \
                       -d '{
                       "title": "string⏪",
                     }'

works flawlessly.

What is the error message (if any)?

400 Bad request - check your parameters
the trace shows the original error as Incorrect string value: '\xF0\x9F\x8E\xAA \xE2...' for column d1.nc_2fmg__testemails.subject at row 1"

Please share the workflow

Share the output returned by the last node

ERROR: UNKNOWN ERROR - check the detailed error for more information

Bad request - please check your parameters

Information on your n8n setup

  • n8n version: latest docker container for arm7
  • Database you’re using (default: SQLite): mariadb (linuxserver.io latest for arm7)
  • Running n8n with the execution process [own(default), main]: ?
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: Docker on Rapsberry Pi4 32 bit

Hi @Pierric, welcome to the community :tada:

I am sorry to hear you’re having trouble here. I just gave this a go on my end but could add your example string to my table as expected using the current n8n version 0.164.1 and the default NocoDB setup (docker run -d --name nocodb -p 8080:8080 nocodb/nocodb:latest):

image

So I wonder if the source of the problem could be specific to your data. Would you be able to share an example workflow not relying on data from your own Gmail account using which this problem can be reproduced?

Hi !

Thank you for your reply @MutedJam !

I have been exploring this issue from both the n8n and the NocoDB front, and finally concluded that the root cause lies on the side of NocoDB, and more precisely, of MariaDB itself.

As you suspect it depends on the data. The example I gave in my first post was a 3-byte unicode character, which does not cause any issue. However the data I had the problem with included some 4-byte unicode characters. Any such characters will cause a default NocoDB/MariaDB installation to crash, returning an error through the NocoDB API. Apparently this is because MariaDB’s understanding of the UTF-8 character set does not include 4-byte characters. I did not suspect there was such a nuance in the world of unicode characters, which is why I was so confused about what was happening.

I have suggested a workaround on the nocoDB side here.

I will mark this reply as the solution, for reference.

Cheers,
Pierric.

1 Like

That’s a great learn, thanks so much for investigating and sharing your results @Pierric!