How to Properly Convert Big5 Encoded HTTP Response to UTF-8 Without Data Loss?

Questions
,
1

Hi n8n community,

I tried to retrieve webpage information using an HTTP request, but because the webpage’s charset is set to Big5, the retrieved data appears as garbled text.

Currently, I’ve tried saving the content using Convert to File and Read/Write Files from Disk, then using Execute Command to run iconv -f big5 -t utf-8, but after executing the command, the content of the .txt file is cleared instead of being successfully converted.

Is there any way to resolve this?

Information on your n8n setup

  • n8n version:1.86.1
  • Database (default: SQLite):default
  • n8n EXECUTIONS_PROCESS setting (default: own, main):default
  • Running n8n via (Docker, npm, n8n cloud, desktop app):Docker
  • Operating system:Ubuntu 20.04.6
2

This is probably because the iconv command doesn’t have an “in place” option like sed or some other command line utilities.

See if this StackExchange post helps.

tl/dr → You need to do it in 2 steps. 1) iconv(convert and create converted output file) 2) move/rename files

Another alternative, if the web server you are fetching content from will cooperate, is to add an accept header and ask/force the server to do the conversion for you.

Accept: text/html;charset=UTF-8

or

Accept-Charset: ISO-8859-1
1 Like
3

Thank you for your response. I resolved the issue using a different method.
I configured the Response Format in the HTTP request to output as a File, and then used this file with the command iconv -f big5 -t utf-8, which successfully performed the conversion.

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.