Only add new rows of a CSV in Mysql table

In my workflow, a CSV file is loaded daily from an FTP server, in which data is logged progressively. The data should then be imported into a MySQL database.

This also works with my workflow, but all data is re-imported each time. However, only the newly added data records should be imported. What is the best way to do this?

Information on your n8n setup

  • **n8n version: 1.25.1
  • **Database (default: SQLite): MySQL
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • **Running n8n via (Docker, npm, n8n cloud, desktop app): npm
  • **Operating system: Ubuntu 22 LTS

Hi @duck
you should use the “Insert or Update” operation in the MySql node and pick a unique field to be sure you don’t create copies of your log data. I don’t know how your data is structured but probably you have a unique timestamp for your log entries that you could use as unique field.

Let me know if this helps
All the best

1 Like

Thank you for your answer. The data is in the following format:

Logging Intervall is 1h, so I have no unique ID, because date and time are in separated columns. How can I combine two columns (03.11.23;22:47:26 to 03.11.23 22:47:26) for a unique id?

And second question. Where can I make the setting with the unique id that only new data is imported? In the MySQL block?

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.