Only add new rows of a CSV in Mysql table

Questions
1

In my workflow, a CSV file is loaded daily from an FTP server, in which data is logged progressively. The data should then be imported into a MySQL database.

This also works with my workflow, but all data is re-imported each time. However, only the newly added data records should be imported. What is the best way to do this?

image
image872Ă—199 19.2 KB

Information on your n8n setup

  • **n8n version: 1.25.1
  • **Database (default: SQLite): MySQL
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • **Running n8n via (Docker, npm, n8n cloud, desktop app): npm
  • **Operating system: Ubuntu 22 LTS
2

Hi @duck
you should use the “Insert or Update” operation in the MySql node and pick a unique field to be sure you don’t create copies of your log data. I don’t know how your data is structured but probably you have a unique timestamp for your log entries that you could use as unique field.

Let me know if this helps
All the best

3

Thank you for your answer. The data is in the following format:
03.11.23;22:47:26;437;1;1;24;55;86;654;6;28;1;1;391;0;120;0;120

Logging Intervall is 1h, so I have no unique ID, because date and time are in separated columns. How can I combine two columns (03.11.23;22:47:26 to 03.11.23 22:47:26) for a unique id?

And second question. Where can I make the setting with the unique id that only new data is imported? In the MySQL block?