General logging questions

[johna]

Hi there, we’re evaluating n8n to see if it is the right fit for our use case. We’ve taken a look at Logging - n8n Documentation and have a few questions:

1. Are logs roll your own or are there some templates/examples that we can leverage to get started?
2. Are there any examples of logs and code that generates logs?
3. Can workflow template definition JSON be accessed programmatically? It looks like there’s a way to do this through the CLI:
   n8n export:workflow --all
4. Is there a way to log the workflow definition JSON when a workflow kicks off?
5. Is there a way to programmatically access human readable names for node types? For example, if I have a node of type n8n-nodes-base.httpRequest, can I programmatically get the human readable name of HTTP Request?
6. Is there a way to programmatically access user data?

[MutedJam]

Hi @johna, welcome to the community!

Quite a bit to unpack here

1. Are logs roll your own or are there some templates/examples that we can leverage to get started?
   Yes, by default only the last 100 log files (each file containing 16 MB of data) are kept. You can adjust N8N_LOG_FILE_MAXSIZE and N8N_LOG_FILE_MAXCOUNT to configure this behaviour.
2. Are there any examples of logs and code that generates logs?
   No, but you could simply spin up a test instance using docker by running something like docker run -it --rm --name n8nlogging -e N8N_LOG_LEVEL=debug -p 5678:5678 n8nio/n8n:0.178.2 and take a look at the real-world output. Here is the debug output from me playing around with a simple webhook workflow just now:

2022-05-27T09:52:06.465Z | debug    | Received webhoook "GET" for path "30c1ba65-af24-42d4-aae9-35fae3631cf3" {"file":"ActiveWorkflowRunner.js","function":"executeWebhook"}
2022-05-27T09:52:06.495Z | verbose  | Started execution of workflow "Example" from webhook with execution ID 3 {"executionId":"3","file":"WebhookHelpers.js","function":"executeWebhook"}
2022-05-27T09:52:08.191Z | debug    | Received child process message of type start for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.216Z | verbose  | Initializing n8n sub-process {"pid":41,"workflowId":1,"file":"WorkflowRunnerProcess.js","function":"runWorkflow"}
2022-05-27T09:52:08.283Z | verbose  | Workflow execution started {"workflowId":1,"file":"WorkflowExecute.js","function":"processRunExecutionData"}
2022-05-27T09:52:08.288Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.288Z | debug    | Executing hook (hookFunctionsPush) {"executionId":"3","workflowId":1,"file":"WorkflowExecuteAdditionalData.js","function":"workflowExecuteBefore"}
2022-05-27T09:52:08.288Z | debug    | Start processing node "Webhook" {"node":"Webhook","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.289Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.289Z | debug    | Running node "Webhook" started {"node":"Webhook","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.291Z | debug    | Running node "Webhook" finished successfully {"node":"Webhook","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.292Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.293Z | debug    | Start processing node "HTTP Request" {"node":"HTTP Request","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.294Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.294Z | debug    | Running node "HTTP Request" started {"node":"HTTP Request","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.298Z | debug    | Proxying request to axios {"file":"NodeExecuteFunctions.js","function":"proxyRequestToAxios"}
2022-05-27T09:52:08.428Z | debug    | Running node "HTTP Request" finished successfully {"node":"HTTP Request","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.429Z | debug    | Start processing node "Item Lists" {"node":"Item Lists","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.429Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.429Z | debug    | Running node "Item Lists" started {"node":"Item Lists","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.430Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.432Z | debug    | Running node "Item Lists" finished successfully {"node":"Item Lists","workflowId":1,"file":"WorkflowExecute.js"}
2022-05-27T09:52:08.433Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.433Z | verbose  | Workflow execution finished successfully {"workflowId":1,"file":"WorkflowExecute.js","function":"processSuccessExecution"}
2022-05-27T09:52:08.434Z | debug    | Received child process message of type processHook for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.434Z | debug    | Executing hook (hookFunctionsSave) {"executionId":"3","workflowId":1,"file":"WorkflowExecuteAdditionalData.js","function":"workflowExecuteAfter"}
2022-05-27T09:52:08.435Z | debug    | Save execution data to database for execution ID 3 {"executionId":"3","workflowId":1,"finished":true,"stoppedAt":"2022-05-27T09:52:08.433Z","file":"WorkflowExecuteAdditionalData.js","function":"workflowExecuteAfter"}
2022-05-27T09:52:08.437Z | debug    | Received child process message of type end for execution ID 3. {"executionId":"3","file":"WorkflowRunner.js"}
2022-05-27T09:52:08.450Z | debug    | Executing hook (hookFunctionsPush) {"executionId":"3","workflowId":1,"file":"WorkflowExecuteAdditionalData.js","function":"workflowExecuteAfter"}

3. Can workflow template definition JSON be accessed programmatically? It looks like there’s a way to do this through the CLI:
   Yes, that would be one possible way. We are also working on an official REST API through which this will be possible (and you can also use the UI API for this as shown here, though this is undocumented + unsupported).
4. Is there a way to log the workflow definition JSON when a workflow kicks off?
   The workflow definition isn’t part of the logs, but it’s stored with the execution data (in the workflowData field of the execution_entity table, the full table structure is documented here) if you choose to save such data.
5. Is there a way to programmatically access human readable names for node types? For example, if I have a node of type n8n-nodes-base.httpRequest, can I programmatically get the human readable name of HTTP Request?
   There’s no in-platform way to resolve the human-readable name I am aware of unfortunately. The given name of a node would be part of the aforementioned workflow JSON though.
6. Is there a way to programmatically access user data?
   Are you referring to the user credentials stored in n8n? These could be read via the CLI (a command like n8n export:credentials --all --decrypted would do the job). If you’re looking for n8n users, these would be stored in the users table of your n8n database.

[alexpti4ka]

@MutedJam

Started execution of workflow "Example" from webhook with execution ID 3 {"executionId":"3","file":"WebhookHelpers.js","function":"executeWebhook"}

Hello! Wondering if there’s a way I can search through log files for specific user or transaction id? I’m debugging my working workflow with a lot of entities and it’s almost impossible to search through logs by timestamp only.
Thank you in advance!

[MutedJam]

Hi @alexpti4ka, there is currently no way built into n8n to search through the actual execution data.

You could, however, query the execution_entity table if you are using a suitable database. The simplest possible (but rather expensive) query I could think of would be something like this:

SELECT
  "workflowId",
  "id" AS "executionId"
FROM
  execution_entity
WHERE
  "workflowData"::text ILIKE '%alexpti4ka%'
  AND "workflowId" IS NOT NULL;

That is assuming you are using Postgres and want to search for alexpti4ka. It also works right in n8n if you want to automate such lookups:

[alexpti4ka]

Thanks a lot, I guess this can do the job.
I just wonder how I can set up a proper Postgres DB?
I’m running a default instance from your manual.

As far as I understand, this step defines my db path

mkdir /root/n8n/

However I see no db inside this folder, just a letsencrypt file.

I’ve also inspected my .env and docker-compose.yml files and saw neither db mentions nor credentials to access it. Could you please explain how I can access n8n’s db to run this query?

[MutedJam]

By default n8n would use an SQLite database. Any other database connection would have to be configured through environment variables.

Migrating from one database to another is a bit of effort though. It would require you to export your workflows and credentials from your n8n instance connected to the old database and then import it again once n8n is up and running connected to the new database. Our CLI can help with that if that’s too much to transfer manually.

[alexpti4ka]

I’ll check that up, thanks.
Maybe there’s a way I can send an execution id to my database at the end of the workflow?
Without modifying default db I mean.
Something like {{get[executionId]}} code?

[MutedJam]

Your database would hold the execution ID by default, so you wouldn’t need to manually store it in there. The id field in the execution_entity is the execution ID you can see in the frontend.

[alexpti4ka]

Oh I see! But what are the credentials for getting inside the default SQLite db?

[MutedJam]

The SQLite database wouldn’t require credentials. Using a suitable you can open the database.sqlite file which usually lives in the hidden .n8n folder of your home directory.

This is how it would look like in DB Browser:

[alexpti4ka]

Thanks, I checked that up, looks good for local instance.
But when I tried to run sqlite3 at my Ubuntu server it sees no content inside database.sqlite

[Jon]

Hey @alexpti4ka,

There is no config file in that folder either are you sure that is the correct path and that n8n is configured to use it?

I would expect to see… database.sqlite, config and maybe a folder called binaryData.

[alexpti4ka]

Hey! I’m afraid there’s nothing like you described. Maybe those files are somehow hidden or I’m searching the wrong place?

[Jon]

That could be it, I can see a docker compose file there is that for n8n?