Retrieve LLM Token Usage in AI Agents

Feature Requests
,
1

Hello!

I would like to extract the completition and prompt tokens info when using AI Agents. It would be highly beneficial to keep track of my costs.

I have tried different workarounds presented here Similar post 1 and Similar post 2 but they don’t work for AI Agents. There is no direct way to do it with N8N nodes.

Please let me know if anyone has managed to successfully retrieve this tokens info.

Thanks!

2

Joining the request. I have no idea how to count app usage per user.
Even though its number is shown in the Chat Model section, unclear how to get it from there.

2 Likes
3

Same here. This isn’t apart of the AI agent node from what I’ve seen… However, I think it should be apart of it. It could be handled similarly to the “tools” leg. Maybe it should be a “log” leg or similar. For instance, I would like to know the input/output tokens per request or maybe some logs around each request.

2 Likes
4

#following
This is a feature I would need too.
Suggestion to the n8n_team: Have the AI Agent output its token usage

3 Likes
5

Voting up here too. For many use cases it would make a real difference knowing the token usage. For some projects I just dropped the default llm model node or even the agente for that matter

6 Likes
6

Yes, this is a very important feature for those looking to control the operational costs of the flow or create a log of what the client is consuming in tokens. The use of AI is closely related to performance and efficiency, and through tokens, we can make many optimizations.

3 Likes
7

I got here looking for the exact same thing :slight_smile: - so another vote for this pls.

2 Likes
8

another vote here :raised_back_of_hand:

2 Likes
9

Essential feature to track costs!

5 Likes
10

Also would really help me out, currently looking for a workaround!

11

this be great and way to send calls to langsmith or helicone

1 Like
12

Have we seen the n8n team providing feature requested by the community in the past?

I mean I’m using n8n only since a few months, and off course it has its limitations, but I don’t know what’s the level of responsiveness of the devs. In your experience, does it append? Often? And on what kind of delay on average?

13

I want to second this request as well, but maybe take it a bit simpler:
You can use an LLM proxy such as LiteLLM to do a lot of things like measuring usage, associating costs etc.
I think with the focus shift of n8n from an integration platform to an AI platform, having any sort of observability will make n8n increasingly valuable in the enterprise tier as well.
Consider integrating with Datadog LLM observability.

It should be relatively straightforward to collect token usage from an AI agent and have that be exposed as metrics, that would at least be the simplest form.

14

We hope this feature relases soon

15

I hope this will be done soon

16

I’m also looking for this feature

17

+1 for the feature

18

I’m looking for a solution like this too

19

It is worthy to be noted here that n8n can send all data to LangSmith which can track costs, and we are using it that way.

However, the downside here is: I have no idea how to distinguish data by workflow or by any other thing except for the LLM model.
Basically it is tracking “everything per n8n instance”.

20

Here’s a workaround:

You can use an n8n: Get Execution node with the “Include Execution Details” option enabled.

This will retrieve information about the entire execution, including Chat Model usage details in the following format:

{
  "tokenUsage": {
    "completionTokens": <number>,
    "promptTokens": <number>,
    "totalTokens": <number>
  }
}

Therefore, you can set up a separate workflow that you can trigger with the execution ID after your AI Agent workflow completes.

It might sound a bit cumbersome, but it should do the trick.

5 Likes