librechat.ai/content/docs/configuration/token_usage.mdx

---
title: Token Usage
icon: Coins
description: This covers how to track and control your token usage for the OpenAI/Plugins endpoints in LibreChat. You will learn how to view your transactions, enable user balances, and add credits to your account.
---

## Intro

As of `v0.6.0`, LibreChat accurately tracks token usage for the OpenAI/Plugins endpoints. All token transactions are stored in the "Transactions" collection in your database. In future releases, you'll be able to toggle viewing how much a conversation costs.

Currently, you can limit user token usage by enabling user balances. Instead of configuring token credit limits via environment variables, you now set these options in your `librechat.yaml` file under the **balance** section.

## Transaction Configuration

The transaction system controls whether token usage records are saved to the database. This can be configured separately from the balance system.

### Transaction Settings

```yaml filename="librechat.yaml"
version: 1.2.9

# Transaction settings
# Controls whether to save transaction records to the database
# Default is true (enabled)
transactions:
  enabled: false
```

**Important:** When `balance.enabled` is set to `true`, transaction recording is automatically enabled regardless of the `transactions.enabled` setting. This ensures that balance tracking functions correctly by maintaining a complete record of all token usage.

    Check out the [Transactions Configuration](/docs/configuration/librechat_yaml/object_structure/transactions) page for more details.

## Balance Configuration

The balance system in LibreChat allows administrators to configure how token credit balances are managed for users. All balance settings are now managed in your YAML configuration under the `balance` object.

**Note:** This replaces the previous environment variables (`CHECK_BALANCE` and `START_BALANCE`) and provides a more structured way to manage user balances.

### Complete Balance Settings

```yaml filename="librechat.yaml"
version: 1.3.4

# Balance settings
balance:
  enabled: true                # Enable token credit balances for users
  startBalance: 20000          # Initial tokens credited upon registration
  autoRefillEnabled: false     # Enable automatic token refills
  refillIntervalValue: 30      # Numerical value for refill interval
  refillIntervalUnit: "days"   # Time unit for refill interval (days, hours, etc.)
  refillAmount: 10000          # Tokens added during each refill
```

### Balance Settings Explained

- **enabled**: Activates token credit tracking and balance management for users. When set to `true`, the system will track token usage and enforce balance limits.

- **startBalance**: Specifies the initial number of tokens credited to a user upon registration. This is the starting balance for all new users.

- **autoRefillEnabled**: Determines whether automatic refilling of token credits is enabled. When set to `true`, the system will automatically add credits to user balances based on the refill interval.

- **refillIntervalValue**: Specifies the numerical value for the interval at which token credits are automatically refilled. Works in conjunction with `refillIntervalUnit`.

- **refillIntervalUnit**: Specifies the time unit for the refill interval. Supported values include "seconds", "minutes", "hours", "days", "weeks", and "months".

- **refillAmount**: Specifies the number of tokens to be added to the user's balance during each automatic refill.

Check out the [Balance Configuration](/docs/configuration/librechat_yaml/object_structure/balance) page for more details.

## How Auto-Refill Works

When a user's balance is tracked and **autoRefill** is enabled, the system will automatically add credits to the balance only when the specified time interval has passed since the last refill. This is achieved by comparing the current date with the `lastRefill` date plus the specified interval.

### Auto-Refill Process

1. When a user attempts to spend tokens, the system checks if the current balance is sufficient
2. If the balance would drop to zero or below after the transaction, the system checks if auto-refill is enabled
3. If auto-refill is enabled, the system checks if the time interval since the last refill has passed:
   - The system compares the current date with `lastRefill + refillInterval`
   - If the interval has passed, tokens are added to the user's balance
   - The `lastRefill` date is updated to the current date
4. The transaction proceeds if the balance is sufficient (either originally or after refill)

### Supported Time Units

The `refillIntervalUnit` can be set to any of the following values:
- "seconds"
- "minutes"
- "hours"
- "days"
- "weeks"
- "months"

For example, if `refillIntervalValue` is set to 30 and `refillIntervalUnit` is `days`, the system will add `refillAmount` tokens to the user's balance only if 30 days have passed since the last refill.

### Balance Synchronization

When a user logs in, the system automatically synchronizes their balance settings with the current global balance configuration. This ensures that any changes to the balance configuration are applied to all users.

The synchronization process:
1. Checks if the user has a balance record
2. If no record exists, creates one with the current `startBalance`
3. Updates the user's auto-refill settings to match the global configuration
4. Ensures the user's refill interval and amount match the global settings

## Managing Token Balances

You can manually add or set user balances. This is especially useful during development or if you plan to build out a full balance-accruing system in the future (for example, via an admin dashboard).

### Adding Balances

```bash filename="To manually add balances, run the following command:"
# Local Development
npm run add-balance

# Docker (default setup)
docker-compose exec api npm run add-balance

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run add-balance"
```

```bash filename="You can also specify the email and token credit amount to add, e.g.:"
# Local Development
npm run add-balance danny@librechat.ai 1000

# Docker (default setup)
docker-compose exec api npm run add-balance danny@librechat.ai 1000

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run add-balance danny@librechat.ai 1000"
```

### Setting Balances

Additionally, you can set a balance for a user. An existing balance will be overwritten by the new balance.

```bash filename="To manually set balances, run the following command:"
# Local Development
npm run set-balance

# Docker (default setup)
docker-compose exec api npm run set-balance

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run set-balance"
```

```bash filename="You can also specify the email and token credit amount to set, e.g.:"
# Local Development
npm run set-balance danny@librechat.ai 1000

# Docker (default setup)
docker-compose exec api npm run set-balance danny@librechat.ai 1000

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run set-balance danny@librechat.ai 1000"
```

### Listing of balances

```bash filename="To see the balances of your users, you can run:"
# Local Development
npm run list-balances

# Docker (default setup)
docker-compose exec api npm run list-balances

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run list-balances"
```

This works well to track your own usage for personal use; 1000 credits = $0.001 (1 mill USD)

## Notes on Token Usage and Balance

- With summarization enabled, you will be blocked from making an API request if the cost of the content that you need to summarize + your messages payload exceeds the current balance
- Counting Prompt tokens is really accurate for OpenAI calls, but not 100% for plugins (due to function calling). It is really close and conservative, meaning its count may be higher by 2-5 tokens.
- The system allows deficits incurred by the completion tokens. It only checks if you have enough for the prompt Tokens, and is pretty lenient with the completion. The graph below details the logic
- The above said, plugins are checked at each generation step, since the process works with multiple API calls. Anything the LLM has generated since the initial user prompt is shared to the user in the error message as seen below.
- There is a 150 token buffer for titling since this is a 2 step process, that averages around 200 total tokens. In the case of insufficient funds, the titling is cancelled before any spend happens and no error is thrown.

![image](https://github.com/danny-avila/LibreChat/assets/110412045/78175053-9c38-44c8-9b56-4b81df61049e)

## More details
source: [LibreChat/discussions/1640](https://github.com/danny-avila/LibreChat/discussions/1640#discussioncomment-8251970)

> "rawAmount": -000, // what's this?

Raw amount of tokens as counted per the tokenizer algorithm.

>    "tokenValue": -00000, // what's this?

Token credits value. 1000 credits = $0.001 (1 mill USD)

> "rate": 00, // what's this?

The rate at which tokens are charged as credits.

For example, gpt-3.5-turbo-1106 has a rate of 1 for user prompt (input) and 2 for completion (output)

| Model                 | Input                | Output               |
|-----------------------|----------------------|----------------------|
| gpt-3.5-turbo-1106    | $0.0010 / 1K tokens  | $0.0020 / 1K tokens  |


Given the provided example:
```sh
    "rawAmount": -137
    "tokenValue": -205.5
    "rate": 1.5
```

```math
\text{Token Value} = (\text{Raw Amount of Tokens}) \times (\text{Rate})
```

```math
137 \times 1.5 = 205.5
```

And to get the real amount of USD spend based on **Token Value**:

```math
\frac{\text{Token Value}}{1,000,000} = \left(\frac{\text{Raw Amount of Tokens} \times \text{Rate}}{1,000,000}\right)
```

```math
\frac{205.5}{1,000,000} = \$0.0002055 \text{ USD}
```

The relevant file for editing rates is found in `api/models/tx.js`

There will be more customization for this soon from the `librechat.yaml` file.

## Preview

![image](https://github.com/danny-avila/LibreChat/assets/110412045/39a1aa5d-f8fc-43bf-81f2-299e57d944bb)

![image](https://github.com/danny-avila/LibreChat/assets/110412045/e1b1cc3f-8981-4c7c-a5f8-e7badbc6f675)

## Additional Notes

- With summarization enabled, API requests are blocked if the cost of the content plus the messages payload exceeds the current balance.
- The system is lenient with completion tokens, focusing primarily on prompt tokens for balance checks.
- A buffer is added for titling (approximately 150 tokens) to account for the two-step process.
- Token credits translate to monetary value (e.g., 1000 credits = $0.001 USD).

For more details and customizations, please refer to the [LibreChat Documentation](https://www.librechat.ai/docs/configuration/librechat_yaml).