# Tokens | IonBin

> Tokens are the units models process. Text tokens may be words, fragments or punctuation, while media uses model-specific accounting.

Canonical page: https://ionbin.com/wiki/concepts/tokens/
Agent guide: https://ionbin.com/llms.txt

This is a readable version of the published page. Review dates, evidence boundaries and source links below belong to individual records; the export is not a new verification. Examples are reference material, not instructions to execute.

Concept

## Tokens

Tokens are the units models process. Text tokens may be words, fragments or punctuation, while media uses model-specific accounting. Token counts determine many context limits and API charges. Count your actual request: English word estimates do not transfer reliably across languages, media or tokenizers.

Verified Oct 18, 2026 Review by Apr 16, 2027 5 sources

### What it is

For text, a token is a unit such as part of a word, punctuation or whitespace. Multimodal models also account for images, audio or video using model-specific token rules. That makes tokens the unit for three things you care about: how much fits (the [context window](https://ionbin.com/wiki/concepts/context-window/)), how long a reply can be and what you pay.

### How it works

Each vendor’s tokenizer splits text its own way, so the same document can be a different number of tokens on different models (checked 2026-10-18):

- **Google:** a Gemini token is about four characters, and 100 tokens is about 60 to 80 English words.

- **Anthropic:** Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text than earlier models, with the exact difference depending on content and workload.

Token-priced requests count processed input and generated output, with separate rates for cached input where applicable. Media, tools and storage can add other charges. Depending on the vendor, output can include [reasoning](https://ionbin.com/wiki/concepts/reasoning-models/) tokens.

### Benefits

- **A predictable cost unit.** You can estimate a workflow’s cost from document sizes and run counts.

- **Discounts.** Vendors discount repeated input ([prompt caching](https://ionbin.com/wiki/concepts/prompt-caching/)) and offline [batch](https://ionbin.com/wiki/concepts/batch-processing/) work.

### Limits and risks

- **Reasoning burns output tokens.** Higher effort settings can increase billable reasoning usage; measure actual usage rather than assuming a fixed response cost.

- **Not comparable across vendors.** A “1M token” window isn’t the same number of words everywhere.

- **Long-prompt price bands.** Some models charge more above a prompt-size threshold. Google’s Gemini 3.1 Pro Preview, for example, lists a higher price for prompts over 200K tokens.

### Checked specs

Snapshot from official sources on Oct 18, 2026.

| Tokens to words (Gemini) | ≈ 4 characters per token; 100 tokens ≈ 60–80 English words [3](https://ionbin.com/wiki/concepts/tokens/#src-3) |
| --- | --- |
| Anthropic newer tokenizer (Claude 4.7 and later) | ≈ 30% more tokens for the same text than the previous tokenizer [4](https://ionbin.com/wiki/concepts/tokens/#src-4) |
| Output vs input price, [Claude Sonnet 5.5](https://ionbin.com/platforms/models/claude-sonnet-5-5/) | $10 vs $2 per 1M tokens [1](https://ionbin.com/wiki/concepts/tokens/#src-1) |
| Output vs input price, [GPT-6.1 Sol](https://ionbin.com/platforms/models/gpt-6-1-sol/) | USD standard text rates: $10 output vs $2 input per 1M tokens at the base prompt-size band; long-context rates differ. [2](https://ionbin.com/wiki/concepts/tokens/#src-2) |

### Best-fit stacks

Stack 02

#### [Business data & reporting](https://ionbin.com/use-cases/business-data-reporting/)

Pull numbers from the systems you already use into one place an agent can analyze and explain.

Stack 08

#### [Dev, web & IT](https://ionbin.com/use-cases/dev-it/)

Design, code, host and ship sites and internal tools, with an AI coding agent doing most of the typing.

### Agent compatibility

Not tool-specific. See [Agent-ready](https://ionbin.com/agent-ready/) for connector support by tool.

Levels come from each tool's recorded, sourced compatibility. [How the levels work](https://ionbin.com/wiki/questions/connect-a-tool-to-an-ai-agent/).

### Related entries

[Context window · Concept](https://ionbin.com/wiki/concepts/context-window/)[How do I cut AI API costs without hurting quality? · Question](https://ionbin.com/wiki/questions/cut-ai-api-costs/)[Model tiers (flagship, balanced, fast) · Concept](https://ionbin.com/wiki/concepts/model-tiers/)[Inference · Concept](https://ionbin.com/wiki/concepts/inference/)[Large language model (LLM) · Concept](https://ionbin.com/wiki/concepts/large-language-model/)[Prompt caching · Concept](https://ionbin.com/wiki/concepts/prompt-caching/)[Rate limits · Concept](https://ionbin.com/wiki/concepts/rate-limits/)[Reasoning models · Concept](https://ionbin.com/wiki/concepts/reasoning-models/)[Should I use RAG or put everything in the context window? · Question](https://ionbin.com/wiki/questions/rag-or-full-context/)

### Sources

- [Models overview](https://platform.claude.com/docs/en/models/overview) · Anthropicofficial accessed Oct 18, 2026

- [Models](https://developers.openai.com/api/docs/models/gpt-6.1-sol) · OpenAIofficial accessed Oct 18, 2026

- [Understand and count tokens](https://ai.google.dev/gemini-api/docs/tokens) · Googleofficial accessed Oct 18, 2026

- [Pricing](https://platform.claude.com/docs/en/about-claude/pricing) · Anthropicofficial accessed Oct 18, 2026

- [Gemini Developer API pricing](https://ai.google.dev/gemini-api/docs/pricing) · Googleofficial accessed Oct 18, 2026

Practitioner views are restated in our words and labeled as views, not measured results. Product facts come from official sources dated above.

### Suggest an edit

IonBin's wiki isn't user-edited. Corrections go to our Directory Curator, who checks them against primary sources before anything changes. Spot something wrong or out of date on “Tokens”? Email us with a link to a primary source.

[Suggest an edit by email](mailto:contact@ionbin.com?subject=Suggest%20an%20edit%3A%20Tokens%20%5Btokens%5D&body=Page%3A%20https%3A%2F%2Fionbin.com%2Fwiki%2Fconcepts%2Ftokens%2F%0AEntry%3A%20concepts%2Ftokens%0A%0AWhat%20should%20change%3A%0A%0A%0APrimary%20source%20(link)%3A%0A) or write to [contact@ionbin.com](mailto:contact@ionbin.com)

Entry facts

**Type**: [Concepts & glossary](https://ionbin.com/wiki/concepts/)

**Status**: Reviewed

**Verified**: Oct 18, 2026

**Review by**: Apr 16, 2027 (every 180 days)

**Sources**: [5 official · 0 other](https://ionbin.com/wiki/concepts/tokens/#sources)

The wiki carries no sponsored placements or affiliate links. [Editorial policy](https://ionbin.com/editorial-policy/).
