IonBin
All pages
Search
Concept

Tokens

Tokens are the units models process. Text tokens may be words, fragments or punctuation, while media uses model-specific accounting. Token counts determine many context limits and API charges. Count your actual request: English word estimates do not transfer reliably across languages, media or tokenizers.

Verified Oct 18, 2026 Review by Apr 16, 2027 5 sources

What it is

For text, a token is a unit such as part of a word, punctuation or whitespace. Multimodal models also account for images, audio or video using model-specific token rules. That makes tokens the unit for three things you care about: how much fits (the context window), how long a reply can be and what you pay.

How it works

Each vendor’s tokenizer splits text its own way, so the same document can be a different number of tokens on different models (checked 2026-10-18):

  • Google: a Gemini token is about four characters, and 100 tokens is about 60 to 80 English words.
  • Anthropic: Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text than earlier models, with the exact difference depending on content and workload.

Token-priced requests count processed input and generated output, with separate rates for cached input where applicable. Media, tools and storage can add other charges. Depending on the vendor, output can include reasoning tokens.

Benefits

  • A predictable cost unit. You can estimate a workflow’s cost from document sizes and run counts.
  • Discounts. Vendors discount repeated input (prompt caching) and offline batch work.

Limits and risks

  • Reasoning burns output tokens. Higher effort settings can increase billable reasoning usage; measure actual usage rather than assuming a fixed response cost.
  • Not comparable across vendors. A “1M token” window isn’t the same number of words everywhere.
  • Long-prompt price bands. Some models charge more above a prompt-size threshold. Google’s Gemini 3.1 Pro Preview, for example, lists a higher price for prompts over 200K tokens.

Checked specs

Snapshot from official sources on Oct 18, 2026.

Tokens to words (Gemini)≈ 4 characters per token; 100 tokens ≈ 60–80 English words [3]
Anthropic newer tokenizer (Claude 4.7 and later)≈ 30% more tokens for the same text than the previous tokenizer [4]
Output vs input price, Claude Sonnet 5.5$10 vs $2 per 1M tokens [1]
Output vs input price, GPT-6.1 SolUSD standard text rates: $10 output vs $2 input per 1M tokens at the base prompt-size band; long-context rates differ. [2]

Best-fit stacks

Stack 02

Business data & reporting

Pull numbers from the systems you already use into one place an agent can analyze and explain.

Stack 08

Dev, web & IT

Design, code, host and ship sites and internal tools, with an AI coding agent doing most of the typing.

Agent compatibility

Not tool-specific. See Agent-ready for connector support by tool.

Levels come from each tool's recorded, sourced compatibility. How the levels work.

Sources

  1. Models overview · Anthropicofficial accessed Oct 18, 2026
  2. Models · OpenAIofficial accessed Oct 18, 2026
  3. Understand and count tokens · Googleofficial accessed Oct 18, 2026
  4. Pricing · Anthropicofficial accessed Oct 18, 2026
  5. Gemini Developer API pricing · Googleofficial accessed Oct 18, 2026

Practitioner views are restated in our words and labeled as views, not measured results. Product facts come from official sources dated above.

Suggest an edit

IonBin's wiki isn't user-edited. Corrections go to our Directory Curator, who checks them against primary sources before anything changes. Spot something wrong or out of date on “Tokens”? Email us with a link to a primary source.

Suggest an edit by email or write to contact@ionbin.com