IonBin
All pages
Search
Concept

Inference

Inference is using a trained AI model to produce an output. A chat reply or agent task may involve several model calls. Usage, model choice and serving setup affect cost and speed; API billing can include tokens, media, tools or compute.

Verified Oct 18, 2026 Review by Apr 16, 2027 3 sources

What it is

IBM defines AI inference as using a trained model to make predictions on new data. Training adjusts model parameters; later training or fine-tuning can update them again. Inference applies the current parameters to an input.

How it works

  1. The input is prepared for the model. Language models use tokens; other models can take images, audio or numeric features.
  2. The model computes an output on suitable hardware, often GPUs for large language models. One agent task can involve several model calls.
  3. Many text APIs charge for input and output tokens; other models use image, audio, request or compute units. Self-hosting adds hardware, operations and capacity costs. Cloudflare Workers AI offers a model catalog on serverless GPUs with usage-based pricing; check each model license.

A worked example, using published list prices (checked 2026-10-18). Anthropic lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. A request with 5,000 input tokens and 1,000 output tokens costs about $0.01 + $0.01 = $0.02. Run it 1,000 times a day and that’s about $20 a day. This is USD token-only arithmetic at the listed base rates, excluding cache pricing, batch discounts, extra tool charges and additional model calls. Count all billable output, including reasoning where applicable, and re-check the pricing page before budgeting.

Benefits

  • Understanding inference cost helps you choose the right model tier.
  • Explains latency differences between models.
  • Frames the build-versus-buy choice for hosting.

Limits and risks

  • Costs scale with usage, so heavy automations need budgets.
  • Longer inputs can raise cost and latency; caching and serving settings affect the result.
  • Self-hosted inference needs capacity planning.

Agent compatibility

Not tool-specific. See Agent-ready for connector support by tool.

Levels come from each tool's recorded, sourced compatibility. How the levels work.

Sources

  1. What is AI inference? · IBMreference accessed Oct 18, 2026
  2. Workers AI docs · Cloudflareofficial accessed Oct 18, 2026
  3. Models overview · Anthropicofficial accessed Oct 18, 2026

Practitioner views are restated in our words and labeled as views, not measured results. Product facts come from official sources dated above.

Suggest an edit

IonBin's wiki isn't user-edited. Corrections go to our Directory Curator, who checks them against primary sources before anything changes. Spot something wrong or out of date on “Inference”? Email us with a link to a primary source.

Suggest an edit by email or write to contact@ionbin.com