How to choose an LLM for your business

Which criteria actually matter when choosing a model?

Choosing an LLM means deciding which language model runs a given business task, through which access route and under which data terms. For the concept itself, see our LLM glossary page.

Public comparison tables are a useful starting point, but they do not measure your task. The criteria that usually decide the choice are:

• Task quality: classification, summarisation, document extraction, tool use and long reasoning give different results on different models.
• Language: how well the model handles your languages, formal correspondence and industry terms.
• Context length: matters when long contracts or technical specifications must be read. Current limits are listed per model in vendor documentation.
• Latency and volume: response time matters in live chat; unit cost matters in batch work.
• Cost driver: input and output token volume, caching and batch processing options.
• Data terms: whether data is used for training, how long it is retained and where it is processed.

Model families usually come in tiers. Anthropic offers Claude in tiers such as Haiku, Sonnet and Opus; Google offers Gemini in Pro, Flash and Flash-Lite tiers. The fast, low-cost tier is often enough for classification and routing; the large tier earns its cost on hard reasoning. Model names and versions change quickly, so this article stays at family level as of the time of writing.

Direct API or a cloud provider?

You can reach the same model by more than one route. Anthropic’s models page lists Claude models on the Claude API as well as on Amazon Bedrock, Google Cloud and Microsoft Foundry. OpenAI models are also available through Azure, and Gemini models through Google Cloud.

The cloud provider route comes with your existing billing, identity management, network rules and region choices. If your company already runs on Azure or Google Cloud, security approval is usually shorter. The direct API route often gets new features first and is simpler to set up. In many cases the decision depends more on your existing cloud infrastructure than on the model.

How do you compare retention, training use and region?

These are three separate questions and each needs confirming.

Training use. OpenAI’s enterprise privacy page states that business data from its API and enterprise products is not used for model training by default.

Retention. The same page says API inputs and outputs may be retained for up to 30 days to provide the service and identify abuse, and that zero data retention can be requested for eligible endpoints.

Region. In Microsoft Foundry the deployment type decides where data is processed: in any Azure region for Global types, within the US, EU or Asia Pacific data zone for Data Zone types, and within the chosen Azure geography for Standard. On Google Cloud, global endpoints carry no data residency guarantee, while the EU multi-region endpoint keeps processing within EU member states.

For companies in Europe, GDPR transfer rules apply when personal data leaves the EEA. If you also process data of people in Türkiye, KVKK (Türkiye’s data protection law) has its own rules on transfers abroad. Decide which data reaches the model, whether it needs masking and on what legal basis it is transferred, together with your legal counsel.

Closed model or open model?

Open-weight models can run on your own servers or in a private cloud environment. Data does not leave, and the model version is under your control. In return, GPU capacity, updates, security patches and monitoring become your responsibility.

An open model usually makes sense when data cannot leave under any circumstances, when volume is steady and high, or when you want to adapt a small model to a narrow task with fine-tuning. When volume is low or variable, a closed model through an API usually carries less operational load.

The table below puts the options side by side and relies only on vendor documentation. For quality and cost it says how to test rather than giving a measurement, because those values change with the task and over time.

Criterion	OpenAI API	Anthropic Claude	Google Gemini	Azure (Foundry)	Open model, self-hosted
Task quality	Measure with your test set	Measure with your test set	Measure with your test set	Depends on hosted model	Depends on model size; test it
Your languages	Test with your examples	Test with your examples	Test with your examples	Depends on hosted model	Check the model’s language coverage
Tiers	Models in several sizes	Haiku, Sonnet, Opus tiers	Pro, Flash, Flash-Lite tiers	Models in the catalogue	You choose the size
Access route	Direct API, Azure	Claude API, Bedrock, Google Cloud, Foundry	Gemini API, Google Cloud	Inside your Azure subscription	Your own infrastructure
Training use	Not by default (enterprise privacy page)	Confirm in contract	Confirm in contract	Confirm in contract	Data stays with you
Region option	Confirm in current docs	Regional endpoints via cloud providers	EU multi-region endpoint on Google Cloud	Global, Data Zone, Standard	Wherever your servers are
Cost driver	Token volume	Token volume, caching	Token volume	Tokens or reserved capacity	GPUs and an operations team

How do you test quality and cost on your own data?

1. Collect 30 to 100 real examples per task. Customer emails, invoice lines, contract clauses. Mask personal data.
2. Write the expected output. The right label for classification, the right fields for extraction, a checklist for summaries.
3. Run two or three candidates with the same prompt. Make small adjustments following each vendor’s prompting guidance, but do not change the test set.
4. Record quality, latency and token use together. Calculate cost by multiplying by your own monthly volume.
5. Read the failures. The type of error matters as much as the score: a grammar error, a wrong field or invented information?

Choose the embedding model the same way, separately. Search quality often depends more on the embedding choice than on the chat model.

Finally, one model or a model per task? A single model is simpler at first. As volume grows, splitting tasks makes sense: a fast, low-cost tier to classify incoming requests and a large tier for the hard cases. If all model calls go through one layer, you can add this routing later. The difference shows most in high-volume work such as document analysis or content automation.

Where should you start?

Write down the task and the data rules first, then look at models. For SMEs, one vendor and a good test set is often enough; technology and software teams tend to need multiple models and routing earlier.

At Corrplus we choose models during discovery with a side-by-side test on the client’s own examples, and record the result together with the data and region assessment. For details see LLM integration and AI consulting.