AI & Machine Learning
Llama
Open-weight models running on your own infrastructure — for the teams who need data control, not another API key.
Overview
Llama is Meta’s family of open-weight large language models. The word that matters is open-weight: you can download the model’s parameters and run them yourself, on your own servers, in your own cloud account, or on hardware that never touches the public internet. That is the whole point of this page. Everything else — the quality of the models, the size of the ecosystem, the tooling — follows from that one property. If a hosted API like a frontier model behind a vendor’s endpoint is a car you hire by the mile, Llama is a car you buy: more responsibility, more control, and the economics flip once you drive far enough.
We reach for Llama and the broader open-weight ecosystem when a team has a reason to keep the model on infrastructure it controls. Usually that reason is data: regulated industries, personal or clinical data, or contractual commitments that make sending prompts to a third-party API a non-starter. Sometimes it is cost — at high, steady volume, per-token API fees add up and running your own inference can be cheaper. Sometimes it is control — you want a model that will not change underneath you, that you can fine-tune deeply on your own data, and that you can pin to a known version forever. We serve these models with the mature open-source stack the community has built around them: vLLM and Ollama for inference, Hugging Face for weights and tooling, and orchestration on Kubernetes or a managed platform like AWS Bedrock where it fits.
The honest framing runs through everything below. Choosing an open model means taking on the operational burden that a hosted API absorbs for you: GPUs to provision, an inference server to run and scale, updates to manage, and an MLOps practice to keep it all healthy. Open weights have closed much of the gap with the top hosted frontier models, but on the very hardest reasoning and coding tasks they still trail. And Llama’s licence, while genuinely permissive for most commercial use, is not an OSI-approved open-source licence — there are conditions, and we read them with you rather than waving them away. We recommend Llama when privacy, control or scale-economics make it the right tool, and we say so plainly when a hosted model would serve you better.
Best for — Teams that must run a capable language model on infrastructure they control — for compliance, data privacy, deep fine-tuning or scale-economics — and want senior engineers to own the serving, tuning and MLOps rather than discover them in production.
Why teams choose Llama
Your data never leaves your boundary
Because the model runs on your own hardware or inside your own cloud account, prompts and completions stay within your network perimeter. For regulated or sensitive workloads this is not a nice-to-have — it is often the only architecture your compliance team will accept, and it removes an entire category of third-party data-processing risk.
No per-token API fees and no vendor lock-in
You pay for compute, not for tokens. At sufficient volume that changes the cost curve entirely, and because the weights are yours, no vendor can raise a price, deprecate a model, or change terms out from under you. You can migrate between clouds or move on-premise carrying the same model.
A model you can genuinely make your own
Open weights mean you can fine-tune on your own data, distil a smaller specialised model, or quantise to fit modest hardware. You are not limited to prompting a black box — you can change the model’s behaviour at the parameter level and own the result permanently.
Why businesses choose Llama
- A compliance, data-residency or privacy requirement means inference has to run on infrastructure you control, and a hosted API is off the table regardless of its capability.
- You want to fine-tune deeply on proprietary data and own a specialised model permanently, rather than prompt a model you cannot change and cannot keep.
- Your volume is high and steady enough that self-hosted inference is genuinely cheaper per request, and you would rather own that cost curve than rent it a token at a time.
- You value predictability: a pinned, validated model version that behaves the same in two years as it does today, insulated from a vendor’s roadmap and deprecation schedule.
What we build with Llama
The capabilities this technology is genuinely strong at — and what we most often build with it.
Self-hosted inference with vLLM
For production serving we most often reach for vLLM, whose paged-attention KV cache and continuous batching push far more throughput out of a given GPU than a naive server. We size the deployment to your traffic, tune batching and context limits for your workload, and expose an OpenAI-compatible endpoint so application code can talk to your own model the way it would talk to a hosted one.
Local and low-friction serving with Ollama
For development, on-device experiments, air-gapped machines, and smaller deployments, Ollama makes running a Llama model genuinely simple — pull a model, run it, and hit a local API. We use it to prototype quickly and to stand up self-contained inference where a full GPU serving cluster would be overkill.
Fine-tuning and LoRA adapters
We adapt Llama to your domain with parameter-efficient fine-tuning — LoRA and QLoRA adapters trained on your data — or full fine-tuning where the task warrants it. Adapters are small, cheap to train and swappable, so you can maintain several specialised behaviours over one base model rather than paying to host many separate models.
Quantisation for modest hardware
Running a large model at full precision is expensive. We quantise — GPTQ, AWQ, or GGUF for CPU and mixed setups — to shrink a model’s memory footprint so it fits a smaller, cheaper GPU with a measured, acceptable quality trade-off. We benchmark the quantised model against your real tasks rather than trusting that a smaller number is close enough.
Retrieval-augmented generation on your data
Most useful Llama deployments pair the model with a retrieval layer: your documents embedded into a vector database and pulled into the prompt at query time, so the model answers from your knowledge rather than its training. We build this with the open stack — an embedding model, a vector store, and an orchestration layer — all of it running inside your boundary alongside the model.
Right-sizing across the Llama family
Llama ships in several sizes, and the smallest capable model is almost always the right answer. We match model size to the task — a small model for classification, extraction or routing; a larger one only where the task genuinely needs it — which cuts GPU cost and latency without the capability loss teams fear when they reflexively pick the biggest model.
Use cases
On-premise assistants for regulated data
A question-answering or drafting assistant over sensitive internal documents — legal, clinical, financial — where the data cannot leave the organisation. The model and its retrieval layer run entirely inside the client’s network, so no prompt is ever sent to a third party.
High-volume text processing at controlled cost
Classification, extraction, summarisation or moderation running over large, steady volumes of text, where a self-hosted model keeps GPUs busy enough that the per-request cost falls well below a per-token hosted API.
Domain-specialised fine-tuned models
A base Llama fine-tuned on a client’s own corpus and task format — a specific tone of voice, a structured output schema, or expertise in a narrow domain — producing a model that outperforms a generic hosted one on that one job and belongs entirely to the client.
Private RAG over a knowledge base
A retrieval-augmented assistant grounded in a company’s documentation, tickets or wiki, with embeddings and the model both hosted internally, so staff get accurate answers from current internal knowledge without that knowledge ever being exposed to an outside service.
When Llama is the right choice
- You have a data-residency or privacy constraint that rules out sending prompts to a third-party API — regulated data, personal or clinical records, or a contractual commitment that inference stays inside your own VPC or on-premise. This is the strongest reason to run open weights, and Llama’s central advantage.
- You need to fine-tune deeply on your own data — full fine-tuning or LoRA adapters that bake domain knowledge, tone, or a task format into the model — rather than relying on prompting and retrieval alone against a model you cannot modify.
- You are running at high, sustained volume where per-token API fees have become a material line item, and you have the scale to keep GPUs busy enough that self-hosted inference is genuinely cheaper per request.
- You want a model that will not change beneath you: a pinned version you can validate once and rely on for years, free of a vendor deprecating an endpoint or silently shifting behaviour between releases.
- Wrong for: a team that wants the highest possible capability on the hardest tasks with zero operations. If you need frontier-grade reasoning and have no appetite for running GPUs, a hosted frontier model will give you more capability and less work. Open weights are a deliberate trade of convenience for control — if you do not need the control, you are buying the burden for nothing.
Llama: pros and cons
Strengths
- Full data control: inference runs inside your own VPC or on-premise, so sensitive prompts and outputs never cross a network boundary you do not own — the decisive advantage for regulated and privacy-sensitive work.
- No per-token pricing and no lock-in: you pay for compute you control, the weights are yours to keep, and you can move between clouds or on-premise without re-platforming your intelligence layer.
- Deep customisation: full fine-tuning and LoRA let you adapt the model to your domain and task far beyond what prompting a hosted model allows, and quantisation lets a tuned model run on comparatively modest GPUs.
- A mature open ecosystem: weights, tooling and serving are well supported — vLLM and Ollama for inference, Hugging Face for models and libraries, and first-class paths through AWS Bedrock and Kubernetes — so you are building on a well-trodden stack, not a research artefact.
Trade-offs
- You own the operational burden: provisioning and paying for GPUs, running and scaling an inference server, patching, monitoring and an MLOps practice to keep it all healthy. A hosted API absorbs every bit of this for you; with open weights it is your job.
- Open weights still trail the very top hosted frontier models on the hardest reasoning, coding and agentic tasks. The gap has narrowed a lot, but if absolute capability on difficult problems is the priority, a leading hosted model is usually still ahead.
- The licence is permissive but not open source by the OSI definition. Llama’s community licence carries conditions — including terms that apply to very large-scale deployments and restrictions on using outputs to train competing models — and those need reading against your actual use, not assumed away.
- Total cost of ownership only beats hosted APIs above a real scale threshold. Idle or lightly used GPUs are expensive, and below that threshold a pay-per-token API is often cheaper as well as simpler. Self-hosting is not automatically the frugal choice.
Serving architecture
A Llama deployment is really three layers, and we design each deliberately. There is the inference tier — the GPUs running vLLM or a comparable server, sized to your concurrency and latency targets and exposing a stable, OpenAI-compatible API. There is the application tier that owns prompting, retrieval, guardrails and business logic, kept separate so the model can be upgraded or swapped without rewriting the app. And there is the data tier — a vector database for retrieval and stores for prompts, evaluations and logs — all of it living inside the same trust boundary as the model, because the entire reason for self-hosting is defeated if data leaks out at the edges.
We keep the inference tier stateless and horizontally scalable, and we treat the model as a pinned artefact: a specific version and a specific quantisation, validated once against your tasks and version-controlled so a deployment is reproducible. GPU memory is the binding constraint, so context length, batch size and the number of concurrently loaded models are chosen against real hardware limits rather than hopeful defaults. Where a workload mixes cheap small-model calls with occasional large-model calls, we route between models rather than sending everything to the biggest one — most requests do not need it, and paying as if they do is the most common way self-hosted inference becomes needlessly expensive.
Performance & throughput
Inference performance on open weights is overwhelmingly a function of how you serve, not just which model you pick. A production server like vLLM, with continuous batching and an efficient KV cache, can serve several times the throughput of a naive loop on identical hardware, so the first performance work is almost always in the serving layer. We measure the numbers that actually matter to your users — time to first token, tokens per second, and throughput under concurrent load — on your hardware with your prompts, rather than quoting a benchmark run under conditions you will never see.
The main levers are model size, quantisation and batching, and they trade against each other. A smaller or quantised model is faster and cheaper but gives up some quality; larger batches raise total throughput but can hurt the latency any single user feels. We tune these against your requirements explicitly — an interactive assistant and a bulk overnight processing job want opposite settings — and we validate that a quantised or smaller model still passes on your real tasks before we let it save you money. Fast and wrong is not a saving.
Security & data control
The security case for open weights is the data boundary itself: run the model inside your own VPC or on-premise and sensitive prompts and completions never traverse a service you do not control. We build to keep that guarantee real — the model, the retrieval store and the logs all inside one trust boundary, network egress locked down, and secrets and credentials handled through your existing infrastructure rather than baked into a container. Self-hosting is only a privacy win if nothing quietly phones home, so we check that it does not.
On top of that we apply the discipline any LLM system needs. Prompts and any tool inputs are treated as untrusted, with guarding against prompt injection and strict validation before model output is ever allowed near a shell, a query or a filesystem. Access to the inference endpoint is authenticated and rate-limited so it cannot be used as an open relay, and for regulated workloads we log prompts and outputs for audit — inside the boundary — so you can demonstrate what the model was asked and what it produced. We are also candid that self-hosting shifts security work onto you: patching the serving stack and the base images is now your responsibility, and we set that up to be maintainable rather than a forgotten box.
Scalability & cost
Because we keep the inference tier stateless, scaling out is a matter of running more replicas behind a load balancer — typically a Kubernetes deployment that adds GPU-backed pods as concurrency rises. The real subtlety is that GPUs are expensive and do not scale to zero cheaply: an idle GPU still costs money, so autoscaling a language model is a genuinely different problem from autoscaling a web service. We design for your actual traffic shape, using batching to raise utilisation, right-sized models to lower the per-request cost, and, where load is bursty or unpredictable, a hybrid where a hosted API absorbs spikes so you are not paying to keep peak capacity idle.
This is also where the honest cost conversation lives. Self-hosted inference beats a per-token API only when your GPUs are busy enough to justify their standing cost, so we model your expected volume against real GPU pricing before recommending the approach — and we will tell you if you are below the threshold where it pays. Above it, owning the compute gives you a flat, predictable cost curve and the freedom to move between clouds or on-premise; below it, a hosted API is usually cheaper and simpler, and we would rather say so than sell you a GPU cluster you will underuse.
Llama integrations & ecosystem
The technologies we most often pair with it — each links to how we work with it.
How we work
We start with the question most vendors skip: should you be self-hosting at all? We look at your data constraints, your expected volume and your capability requirements, and we model the cost of open weights against a hosted API honestly. If a hosted model is the better answer for you, we say so before you have spent anything on GPUs. When open weights are right, we then choose the smallest model that clears your quality bar on your real tasks, because model size drives every downstream cost — hardware, latency and the size of the operational problem.
From there we build in thin, verifiable slices: a serving endpoint you can actually call, an evaluation set drawn from your real work, and a retrieval or fine-tuning layer added only where it earns its place. The engineers who design the deployment are the ones who run it, which is what we mean by operating what we build — self-hosted inference has a real operational tail, and we would rather set it up to be maintainable and hand you something your team can own than leave you with a clever demo and a pager you do not understand.
The service behind it
Delivered throughAI DevelopmentWhat we build with Llama
The disciplines this technology most often shows up in — from a first build to taking over and stabilising an existing one.
How we deliver
- 01
Discover
We map the system, the constraints and the business it serves — including the parts nobody documented.
Architecture brief
- 02
Architect
Decisions get made, written down and defended before a line of production code exists.
Decision records
- 03
Build
Short cycles against working software. You see progress in the product, not in a status deck.
Shipping increments
- 04
Operate
Monitoring, incident response and iteration. The system is alive, so the engagement is too.
Runbooks & SLOs
Industries we use Llama in
Domain knowledge changes what gets built. A few of the sectors we know before the first meeting.
Also in AI & Machine Learning
Why teams choose us for Llama
Senior engineers only
Self-hosting a large model is unforgiving of guesswork — GPU sizing, quantisation trade-offs and serving throughput all punish inexperience quietly until production exposes them. The people making these decisions on your project have made them before and have run the result; there are no juniors learning on your GPUs.
We operate what we build
Open-weight models come with a real operational tail: inference to keep healthy, GPUs to keep utilised, base images to patch. We run what we ship, so we design for that eighteen-month reality — maintainable serving, sane autoscaling, and monitoring you can act on — not a demo that falls over the first time traffic is real.
Honest about build versus rent
Self-hosting is the right answer less often than the hype suggests. We model the cost and capability against a hosted API on your actual numbers, and if a hosted frontier model would serve you better and cheaper, we tell you before you buy hardware. We would rather lose the GPU project than sell you an idle cluster.
Fluent across the open-model stack
A Llama deployment is never just the model. We bring the surrounding pieces — vLLM and Ollama for serving, LoRA and quantisation for adaptation, a vector database and orchestration for retrieval, and Kubernetes for scale — so the whole system is built by one team rather than stitched together from parts.
Typical timeline
- 01
Feasibility & model selection
One to two weeks establishing whether open weights are the right choice, modelling cost against a hosted API on your real volume, and selecting the smallest Llama model that clears your quality bar on tasks drawn from your actual work.
- 02
Serving & evaluation baseline
Two to three weeks standing up a self-hosted inference endpoint on your infrastructure, with an evaluation set that measures the model on your tasks, so quality and cost are grounded in numbers before the build grows.
- 03
Fine-tuning & retrieval
Adding the layers that make the model yours — LoRA or full fine-tuning on your data, a retrieval pipeline over your documents, quantisation where it saves money — each validated against the evaluation baseline rather than assumed to help.
- 04
Hardening & handover
Load testing under representative concurrency, security and data-boundary review, autoscaling and monitoring, and documentation so your team can operate, patch and extend the deployment rather than depend on us to keep it alive.
How pricing works
- A fixed-scope build for a well-defined deployment — a self-hosted inference endpoint, a fine-tuned model, or a private RAG assistant — quoted once we understand your data, volume and hardware.
- An open-model feasibility assessment: we model the cost and capability of self-hosting against a hosted API on your actual numbers, so the build/rent decision is made on evidence rather than assumption, priced as a short engagement.
- Monthly senior engagement for ongoing work — evolving the model, the serving stack and the MLOps practice — where you want continuity and someone who owns the operational reality rather than a one-off deliverable.
Hire Llama engineers
Need Llama capacity on your own team? We embed named senior engineers into your existing team — reporting to your leads, working in your rituals — so you add capacity without a hiring cycle.
Hire Llama engineersCommon questions
Is Llama actually open source, and can I use it commercially?
Llama is open-weight and permissively licensed for most commercial use, but it is not open source by the OSI definition — that distinction matters and we do not blur it. Meta’s community licence carries conditions, including terms that trigger for very large-scale deployments and restrictions around using the model’s outputs to train competing models. For the overwhelming majority of businesses it is perfectly usable commercially, but we read the current licence against your specific use before committing, rather than assuming it is unconditionally free.
Should I self-host Llama or just use a hosted API?
Self-host when you have a reason to: a data or compliance constraint that keeps inference inside your own boundary, a need to fine-tune deeply and own the model, or volume high enough that self-hosted inference is genuinely cheaper per request. Use a hosted API when you want the highest capability with no operations, or when your volume is modest enough that paying per token is cheaper than keeping a GPU running. The decision turns on your actual numbers, and we model it honestly before recommending either way.
Will an open model be as capable as a top hosted frontier model?
For most practical tasks — extraction, classification, summarisation, retrieval-grounded answering, and a great deal of drafting — a well-chosen, well-served open model is entirely capable and the gap is irrelevant. On the very hardest reasoning, coding and agentic problems, the leading hosted frontier models are still ahead, though by less than they were. We benchmark candidate models against your real tasks rather than a leaderboard, because what matters is whether a model is good enough for your work, not whether it tops an abstract ranking.
What hardware do I need to run Llama?
It depends entirely on which model and how you serve it. A small model, quantised, can run on a single modest GPU or even a capable CPU for light use, while a large model at full precision needs serious GPU memory. Quantisation and picking the smallest model that clears your quality bar are the two biggest levers on hardware cost, and we size the deployment against your real concurrency and latency targets rather than provisioning for a worst case you will never hit. Getting this right is most of the cost story.
Is fine-tuning worth it, or is retrieval enough?
Often retrieval is enough and is the right first move — pulling your documents into the prompt at query time gives the model current, specific knowledge without any training, and it is cheaper and easier to keep up to date. Fine-tuning earns its place when you need to change the model’s behaviour rather than its knowledge: a consistent tone, a strict output format, or fluency in a narrow domain that prompting cannot reliably produce. In practice the strong deployments combine both — a fine-tuned model for behaviour and retrieval for facts — and we recommend fine-tuning only where an evaluation shows it actually helps.
Building on Llama?
A technical conversation with the engineers who would do the work. If we are not the right fit, we will say so on the call.