Solutions
Build your own, adapt open-weight models, or use private frontier models. We work with all three, deploy where you decide, route between them through one gateway and test them on your own cases.
01Overview
Almost every organization we meet wants one of three things. Some want to build their own models in-house, because the data is theirs and the use case is narrow. Some want to use open-weight models, the kind published on Hugging Face, because they can run them anywhere and adapt them. Some want the best private models from the frontier labs, because quality matters most and they do not want to run GPUs.
All three are reasonable. None of them is right for every task. A bank may use a frontier model to draft internal research, an open-weight model on its own servers to read customer files, and a small in-house model to score transactions. The mistake is picking one path for the whole company and then bending every use case to fit it.
Boltcliq is model-agnostic. We build with all three, deploy where you decide, route between them through one gateway, and evaluate them on your own cases. This page is a technical explainer: what each path involves, what to check, which tools matter, and where the traps are. It is written for engineers and for the people who sign the contracts.
A note on names. We mention model families, labs and tools so that you can see what we work with. A name on this page means we design for compatibility. It does not mean the vendor endorses us, partners with us or has reviewed this page. Licences, prices and capabilities change often, so confirm current terms with the source before you commit.
02Compatibility, not endorsement
Cloud and hardware
Names and logos show compatibility only. They do not imply partnership, endorsement or certification by the companies shown.
03The three paths
| Build or adapt (in-house and open-weight) | Use a private frontier model |
|---|---|
| Cost: capital and engineering up front, then GPU-hours or hardware. Cheap per call at steady, high utilisation. | Cost: pay per token, nothing up front. Cheap at low or spiky volume, grows with usage. |
| Control: you choose the weights, version, location and update date. Nothing changes unless you change it. | Control: the lab sets the model, versions and retirement dates. You control the prompt and the contract. |
| Privacy: data can stay inside your boundary, including air-gapped sites. | Privacy: data goes to a provider under contract terms. Retention, region and training use must be checked. |
| Quality ceiling: strong open models are close for many tasks, but the very hardest reasoning tasks often favour frontier models. Confirm on your own cases. | Quality ceiling: usually the highest on broad, hard tasks, and it changes every few months. |
| Speed to value: weeks for adaptation, months for a model built from scratch. | Speed to value: days. An API key and a prompt. |
| Lock-in: low on weights you hold. Some tooling and hardware lock-in remains. | Lock-in: moderate. Prompts, tool formats and fine-tunes are tied to one vendor unless you abstract them. |
| Skills needed: ML engineering, data engineering, GPU operations, evaluation. | Skills needed: prompt design, integration, evaluation, contract and risk review. |
04Decision guide
Situation
The task is narrow and valuable: scoring, forecasting, speech or vision in a local language, document classification at volume. You have labelled data nobody else has.
What it does
We design, train and validate a compact model, often a few million to a few billion parameters, with a repeatable training recipe, and deploy it on hardware you choose.
Human decides
You decide whether the data may be used for training, what accuracy is acceptable, and who owns the weights.
05Path two
Open-weight means you can download the weights and run them yourself. It does not always mean open source. Training data and code are often not released, and licences differ. This is a snapshot of families we work with. New releases arrive monthly, so treat it as a map, not a ranking.
A widely used family of general language models, with strong tooling support. Released under a community licence that carries usage conditions, including thresholds for very large services and an acceptable use policy. Confirm current terms.
Compact and efficient models from Mistral AI, including mixture-of-experts designs. Some releases use Apache-2.0, others use research or commercial licences. Check each release separately.
A broad family covering general language, code, vision-language and embeddings, with good multilingual coverage. Several sizes use Apache-2.0, others use a custom licence. Confirm the licence for the exact checkpoint.
Reasoning-focused and general models, some under MIT-style terms. Large mixture-of-experts checkpoints need serious hardware, but distilled smaller versions exist. Review provenance and your own policy on model origin.
Smaller models designed to run on modest hardware. Released under Google's own terms of use that include a prohibited use policy. Confirm current terms before commercial use.
Small language models trained with a strong focus on data quality, useful for on-device and low-cost serving. Several releases use the MIT licence. Confirm per release.
Whisper is an open speech recognition model with permissive licensing at the time of writing. Quality in low-resource African languages varies a lot, so measure it, and plan to fine-tune with your own labelled audio.
Open embedding models such as BGE, E5 and GTE style families, plus open rerankers, power retrieval. They are small, fast and easy to host. Multilingual coverage matters more than leaderboard rank for local-language corpora.
Open vision-language, segmentation and detection models (for example SAM and YOLO style families, ViT backbones), plus OLMo-style fully open research models. Licences range from permissive to copyleft. Check each.
06Open-weight due diligence
| What it usually allows | What to check | |
|---|---|---|
| Apache-2.0 / MIT | Commercial use, modification and redistribution, with notice requirements. Apache-2.0 adds a patent grant. | That the licence applies to the weights and not only the code. That attribution notices ship with your product. |
| Community licences with usage limits | Commercial use for most organizations, often with an acceptable use policy and a user-count threshold. | Thresholds, naming and attribution rules, limits on using outputs to train other models, and what happens if you grow past a limit. |
| OpenRAIL-style licences | Broad use with behavioural restrictions that pass down to derivatives. | The use restrictions list, how it flows to your customers, and whether your use case touches a restricted area. |
| Research or non-commercial | Evaluation and academic work. | Do not ship in a product. Look for a separate commercial grant. |
| Training data and outputs | Varies. Some licences speak to outputs, many do not speak to training data provenance. | Data provenance statements, opt-out handling, and your own sector rules on data origin. |
07Open-weight supply chain
The hub is an excellent resource and also an open registry that anyone can upload to. Treat a downloaded model like a downloaded dependency: pin it, scan it, and store a copy you control.
Check the licence field, intended use, training data notes, known limits, languages and evaluation claims. Treat claims as hints, not proof. A missing card is a warning.
Safetensors files hold only tensors and cannot execute code on load. Python pickle files (.bin, .pt, .pkl) can run arbitrary code when loaded. Prefer safetensors, and refuse pickles unless they are scanned and from a trusted source.
Never enable trust_remote_code on an untrusted repository without reading the code first.
Download by commit hash, not by branch name, so the weights cannot change under you. Record the hash, the licence and the date in your model register.
Run a pickle scanner and a malware scan on every artifact, verify file hashes, and check the tokenizer and config files for unexpected custom code. Run the first load in a sandbox with no network.
Copy the approved revision into your own artifact store, such as an S3-compatible bucket or an OCI registry. Production pulls from the mirror only. Air-gapped sites receive the mirror by approved media transfer.
Datasets from the hub get the same treatment: licence check, pinned revision, and a record of what was filtered. Your own training data is versioned and hashed so a model can be traced to the exact data behind it.
08Adaptation techniques
Start with the cheapest technique that solves the problem. Most projects need retrieval and good prompts first. Fine-tuning comes after you have an evaluation set that proves it is needed.
Index your documents in PostgreSQL with the pgvector extension, retrieve relevant passages at question time and give them to the model with citations. Knowledge stays in your database, can be updated in minutes and respects access rules.
Fresh, traceable answers without retraining.
Parameter-efficient fine-tuning trains small adapter matrices and leaves the base weights frozen. QLoRA does it on a quantised base, so a single workstation GPU can tune models that would otherwise need a cluster. Adapters are small files you can version and swap.
Domain behaviour at a fraction of full tuning cost.
Train on prompt and response pairs that show the format, tone and judgement you want. Quality beats quantity: a few thousand clean, reviewed examples often beat a large noisy set.
Reliable structure and style.
Methods such as DPO and related approaches teach a model to prefer better answers over worse ones using ranked pairs. Useful for tone, refusal behaviour and reducing unwanted patterns. Needs careful preference data.
Shapes behaviour that examples alone do not.
Use a large teacher model, where its licence permits, to produce training data for a small student model that handles one task cheaply. Check the teacher's terms: some forbid using outputs to train competing models.
Frontier-level behaviour on a narrow task, at small-model cost.
Quantisation stores weights in fewer bits (GGUF for llama.cpp, AWQ and GPTQ for GPU serving) so models fit smaller hardware, with some quality loss that you must measure. Merging combines weights or adapters from related models. It can help, and it can quietly break behaviour.
Fit the hardware you actually have.
09Path one
Building a model means you train it on your data and own the result. It does not mean training a general chatbot from scratch. That costs enormous compute and data, and almost no organization needs it. Be honest about this early.
Where building makes sense: a domain-specific small model such as a classifier, a ranker or an extractor. Speech recognition or text-to-speech for languages the big models serve poorly. Vision for local conditions, such as crop disease, damaged roads or medical images. Tabular and forecasting models for demand, credit, churn, energy load or fraud. These tasks are well served by models with millions to a few billion parameters.
Continued pretraining sits between adapting and building. You take an open base model and keep training it on a large body of domain text, such as legal, medical or a language it saw little of. It needs far more data and compute than fine-tuning, and it can cause the model to forget things. Do it only when retrieval and fine-tuning clearly fall short.
Training from scratch is rarely justified for language models. The cost is in data curation, compute, failed runs and the team to run them. For tabular, vision and speech tasks it is more common and much cheaper.
Free 30 minutes with an engineer.
10Path three
Frontier models from OpenAI, Anthropic, Google and other labs are often the strongest option for hard, general tasks. You can reach them through the lab's own API, or through a cloud provider's private endpoint such as Amazon Bedrock, Azure OpenAI or Google Vertex AI. Availability of each model on each route changes, so confirm it.
The fastest route and the newest models. Data goes to the lab's infrastructure under its commercial terms. Check retention, training use, region and subprocessors.
Models served through your existing cloud agreement, with network controls such as private links, your own keys and regional hosting. Often easier for procurement and for data residency discussions.
Cost grows with usage. Models are versioned and retired on the lab's schedule. Outages and rate limits are outside your control. Some data should never leave the building regardless of contract.
11Before you send a single record
Your data protection officer and your counsel should lead this, with engineering support. Boltcliq helps you structure the questions and test the answers, and we do not give legal advice. The aim is simple: know exactly what happens to every prompt and every output.
Ask each provider in writing and keep the answers with the contract. Terms differ between consumer products, API products and cloud-hosted versions of the same model, so ask about the specific route you will use.
12Routing through Monolith
Monolith is the tool and model gateway in the Boltcliq platform. It is where provider routing, fallbacks, redaction and cost caps live, so that applications call one interface and policy decides which model answers. This is how a request flows.
An application or agent sends a request to the gateway using a familiar OpenAI-compatible interface, with a task label and the identity of the caller.
Odin authenticates the caller and checks what this identity may do. Policy decides which models are allowed for this data class, region and budget.
For tasks marked routine, the gateway tries a small local or open-weight model first. Cheap, fast, and inside your boundary.
If the answer fails a confidence or validation check, and policy allows, the request escalates to a larger or frontier model. Sensitive classes never escalate outside the boundary.
Before any request leaves the boundary, the gateway removes or masks defined fields such as names, account numbers and national IDs. Mappings stay local so answers can be restored.
Cyclopes records who called what, which model answered, the cost and the outcome. Escalated cases feed the next evaluation set and the next fine-tune.
13Illustrative interface
The idea is stable: applications ask for a task, not a vendor, and policy picks the model.
OpenAI-compatible chat API
api · Lets existing SDKs and tools point at the gateway.
Model registry and routing policy
api · Declares allowed models per task and data class.
Eval and cost reports
channel · Per route, per team, per month.
Connectors to hub mirrors and private endpoints
connector
# Illustrative only.
curl https://gateway.example.internal/v1/chat/completions \
-H "Authorization: Bearer $AGENT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "route:customer-support-draft",
"messages": [
{"role": "user", "content": "Summarise this complaint and suggest a reply."}
],
"metadata": {
"data_class": "customer-confidential",
"max_cost_usd": 0.02
}
}'
# Policy resolves "route:customer-support-draft" to, for example:
# 1. a local open-weight model on your own servers
# 2. a frontier model via a private endpoint, only if the data class allows it
# The response records which model answered and what it cost.14Running models
Serving is where many pilots stall. A model that works in a notebook still has to meet latency, uptime and cost targets for real users.
A rough rule: weights need about the parameter count times bytes per parameter. A 7 billion parameter model in 16-bit needs about 14 GB just for weights. At 4-bit it needs about 3.5 to 4 GB. Add memory for the KV cache, which grows with context length and the number of concurrent users, plus runtime overhead. A 70 billion parameter model at 4-bit needs roughly 35 to 40 GB for weights before the cache, so it spans one large GPU or two smaller ones. Treat these as estimates and test on the real stack.
Data-centre GPUs with high memory and fast interconnect suit large models and heavy traffic. Mid-range data-centre or workstation GPUs suit small and medium models. Consumer GPUs can serve small models for tests and low traffic, but check the vendor licence terms for data-centre use.
Quantised small models (roughly 1 to 8 billion parameters) run on modern server CPUs with llama.cpp or ONNX Runtime. Throughput is lower and latency higher, but it works with no GPU. A fit for back-office batch jobs, branch sites and fallbacks.
Continuous batching serves many requests on the same GPU at once. It is the biggest lever on cost per token. Better utilisation means lower unit cost, but it needs enough traffic to fill the batch.
Prefix caching reuses work on shared system prompts and documents. Response caching avoids repeat calls for identical questions. Embedding caches avoid recomputing vectors. Cache keys must respect access rules.
Scale replicas on queue depth and latency, not CPU. GPU nodes start slowly, so keep warm capacity and scale to zero only for non-urgent workloads. Plan a fallback route for when capacity runs out.
15Measure on your own cases
Public leaderboards tell you little about your documents, your customers and your languages. The only evidence that counts is a test built from your own work.
A curated set of real inputs with expert-approved answers or grading rules. Start with 100 to 300 cases and grow it. Cover easy, hard and adversarial cases, and each language you serve. Keep a hidden slice that is never used for tuning.
A strong model can grade answers at scale, and this is useful for screening. Judges favour longer answers, favour their own style and can be wrong in specialist domains and low-resource languages. Calibrate against human grades and keep a human-reviewed sample.
Every change to a prompt, adapter, quantisation, base model or retrieval index runs the golden set before release. If a score drops below the threshold, the release stops. This is what makes a model swap safe.
Test for prompt injection, data leakage, jailbreaks, bias, harmful output and unsafe tool use. Include local-language and code-mixed attacks, which generic safety tests miss. Repeat after every significant change.
16Tokens versus GPU-hours
We do not publish a break-even number, because there is no honest universal one. It depends on utilisation, model size, context length, hardware prices in your region and what you count as cost.
You pay for input and output tokens. No idle cost, no ops team, and prices have tended to fall over time. Cost grows linearly with usage, and long documents or agent loops can multiply tokens quickly.
You pay for capacity whether it is used or not. At steady, high utilisation the cost per token can be much lower. At low utilisation it can be much higher, because an idle GPU still costs money.
Compare the monthly API bill for a realistic workload with the monthly cost of the smallest hardware that meets your latency target, including people. The crossover depends on utilisation. We build this model with your numbers and show the assumptions.
Evaluation runs, data preparation, re-tuning after model updates, security reviews, monitoring and failover capacity. In-house paths shift cost from tokens to people and time.
17Boundary
Does
Does not
18FAQ
Experience you can lean on
Immersed in technology and across many different verticals, so the people on your project have done this before.
Open, Private and In-House Models
Tell us the problem. An engineer, not a salesperson, answers.
Build your own, adapt open-weight models, or use private frontier models. We work with all three, deploy where you decide, route between them through one gateway and test them on your own cases.
