AI Engineering

What the $349 benchmark actually produces, run on our own hardware: Qwen3-8B on one RTX PRO 6000 Blackwell (96 GB), identical prompts and settings across vLLM 0.27.1, SGLang 0.5.9 and llama.cpp, then an FP8 pass on the winner to isolate quantization. Greedy decoding, token counts matched across engines before timing was compared.

Findings: stack choice barely matters for one user (83 to 96 tok/s everywhere) but is a 4x decision under concurrent load; time to first token differs 4x between engine families; FP8 gave a clean 1.5x with zero factual regressions on a fixed 20-prompt check. Getting FP8 to run on this workstation-class Blackwell chip (sm_120) required routing around a kernel assertion in the default FP8 path, which is exactly the class of work the kernel tier exists for.

Every number ships with raw CSVs, an environment manifest and a one-command reproduction script. The script was re-run end to end after the report was written: all figures reproduced within 6 percent. Read the full sample report or download the raw evidence bundle. To see what numbers like these do to your serving bill, use the self-hosted LLM cost calculator.

Fixed scope: one model, one target GPU, up to three candidate serving stacks chosen from vLLM, SGLang, TensorRT-LLM, ONNX Runtime and llama.cpp, plus one quantization pass (FP8, INT8, INT4, AWQ or GPTQ, whichever fits the model and hardware).

• a Benchmark report: tokens per second, p50 and p99 latency, VRAM use, and cost per 1M tokens, before and after.
• b The tuned configuration files for the recommended stack.
• c Reproduction scripts. Every number in the report can be re-run in your environment.

Delivered within 5 days after access is in place. If the benchmark shows your current setup is already at the practical limit, the report proving that is the deliverable.

Quoted only after C, and only when the C report has isolated a bottleneck that kernel work can move. Scope: CUDA or Triton kernels written for your hardware generation, fused ops, and custom quantization kernels.

• a The contract states the target speedup before work starts.
• b If the target is missed, the final payment is waived.
• c A time cap is written into the contract, so the work cannot run open ended.

Deliverables: kernel source, an integration patch for your stack, before and after benchmarks, and tests.

The required first step for any build. We audit the data, train a baseline, and state what is realistically achievable before you commit to the full build.

• a Data audit: volume, quality, and label state.
• b A trained baseline and its measured result.
• c The metric range the full build can realistically reach.
• d A fixed quote and a draft spec for the build itself.

Delivered within 5 days. We do not sell a build without this step.

The exact tier is fixed in the feasibility report. Typical builds: image and text classification or detection, domain fine-tuning of an open LLM with LoRA, document and OCR extraction pipelines, and tabular prediction models. Work outside this range is declined rather than improvised.

• a A containerized inference server (Docker image) ready to run on your infrastructure.
• b The trained weights.
• c An evaluation report against the acceptance metric agreed in A.
• d A deployment guide. We do not host your model; a 30-day staging option is the only exception.

Acceptance is the metric agreed in writing during A, stated as an improvement over the baseline. If the delivered model misses it, the final payment is waived.