Why we use Triton, Ray,
and absolutely nothing else.

This post is for the platform engineer interviewing at Drape next week, the architect at another vertical-AI company doing infrastructure due diligence on us, and the very small number of founders building generative pipelines who want to know what we actually run on. Everyone else is welcome to skim, but the substance is technical.

Drape's generation pipeline serves about 600,000 editorial frames per month across 184 ateliers, with a p95 wall-time of 3 minutes 41 seconds from source upload to first frame ready. The whole pipeline runs on a stack of fourteen direct dependencies. The largest one is PyTorch. The second largest is FastAPI. Everything else is small.

We did not start here. We started, like most ML platforms, with everything. We had Kubeflow, MLflow, Weights & Biases, Hugging Face Inference Endpoints, Modal, Replicate (as a fallback), Triton Inference Server (the NVIDIA one), Ray Serve, Ray Tune, Ray Data, Kubernetes with Karpenter autoscaling, Argo for workflows, Prefect for batch, Dagster for ETL, Airflow because we were not sure, and Lambda for "glue."

By month six it was unmaintainable. Every alerting page involved correlating five dashboards. Every cold-start GPU was a thirty-minute investigation across three frameworks. Every release felt like deploying a small Linux distribution.

So we ripped almost all of it out.

What we kept, in order of how mission-critical each is to our wall-time SLA:

Triton (the OpenAI kernel language, not the NVIDIA server). Our custom CUDA kernels for the fabric-aware diffusion attention are written in Triton. About 1,400 lines of code that does 60% of the work on every frame. Triton gives us the performance of hand-written CUDA with the iteration speed of Python. We have rewritten the same kernel four times in eighteen months as the model architecture has evolved. Each rewrite took a day and gave us a 1.4×–2.1× speedup.

Ray Serve. Our serving layer. We run a single Ray cluster that hosts every model and routes every request. Ray's actor model is the right primitive for our workload — long-lived GPU-resident processes that take incoming JSON requests, run inference, and return base64 frames. We tried FastAPI + GPU multiprocessing first. It was unmaintainable. Ray Serve replaced it and removed about 4,000 lines of custom orchestration code.

PyTorch. Obviously. We are on PyTorch 2.4 with torch.compile on every forward pass. We have not had a real reason to evaluate JAX. JAX is faster on TPUs which we do not use. PyTorch is faster on H100s which we do use.

FastAPI. Our HTTP boundary. Every Drape API request lands on a FastAPI instance, gets validated, and is forwarded to Ray. FastAPI is fast enough, type-safe enough, and Pydantic v2 is genuinely good. We have never needed anything else.

Postgres. One Postgres database, hosted on Hetzner managed Postgres. Everything goes here — accounts, brand kits, generation metadata, billing, audit log. We have not yet needed Redis, Elasticsearch, Kafka, ClickHouse, or BigQuery. We will when we do.

S3 (technically MinIO on Hetzner). Source photos in, generated frames out. Lifecycle policies handle archival. We do not use a CDN — frames are streamed direct from our Frankfurt region via the studio app. Latency is acceptable for our customers, who are not waiting on a low-bandwidth phone connection to download a 4K editorial frame.

That is the entire critical path. Six things. The remaining eight dependencies are smaller — Stripe for billing, Brevo for transactional email, Plausible for analytics, OpenTelemetry for tracing, Sentry for errors, GitHub Actions for CI, Tailwind for the dashboard, Next.js for the studio app. Total surface area: about thirty thousand lines of code we own.

Every dependency you add is a bet that the maintainers of that dependency will care about your problem for the next five years. Most of them will not.

What we removed, and why

Kubernetes. We ran on Kubernetes for six months. We removed it. Kubernetes is a marvelous piece of software, and it is built for organizations with thirty engineers managing a thousand services. We are twelve engineers managing four services. The complexity tax was not justified. We now run on Docker Compose on dedicated Hetzner GPU servers, with HAProxy in front. We deploy by SSHing in and running docker compose up -d. It takes six seconds. Nobody pages anyone in the middle of the night because a sidecar crashed.

Kubeflow, MLflow, Weights & Biases. We had all three. We now use a Postgres table called experiments with twelve columns. It is searchable. It is exportable. It is owned by us. It is enough.

Argo, Prefect, Dagster, Airflow. We had four workflow engines simultaneously because nobody had agreed on one. We removed all four. The work that needed workflow orchestration — training runs, batch generations, dataset preparation — is now twelve cron entries calling Python scripts. Each script is idempotent. Each script logs to Postgres. Each script can be re-run by hand if it fails. Total complexity: zero new abstractions.

Modal and Replicate. Both excellent products. Both wrong for us once we had committed to a single base model and a single deployment target. They are right for someone running fifteen models with bursty workload. We run one model family with steady workload.

The boring infrastructure principle

The reason we removed all of this is not aesthetic. It is operational. Every dependency in your stack is a thing that can break, that you need to upgrade, that gets a CVE, that introduces a config knob that you eventually have to reason about at 3 AM during an incident.

Big companies can afford the complexity because they have the people. We do not have the people, and more importantly, we do not want the people. We want twelve engineers who can each hold the full stack in their head, ship features end-to-end, and go home at 19:00 without an on-call rotation that wakes them at 4 AM because Argo's controller had a memory leak.

Boring infrastructure is what makes this possible. PyTorch is boring. FastAPI is boring. Postgres is boring. Docker Compose on a dedicated server is boring. The boring choices are what give you the budget — in attention, in operational headspace, in calendar time — to do the unboring work, which in our case is training models on the right fabrics.

Use boring tools so you can do interesting work. Use interesting tools and you will spend your interesting hours debugging them.

What we will add next, and what we will not

We are evaluating two things for the next six months. First, vLLM as a possible replacement for our custom Ray Serve inference path — vLLM's continuous batching is genuinely better than what we have, and it is mature enough now to be considered "boring." We will probably adopt it in Q3.

Second, a proper feature store. Right now our training data pipeline is "rsync from one server to another." We have outgrown that. We are evaluating Feast and LakeFS. We will pick one and stick with it for five years.

What we will not add: Kubernetes (will not return), a service mesh (do not need one), a vector database that is not Postgres pgvector (works fine at our scale), an LLM gateway (we serve one model family), or any of the new "AI infrastructure" platforms announced on Hacker News last week. We will revisit them in 2028 if they are still around.

Hiring note

If reading the above made you feel that this is the kind of engineering organization you want to work in, we are hiring a senior platform engineer in Bucharest right now. Email me directly — daria@drape.studio. The bar is high and the work is real and we go home at 19:00.