DBU markup on every run
You pay platform units on top of the VMs you already rent. The markup scales with usage, not with value.
For enterprise data platform teamsKeep Databricks.
Keep Databricks.
Half the compute bill.
Your workspace, notebooks, job definitions and catalog stay exactly where they are, untouched. flow-switch.io only reroutes the compute for your jobs: they are converted automatically and run on your own Spark runtime, in your own cloud account, so you stop paying DBUs for job execution.
That is net of our fee — here is the arithmetic →- Same notebooks, jobs and schedules — nothing to relearn.
- Conversion is automatic: no migration project, no rewrite.
- Roll back to Databricks compute at any time.
The problem
Your Spark bill grew faster than your data
Enterprise data platforms end up paying twice for compute: once to the cloud, once to the vendor on top. Meanwhile the jobs themselves have not changed in years.
Locked into one runtime
Notebooks, job specs and cluster policies are tied to a single vendor. Moving anything means rewriting everything.
Idle clusters, unclear ownership
Autoscaling that never scales down, jobs nobody owns, and a monthly invoice no one can attribute to a team.
Procurement pressure
Finance asks for a 30% platform saving. Engineering has no lever that does not risk the nightly load.
How it works
Wrap, run, verify
The job wrapper reads your existing job definitions and emits an equivalent workload for the flow-switch.io runtime. Sources and sinks stay exactly where they are.
- 01
Point us at your workspace
Read-only access to job definitions, cluster configs and run history. Nothing leaves your tenant.
- 02
Automatic wrapping
Jobs, schedules and cluster policies are translated into portable Spark workloads with the same dependencies and secrets wiring.
- 03
Shadow run and diff
Both platforms run the same workload against the same input. We compare row counts, schemas and runtime before anything is cut over.
- 04
Cut over, team by team
Move one pipeline group at a time. Roll back to the original job at any point — the definition is never deleted.
What a wrapped job looks like
One short file per job. It points at the job definition — wherever it lives — states the compute you want it to run on, and keeps the way back open.
Nothing else in your repository changes: the notebook, the transformations and the table names are untouched.
sap-daily-ingest.fsio.yaml
# flow-switch.io job wrapper - the only file that gets added.# Notebooks, code and schedules stay exactly as they are.apiVersion: flow-switch.io/v1kind: WrappedJobmetadata: name: sap-daily-ingest team: data-platform # tags runs for cost attributionspec: source: # read-only; nothing is edited repo: github.com/acme/data-platform job: resources/sap_daily_ingest.job.yaml runtime: driver: { cores: 3, memory: 10g } executors: { count: 2, cores: 3, memory: 10g } auth: azureWorkloadIdentity: sap-daily-ingest-uami # federated via Azure MI, no secrets stored rollback: toDatabricks: true # one command to switch backThe platform
A runtime your platform team can actually own
Open Spark, no fork
Apache Spark 4.0 as released. No proprietary runtime, no closed extensions to reverse engineer when something breaks.
Runs in your cloud
Deployed into your AKS/EKS/GKE cluster, your VNet, your subscription. We never hold your data or your keys.
Job-level cost attribution
Every run is tagged to a team, pipeline and cost centre. Finance gets the breakdown without a spreadsheet exercise.
Delta Lake compatible
Reads and writes the same Delta tables in the same storage, so downstream consumers stay where they are. Converted jobs address those tables by path rather than through Unity Catalog — the FAQ covers exactly what that changes.
Orchestrator agnostic
Trigger from Airflow, ADF, Dagster or the built-in scheduler. Existing DAGs keep their shape.
Observability included
Spark UI, structured run logs and metrics exported to the monitoring stack you already run.
Outcomes
What teams see after the switch
0.0–0.0×cheaper per run than Databricks Classic Jobs
0.0–0.0×cheaper per run than Databricks Serverless
0%faster than Classic on sustained load, same VM SKUs
0pipeline rewrites required
Measured on an 80,000,784-row, 5-table SAP → Delta Lake benchmark across three workload shapes — see the matrix below. Per-run figures are compute-only and exclude our fee; the pricing section restates them net. Your numbers depend on job mix; the assessment models them before you commit.
Benchmarks
Three workload shapes, three computes, billed numbers
Identical PySpark code, identical dataset, identical VM SKUs where the platform exposes them. DBU quantities come from billed usage records, not estimates.
Single-pass ingest
8M-row slice · no joins · measures per-run overhead: scheduling and I/O setup
−79.8% cost vs Serverless · −63.8% vs Classic
| Metric | Databricks Serverless auto-scaled · $0.50/DBU, all-in | Databricks Classic Job 2× D4ds_v4 · $0.30/DBU + VM | flow-switch.io on AKS 2× D4ds_v4 · VM only, no DBU |
|---|---|---|---|
| Duration | 29.5 s | 76.5 s | 52.7 s |
| DBU cost | $0.1726 | $0.0327 | no DBU |
| VM cost | — | $0.0635 | $0.0080 |
| Total, warm compute | $0.1726 | $0.0961 | $0.0080 |
| Total, incl. spin-up | $0.1726 | $0.0961 | $0.0348 |
| Avg / peak CPU | sandboxed | 30.3% / 33.7% | 43.2% / 67.1% |
| Avg / peak memory | sandboxed | 59.4% / 73.6% | 20.3% / 39.4% |
| Network moved | — | 0.102 GB | 0.078 GB |
Under sustained load the runtimes tie
1959.8 s vs 2107.6 s — flow-switch.io came in 7% ahead of Classic on identical work and identical VM SKUs. Spin-up, disk cache and scheduling differences wash out once a job is execution-bound.
Serverless buys speed, not efficiency
It finished the saturation run ~7× faster by scaling out — and billed 3.633 DBU for it, roughly 6× the flow-switch.io cost for the same result.
The hardware really was saturated
Peak CPU hit 99–100% and peak memory 83–92%. Pool averages look lower only because the driver node idles: the Classic worker averaged 92.6% and the flow-switch.io executor 79.8%.
What we gave up
Open-source Spark has no Databricks disk cache. A naive DISK_ONLY persist to compensate ran 76% slower (405 s vs 230 s), so we did not adopt it. Classic keeps the memory-efficiency edge.
How this was measured
- Identical PySpark code on all three computes; per-table row counts verified identical for every scenario. Only the write target differs — Unity Catalog tables on Databricks, ADLS Delta paths on flow-switch.io. That is the product difference today rather than a benchmark shortcut; the FAQ sets out what it means for governance.
- Duration is measured inside the job, first read to last write, so it excludes startup on every platform.
- DBU quantities are billed values from system.billing.usage matched on job_run_id — not estimates. Prices: $0.30/DBU Classic, $0.50/DBU Serverless, VM $0.272/node-hour for D4ds_v4 in westeurope.
- The "incl. spin-up" row makes the three columns lifecycle-equivalent: Databricks totals already carry their 3–7 min cluster start, so the flow-switch.io figure adds measured node scale-up (~90 s) plus pod scheduling and image pull. All savings are computed from those figures.
- Utilisation samples come from the kubelet stats API (~15 s cadence) on AKS and system.compute.node_timeline (1 min) on Classic, both filtered to the in-job window. Serverless is sandboxed, so only billed DBU is available.
- Not modelled: ADLS transaction costs, AKS control plane (free tier), OS disks, intra-region bandwidth (free). Structured Streaming scenarios are still pending.
Full run records, queries and raw samples are available under NDA — ask on the call.
Deployment
Your account. Your network. Your data.
flow-switch.io is installed into infrastructure you already own and audit. There is no data plane on our side.
Your cloud account
Your identity providerSSO/SCIM through Entra ID, Okta or your existing OIDC provider.
Your Kubernetes clusterRuntime operators, driver and executor pods scheduled on your node pools.
Your storageADLS, S3 or GCS accessed with your managed identity — credentials never leave the cluster.
Your monitoringLogs and metrics shipped to the stack your SRE team already operates.
Security & compliance
Built for the review board, not around it
- Deployed inside your tenant — no customer data on flow-switch.io infrastructure
- Private networking only; no public ingress required
- Managed identity / workload identity federation, no long-lived secrets
- Hardened, minimal container images — a deliberately small supply-chain attack surface
- Audit log of every conversion, run and configuration change
- Role-based access aligned to your existing groups
- Architecture and DPIA documentation available under NDA
Working towards SOC 2 and ISO 27001 — neither certified yet
We would rather tell you where we are than imply a badge we do not hold. The audit scope is deliberately small: because the runtime is deployed inside your tenant, no customer data reaches flow-switch.io systems, so the controls under audit cover our own engineering and access processes rather than a platform holding your data. In the meantime we answer security questionnaires directly and share the architecture pack under NDA.
Security questionnaire, architecture pack and reference calls available on request.
Pricing
What it costs, and how the number is set
One meter, priced three ways. The assessment measures your converted job count and the per-run Databricks counterfactual from billed usage records. That measurement is the quote — and afterwards, the invoice.
In-house
We deliver the platform into your organisation as inner source. Your team runs it from day one and keeps running it if we disappear.
- Fee shape
- Fixed delivery fee, phased by pipeline group, plus an annual subscription for updates and support.
- The subscription buys
- Runtime releases that target cost per run, not only security patches — see below. Your fork stays current without a migration project of your own.
- Metered on
- Converted job count band, fixed by the assessment. No usage metering — we do not audit a system we handed over.
- Predictable
- Fully. Both numbers are known before the first pipeline moves.
- Priced against
- Building it yourself: roughly 2.5 FTE indefinitely, and two to three quarters before the first job moves.
- Fits
- Sovereignty or export-control constraints, and platform orgs with the bench to own a runtime.
In your tenant
We deploy and operate the runtime inside your own cloud account. No data plane on our side, no change to where your data lives.
- Fee shape
- 25% of realised savings for the first 12–18 months, then a fixed annual platform fee.
- Metered on
- Every run carries a Databricks counterfactual computed from node-seconds and the DBU rates that applied to that job before the move. Same method as the benchmark.
- Predictable
- Variable in year one and capped; fixed from first renewal, anchored on the measured year-one fee.
- Ceiling
- Never more than 15% of your previous Databricks jobs bill, whatever the savings turn out to be.
- Fits
- Most enterprises: you want the saving without adding a runtime to your team’s backlog.
Cloud
Hosted by us, as SaaS. The fastest way to run a real workload — and deliberately the least economical at scale.
- Fee shape
- Published per-vCPU-hour, all-in, listed next to the underlying VM rate so you can see the multiple.
- Metered on
- Actual consumption. No minimum, no commitment.
- Improvements
- Runtime releases land on your workloads automatically. When a release cuts cost per run, your invoice falls with it — there is nothing to upgrade.
- Predictable
- Variable, and visible per run in the console.
- Honest limit
- Net saving lands around 35–45% against Classic rather than 63%, because we carry the infrastructure and the margin.
- Fits
- Dev and test, proofs of concept, and teams below the crossover point — we will tell you where yours is.
Worked example
The arithmetic on a $1.2M jobs bill
70% Classic / 30% Serverless, derived from the published batch_long run: $0.1589 → $0.0589 per run against Classic, $0.3156 → $0.0589 against Serverless, spin-up included on both sides.
| Line | Databricks today | On flow-switch.io |
|---|---|---|
| Classic workloads | $840,000 | $311,000 |
| Serverless workloads | $360,000 | $67,000 |
| Infrastructure subtotal | $1,200,000 | $378,000 |
| flow-switch.io fee — 25% of realised saving | — | $206,000 |
| Total you pay | $1,200,000 | $584,000 |
Net saving $616,000 — 51.3% of the bill, after our fee
Why 25%, and not 40%
Net saving is gross saving multiplied by one minus our share. At the 62.9% gross saving we measure against Classic, any fee above ~20% of savings breaks a "half the bill" claim; on a mixed estate the ceiling is 27%. We set 25% so the headline stays true on the estates we actually see.
A cap, so procurement can budget the worst case
The fee is capped as a share of your previous Databricks jobs bill. If savings run higher than modelled, you keep the difference rather than paying us more for it.
A floor, so we decline deals that are too small
There is a minimum annual commitment. Below it the migration costs you more in change process than it returns, and we would rather say that on the first call.
A sunset on the counterfactual
Nobody believes a synthetic Databricks price for a workload that has not run there in a year. Share-of-savings converts to a fixed fee at first renewal, escalating with run volume — not with savings.
What the subscription actually buys
Updates are not only security patches. Every release targets cost per run on jobs you have already moved, so the fee you pay in year three should be buying a cheaper bill than it bought in year one.
- Spark version tracking, so upstream performance work reaches your jobs without a migration project of your own.
- Scheduler, autoscaling and I/O improvements, measured on the same benchmark matrix we publish — a release that regresses cost per run does not ship.
- Job-wrapper coverage: job types flagged as unsupported in your assessment come into scope over time, at no extra fee.
- Every release ships with its before/after cost per run on your own workload shapes, so you can see whether the subscription paid for itself.
If the assessment does not model a net saving after our fee, we tell you on the call and there is no engagement. We would rather lose the deal than defend a number later.
How the fee is metered
- The meter is the one behind the benchmark: node-seconds from the runtime, priced against the Databricks rates that applied to that job before the move — $0.30/DBU Classic, $0.50/DBU Serverless — so every invoice line carries a counterfactual you can audit.
- Savings are computed on moved jobs only. The worked example assumes full wrapper coverage; the assessment gives you your real coverage percentage and re-runs the table against it before you commit.
- The fee is charged on realised savings, not modelled ones. A job that runs less often than expected costs you less and earns us less.
- Spin-up is included on both sides of the comparison, exactly as in the benchmark. We do not bill for a saving created by hiding cluster start time.
- Scope is Spark jobs compute. SQL warehouses, notebooks, ML and DLT stay where they are — we do not touch them and we do not claim savings on them.
- Cloud rates are published. In-house and in-tenant terms are quoted from your assessment rather than from a rate card that would not apply to you.
- Runtime improvements shipped after cutover lower your cost per run. On share-of-savings that means our fee falls with your bill; on a fixed fee you keep the whole improvement.
The assessment output is the quote. Bring your job list to the call and we will scope it.
Engagement
How a rollout runs
- 01
Discovery call
45 minutes. Your job mix, your constraints, whether this is a fit at all.
- 02
Assessment
We model wrapper coverage and expected savings against your actual run history.
- 03
Pilot
One pipeline group runs in shadow mode next to production until the diffs are clean.
- 04
Rollout
Team-by-team cutover with rollback available at every step.
Commercial terms follow the assessment, in one of the three shapes above — nothing is committed before the numbers are yours.
FAQ
Questions the review board asks
Do we have to rewrite our notebooks?
No. The job wrapper translates job and cluster definitions; the Spark code itself runs unchanged. Where an API has no open-source equivalent, the assessment flags it before you commit.
What happens to the jobs we keep on Databricks?
Nothing. Most teams move a subset first and run both platforms side by side indefinitely. The original job definitions are never deleted.
Where does our data go?
Nowhere new. The runtime is deployed into your cloud account and reads and writes the same storage accounts as before. What changes is how a moved job addresses those tables, not where they live — the next answer covers it.
What happens to Unity Catalog?
Today a converted job addresses Delta tables by storage path rather than through Unity Catalog. The files, the format and the storage account are unchanged, so the tables stay readable by everything that reads them now, Databricks included — most teams keep them registered in UC as external tables so names and downstream queries do not move. What does change on a moved job is where governance is enforced: storage permissions on the runtime’s managed identity take the place of UC table grants, and UC will not capture lineage or apply row filters and column masks for that job. Native Unity Catalog support is on the roadmap, and we would rather not put a date on it here. The assessment flags every job where this matters, and governance-critical pipelines are usually the ones we recommend leaving on Databricks.
Who operates the runtime?
Your platform team owns it, with our support. Everything is standard Kubernetes and open-source Spark, so it stays operable without us.
Can we reproduce your benchmark numbers?
Yes — that is the point of publishing the method alongside them. DBU figures come from billed usage records rather than estimates, spin-up is included on every column, and the workload is the same PySpark code on all three computes. We will walk through the run records with your team and re-run the matrix on your own job shapes during the assessment.
How long does a migration take?
Assessment to first shadow run is typically two weeks. Full rollout depends on how many pipeline groups you move and how fast your change process allows.
What does it cost?
Three shapes, depending on who runs the platform. In-house: a fixed delivery fee plus an annual subscription that keeps the runtime current — releases target cost per run, not only security patches — priced against what building it yourself would cost. In your tenant: 25% of realised savings, metered per run against a Databricks counterfactual from billed usage records, capped as a share of your previous bill and converting to a fixed fee at renewal. Cloud: a published per-vCPU-hour rate, listed next to the underlying VM rate so you can see the multiple. In all three the fee comes out of savings we can show you — if the assessment does not model a net saving after our fee, we say so on the call and there is no engagement.
Book a call
Bring your job list. We will tell you what moves.
45 minutes with an engineer, not an SDR. Come with a rough count of jobs and clusters and you will leave with an estimate of what is convertible.
- A walkthrough of the job wrapper against your job types
- An honest read on which workloads should stay where they are
- A scoped assessment plan if it looks like a fit
Pick a time
The calendar has not loaded: it comes from Calendly, and nothing is fetched from them until you agree.
Or skip it entirely and mail us — we will send times by hand.