Compiler Engineer
Build high-performance inference engines and model compilers. Experience with LLVM, MLIR, or similar frameworks preferred.
About the Role
The compiler team builds the high-throughput, low-latency execution engine that powers Hanzo Engine and edge inference. You will work on optimizing compute graphs, lowering neural network intermediate representations, and writing hardware-specific backends for GPUs, TPUs, and emerging NPUs.
Hanzo builds an open AI cloud — compilers, models, distributed infrastructure, and agent tooling. Our codebases are predominantly open source, and team members operate with high autonomy and end-to-end ownership.
Key Responsibilities
Develop optimizing compilers and graph transformation passes for LLM inference pipelines.
Work with MLIR, LLVM, and custom IRs to generate performant machine code and kernel invocations.
Optimize KV cache memory management, speculative decoding, and paged attention engines.
Benchmark, profile, and push the Pareto frontier of tokens per second per dollar.
What We're Looking For
Strong proficiency in C++ and/or Rust, with deep knowledge of compiler architecture (frontends, IRs, passes, backends).
Familiarity with MLIR, LLVM, or ML graph compilation tools (TVM, XLA, TensorRT, TorchDynamo).
Understanding of GPU hardware architectures (compute capabilities, tensor cores, shared memory, warp scheduling).
Rigorous approach to performance benchmarking, micro-benchmarks, and regression testing.
Nice to Have
Experience writing custom CUDA or Metal kernels for flash attention or fused operations.
Contributions to open source compiler or runtime projects.