Cryptography Primitives Acceleration (CPA Tutorial)
Mar 22: Morning @ ASPLOS'26
Artificial Intelligence (AI) is driving a new industrial revolution, transforming how we create, exchange, and safeguard information. From chatbots and autonomous vehicles to enterprise assistants and AI coders, human workflows are increasingly translated into digital tokens—a process that makes the world effectively tokenized. Yet this transformation exposes sensitive data at an unprecedented scale, making privacy preservation a defining challenge of the AI era. Current privacy solutions, however, remain prohibitively expensive. Software-based encryption can slow computation by 100–10,000×, while hardware isolation demands millions of dollar fabrication costs. These barriers make privacy protection over prohibitive, leaving the broader AI revolution unprotected. This tutorial gives you a tour of the state-of-the-art in cryptography primitives acceleration, and how to make privacy protection as efficient and accessible as unprotected AI.
In this tutorial, you will learn:
- (a) How to convert an AI model into Homomorphic Encryption (HE), enabling privacy-preserving AI inference, and how to make a compiler to do so?
- (b) How to deploy HE on TPU hardware to achieve SotA throughput and energy efficiency?
- (c) How to further extend HE to arbitrary-precision cryptography primitives, enabling Zero-Knowledge Proofs (ZKPs)?
- (d) How to write high-performance JAX kernels, optimize them to run on distributed TPUs?
Figure 1: CPA overview, turning TPU as a crypto accelerator, achieving SoTA throughput and energy efficiency.
CPA Resources
CROSS
Enabling Google TPUs for HE, achieving SoTA throughput and energy efficiency.
HEIR
Google's MLIR-based compiler toolchain for converting AI models into HE.
SmartPAF
Non-polynomial operators approximations to make model crypto-friendly.
MORPH
Enabling TPUs for ZKP (arbitrary-precision MSM and NTT, stay tuned).
Intel HERACLES
Encrypted Computing SDK.
Agenda (Mar 22, 2026, ASPLOS'26)
Welcome
Keynote: Privacy-Preserving AI Computing
Introduction to Verifiable Privacy-preserving AI via HE and ZKP
Overview of HE compilation stacks.
HEIR Compiler – Encrypt Model with better Performance / less Accuracy Degradation
- Overview of components for compiling ML models
- Hands on: Model rearchitecture
- Hands on: Arithmetization (polynomial approximation, packing/kernel selection)
- Hands on: Scheme selection, noise analysis, ciphertext management
- Hands on: Accuracy degradation considerations
- Hands on: Differences among schemes (CGGI, CKKS/BGV/BFV)
- Hands on: Life cycle of model compilation using CKKS
- Hands on: Export to jaxite (CROSS)
CROSS – Enabling Google TPU for Homomorphic Encryption
Memory, Computation and Accuracy overhead of Cryptography primitives (HE and ZKP).
TPU-specific Memory and Compute Optimizations for HE.
- Hands on: Encode, Encryption, Decryption, Decode
- Hands on: HE Kernels (HEAdd, HEMul, HERotation, HERescale, NTT, Keyswitch)
- Hands on: Parameter Selection (32-bit logq, extended via rescaling)
- Hands on: MNIST inference benchmark
- Hands on: Similarity search benchmark
MORPH – Enabling Google TPU for ZKP
TPU-specific Memory and Compute Optimizations for Multi-scalar Multiplication.
- Hands on: Efficient Modular Reduction and Multiplication for Big Integers (>256 bits)
- Hands on: Different dataflow choices (Bucket-level, Window-level, Hybrid parallelism)
Hardware – Intel’s FHE Accelerator ASIC HERACLES
High-level overview of architecture and ISA, Polynomial ISA, Memory sub-system, formatting.
- Hands on: Kernel generation using Encrypted Computing SDK
- Hands on: Simulation of functional correctness and performance estimation
- Hands on: All basic FHE operations and MNIST inference benchmark
Future Work / Discussion / Call for action
Organizers
Jianming Tong
Georgia Institute of Technology
5th-year PhD candidate, enabling systems for Agentic Cryptography. One system for both AI and cryptography primitives.
Tushar Krishna
Georgia Institute of Technology
Associate Professor in ECE. Research spans computer architecture, interconnection networks, distributed systems, and AI/ML accelerator systems.
Jeremy Kun
Staff software engineer and tech lead on HEIR compiler. Author of "A Programmer's Introduction to Mathematics".
Anupam Golder
Intel Corporation
Research scientist at Intel CRL developing ASIC microarchitectures for cryptographic accelerators like FHE.
Jingtian Dang
Georgia Institute of Technology
2nd-year PhD student. Leads the project to map ZKP primitives onto Google TPUs.
Shruthi Gorantala
Co-founder and lead for FHE team at Google. Driving standardization of FHE benchmarking.
Asra Ali
Software Engineer on Privacy, Safety, and Security. Focus on developing a transpiler for FHE.
Baiyu Li
Research Scientist studying lattice-based cryptography and secure computation. PhD from UC San Diego.
Simon Langowski
MIT
PhD student at MIT working on cryptography for systems to innovate efficient algorithms for crypto workloads.
Citation
If you find this tutorial helpful, feel free to:
- Star CROSS repo at https://github.com/EfficientPPML/CROSS
- Star the HEIR repo at https://github.com/google/heir
- Star the Jaxite repo at https://github.com/google/jaxite
- Cite our paper with biblatex below:
@inproceedings{tong2025CROSS,
author = {Jianming Tong and Tianhao Huang and Jingtian Dang and Leo de Castro and Anirudh Itagi and Anupam
Golder and Asra Ali and Jevin Jiang and Jeremy Kun and Arvind and G. Edward Suh and Tushar Krishna},
title = {Leveraging ASIC AI Chips for Homomorphic Encryption},
year = {2026},
publisher = {2026 IEEE International Symposium on High Performance Computer Architecture (HPCA)},
address = {Australia},
keywords = {AI ASICs, TPU, Fully Homomorphic Encryption},
location = {Australia},
series = {HPCA'26} }@misc{ali2025heiruniversalcompilerhomomorphic,
title={HEIR: A Universal Compiler for Homomorphic Encryption},
author={Asra Ali and Jaeho Choi and Bryant Gipson and Shruthi Gorantala and Jeremy Kun and Wouter Legiest and Lawrence Lim and Alexander Viand and Meron Zerihun Demissie and Hongren Zheng},
year={2025},
eprint={2508.11095},
archivePrefix={arXiv},
primaryClass={cs.CR},
url={https://arxiv.org/abs/2508.11095},
}
