CIS Seminar Series 2026 -Jingtong Hu Ph.D.-University of Pittsburgh

About

How to Make LLMs Cool: Breaking the Thermal Wall through Thermally Adaptive Algorithms and Human-in-the-Loop Pacing for On-Device Inference

ABSTRACT

On-device LLM inference is driven by critical needs for privacy, low-latency safety, and off-grid availability. Yet sustained inference drives mobile GPU temperatures past safe thresholds within minutes, forcing devices into aggressive throttling that destroys the very benefits motivating local execution. Current thermal management treats this as a hardware-only problem — reactive frequency scaling that is blind to the workload it serves. In this talk, I will present three systems that rethink thermal management as a cross-layer, workload-aware design objective spanning model internals and human perception.

First, I will introduce GreenDynaMix, a runtime that converts MoE expert precision into a dynamic thermal control knob. A dual-PID control loop couples DVFS-driven frequency reduction with sensitivity-aware precision demotion, compensating for throttling-induced throughput loss while maintaining output quality. Second, I will present TRAIL, which extends this idea to Vision-Language Models by dynamically replacing thermally redundant transformer layers with pre-computed linear projections, recovering over 50% of throttling-induced throughput loss while maintaining 95.9% accuracy on VLM benchmarks. Third, I will describe GreenRead, a perceptual throughput pacing framework that uses gaze tracking to match token generation speed to human reading rates, eliminating "wasted heat" from tokens generated faster than users can consume them. Combining GreenRead with GreenDynaMix on a Samsung Galaxy S25 yields thermal benefits exceeding either technique alone, pointing toward a vertically integrated approach where a single algorithmic decision propagates thermal relief across the entire inference stack.

BIOGRAPHY

Jingtong Hu is a Professor in the Department of Electrical and Computer Engineering at the University of Pittsburgh. His research focuses on embedded systems, on-device AI, and digital health, with an emphasis on human-centric ambient intelligence. His work has received three best paper awards, including the Donald O. Pederson Best Paper Award from IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, along with five best paper nominations from DAC, ASP-DAC, and ESWEEK. He is also a recipient of the IEEE TCCPS Mid-Career Award, DAC Under-40 Innovators Award, JSPS Invitational Fellowship, Carl Friedrich von Siemens Research Fellowship, Humboldt Research Fellowship, Outstanding New Faculty Awards, and ACM SIGDA Meritorious Service Award. He served as an Executive Committee member and Education Chair for ACM SIGDA, Executive Committee member for DAC, and as an Associate Editor for IEEE Embedded Systems Letters, ACM Transactions on Design Automation of Electronic Systems, ACM Transactions on Cyber-Physical Systems, ACM Transactions on Embedded Computing, and ACM Transactions on Health. His research has been sponsored by NSF, NIH, ARL, AFRL, Meta, Amazon, and others.