Guan-Horng Liu

Hi, I am Guan-Horng Liu (I go by "Guan" ), a fourth-year Machine Learning PhD in Georgia Tech advised by Evangelos A. Theodorou. I also work closely with Molei Tao from Georgia Tech and Valentin De Bortoli from CNRS.

My research concerns scalable computational methods for Neural dynamics, which include, e.g., Neural ODEs, Neural SDEs (diffusion models), interpreting DNNs as discrete-time systems, or mean-field population interaction. The high dimensionality and over-parameterization of these dynamics pose interesting challenges to traditional computational methods. In return, studying these parametrized models from a pure dynamical standpoint enriches theoretical and algorithmic improvements grounded on the first principles of optimal control, stochastic calculus, game theory, optimal transport, and stochastic physics. This new line of research has received spotlight/oral presentations in ICLR’21, ICML’21, and NeurIPS’21.

I am fortunate to work in Nvidia Research during 2022 Summer, developing robust diffusion models with Weili Nie, Arash Vahdat, De-An Huang, and Anima Anandkumar. See my full CV here (updated in Sep 2022).

Contact: ghliu [at] gatech [dot] edu
Follow: Google Scholar | LinkedIn | ghliu | @guanhorng_liu

Updates

Sep, 2022	Our Deep Generalized Schrödinger Bridge is accepted to NeurIPS 2022.
Aug, 2022	I will give an invited talk at Score Models Workshop in NeurIPS 2022!
Jun, 2022	I have joined Nvidia Research for an internship this summer.
Feb, 2022	Our Likelihood training of Schrödinger Bridge is accepted to ICLR’22. Code is released here!
Jan, 2022	I am very fortunate to receive GaTech AE Graduate Fellowship. Thanks GaTech AE!
Dec, 2021	I give a talk on Higher-order Optimization of Neural ODEs at DataSig.
Oct, 2021	I give a talk on Optimal Control Theoretic Neural Optimizer at GaTech ML PhD Seminar.
Sep, 2021	Our Second-order Neural ODE Optimizer is accepted to NeurIPS’21 as a Spotlight (3.0%).
May, 2021	Our Dynamic Game Theoretic Neural Optimizer is accepted to ICML’21 as an Oral (3.0%).
Jan, 2021	Our Differential Dynamic Programming Neural Optimizer is accepted to ICLR’21 as a Spotlight (3.8%).

Publications

2022

NeurIPS 2022

Deep Generalized Schrödinger Bridge

Guan-Horng Liu, Tianrong Chen*, Oswin So*, and Evangelos A Theodorou

Advances in Neural Information Processing Systems, 2022

bibtex abstract arXiv

Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling the collective behavior of individual agents interacting stochastically with a large population. In this work, we aim at solving a challenging class of MFGs in which the differentiability of these interacting preferences may not available to the solver, and the population is urged to converge exactly to some desired distribution. These setups are, despite being well-motivated for practical purposes, complicated enough to paralyze most (deep) numerical solvers. Nevertheless, we show that Schrödinger Bridge — as an entropy-regularized optimal transport model — can be generalized to accepting mean-field structures, hence solving these MFGs. This is achieved via the application of Forward-Backward Stochastic Differential Equations theory, which, intriguingly, leads to a computational framework with a similar structure to Temporal Difference learning. As such, it opens up novel algorithmic connections to Deep Reinforcement Learning that we leverage to facilitate practical training. We show that our proposed objective function provides necessary and sufficient conditions to the mean-field problem. Our method, named Deep Generalized Schrödinger Bridge (DeepGSB), not only outperforms prior methods in solving classical population navigation MFGs, but is also capable of solving 1000-dimensional opinion depolarization, setting a new state-of-the-art numerical solver for high-dimensional MFGs. Our code will be made available at https://github.com/ghliu/DeepGSB.

@inproceedings{liu2022deepgsb,
title={Deep Generalized Schrödinger Bridge},
author={Liu, Guan-Horng and Chen, Tianrong and So, Oswin and Theodorou, Evangelos A},
booktitle={Advances in Neural Information Processing Systems},
year={2022},
}
ICLR 2022

Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory

Tianrong Chen*, Guan-Horng Liu*, and Evangelos A Theodorou

International Conference on Learning Representations (* Equal contribution), 2022

bibtex abstract arXiv PDF Code

Schrödinger Bridge (SB) is an entropy-regularized optimal transport problem that has received increasing attention in deep generative modeling for its mathematical flexibility compared to the Scored-based Generative Model (SGM). However, it remains unclear whether the optimization principle of SB relates to the modern training of deep generative models, which often rely on constructing log-likelihood objectives.This raises questions on the suitability of SB models as a principled alternative for generative applications. In this work, we present a novel computational framework for likelihood training of SB models grounded on Forward-Backward Stochastic Differential Equations Theory - a mathematical methodology appeared in stochastic optimal control that transforms the optimality condition of SB into a set of SDEs. Crucially, these SDEs can be used to construct the likelihood objectives for SB that, surprisingly, generalizes the ones for SGM as special cases. This leads to a new optimization principle that inherits the same SB optimality yet without losing applications of modern generative training techniques, and we show that the resulting training algorithm achieves comparable results on generating realistic images on MNIST, CelebA, and CIFAR10. Our code is available at https://github.com/ghliu/SB-FBSDE.

@inproceedings{chen2021likelihood,
title={Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory},
author={Chen, Tianrong and Liu, Guan-Horng and Theodorou, Evangelos A},
booktitle={International Conference on Learning Representations},
year={2022},
}

2021

NeurIPS 2021

Second-order Neural ODE Optimizer

Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

Advances in Neural Information Processing Systems, 2021

Spotlight presentation (acceptance rate 3.0%)

bibtex abstract arXiv PDF Code Poster Slides Video Talk @ DataSig

We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient computation by solving a backward ODE, deriving efficient second-order methods becomes highly nontrivial. Nevertheless, inspired by the recent Optimal Control (OC) interpretation of training deep networks, we show that a specific continuous-time OC methodology, called Differential Programming, can be adopted to derive backward ODEs for higher-order derivatives at the same O(1) memory cost. We further explore a low-rank representation of the second-order derivatives and show that it leads to efficient preconditioned updates with the aid of Kronecker-based factorization. The resulting method converges much faster than first-order baselines in wall-clock time, and the improvement remains consistent across various applications, e.g. image classification, generative flow, and time-series prediction. Our framework also enables direct architecture optimization, such as the integration time of Neural ODEs, with second-order feedback policies, strengthening the OC perspective as a principled tool of analyzing optimization in deep learning.

@inproceedings{liu2021second,
title={Second-order Neural ODE Optimizer},
author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
booktitle={Advances in Neural Information Processing Systems},
year={2021},
}
ICML 2021

Dynamic Game Theoretic Neural Optimizer

Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

International Conference on Machine Learning, 2021

Oral presentation (acceptance rate 3.0%)

bibtex abstract arXiv PDF Poster Slides Video

The connection between training deep neural networks (DNNs) and optimal control theory (OCT) has attracted considerable attention as a principled tool of algorithmic design. Despite few attempts being made, they have been limited to architectures where the layer propagation resembles a Markovian dynamical system. This casts doubts on their flexibility to modern networks that heavily rely on non-Markovian dependencies between layers (e.g. skip connections in residual networks). In this work, we propose a novel dynamic game perspective by viewing each layer as a player in a dynamic game characterized by the DNN itself. Through this lens, different classes of optimizers can be seen as matching different types of Nash equilibria, depending on the implicit information structure of each (p)layer. The resulting method, called Dynamic Game Theoretic Neural Optimizer (DGNOpt), not only generalizes OCT-inspired optimizers to richer network class; it also motivates a new training principle by solving a multi-player cooperative game. DGNOpt shows convergence improvements over existing methods on image classification datasets with residual networks. Our work marries strengths from both OCT and game theory, paving ways to new algorithmic opportunities from robust optimal control and bandit-based optimization.

@inproceedings{liu2021dynamic,
title={Dynamic Game Theoretic Neural Optimizer},
author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
booktitle={International Conference on Machine Learning},
year={2021},
}
ICLR 2021

DDPNOpt: Differential Dynamic Programming Neural Optimizer

Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

International Conference on Learning Representations, 2021

Spotlight presentation (acceptance rate 3.8%)

bibtex abstract arXiv PDF Poster Slides Video

Interpretation of Deep Neural Networks (DNNs) training as an optimal control problem with nonlinear dynamical systems has received considerable attention recently, yet the algorithmic development remains relatively limited. In this work, we make an attempt along this line by reformulating the training procedure from the trajectory optimization perspective. We first show that most widely-used algorithms for training DNNs can be linked to the Differential Dynamic Programming (DDP), a celebrated second-order method rooted in the Approximate Dynamic Programming. In this vein, we propose a new class of optimizer, DDP Neural Optimizer (DDPNOpt), for training feedforward and convolution networks. DDPNOpt features layer-wise feedback policies which improve convergence and reduce sensitivity to hyper-parameter over existing methods. It outperforms other optimal-control inspired training methods in both convergence and complexity, and is competitive against state-of-the-art first and second order methods. We also observe DDPNOpt has surprising benefit in preventing gradient vanishing. Our work opens up new avenues for principled algorithmic design built upon the optimal control theory.

@inproceedings{liu2021differential,
title={DDPNOpt: Differential Dynamic Programming Neural Optimizer},
author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
booktitle={International Conference on Learning Representations},
year={2021},
}
arxiv

Spatio-Temporal Differential Dynamic Programming for Control of Fields

Ethan N Evans, Oswin So, Andrew P Kendall, Guan-Horng Liu, and Evangelos A Theodorou

arXiv preprint (in submission), 2021

bibtex abstract arXiv

We consider the optimal control problem of a general nonlinear spatio-temporal system described by Partial Differential Equations (PDEs). Theory and algorithms for control of spatio-temporal systems are of rising interest among the automatic control community and exhibit numerous challenging characteristic from a control standpoint. Recent methods focus on finite-dimensional optimization techniques of a discretized finite dimensional ODE approximation of the infinite dimensional PDE system. In this paper, we derive a differential dynamic programming (DDP) framework for distributed and boundary control of spatio-temporal systems in infinite dimensions that is shown to generalize both the spatio-temporal LQR solution, and modern finite dimensional DDP frameworks. We analyze the convergence behavior and provide a proof of global convergence for the resulting system of continuous-time forward-backward equations. We explore and develop numerical approaches to handle sensitivities that arise during implementation, and apply the resulting STDDP algorithm to a linear and nonlinear spatio-temporal PDE system. Our framework is derived in infinite dimensional Hilbert spaces, and represents a discretization-agnostic framework for control of nonlinear spatio-temporal PDE systems.

@article{evans2021spatio,
title={Spatio-Temporal Differential Dynamic Programming for Control of Fields},
author={Evans, Ethan N and So, Oswin and Kendall, Andrew P and Liu, Guan-Horng and Theodorou, Evangelos A},
journal={arXiv preprint arXiv:2104.04044},
year={2021},
}
RSS 2021

Variational Inference MPC using Tsallis Divergence

Ziyi Wang, Oswin So, Jason Gibson, Bogdan Vlahov, Manan S Gandhi, Guan-Horng Liu, and Evangelos A Theodorou

Robotics: Science and Systems, 2021

bibtex abstract arXiv

In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using thenon-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive PathIntegral Control, Cross Entropy Method, and Stein VariationalInference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

@inproceedings{wang2021variational,
title={Variational Inference MPC using Tsallis Divergence},
author={Wang, Ziyi and So, Oswin and Gibson, Jason and Vlahov, Bogdan and Gandhi, Manan S and Liu, Guan-Horng and Theodorou, Evangelos A},
booktitle={Robotics: Science and Systems},
year={2021},
}

2020

arxiv

A Differential Game Theoretic Neural Optimizer for Training Residual Networks

Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

arXiv preprint, 2020

bibtex abstract arXiv

Connections between Deep Neural Networks (DNNs) training and optimal control theory has attracted considerable attention as a principled tool of algorithmic design. Differential Dynamic Programming (DDP) neural optimizer is a recently proposed method along this line. Despite its empirical success, the applicability has been limited to feedforward networks and whether such a trajectory-optimization inspired framework can be extended to modern architectures remains unclear. In this work, we derive a generalized DDP optimizer that accepts both residual connections and convolution layers. The resulting optimal control representation admits a game theoretic perspective, in which training residual networks can be interpreted as cooperative trajectory optimization on state-augmented dynamical systems. This Game Theoretic DDP (GT-DDP) optimizer enjoys the same theoretic connection in previous work, yet generates a much complex update rule that better leverages available information during network propagation. Evaluation on image classification datasets (e.g. MNIST and CIFAR100) shows an improvement in training convergence and variance reduction over existing methods. Our approach highlights the benefit gained from architecture-aware optimization.

@article{liu2020differential,
title={A Differential Game Theoretic Neural Optimizer for Training Residual Networks},
author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
journal={arXiv preprint arXiv:2007.08880},
year={2020},
}

2019

arxiv

Deep learning theory review: An optimal control and dynamical systems perspective

Guan-Horng Liu, and Evangelos A Theodorou

arXiv preprint (in submission), 2019

bibtex abstract arXiv Code

Attempts from different disciplines to provide a fundamental understanding of deep learning have advanced rapidly in recent years, yet a unified framework remains relatively limited. In this article, we provide one possible way to align existing branches of deep learning theory through the lens of dynamical system and optimal control. By viewing deep neural networks as discrete-time nonlinear dynamical systems, we can analyze how information propagates through layers using mean field theory. When optimization algorithms are further recast as controllers, the ultimate goal of training processes can be formulated as an optimal control problem. In addition, we can reveal convergence and generalization properties by studying the stochastic dynamics of optimization algorithms. This viewpoint features a wide range of theoretical study from information bottleneck to statistical physics. It also provides a principled way for hyper-parameter tuning when optimal control theory is introduced. Our framework fits nicely with supervised learning and can be extended to other learning problems, such as Bayesian learning, adversarial training, and specific forms of meta learning, without efforts. The review aims to shed lights on the importance of dynamics and optimal control when developing deep learning theory.

@article{liu2019deep,
title={Deep learning theory review: An optimal control and dynamical systems perspective},
author={Liu, Guan-Horng and Theodorou, Evangelos A},
journal={arXiv preprint arXiv:1908.10920},
year={2019},
}

2017

CoRL 2017

Learning end-to-end multimodal sensor policies for autonomous navigation

Guan-Horng Liu, Avinash Siravuru, Sai Prabhakar, Manuela Veloso, and George Kantor

Conference on Robot Learning, 2017

bibtex abstract arXiv PDF Video

Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called Sensor Dropout, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion. Simulation results of the multisensory policy, as visualized in TORCS racing game, can be seen here: this https URL.

@inproceedings{liu2017learning,
title={Learning end-to-end multimodal sensor policies for autonomous navigation},
author={Liu, Guan-Horng and Siravuru, Avinash and Prabhakar, Sai and Veloso, Manuela and Kantor, George},
booktitle={Conference on Robot Learning},
year={2017},
}
CMU Thesis

High Dimensional Planning and Learning for Off-Road Driving

Guan-Horng Liu

CMU Robotics Institute Master Thesis, 2017

bibtex PDF

@article{liu2017high,
title={High Dimensional Planning and Learning for Off-Road Driving},
author={Liu, Guan-Horng},
journal={CMU Robotics Institute Master Thesis},
year={2017},
}
RLDM 2017

Multi-modal Deep Reinforcement Learning with a Novel Sensor-based Dropout

Guan-Horng Liu, Avinash Siravuru, Sai Prabhakar, George Kantor, and Manuela Veloso

Multi-disciplinary Conference on Reinforcement Learning and Decision Making, 2017

bibtex PDF

@article{liu2017higi,
title={Multi-modal Deep Reinforcement Learning with a Novel Sensor-based Dropout},
author={Liu, Guan-Horng and Siravuru, Avinash and Prabhakar, Sai and Kantor, George and Veloso, Manuela},
journal={Multi-disciplinary Conference on Reinforcement Learning and Decision Making},
year={2017},
}

2014

JBE 2014

A bio-inspired hopping kangaroo robot with an active tail

Guan-Horng Liu, Hou-Yi Lin, Huai-Yu Lin, Shao-Tuan Chen, and Pei-Chun Lin

Journal of Bionic Engineering, 2014

bibtex PDF

@article{liu2014bio,
title={A bio-inspired hopping kangaroo robot with an active tail},
author={Liu, Guan-Horng and Lin, Hou-Yi and Lin, Huai-Yu and Chen, Shao-Tuan and Lin, Pei-Chun},
journal={Journal of Bionic Engineering},
year={2014},
}
RSJ 2014

Autonomous Control of the WAM-V Catamaran Type Unmanned Surface Vehicle: Propulsion System Design

Guan-Horng Liu, Andre Yuji YASUTOMI, Alexis HOLGADO, and Edwardo F FUKUSHIMA

Annual Conference of the Robotics Society of Japan, 2014

bibtex PDF

@article{liu2014auto,
title={Autonomous Control of the WAM-V Catamaran Type Unmanned Surface Vehicle: Propulsion System Design},
author={Liu, Guan-Horng and YASUTOMI, Andre Yuji and HOLGADO, Alexis and FUKUSHIMA, Edwardo F},
journal={Annual Conference of the Robotics Society of Japan},
year={2014},
}

2013

SII 2013

Design of a kangaroo robot with dynamic jogging locomotion

Guan-Horng Liu, Hou-Yi Lin, Huai-Yu Lin, Shao-Tuan Chen, and Pei-Chun Lin

Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, 2013

Best Paper Award

bibtex PDF

@inproceedings{liu2013design,
title={Design of a kangaroo robot with dynamic jogging locomotion},
author={Liu, Guan-Horng and Lin, Hou-Yi and Lin, Huai-Yu and Chen, Shao-Tuan and Lin, Pei-Chun},
booktitle={Proceedings of the 2013 IEEE/SICE International Symposium on System Integration},
year={2013},
}