Guan-Horng Liu

Publications

2022

1. ICLR 2022

   Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory

   Tianrong Chen*, Guan-Horng Liu*, and Evangelos A Theodorou

   International Conference on Learning Representations (* Equal contribution), 2022

   bibtex abstract arXiv PDF Code

   Schrödinger Bridge (SB) is an entropy-regularized optimal transport problem that has received increasing attention in deep generative modeling for its mathematical flexibility compared to the Scored-based Generative Model (SGM). However, it remains unclear whether the optimization principle of SB relates to the modern training of deep generative models, which often rely on constructing log-likelihood objectives.This raises questions on the suitability of SB models as a principled alternative for generative applications. In this work, we present a novel computational framework for likelihood training of SB models grounded on Forward-Backward Stochastic Differential Equations Theory - a mathematical methodology appeared in stochastic optimal control that transforms the optimality condition of SB into a set of SDEs. Crucially, these SDEs can be used to construct the likelihood objectives for SB that, surprisingly, generalizes the ones for SGM as special cases. This leads to a new optimization principle that inherits the same SB optimality yet without losing applications of modern generative training techniques, and we show that the resulting training algorithm achieves comparable results on generating realistic images on MNIST, CelebA, and CIFAR10. Our code is available at https://github.com/ghliu/SB-FBSDE.

   @inproceedings{chen2021likelihood,
     title={Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory},
     author={Chen, Tianrong and Liu, Guan-Horng and Theodorou, Evangelos A},
     booktitle={International Conference on Learning Representations},
     year={2022},
   }

2021

1. NeurIPS 2021

   Second-order Neural ODE Optimizer

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   Advances in Neural Information Processing Systems, 2021

   Spotlight presentation (acceptance rate 3.0%)

   bibtex abstract arXiv PDF Code Poster Slides Video Talk @ DataSig

   We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient computation by solving a backward ODE, deriving efficient second-order methods becomes highly nontrivial. Nevertheless, inspired by the recent Optimal Control (OC) interpretation of training deep networks, we show that a specific continuous-time OC methodology, called Differential Programming, can be adopted to derive backward ODEs for higher-order derivatives at the same O(1) memory cost. We further explore a low-rank representation of the second-order derivatives and show that it leads to efficient preconditioned updates with the aid of Kronecker-based factorization. The resulting method converges much faster than first-order baselines in wall-clock time, and the improvement remains consistent across various applications, e.g. image classification, generative flow, and time-series prediction. Our framework also enables direct architecture optimization, such as the integration time of Neural ODEs, with second-order feedback policies, strengthening the OC perspective as a principled tool of analyzing optimization in deep learning.

   @inproceedings{liu2021second,
     title={Second-order Neural ODE Optimizer},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     booktitle={Advances in Neural Information Processing Systems},
     year={2021},
   }

2. ICML 2021

   Dynamic Game Theoretic Neural Optimizer

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   International Conference on Machine Learning, 2021

   Oral presentation (acceptance rate 3.0%)

   bibtex abstract arXiv PDF Poster Slides Video

   The connection between training deep neural networks (DNNs) and optimal control theory (OCT) has attracted considerable attention as a principled tool of algorithmic design. Despite few attempts being made, they have been limited to architectures where the layer propagation resembles a Markovian dynamical system. This casts doubts on their flexibility to modern networks that heavily rely on non-Markovian dependencies between layers (e.g. skip connections in residual networks). In this work, we propose a novel dynamic game perspective by viewing each layer as a player in a dynamic game characterized by the DNN itself. Through this lens, different classes of optimizers can be seen as matching different types of Nash equilibria, depending on the implicit information structure of each (p)layer. The resulting method, called Dynamic Game Theoretic Neural Optimizer (DGNOpt), not only generalizes OCT-inspired optimizers to richer network class; it also motivates a new training principle by solving a multi-player cooperative game. DGNOpt shows convergence improvements over existing methods on image classification datasets with residual networks. Our work marries strengths from both OCT and game theory, paving ways to new algorithmic opportunities from robust optimal control and bandit-based optimization.

   @inproceedings{liu2021dynamic,
     title={Dynamic Game Theoretic Neural Optimizer},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     booktitle={International Conference on Machine Learning},
     year={2021},
   }

3. ICLR 2021

   DDPNOpt: Differential Dynamic Programming Neural Optimizer

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   International Conference on Learning Representations, 2021

   Spotlight presentation (acceptance rate 3.8%)

   bibtex abstract arXiv PDF Poster Slides Video

   Interpretation of Deep Neural Networks (DNNs) training as an optimal control problem with nonlinear dynamical systems has received considerable attention recently, yet the algorithmic development remains relatively limited. In this work, we make an attempt along this line by reformulating the training procedure from the trajectory optimization perspective. We first show that most widely-used algorithms for training DNNs can be linked to the Differential Dynamic Programming (DDP), a celebrated second-order method rooted in the Approximate Dynamic Programming. In this vein, we propose a new class of optimizer, DDP Neural Optimizer (DDPNOpt), for training feedforward and convolution networks. DDPNOpt features layer-wise feedback policies which improve convergence and reduce sensitivity to hyper-parameter over existing methods. It outperforms other optimal-control inspired training methods in both convergence and complexity, and is competitive against state-of-the-art first and second order methods. We also observe DDPNOpt has surprising benefit in preventing gradient vanishing. Our work opens up new avenues for principled algorithmic design built upon the optimal control theory.

   @inproceedings{liu2021differential,
     title={DDPNOpt: Differential Dynamic Programming Neural Optimizer},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     booktitle={International Conference on Learning Representations},
     year={2021},
   }

4. arxiv

   Spatio-Temporal Differential Dynamic Programming for Control of Fields

   Ethan N Evans, Oswin So, Andrew P Kendall, Guan-Horng Liu, and Evangelos A Theodorou

   arXiv preprint (in submission), 2021

   bibtex abstract arXiv

   We consider the optimal control problem of a general nonlinear spatio-temporal system described by Partial Differential Equations (PDEs). Theory and algorithms for control of spatio-temporal systems are of rising interest among the automatic control community and exhibit numerous challenging characteristic from a control standpoint. Recent methods focus on finite-dimensional optimization techniques of a discretized finite dimensional ODE approximation of the infinite dimensional PDE system. In this paper, we derive a differential dynamic programming (DDP) framework for distributed and boundary control of spatio-temporal systems in infinite dimensions that is shown to generalize both the spatio-temporal LQR solution, and modern finite dimensional DDP frameworks. We analyze the convergence behavior and provide a proof of global convergence for the resulting system of continuous-time forward-backward equations. We explore and develop numerical approaches to handle sensitivities that arise during implementation, and apply the resulting STDDP algorithm to a linear and nonlinear spatio-temporal PDE system. Our framework is derived in infinite dimensional Hilbert spaces, and represents a discretization-agnostic framework for control of nonlinear spatio-temporal PDE systems.

   @article{evans2021spatio,
     title={Spatio-Temporal Differential Dynamic Programming for Control of Fields},
     author={Evans, Ethan N and So, Oswin and Kendall, Andrew P and Liu, Guan-Horng and Theodorou, Evangelos A},
     journal={arXiv preprint arXiv:2104.04044},
     year={2021},
   }

5. RSS 2021

   Variational Inference MPC using Tsallis Divergence

   Ziyi Wang, Oswin So, Jason Gibson, Bogdan Vlahov, Manan S Gandhi, Guan-Horng Liu, and Evangelos A Theodorou

   Robotics: Science and Systems, 2021

   bibtex abstract arXiv

   In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using thenon-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive PathIntegral Control, Cross Entropy Method, and Stein VariationalInference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

   @inproceedings{wang2021variational,
     title={Variational Inference MPC using Tsallis Divergence},
     author={Wang, Ziyi and So, Oswin and Gibson, Jason and Vlahov, Bogdan and Gandhi, Manan S and Liu, Guan-Horng and Theodorou, Evangelos A},
     booktitle={Robotics: Science and Systems},
     year={2021},
   }

2020

1. arxiv

   A Differential Game Theoretic Neural Optimizer for Training Residual Networks

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   arXiv preprint, 2020

   bibtex abstract arXiv

   Connections between Deep Neural Networks (DNNs) training and optimal control theory has attracted considerable attention as a principled tool of algorithmic design. Differential Dynamic Programming (DDP) neural optimizer is a recently proposed method along this line. Despite its empirical success, the applicability has been limited to feedforward networks and whether such a trajectory-optimization inspired framework can be extended to modern architectures remains unclear. In this work, we derive a generalized DDP optimizer that accepts both residual connections and convolution layers. The resulting optimal control representation admits a game theoretic perspective, in which training residual networks can be interpreted as cooperative trajectory optimization on state-augmented dynamical systems. This Game Theoretic DDP (GT-DDP) optimizer enjoys the same theoretic connection in previous work, yet generates a much complex update rule that better leverages available information during network propagation. Evaluation on image classification datasets (e.g. MNIST and CIFAR100) shows an improvement in training convergence and variance reduction over existing methods. Our approach highlights the benefit gained from architecture-aware optimization.

   @article{liu2020differential,
     title={A Differential Game Theoretic Neural Optimizer for Training Residual Networks},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     journal={arXiv preprint arXiv:2007.08880},
     year={2020},
   }

2019

1. arxiv

   Deep learning theory review: An optimal control and dynamical systems perspective

   Guan-Horng Liu, and Evangelos A Theodorou

   arXiv preprint (in submission), 2019

   bibtex abstract arXiv Code

   Attempts from different disciplines to provide a fundamental understanding of deep learning have advanced rapidly in recent years, yet a unified framework remains relatively limited. In this article, we provide one possible way to align existing branches of deep learning theory through the lens of dynamical system and optimal control. By viewing deep neural networks as discrete-time nonlinear dynamical systems, we can analyze how information propagates through layers using mean field theory. When optimization algorithms are further recast as controllers, the ultimate goal of training processes can be formulated as an optimal control problem. In addition, we can reveal convergence and generalization properties by studying the stochastic dynamics of optimization algorithms. This viewpoint features a wide range of theoretical study from information bottleneck to statistical physics. It also provides a principled way for hyper-parameter tuning when optimal control theory is introduced. Our framework fits nicely with supervised learning and can be extended to other learning problems, such as Bayesian learning, adversarial training, and specific forms of meta learning, without efforts. The review aims to shed lights on the importance of dynamics and optimal control when developing deep learning theory.

   @article{liu2019deep,
     title={Deep learning theory review: An optimal control and dynamical systems perspective},
     author={Liu, Guan-Horng and Theodorou, Evangelos A},
     journal={arXiv preprint arXiv:1908.10920},
     year={2019},
   }

2017

1. CoRL 2017

   Learning end-to-end multimodal sensor policies for autonomous navigation

   Guan-Horng Liu, Avinash Siravuru, Sai Prabhakar, Manuela Veloso, and George Kantor

   Conference on Robot Learning, 2017

   bibtex abstract arXiv PDF Video

   Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called Sensor Dropout, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion. Simulation results of the multisensory policy, as visualized in TORCS racing game, can be seen here: this https URL.

   @inproceedings{liu2017learning,
     title={Learning end-to-end multimodal sensor policies for autonomous navigation},
     author={Liu, Guan-Horng and Siravuru, Avinash and Prabhakar, Sai and Veloso, Manuela and Kantor, George},
     booktitle={Conference on Robot Learning},
     year={2017},
   }

2. CMU Thesis

   High Dimensional Planning and Learning for Off-Road Driving

   Guan-Horng Liu

   CMU Robotics Institute Master Thesis, 2017

   bibtex PDF

   @article{liu2017high,
     title={High Dimensional Planning and Learning for Off-Road Driving},
     author={Liu, Guan-Horng},
     journal={CMU Robotics Institute Master Thesis},
     year={2017},
   }

3. RLDM 2017

   Multi-modal Deep Reinforcement Learning with a Novel Sensor-based Dropout

   Guan-Horng Liu, Avinash Siravuru, Sai Prabhakar, George Kantor, and Manuela Veloso

   Multi-disciplinary Conference on Reinforcement Learning and Decision Making, 2017

   bibtex PDF

   @article{liu2017higi,
     title={Multi-modal Deep Reinforcement Learning with a Novel Sensor-based Dropout},
     author={Liu, Guan-Horng and Siravuru, Avinash and Prabhakar, Sai and Kantor, George and Veloso, Manuela},
     journal={Multi-disciplinary Conference on Reinforcement Learning and Decision Making},
     year={2017},
   }

2014

1. JBE 2014

   A bio-inspired hopping kangaroo robot with an active tail

   Guan-Horng Liu, Hou-Yi Lin, Huai-Yu Lin, Shao-Tuan Chen, and Pei-Chun Lin

   Journal of Bionic Engineering, 2014

   bibtex PDF

   @article{liu2014bio,
     title={A bio-inspired hopping kangaroo robot with an active tail},
     author={Liu, Guan-Horng and Lin, Hou-Yi and Lin, Huai-Yu and Chen, Shao-Tuan and Lin, Pei-Chun},
     journal={Journal of Bionic Engineering},
     year={2014},
   }

2. RSJ 2014

   Autonomous Control of the WAM-V Catamaran Type Unmanned Surface Vehicle: Propulsion System Design

   Guan-Horng Liu, Andre Yuji YASUTOMI, Alexis HOLGADO, and Edwardo F FUKUSHIMA

   Annual Conference of the Robotics Society of Japan, 2014

   bibtex PDF

   @article{liu2014auto,
     title={Autonomous Control of the WAM-V Catamaran Type Unmanned Surface Vehicle: Propulsion System Design},
     author={Liu, Guan-Horng and YASUTOMI, Andre Yuji and HOLGADO, Alexis and FUKUSHIMA, Edwardo F},
     journal={Annual Conference of the Robotics Society of Japan},
     year={2014},
   }

2013

1. SII 2013

   Design of a kangaroo robot with dynamic jogging locomotion

   Guan-Horng Liu, Hou-Yi Lin, Huai-Yu Lin, Shao-Tuan Chen, and Pei-Chun Lin

   Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, 2013

   Best Paper Award

   bibtex PDF

   @inproceedings{liu2013design,
     title={Design of a kangaroo robot with dynamic jogging locomotion},
     author={Liu, Guan-Horng and Lin, Hou-Yi and Lin, Huai-Yu and Chen, Shao-Tuan and Lin, Pei-Chun},
     booktitle={Proceedings of the 2013 IEEE/SICE International Symposium on System Integration},
     year={2013},
   }