Guan-Horng Liu

Publications

1. Game Theoretic Neural ODE Optimizer

   Panagiotis Theodoropoulos, Guan-Horng Liu*, Tiarnong Chen*, and Evangelos A Theodorou

   ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023

   arxiv

   @inproceedings{theodoropoulos2023game,
     title={Game Theoretic Neural ODE Optimizer},
     author={Theodoropoulos, Panagiotis and Liu*, Guan-Horng and Chen*, Tiarnong and Theodorou, Evangelos A},
     booktitle={International Conference on Machine Learning},
     year={2023},
   }

2. Improved Sampling via Learned Diffusions

   Lorenz Richter*, Julius Berner*, and Guan-Horng Liu

   ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023

   arxiv

   Recently, a series of papers proposed deep learning-based approaches to sample from unnormalized target densities using controlled diffusion processes. In this work, we identify these approaches as special cases of the Schrödinger bridge problem, seeking the most likely stochastic evolution between a given prior distribution and the specified target. We further generalize this framework by introducing a variational formulation based on divergences between path space measures of time-reversed diffusion processes. This abstract perspective leads to practical losses that can be optimized by gradient-based algorithms and includes previous objectives as special cases. At the same time, it allows us to consider divergences other than the reverse Kullback-Leibler divergence that is known to suffer from mode collapse. In particular, we propose the so-called logvariance loss, which exhibits favorable numerical properties and leads to significantly improved performance across all considered approaches.

   @inproceedings{richter2023improved,
     title={Improved Sampling via Learned Diffusions},
     author={Richter*, Lorenz and Berner*, Julius and Liu, Guan-Horng},
     booktitle={International Conference on Machine Learning},
     year={2023},
   }

3. I²SB: Image-to-Image Schrödinger Bridge

   Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A Theodorou, Weili Nie^†, and Anima Anandkumar^†

   International Conference on Machine Learning (ICML), 2023

   arxiv | code | project

   We propose Image-to-Image Schrödinger Bridge (I²SB), a new class of conditional diffusion models that directly learn the nonlinear diffusion processes between two given distributions. These diffusion bridges are particularly useful for image restoration, as the degraded images are structurally informative priors for reconstructing the clean images. I²SB belongs to a tractable class of Schrödinger bridge, the nonlinear extension to score-based models, whose marginal distributions can be computed analytically given boundary pairs. This results in a simulation-free framework for nonlinear diffusions, where the I²SB training becomes scalable by adopting practical techniques used in standard diffusion models. We validate I²SB in solving various image restoration tasks, including inpainting, super-resolution, deblurring, and JPEG restoration on ImageNet 256x256 and show that I²SB surpasses standard conditional diffusion models with more interpretable generative processes. Moreover, I²SB matches the performance of inverse methods that additionally require the knowledge of the corruption operators. Our work opens up new algorithmic opportunities for developing efficient nonlinear diffusion models on a large scale.

   @inproceedings{liu2023i2sb,
     title={I{$^2$}SB: Image-to-Image Schr{\"o}dinger Bridge},
     author={Liu, Guan-Horng and Vahdat, Arash and Huang, De-An and Theodorou, Evangelos A and Nie, Weili and Anandkumar, Anima},
     booktitle={International Conference on Machine Learning},
     year={2023},
   }

4. Deep Momentum Multi-Marginal Schrödinger Bridge

   Tianrong Chen, Guan-Horng Liu, Molei Tao, and Evangelos A Theodorou

   preprint (in submission), 2023

   arxiv

   Reconstructing population dynamics using only samples from distributions at coarse time intervals is a crucial challenge. Recent datadriven approaches such as flow-based models or Schrödinger Bridge models have demonstrated appealing performance, yet the inferred sample trajectories either fail to account for the underlying stochasticity or are unnecessarily rigid. In this article, we propose Deep Momentum Multi-Marginal Schrödinger Bridge (DMSB), a novel computational framework that learns the smooth measurevalued spline for stochastic systems without violating the position marginal constraints across time. We first extend the scalable mean matching objective used in the state space SB algorithm into the phase space. We next carefully craft a multi-constraint optimization training method based on Bregman Iteration that enables effective phase space means matching training for the high-dimensional dataset. We demonstrate that the resulting training algorithm significantly outperforms baselines on both synthetic datasets and a real-world single-cell RNA sequence dataset.

   @article{chen2023dmsb,
     title={Deep Momentum Multi-Marginal Schr{\"o}dinger Bridge},
     author={Chen, Tianrong and Liu, Guan-Horng and Tao, Molei and Theodorou, Evangelos A},
     journal={arXiv preprint arXiv:2303.01751},
     year={2023},
   }

1. Deep Generalized Schrödinger Bridge

   Guan-Horng Liu, Tianrong Chen*, Oswin So*, and Evangelos A Theodorou

   Advances in Neural Information Processing Systems (NeurIPS), 2022

   Oral presentation (acceptance rate 1.9%)

   arxiv | code

   Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling the collective behavior of individual agents interacting stochastically with a large population. In this work, we aim at solving a challenging class of MFGs in which the differentiability of these interacting preferences may not available to the solver, and the population is urged to converge exactly to some desired distribution. These setups are, despite being well-motivated for practical purposes, complicated enough to paralyze most (deep) numerical solvers. Nevertheless, we show that Schrödinger Bridge — as an entropy-regularized optimal transport model — can be generalized to accepting mean-field structures, hence solving these MFGs. This is achieved via the application of Forward-Backward Stochastic Differential Equations theory, which, intriguingly, leads to a computational framework with a similar structure to Temporal Difference learning. As such, it opens up novel algorithmic connections to Deep Reinforcement Learning that we leverage to facilitate practical training. We show that our proposed objective function provides necessary and sufficient conditions to the mean-field problem. Our method, named Deep Generalized Schrödinger Bridge (DeepGSB), not only outperforms prior methods in solving classical population navigation MFGs, but is also capable of solving 1000-dimensional opinion depolarization, setting a new state-of-the-art numerical solver for high-dimensional MFGs. Our code will be made available at https://github.com/ghliu/DeepGSB.

   @inproceedings{liu2022deepgsb,
     title={Deep Generalized Schr{\"o}dinger Bridge},
     author={Liu, Guan-Horng and Chen, Tianrong and So, Oswin and Theodorou, Evangelos A},
     booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
     year={2022},
   }

2. Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory

   Tianrong Chen*, Guan-Horng Liu*, and Evangelos A Theodorou

   International Conference on Learning Representations (ICLR), 2022

   arxiv | code

   Schrödinger Bridge (SB) is an entropy-regularized optimal transport problem that has received increasing attention in deep generative modeling for its mathematical flexibility compared to the Scored-based Generative Model (SGM). However, it remains unclear whether the optimization principle of SB relates to the modern training of deep generative models, which often rely on constructing log-likelihood objectives.This raises questions on the suitability of SB models as a principled alternative for generative applications. In this work, we present a novel computational framework for likelihood training of SB models grounded on Forward-Backward Stochastic Differential Equations Theory - a mathematical methodology appeared in stochastic optimal control that transforms the optimality condition of SB into a set of SDEs. Crucially, these SDEs can be used to construct the likelihood objectives for SB that, surprisingly, generalizes the ones for SGM as special cases. This leads to a new optimization principle that inherits the same SB optimality yet without losing applications of modern generative training techniques, and we show that the resulting training algorithm achieves comparable results on generating realistic images on MNIST, CelebA, and CIFAR10. Our code is available at https://github.com/ghliu/SB-FBSDE.

   @inproceedings{chen2021likelihood,
     title={Likelihood Training of Schr{\"o}dinger Bridge using Forward-Backward SDEs Theory},
     author={Chen, Tianrong and Liu, Guan-Horng and Theodorou, Evangelos A},
     booktitle={International Conference on Learning Representations},
     year={2022},
   }

1. Second-order Neural ODE Optimizer

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   Advances in Neural Information Processing Systems (NeurIPS), 2021

   Spotlight presentation (acceptance rate 3.0%)

   arxiv | code | talk (DataSig)

   We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient computation by solving a backward ODE, deriving efficient second-order methods becomes highly nontrivial. Nevertheless, inspired by the recent Optimal Control (OC) interpretation of training deep networks, we show that a specific continuous-time OC methodology, called Differential Programming, can be adopted to derive backward ODEs for higher-order derivatives at the same O(1) memory cost. We further explore a low-rank representation of the second-order derivatives and show that it leads to efficient preconditioned updates with the aid of Kronecker-based factorization. The resulting method converges much faster than first-order baselines in wall-clock time, and the improvement remains consistent across various applications, e.g. image classification, generative flow, and time-series prediction. Our framework also enables direct architecture optimization, such as the integration time of Neural ODEs, with second-order feedback policies, strengthening the OC perspective as a principled tool of analyzing optimization in deep learning.

   @inproceedings{liu2021second,
     title={Second-order Neural ODE Optimizer},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     booktitle={Advances in Neural Information Processing Systems},
     year={2021},
   }

2. Dynamic Game Theoretic Neural Optimizer

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   International Conference on Machine Learning (ICML), 2021

   Oral presentation (acceptance rate 3.0%)

   arxiv

   The connection between training deep neural networks (DNNs) and optimal control theory (OCT) has attracted considerable attention as a principled tool of algorithmic design. Despite few attempts being made, they have been limited to architectures where the layer propagation resembles a Markovian dynamical system. This casts doubts on their flexibility to modern networks that heavily rely on non-Markovian dependencies between layers (e.g. skip connections in residual networks). In this work, we propose a novel dynamic game perspective by viewing each layer as a player in a dynamic game characterized by the DNN itself. Through this lens, different classes of optimizers can be seen as matching different types of Nash equilibria, depending on the implicit information structure of each (p)layer. The resulting method, called Dynamic Game Theoretic Neural Optimizer (DGNOpt), not only generalizes OCT-inspired optimizers to richer network class; it also motivates a new training principle by solving a multi-player cooperative game. DGNOpt shows convergence improvements over existing methods on image classification datasets with residual networks. Our work marries strengths from both OCT and game theory, paving ways to new algorithmic opportunities from robust optimal control and bandit-based optimization.

   @inproceedings{liu2021dynamic,
     title={Dynamic Game Theoretic Neural Optimizer},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     booktitle={International Conference on Machine Learning},
     year={2021},
   }

3. DDPNOpt: Differential Dynamic Programming Neural Optimizer

   Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou

   International Conference on Learning Representations (ICLR), 2021

   Spotlight presentation (acceptance rate 3.8%)

   arxiv

   Interpretation of Deep Neural Networks (DNNs) training as an optimal control problem with nonlinear dynamical systems has received considerable attention recently, yet the algorithmic development remains relatively limited. In this work, we make an attempt along this line by reformulating the training procedure from the trajectory optimization perspective. We first show that most widely-used algorithms for training DNNs can be linked to the Differential Dynamic Programming (DDP), a celebrated second-order method rooted in the Approximate Dynamic Programming. In this vein, we propose a new class of optimizer, DDP Neural Optimizer (DDPNOpt), for training feedforward and convolution networks. DDPNOpt features layer-wise feedback policies which improve convergence and reduce sensitivity to hyper-parameter over existing methods. It outperforms other optimal-control inspired training methods in both convergence and complexity, and is competitive against state-of-the-art first and second order methods. We also observe DDPNOpt has surprising benefit in preventing gradient vanishing. Our work opens up new avenues for principled algorithmic design built upon the optimal control theory.

   @inproceedings{liu2021differential,
     title={DDPNOpt: Differential Dynamic Programming Neural Optimizer},
     author={Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     booktitle={International Conference on Learning Representations},
     year={2021},
   }

4. Spatio-Temporal Differential Dynamic Programming for Control of Fields

   Ethan N Evans, Oswin So, Andrew P Kendall, Guan-Horng Liu, and Evangelos A Theodorou

   preprint (in submission), 2021

   arxiv

   We consider the optimal control problem of a general nonlinear spatio-temporal system described by Partial Differential Equations (PDEs). Theory and algorithms for control of spatio-temporal systems are of rising interest among the automatic control community and exhibit numerous challenging characteristic from a control standpoint. Recent methods focus on finite-dimensional optimization techniques of a discretized finite dimensional ODE approximation of the infinite dimensional PDE system. In this paper, we derive a differential dynamic programming (DDP) framework for distributed and boundary control of spatio-temporal systems in infinite dimensions that is shown to generalize both the spatio-temporal LQR solution, and modern finite dimensional DDP frameworks. We analyze the convergence behavior and provide a proof of global convergence for the resulting system of continuous-time forward-backward equations. We explore and develop numerical approaches to handle sensitivities that arise during implementation, and apply the resulting STDDP algorithm to a linear and nonlinear spatio-temporal PDE system. Our framework is derived in infinite dimensional Hilbert spaces, and represents a discretization-agnostic framework for control of nonlinear spatio-temporal PDE systems.

   @article{evans2021spatio,
     title={Spatio-Temporal Differential Dynamic Programming for Control of Fields},
     author={Evans, Ethan N and So, Oswin and Kendall, Andrew P and Liu, Guan-Horng and Theodorou, Evangelos A},
     journal={arXiv preprint arXiv:2104.04044},
     year={2021},
   }

5. Variational Inference MPC using Tsallis Divergence

   Ziyi Wang, Oswin So, Jason Gibson, Bogdan Vlahov, Manan S Gandhi, Guan-Horng Liu, and Evangelos A Theodorou

   Robotics: Science and Systems (RSS), 2021

   arxiv

   In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using thenon-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive PathIntegral Control, Cross Entropy Method, and Stein VariationalInference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

   @inproceedings{wang2021variational,
     title={Variational Inference MPC using Tsallis Divergence},
     author={Wang, Ziyi and So, Oswin and Gibson, Jason and Vlahov, Bogdan and Gandhi, Manan S and Liu, Guan-Horng and Theodorou, Evangelos A},
     booktitle={Robotics: Science and Systems (RSS)},
     year={2021},
   }

1. Deep learning theory review: An optimal control and dynamical systems perspective

   Guan-Horng Liu, and Evangelos A Theodorou

   preprint (in submission), 2019

   arxiv | code

   Attempts from different disciplines to provide a fundamental understanding of deep learning have advanced rapidly in recent years, yet a unified framework remains relatively limited. In this article, we provide one possible way to align existing branches of deep learning theory through the lens of dynamical system and optimal control. By viewing deep neural networks as discrete-time nonlinear dynamical systems, we can analyze how information propagates through layers using mean field theory. When optimization algorithms are further recast as controllers, the ultimate goal of training processes can be formulated as an optimal control problem. In addition, we can reveal convergence and generalization properties by studying the stochastic dynamics of optimization algorithms. This viewpoint features a wide range of theoretical study from information bottleneck to statistical physics. It also provides a principled way for hyper-parameter tuning when optimal control theory is introduced. Our framework fits nicely with supervised learning and can be extended to other learning problems, such as Bayesian learning, adversarial training, and specific forms of meta learning, without efforts. The review aims to shed lights on the importance of dynamics and optimal control when developing deep learning theory.

   @article{liu2019deep,
     title={Deep learning theory review: An optimal control and dynamical systems perspective},
     author={Liu, Guan-Horng and Theodorou, Evangelos A},
     journal={arXiv preprint arXiv:1908.10920},
     year={2019},
   }

1. Learning end-to-end multimodal sensor policies for autonomous navigation

   Guan-Horng Liu, Avinash Siravuru, Sai Prabhakar, Manuela Veloso, and George Kantor

   Conference on Robot Learning (CoRL), 2017

   arxiv

   Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called Sensor Dropout, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion. Simulation results of the multisensory policy, as visualized in TORCS racing game, can be seen here: this https URL.

   @inproceedings{liu2017learning,
     title={Learning end-to-end multimodal sensor policies for autonomous navigation},
     author={Liu, Guan-Horng and Siravuru, Avinash and Prabhakar, Sai and Veloso, Manuela and Kantor, George},
     booktitle={Conference on Robot Learning (CoRL)},
     year={2017},
   }

2. High Dimensional Planning and Learning for Off-Road Driving

   Guan-Horng Liu

   CMU Robotics Institute Master Thesis, 2017

   @article{liu2017high,
     title={High Dimensional Planning and Learning for Off-Road Driving},
     author={Liu, Guan-Horng},
     journal={CMU Robotics Institute Master Thesis},
     year={2017},
   }

3. Multi-modal Deep Reinforcement Learning with a Novel Sensor-based Dropout

   Guan-Horng Liu, Avinash Siravuru, Sai Prabhakar, George Kantor, and Manuela Veloso

   Multi-disciplinary Conference on Reinforcement Learning and Decision Making (MRLD), 2017

   @article{liu2017higi,
     title={Multi-modal Deep Reinforcement Learning with a Novel Sensor-based Dropout},
     author={Liu, Guan-Horng and Siravuru, Avinash and Prabhakar, Sai and Kantor, George and Veloso, Manuela},
     journal={Multi-disciplinary Conference on Reinforcement Learning and Decision Making (MRLD)},
     year={2017},
   }

1. A bio-inspired hopping kangaroo robot with an active tail

   Guan-Horng Liu, Hou-Yi Lin, Huai-Yu Lin, Shao-Tuan Chen, and Pei-Chun Lin

   Journal of Bionic Engineering (JBE), 2014

   @article{liu2014bio,
     title={A bio-inspired hopping kangaroo robot with an active tail},
     author={Liu, Guan-Horng and Lin, Hou-Yi and Lin, Huai-Yu and Chen, Shao-Tuan and Lin, Pei-Chun},
     journal={Journal of Bionic Engineering (JBE)},
     year={2014},
   }

2. Autonomous Control of the WAM-V Catamaran Type Unmanned Surface Vehicle: Propulsion System Design

   Guan-Horng Liu, Andre Yuji YASUTOMI, Alexis HOLGADO, and Edwardo F FUKUSHIMA

   Annual Conference of the Robotics Society of Japan, 2014

   @article{liu2014auto,
     title={Autonomous Control of the WAM-V Catamaran Type Unmanned Surface Vehicle: Propulsion System Design},
     author={Liu, Guan-Horng and YASUTOMI, Andre Yuji and HOLGADO, Alexis and FUKUSHIMA, Edwardo F},
     journal={Annual Conference of the Robotics Society of Japan},
     year={2014},
   }

1. Design of a kangaroo robot with dynamic jogging locomotion

   Guan-Horng Liu, Hou-Yi Lin, Huai-Yu Lin, Shao-Tuan Chen, and Pei-Chun Lin

   Proceedings of the 2013 IEEE/SICE International Symposium on System Integration (SII), 2013

   Best Paper Award

   @inproceedings{liu2013design,
     title={Design of a kangaroo robot with dynamic jogging locomotion},
     author={Liu, Guan-Horng and Lin, Hou-Yi and Lin, Huai-Yu and Chen, Shao-Tuan and Lin, Pei-Chun},
     booktitle={Proceedings of the 2013 IEEE/SICE International Symposium on System Integration (SII)},
     year={2013},
   }