E. Hazan, and K. Singh. (2022)cite arxiv:2211.09619Comment: Draft; comments/suggestions welcome at nonstochastic.control@gmail.com.
Abstract
This text presents an introduction to an emerging paradigm in control of
dynamical systems and differentiable reinforcement learning called online
nonstochastic control. The new approach applies techniques from online convex
optimization and convex relaxations to obtain new methods with provable
guarantees for classical settings in optimal and robust control.
The primary distinction between online nonstochastic control and other
frameworks is the objective. In optimal control, robust control, and other
control methodologies that assume stochastic noise, the goal is to perform
comparably to an offline optimal strategy. In online nonstochastic control,
both the cost functions as well as the perturbations from the assumed dynamical
model are chosen by an adversary. Thus the optimal policy is not defined a
priori. Rather, the target is to attain low regret against the best policy in
hindsight from a benchmark class of policies.
This objective suggests the use of the decision making framework of online
convex optimization as an algorithmic methodology. The resulting methods are
based on iterative mathematical optimization algorithms, and are accompanied by
finite-time regret and computational complexity guarantees.
%0 Generic
%1 hazan2022introduction
%A Hazan, Elad
%A Singh, Karan
%D 2022
%K MachineLearning Optimization Robotics
%T Introduction to Online Nonstochastic Control
%U http://arxiv.org/abs/2211.09619
%X This text presents an introduction to an emerging paradigm in control of
dynamical systems and differentiable reinforcement learning called online
nonstochastic control. The new approach applies techniques from online convex
optimization and convex relaxations to obtain new methods with provable
guarantees for classical settings in optimal and robust control.
The primary distinction between online nonstochastic control and other
frameworks is the objective. In optimal control, robust control, and other
control methodologies that assume stochastic noise, the goal is to perform
comparably to an offline optimal strategy. In online nonstochastic control,
both the cost functions as well as the perturbations from the assumed dynamical
model are chosen by an adversary. Thus the optimal policy is not defined a
priori. Rather, the target is to attain low regret against the best policy in
hindsight from a benchmark class of policies.
This objective suggests the use of the decision making framework of online
convex optimization as an algorithmic methodology. The resulting methods are
based on iterative mathematical optimization algorithms, and are accompanied by
finite-time regret and computational complexity guarantees.
@misc{hazan2022introduction,
abstract = {This text presents an introduction to an emerging paradigm in control of
dynamical systems and differentiable reinforcement learning called online
nonstochastic control. The new approach applies techniques from online convex
optimization and convex relaxations to obtain new methods with provable
guarantees for classical settings in optimal and robust control.
The primary distinction between online nonstochastic control and other
frameworks is the objective. In optimal control, robust control, and other
control methodologies that assume stochastic noise, the goal is to perform
comparably to an offline optimal strategy. In online nonstochastic control,
both the cost functions as well as the perturbations from the assumed dynamical
model are chosen by an adversary. Thus the optimal policy is not defined a
priori. Rather, the target is to attain low regret against the best policy in
hindsight from a benchmark class of policies.
This objective suggests the use of the decision making framework of online
convex optimization as an algorithmic methodology. The resulting methods are
based on iterative mathematical optimization algorithms, and are accompanied by
finite-time regret and computational complexity guarantees.},
added-at = {2022-11-19T05:48:25.000+0100},
author = {Hazan, Elad and Singh, Karan},
biburl = {https://www.bibsonomy.org/bibtex/22977c09c6a9651745359c2b3e429348b/sairahul},
description = {Introduction to Online Nonstochastic Control},
interhash = {d30cee67ed9a907613b8f9e88c783ae3},
intrahash = {2977c09c6a9651745359c2b3e429348b},
keywords = {MachineLearning Optimization Robotics},
note = {cite arxiv:2211.09619Comment: Draft; comments/suggestions welcome at nonstochastic.control@gmail.com},
timestamp = {2022-11-19T05:48:25.000+0100},
title = {Introduction to Online Nonstochastic Control},
url = {http://arxiv.org/abs/2211.09619},
year = 2022
}