Deep learning as optimal control problems

Martin Benning; Elena Celledoni; Matthias J. Ehrhardt; Brynjulf Owren; Carola Bibiane Schonlieb

doi:10.1016/j.ifacol.2021.06.124

Deep learning as optimal control problems

Martin Benning, Elena Celledoni, Matthias J. Ehrhardt, Brynjulf Owren, Carola Bibiane Schonlieb

Research output: Contribution to journal › Conference article › peer-review

2 Citations (SciVal)

Abstract

We briefly review recent work where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We report here new preliminary experiments with implicit symplectic Runge-Kutta methods. In this paper, we discuss ongoing and future research in this area.

Original language	English
Pages (from-to)	620-623
Number of pages	4
Journal	IFAC-PapersOnLine
Volume	54
Issue number	9
Early online date	16 Jul 2021
DOIs	https://doi.org/10.1016/j.ifacol.2021.06.124
Publication status	Published - 31 Dec 2021
Event	24th International Symposium on Mathematical Theory of Networks and Systems, MTNS 2020 - Cambridge, UK United Kingdom Duration: 23 Aug 2021 → 27 Aug 2021

Keywords

Deep neural networks
Optimal control
Resnet

ASJC Scopus subject areas

Control and Systems Engineering

Access to Document

10.1016/j.ifacol.2021.06.124

Cite this

@article{7df20e567a4840629f99bc7810a6e756,

title = "Deep learning as optimal control problems",

abstract = "We briefly review recent work where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We report here new preliminary experiments with implicit symplectic Runge-Kutta methods. In this paper, we discuss ongoing and future research in this area. ",

keywords = "Deep neural networks, Optimal control, Resnet",

author = "Martin Benning and Elena Celledoni and Ehrhardt, {Matthias J.} and Brynjulf Owren and Schonlieb, {Carola Bibiane}",

year = "2021",

month = dec,

day = "31",

doi = "10.1016/j.ifacol.2021.06.124",

language = "English",

volume = "54",

pages = "620--623",

journal = "IFAC-PapersOnLine",

issn = "2405-8963",

publisher = "Elsevier",

number = "9",

note = "24th International Symposium on Mathematical Theory of Networks and Systems, MTNS 2020 ; Conference date: 23-08-2021 Through 27-08-2021",

}

TY - JOUR

T1 - Deep learning as optimal control problems

AU - Benning, Martin

AU - Celledoni, Elena

AU - Ehrhardt, Matthias J.

AU - Owren, Brynjulf

AU - Schonlieb, Carola Bibiane

PY - 2021/12/31

Y1 - 2021/12/31

N2 - We briefly review recent work where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We report here new preliminary experiments with implicit symplectic Runge-Kutta methods. In this paper, we discuss ongoing and future research in this area.

AB - We briefly review recent work where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We report here new preliminary experiments with implicit symplectic Runge-Kutta methods. In this paper, we discuss ongoing and future research in this area.

KW - Deep neural networks

KW - Optimal control

KW - Resnet

UR - http://www.scopus.com/inward/record.url?scp=85117942598&partnerID=8YFLogxK

U2 - 10.1016/j.ifacol.2021.06.124

DO - 10.1016/j.ifacol.2021.06.124

M3 - Conference article

AN - SCOPUS:85117942598

SN - 2405-8963

VL - 54

SP - 620

EP - 623

JO - IFAC-PapersOnLine

JF - IFAC-PapersOnLine

IS - 9

T2 - 24th International Symposium on Mathematical Theory of Networks and Systems, MTNS 2020

Y2 - 23 August 2021 through 27 August 2021

ER -

Deep learning as optimal control problems

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this