Neural Rough Differential Equations for Long Time Series

James Morrill, Cristopher Salvi, Patrick Kidger, James Foster

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

24 Citations (SciVal)

Abstract

Neural controlled differential equations (CDEs) are the continuous-time analogue of recurrent neural networks, as Neural ODEs are to residual networks, and offer a memory-efficient continuous-time way to model functions of potentially irregular time series. Existing methods for computing the forward pass of a Neural CDE involve embedding the incoming time series into path space, often via interpolation, and using evaluations of this path to drive the hidden state. Here, we use rough path theory to extend this formulation. Instead of directly embedding into path space, we instead represent the input signal over small time intervals through its \textit{log-signature}, which are statistics describing how the signal drives a CDE. This is the approach for solving \textit{rough differential equations} (RDEs), and correspondingly we describe our main contribution as the introduction of Neural RDEs. This extension has a purpose: by generalising the Neural CDE approach to a broader class of driving signals, we demonstrate particular advantages for tackling long time series. In this regime, we demonstrate efficacy on problems of length up to 17k observations and observe significant training speed-ups, improvements in model performance, and reduced memory requirements compared to existing approaches.
Original languageEnglish
Title of host publicationProceedings of the 38th International Conference on Machine Learning
PublisherPMLR
Pages7829-7838
Number of pages10
Volume38
Publication statusPublished - 24 Jul 2021
Externally publishedYes
EventThirty-eighth International Conference on Machine Learning - Virutal only
Duration: 18 Jul 202124 Jul 2021
Conference number: 38
https://icml.cc/Conferences/2021

Publication series

NameProceedings of Machine Learning Research
PublisherPMLR
ISSN (Electronic)2640-3498

Conference

ConferenceThirty-eighth International Conference on Machine Learning
Abbreviated titleICML
Period18/07/2124/07/21
Internet address

Fingerprint

Dive into the research topics of 'Neural Rough Differential Equations for Long Time Series'. Together they form a unique fingerprint.

Cite this