Extract Dynamic Information To Improve Time Series Modeling: a Case Study with Scientific Workflow
In modeling time series data, we often need to augment the existing data
records to increase the modeling accuracy. In this work, we describe a number
of techniques to extract dynamic information about the current state of a large
scientific workflow, which could be generalized to other types of applications.
The specific task to be modeled is the time needed for transferring a file from
an experimental facility to a data center. The key idea of our approach is to
find recent past data transfer events that match the current event in some
ways. Tests showed that we could identify recent events matching some recorded
properties and reduce the prediction error by about 12% compared to the similar
models with only static features. We additionally explored an application
specific technique to extract information about the data production process,
and was able to reduce the average prediction error by 44%.