Repository logo
 

Minimal re-computation for exploratory data analysis in astronomy

Accepted version
Peer-reviewed

Type

Article

Change log

Authors

Small, D 
Kettenis, M 

Abstract

We present a technique to automatically minimise the re-computation when a data processing program is iteratively changed, or added to, as is often the case in exploratory data analysis in radio astronomy. A typical example is flagging and calibration of demanding or unusual observations where visual inspection suggests improvement to the processing strategy. The technique is based on memoization and referentially transparent tasks. We describe a prototype implementation for the CASA data reduction package. This technique improves the efficiency of data analysis while reducing the possibility for user error and improving the reproducibility of the final result.

Description

Keywords

Methods, Data analysis, Functional languages

Journal Title

Astronomy and Computing

Conference Name

Journal ISSN

2213-1337
2213-1345

Volume Title

25

Publisher

Elsevier BV
Sponsorship
European Commission (283393)
European Commission Horizon 2020 (H2020) Research Infrastructures (RI) (653477)