Accelerating the Configuration Tuning of Big Data Analytics with Similarity-aware Multitask Bayesian Optimization
| cam.orpheus.counter | 24 | |
| cam.orpheus.success | Mon Apr 26 07:33:16 BST 2021 - Embargo updated | |
| dc.contributor.author | Fekry, A | |
| dc.contributor.author | Carata, L | |
| dc.contributor.author | Pasquier, T | |
| dc.contributor.author | Rice, A | |
| dc.contributor.orcid | Rice, Andrew [0000-0002-4677-8032] | |
| dc.date.accessioned | 2020-11-11T00:30:34Z | |
| dc.date.available | 2020-11-11T00:30:34Z | |
| dc.date.issued | 2020 | |
| dc.description.abstract | One of the key challenges for data analytics deployment is configuration tuning. The existing approaches for configuration tuning are expensive and overlook the dynamic characteristics of the analytics environment (i.e. frequent changes in workload due to receiving evolving input sizes or change in the underlying cluster environment). Such workload/environment changes can cause significant performance degradation, with retuning the configuration to accommodate those changes can yield up to 85\% potential execution time saving. We propose SimTune, an approach that accommodates such changes through efficient configuration tuning. SimTune combines workload characterization and Multitask Bayesian optimization to identify similarity across workloads and accelerate finding near-optimal configurations. Our experimental results show that SimTune reduces the search time for finding close-to-optimal configurations by 56-73\% (at the median) when compared to existing state-of-the-art techniques. This means that the amortization of the tuning cost happens significantly faster, enabling practical tuning in the rapidly changing environment of distributed analytics. | |
| dc.description.sponsorship | Google Cloud, Amazon AWS | |
| dc.identifier.doi | 10.17863/CAM.59851 | |
| dc.identifier.isbn | 9781728162515 | |
| dc.identifier.issn | 2639-1589 | |
| dc.identifier.uri | https://www.repository.cam.ac.uk/handle/1810/312750 | |
| dc.language.iso | eng | |
| dc.publisher | IEEE | |
| dc.publisher.url | http://dx.doi.org/10.1109/bigdata50022.2020.9378085 | |
| dc.rights | All rights reserved | |
| dc.subject | 4606 Distributed Computing and Systems Software | |
| dc.subject | 46 Information and Computing Sciences | |
| dc.title | Accelerating the Configuration Tuning of Big Data Analytics with Similarity-aware Multitask Bayesian Optimization | |
| dc.type | Conference Object | |
| dcterms.dateAccepted | 2020-10-20 | |
| prism.endingPage | 275 | |
| prism.publicationDate | 2020 | |
| prism.publicationName | Proceedings - 2020 IEEE International Conference on Big Data, Big Data 2020 | |
| prism.startingPage | 266 | |
| pubs.conference-finish-date | 2020-12-13 | |
| pubs.conference-name | 2020 IEEE International Conference on Big Data (Big Data) | |
| pubs.conference-start-date | 2020-12-10 | |
| rioxxterms.licenseref.startdate | 2020-12-10 | |
| rioxxterms.licenseref.uri | http://www.rioxx.net/licenses/all-rights-reserved | |
| rioxxterms.type | Conference Paper/Proceeding/Abstract | |
| rioxxterms.version | AM | |
| rioxxterms.versionofrecord | 10.1109/BigData50022.2020.9378085 |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- paper.pdf
- Size:
- 550.25 KB
- Format:
- Adobe Portable Document Format
- Description:
- Accepted version
- Licence
- http://www.rioxx.net/licenses/all-rights-reserved
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- DepositLicenceAgreementv2.1.pdf
- Size:
- 150.9 KB
- Format:
- Adobe Portable Document Format