ICALEPCS2019 - List of Authors (Wozniak, J.P.)

Paper	Title	Page
WEPHA163	NXCALS - Architecture and Challenges of the Next CERN Accelerator Logging Service	1465
	J.P. Wozniak, C. Roderick CERN, Geneva, Switzerland
	CERN’s Accelerator Logging Service (CALS) is in production since 2003 and stores data from accelerator infrastructure and beam observation devices. Initially expecting 1 TB/year, the Oracle based system has scaled to cope with 2.5 TB/day coming from >2.3 million signals. It serves >1000 users making an average of 5 million extraction requests per day. Nevertheless, with a large data increase during LHC Run 2 the CALS system began to show its limits, particularly for supporting data analytics. In 2016 the NXCALS project was launched with the aim of replacing CALS from Run 3 onwards, with a scalable system using "Big Data" technologies. The NXCALS core is production-ready, based on open-source technologies such as Hadoop, HBase, Spark and Kafka. This paper will describe the NXCALS architecture and design choices, together with challenges faced while adopting these technologies. This includes: write/read performance when dealing with vast amounts of data from heterogenous data sources with strict latency requirements; how to extract, transform and load >1 PB of data from CALS to NXCALS. NXCALS is not CERN-specific and can be relevant to other institutes facing similar challenges.
	Poster WEPHA163 [1.689 MB]
DOI •	reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-WEPHA163
About •	paper received ※ 29 September 2019 paper accepted ※ 09 October 2019 issue date ※ 30 August 2020
Export •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

NXCALS - Architecture and Challenges of the Next CERN Accelerator Logging Service

1465

J.P. Wozniak, C. Roderick
CERN, Geneva, Switzerland

CERN’s Accelerator Logging Service (CALS) is in production since 2003 and stores data from accelerator infrastructure and beam observation devices. Initially expecting 1 TB/year, the Oracle based system has scaled to cope with 2.5 TB/day coming from >2.3 million signals. It serves >1000 users making an average of 5 million extraction requests per day. Nevertheless, with a large data increase during LHC Run 2 the CALS system began to show its limits, particularly for supporting data analytics. In 2016 the NXCALS project was launched with the aim of replacing CALS from Run 3 onwards, with a scalable system using "Big Data" technologies. The NXCALS core is production-ready, based on open-source technologies such as Hadoop, HBase, Spark and Kafka. This paper will describe the NXCALS architecture and design choices, together with challenges faced while adopting these technologies. This includes: write/read performance when dealing with vast amounts of data from heterogenous data sources with strict latency requirements; how to extract, transform and load >1 PB of data from CALS to NXCALS. NXCALS is not CERN-specific and can be relevant to other institutes facing similar challenges.

Poster WEPHA163 [1.689 MB]

DOI •

reference for this paper ※ https://doi.org/10.18429/JACoW-ICALEPCS2019-WEPHA163

About •

paper received ※ 29 September 2019 paper accepted ※ 09 October 2019 issue date ※ 30 August 2020

Export •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)