The Joint Accelerator Conferences Website (JACoW) is an international collaboration that publishes the proceedings of accelerator conferences held around the world.
TY - UNPB AU - Ebner, S.G. AU - Billich, H.R. AU - Brands, H. AU - Panepucci, E.H. AU - Sala, L. ED - Corvetti, Lou ED - Riches, Kathleen ED - Schaa, Volker RW TI - Data Streaming - Efficient Handling of Large and Small (Detector) Data at the Paul Scherrer Institute J2 - Proc. of ICALEPCS2015, Melbourne, Australia, 17-23 October 2015 C1 - Melbourne, Australia T2 - International Conference on Accelerator and Large Experimental Physics Control Systems T3 - 15 LA - english AB - For the latest generation of detectors transmission, persistence and reading of data becomes a bottleneck. Following the traditional pattern acquisition-persistence-analysis leads to a massive delay before information on the data is available. This prevents the efficient use of beamtime for users. Also, sometimes, single nodes cannot keep up in receiving and persisting data. PSI is breaking up with the traditional data acquisition paradigm for its detectors and is focusing on data streaming, to address these issues. Data is immediately streamed out directly after acquisition. The resulting stream is either retrieved by a node next to the storage to persist the data, or split up to enable parallel persistence, as well as online processing and monitoring. The concepts, designs, and software involved in the current implementation for the Pilatus, Eiger , PCO Edge and Gigafrost detectors at SLS, as well as what we are going to use for the Jungfrau detector and the whole beam synchronous data acquisition system at SwissFEL, will be shown. It will be shown how load-balancing, scalability, extensibility and immediate feedback are achieved, while reducing overall software complexity. PB - JACoW CP - Geneva, Switzerland ER -