09 Pre- and Post-Processing
Paper Title Page
WEAAI1 Bringing Large-scale Analytics to Accelerators 116
 
  • N. Malitsky
    BNL, Upton, Long Island, New York, USA
 
  The report presents a new approach for storing and processing both the accelerator control data and the experimental results. It is based on the analysis and consolidation of several modern technologies, such as the EPICS control infrastructure, the SciDB array-oriented data management and analytics platform, the HDF5 file format, and others. The paper overviews the different features of the proposed system and the development of analytics algorithms in the context of the modern light source facilities.  
slides icon Slides WEAAI1 [2.505 MB]  
 
FRABI2 Big Data Analysis and Visualization: What Do Linacs and Tropical Cyclones Have in Common? 299
 
  • E.W. Bethel, S. Byna, J. Chou, E. Cormier-Michel, C.G.R. Geddes, M. Howison, F. Li, P. Prabhat, J. Qiang, O. Rübel, R.D. Ryne, M.F. Wehner, K. Wu
    LBNL, Berkeley, California, USA
 
  Funding: This work was supported by the Director, Office of Science, Office and Advanced Scientific Computing Research, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231.
While there is wisdom in the old adage "the two constants in life are death and taxes," there are unavoidable truths facing modern experimental and computational science. First is the growing "impedence mismatch" between our ability to collect and generate data, and our ability to store, manage, and gain understanding from it. The second is the fact that we cannot continue to rely on the same software technologies that have worked well for the past couple of decades for data management, analysis, and visualization. A third is that these complementary activities must be considered in a holistic, rather than balkanized way. The inseperable interplay between data management, analysis, visualization, and high performance computational infrastructure, are best viewed through the lens of case studies from multiple scientific domains, where teams of computer and accelerator scientists combine forces to tackle challenging data understanding problems.
 
slides icon Slides FRABI2 [3.622 MB]