LLNL, Livermore, California, USA
The Advanced Radiographic Capability (ARC) currently under development for the National Ignition Facility (NIF) will provide short (1-50 picoseconds) ultra high power (>1 Petawatt) laser pulses used for a variety of diagnostic purposes on NIF ranging from a high energy x-ray pulse source for backlighter imaging to an experimental platform for fast-ignition. A single NIF Quad (4 beams) is being upgraded to support experimentally driven, autonomous operations using either ARC or existing NIF pulses. Using its own seed oscillator, ARC generates short, wide bandwidth pulses that propagate down the existing NIF beamlines for amplification before being redirected through large aperture gratings that perform chirped pulse compression, generating a series of high-intensity pulses within the target chamber. This significant effort to integrate the ARC adds 40% additional control points to the existing NIF Quad and will be deployed in several phases over the coming year. This talk discusses some new unique ARC software controls used for short pulse operation on NIF and integration techniques being used to expedite deployment of this new diagnostic.
LLNL, Livermore, California, USA
The National Ignition Facility (NIF) is operated by the Integrated Computer Control System in an object-oriented, CORBA-based system distributed among over 1800 front-end processors, embedded controllers and supervisory servers. At present, NIF operates 24x7 and conducts a variety of fusion, high energy density and basic science experiments. During the past year, the control system was expanded to include a variety of new diagnostic systems, and programmable laser beam shaping and parallel shot automation for more efficient shot operations. The system is also currently being expanded with an Advanced Radiographic Capability, which will provide short (<10 picoseconds) ultra-high power (>1 Petawatt) laser pulses that will be used for a variety of diagnostic and experimental capabilities. Additional tools have been developed to support experimental planning, experimental setup, facility configuration and post shot analysis, using open-source software, commercial workflow tools, database and messaging technologies. This talk discusses the current status of the control and information systems to support a wide variety of experiments being conducted on NIF including ignition experiments.
LLNL, Livermore, California, USA
Used actively by the National Ignition Facility since 2004, the JIRA issue tracking system from Atlassian is now used for 63 different projects. NIF software developers and customers have created over 80,000 requests (issues) for new features and bug fixes. The largest NIF software project in JIRA is the Integrated Computer Control system (ICCS), with nearly 40,000 issues. In this paper, we’ll discuss how JIRA has been customized to meet our software development process. ICCS developed a custom workflow in JIRA for tracking code reviews, recording test results by both developers and a dedicated Quality Control team, and managing the product release process. JIRA’s advanced customization capability have proven to be a great help in tracking key metrics about the ICCS development efforts (e.g. developer workload). ICCS developers store software in a configuration management tool called AccuRev, and document all software changes in each JIRA issue. Specialized tools developed by the NIF Configuration Management team analyze each software product release, insuring that each software product release contains only the exact expected changes.
LLNL, Livermore, California, USA
The Integrated Computer Control System (ICCS) at the National Ignition Facility (NIF) uses Front-End Processors (FEP) controlling over 60,000 devices. Often device faults are not discovered until a device is needed during a shot, creating run-time errors that delay the laser shot. This paper discusses a new ICCS framework feature for FEPs to monitor devices and report its overall health, allowing for problem devices to be identified before they are needed. Each FEP has different devices and a unique definition of healthy. The ICCS software uses an object oriented approach using polymorphism so FEP’s can determine their health status and report it in a consistent way. This generic approach provides consistent GUI indication and the display of detailed information of device problems. It allows for operators to be informed quickly of faults and provides them with the information necessary to pin point and resolve issues. Operators now know before starting a shot if the control system is ready, thereby reducing time and material lost due to a failure and improving overall control system reliability and availability.
LLNL, Livermore, California, USA
The Integrated Computer Control System (ICCS), used by the National Ignition Facility (NIF) provides comprehensive status and control capabilities for operating approximately 100,000 devices through 2,600 processes located on 1,800 servers, front end processors and embedded controllers. Understanding the behaviors of complex, large scale, operational control software, and improving system reliability and availability, is a critical maintenance activity. In this paper we describe the ICCS diagnostic framework, with tunable detail levels and automatic rollovers, and its use in analyzing system behavior. ICCS recently added Splunk as a tool for improved archiving and analysis of these log files (about 20GB, or 35 million logs, per day). Splunk now continuously captures all ICCS log files for both real-time examination and exploration of trends. Its powerful search query language and user interface provides allows interactive exploration of log data to visualize specific indicators of system performance, assists in problems analysis, and provides instantaneous notification of specific system behaviors.
LLNL, Livermore, California, USA
Network packet inspection using port mirroring provides the ultimate tool for understanding complex behaviors in large distributed control systems. The timestamped captures of network packets embody the full spectrum of protocol layers and uncover intricate and surprising interactions. No other tool is capable of penetrating through the layers of software and hardware abstractions to allow the researcher to analyze an integrated system composed of various operating systems, closed-source embedded controllers, software libraries and middleware. Being completely passive, the packet inspection does not modify the timings or behaviors. The completeness and fine resolution of the network captures present an analysis challenge, due to huge data volumes and difficulty of determining what constitutes the signal and noise in each situation. We discuss the development of a deep packet inspection toolchain and application of the R language for data mining and visualization. We present case studies demonstrating off-normal analysis in a distributed real-time control system. In each case, the toolkit pinpointed the problem root cause which had escaped traditional software debugging techniques.