The COSMOS supercomputer, designed to probe the deepest mysteries of our Universe, has had a significant impact on high performance computing back here on Earth. The technical advances it has inspired power applications in a wide range of areas, from the medical sector to Hollywood CGI.
COSMOS is housed at the Stephen Hawking Centre for Theoretical Cosmology (CTC), which studies the two phenomena that are currently of greatest interest to cosmologists: the Big Bang, which marked the birth of our Universe, and black holes and their mergers. The enormous temperatures and densities involved in both types of event violently shake, and thereby unveil, the hidden fabric of our Universe.
The challenge
Information about the Big Bang comes to us via the radiation released at the time, known as cosmic microwave background (CMB) radiation, which still permeates our Universe. Large scale observational efforts, such as the Planck satellite mission, provide us with invaluable data about the CMB. However, analysing this data and comparing it with existing mathematical theory requires immense amounts of computing power. The study of black holes, too, requires computing power that outstrips what is required for most commercial applications of computing.
High performance computing (HPC) facilities are therefore vital to CTC's research programme. This is why CTC has been engaged in long-standing industrial collaborations to design and develop HPC hardware and software. These collaborations have not just furthered CTC research, but have also brought significant benefits to the commercial partners too.
The science
The industrial partners involved are household names in the computing industry: Intel and Hewlett Packard Enterprises (HPE) who in 2014 purchased SGI, a company the CTC had been collaborating with since 1997. The collaborations have enabled CTC researchers to be intimately involved with the design of software and hardware, benefitting from early access to the latest advances and support from expert developers.
The Planck satellite observations provided one impetus for the collaborations. CTC researchers already had a general methodology for analysing the data and confronting it with theory, developed for a previous satellite mission, but faced a huge computational task scaling this up for the Planck satellite analysis. Together with their industrial partners, they defined a unique new HPC architecture. This combined a large shared-memory server from SGI and hardware acceleration from Intel's Xeon Phi co-processors. On delivery in 2012, the COSMOS UV2 supercomputer was the largest shared-memory system in the world. The academic papers CTC researchers were able to produce with its help became some of the most important Planck science publications.
Another type of challenge came from the computer codes CTC researchers needed to run, not just to analyse Plank data, but also to study black holes and mysterious cosmic phenomena known as topological defects. To be effective these codes required dramatic speed-ups, which CTC researchers achieved together with Intel software engineers. In the case of Planck data the speed-up exceeded a factor of a 100 and won the prestigious HPCwire Award in 2015. The codes have since provided new physical insights in areas that were previously intractable.
Apart from theory and data, cosmology relies heavily on visualisations of the dramatic processes it studies. This need has driven a CTC collaboration with Intel to pioneer in-situ visualisation: a new paradigm which analyses data "on the fly", as it is generated. This greatly reduces analysis timescales and has led to a wealth of scientific results, including new insights into the conjectured exponential expansion of the early Universe gleaned from Planck data.
The impact
For the industrial partners, the requirements of CTC research provided specifications to guide new developments, as well as opportunities to stress test and validate them. CTC researchers were intimately involved in the development of SGI supercomputers. The resulting innovations played a key part in the $275,000,000 takeover of SGI by HPE in 2016. Together with CTC researchers HPE then designed and delivered the Superdome Flex server, HPE's flagship in-memory system, in 2017. It is deployed by commercial customers with large data flows worldwide — including Burger King and the India Oil Company.
CTC's computer codes were used to validate and test Intel's newly developed new Xeon Phi coprocessor, as well as follow-on products that have played a key role in recent HPC. By 2017, one such follow-on product, codenamed Knights Landing, powered four of the top 10 systems in the world and 19 of the top 100. CTC's work also helped improve the software needed to run Xeon Phi as well as software developer tools.
Since 2014, CTC's collaborative research with Intel has been at the frontier of visualisation technology. CTC researchers were intimately involved in the development of open-source visualisation libraries OSPray and Catalyst, and the creation of in-situ visualisation capabilities on HPC systems. OSPray is now at the core of the Intel OneAPI Rendering Toolkit, which is widely used in medical data applications, the oil and gas industry, the defence industry, interior space planning software, and Hollywood CGI. If you have watched How to train your dragon: The hidden world, you'll have seen its impressive results.
The collaborations between CTC and its industrial partners are ongoing and expanding. CTC is now an Intel oneAPI Centre of Excellence and receives funding from the ExCALIBUR programme which supports the development of algorithms and infrastructure that benefits UK research. Among other things, this has inspired a new Master's degree in Data Intensive Science at CTC, which will produce the expert scientists needed in both academia and industry.
It is thanks to our desire to understand the Universe that these real-world impacts have been achieved.
The image above is a visualisation of radiation from a cosmic string network created with the help of the Intel oneAPI Rendering Toolkit. Credit: Amelia Drew (CTC, University of Cambridge) and Carson Brownlee (Intel Advanced Visualisation and Rendering)
This article introduces just one of the examples of the Faculty's high impact research that was submitted as part of the Research Excellence Framework (REF) 2021, a major UK exercise to assess the quality of research at UK universities.
