InfiniCortex - From proof-of-concept to production

Gabriel Noaje, Alan Davis, Jonathan Low, Lim Seng, Tan Geok Lian, Lukasz P. Orłowski, Dominic Chien, Liou Sing-Wu, Tan Tin Wee, Yves Poppe, Ban Hon Kim Kenneth, Andrew Howard, David Southwell, Jason Gunthorpe, Marek T. Michalewicz

    Research output: Contribution to journalArticlepeer-review

    3 Citations (Scopus)

    Abstract

    The global effort to build ever more powerful supercomputers is faced with the challenge of ramping up High Performance Computing systems to ExaScale capabilities and, at the same time, keeping the electrical power consumption for a system of that scale at less than 20 MW level. One possible solution, bypassing this local energy limit, is to use distributed supercomputers to alleviate intense power requirements at any single location. The other critical challenge faced by the global computer industry and international scientific collaborations is the requirement of streaming colossal amounts of time-critical data. Examples abound: i) transfer of astrophysical data collected by the Square Kilometre Array to the international partners, ii) streaming of large facilities experimental data through the Pacific Research Platform collaboration of DoE, ESnet and other partners in the US and elsewhere, iii) the Superficilities vision expressed by DoE, iv) new architecture for CERN LHC data processing pipeline focussing on more powerful processing facilities connected by higher throughput connectivity. The InfiniCortex project led by A*STAR Computational Resource Centre demonstrates a worldwide InfiniBand fabric circumnavigating the globe and bringing together, as one concurrent globally distributer HPC system, several supercomputing facilities spanned across four continents (Asia, Australia, Europe and North America). Using global scale InfiniBand connections, with bandwidth utilisation approaching 98% link capacity, we have established a new architectural approach which might lead to the next generation supercomputing systems capable of solving the most complex problems through the aggregation and parallelisation of many globally distributed supercomputers into a single hive-mind of enormous scale.

    Original languageEnglish
    Pages (from-to)87-102
    Number of pages16
    JournalSupercomputing Frontiers and Innovations
    Volume4
    Issue number2
    DOIs
    Publication statusPublished - 2017

    Fingerprint

    Dive into the research topics of 'InfiniCortex - From proof-of-concept to production'. Together they form a unique fingerprint.

    Cite this