The combination of the high-performance psort sorting library and of a carefully tuned desktop-class cluster allowed us to improve the previous record on the Datamation sort benchmark by over an order of magnitude, sorting a million 100 byte records from disk to disk in a few dozen milliseconds. Of the many implementation and configuration choices we faced, the most crucial were judicious data placement and access patterns on disk, adoption of UDP sockets instead of MPI, careful pruning of virtually all system daemons, and rejection of "on demand" frequency scaling.

Datamation: A quarter of a century and four orders of magnitude later / P. Bertasi, M. Bonazza, M. Bressan, E. Peserico (PROCEEDINGS IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING). - In: 2011 IEEE International Conference on Cluster Computing[s.l] : IEEE, 2011. - ISBN 978-1-4577-1355-2. - pp. 605-609 (( convegno IEEE International Conference on Cluster Computing, CLUSTER 2011 tenutosi a Austin nel 2011 [10.1109/CLUSTER.2011.75].

Datamation: A quarter of a century and four orders of magnitude later

M. Bressan;
2011

Abstract

The combination of the high-performance psort sorting library and of a carefully tuned desktop-class cluster allowed us to improve the previous record on the Datamation sort benchmark by over an order of magnitude, sorting a million 100 byte records from disk to disk in a few dozen milliseconds. Of the many implementation and configuration choices we faced, the most crucial were judicious data placement and access patterns on disk, adoption of UDP sockets instead of MPI, careful pruning of virtually all system daemons, and rejection of "on demand" frequency scaling.
cluster; datamation; latency; psort; sorting
Settore INF/01 - Informatica
IEEE Technical Committee on Scalable Computing
Intel
Dell
SGI
Platform Computing
Book Part (author)
File in questo prodotto:
File Dimensione Formato  
Bertasi&2011-CLUSTER.pdf

solo utenti autorizzati

Tipologia: Publisher's version/PDF
Dimensione 294.81 kB
Formato Adobe PDF
294.81 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/922286
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact