In the last decade, we witnessed an increasing interest in High Performance Computing (HPC) infrastructures, which play an important role in both academic and industrial research projects. At the same time, due to the increasing amount of available data, we also witnessed the introduction of new frameworks and applications based on the MapReduce paradigm (e.g., Hadoop). Traditional HPC systems are usually designed for CPU- and memory-intensive applications. However, the use of already available HPC infrastructures for data-intensive applications is an interesting topic, in particular in academia where the budget is usually limited and the same cluster is used by many researchers with different requirements. In this paper, we investigate the integration of Hadoop, and its performance, in an already existing low-budget general purpose HPC cluster characterized by heterogeneous nodes and a low amount of secondary memory per node.

Hadoop on a Low-Budget General Purpose HPC Cluster in Academia / Garza, Paolo; Margara, Paolo; Nepote, Nicolo'; Grimaudo, Luigi; Piccolo, Elio. - 241:(2014), pp. 187-192. ((Intervento presentato al convegno ADBIS Special session on Big Data: New Trends and Applications (BiDaTA 2013) tenutosi a Genova [10.1007/978-3-319-01863-8_21].

Hadoop on a Low-Budget General Purpose HPC Cluster in Academia

GARZA, PAOLO;MARGARA, PAOLO;NEPOTE, NICOLO';GRIMAUDO, LUIGI;PICCOLO, Elio
2014

Abstract

In the last decade, we witnessed an increasing interest in High Performance Computing (HPC) infrastructures, which play an important role in both academic and industrial research projects. At the same time, due to the increasing amount of available data, we also witnessed the introduction of new frameworks and applications based on the MapReduce paradigm (e.g., Hadoop). Traditional HPC systems are usually designed for CPU- and memory-intensive applications. However, the use of already available HPC infrastructures for data-intensive applications is an interesting topic, in particular in academia where the budget is usually limited and the same cluster is used by many researchers with different requirements. In this paper, we investigate the integration of Hadoop, and its performance, in an already existing low-budget general purpose HPC cluster characterized by heterogeneous nodes and a low amount of secondary memory per node.
9783319018621
9783319018638
File in questo prodotto:
File Dimensione Formato  
2514484_draft.pdf

non disponibili

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 246.87 kB
Formato Adobe PDF
246.87 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Garza-etAl-BIDATA2013.pdf

non disponibili

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 334.38 kB
Formato Adobe PDF
334.38 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11583/2514484
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo