The birth of HPC Cuba
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Roefs, Dieter
Enriquez, Hector Cruz
To offer supercomputing services to all Cuban institutes of higher education,research centers and industrial partners is the mission set forth by a groupof ambitious young system administrators working at various universities ofthe lush Caribbean island of Cuba. With the assistance of the Flemish Inter-university Council (VLIR), lots of hard work and hardware donations by GhentUniversity they have succeeded in setting up a nation-spanning architecturecomprised of three datacenters, in Havana (West), Santa Clara (central) andSantiago de Cuba (East). Their campuses are spread geographically across the1000 km long island and are interconnected by a 20 Mbps network that covers upto the furthest regions of the island where remote rural universities aregiven the opportunity to take advantage of the magnificent capacities ofmassive parallel processing. Thus was born "HPC Cuba" the national center for academic supercomputing.While fully depending on Free Software for their operations, HPC Cuba has alsojoined the global Free Software community by producing installation packagesfor software not yet covered by EasyBuild. On a tropical island where access to resources is being limited by a USimposed trade embargo, setting up a datacenter capable of hosting a propersupercomputer "no es fácil" ("it is not easy", a Cuban saying). Come learnabout the specific challenges poised in the Cuban context, calling forresourceful solutions and unique problem-solving capacities standing proof ofthe high standard of the universities of this so-called third world countryand its relentless human resources. KEYWORDS: Cuba, University, Development Coooperation, VLIR-UOS, Free Software,National Center for Supercomputing, Distributed, HPC Cuba
 Next up is Hector and Dieter. We're going to talk about the birth of HPC in Cuba. Good morning. My name is Hector, and I'm happy to be here to tell you what we are doing for making HPC service available for Cuban scientists. The lowest you can see are institutions that are supporting our projects. The BLIR UELS, that is the Flemish University Council. Also, BLRED.CU is a collaboration on projects with Flemish University. And six Cuban universities, where three of them are very involved in this project that we call HPC Cuba. We want to make HPC service available for all Cuban scientists. So our mission is to provide centralized access to HPC infrastructure, also to offer training to our HPC staff, and also to support our users belonging to the higher education system, and also to our research institution in Cuba. The HPC Cuba project that we can also call Cuba Center for Academy Supercomputing. It was sold with a roughly distribution of the HPC infrastructure in three main points in Havana, Santa Clara, and Santiago de Cuba. In three universities, the University of Informatics Science in Havana, the Central University of Las Pillas in Santa Clara, University of Oriente in Santiago de Cuba. Some facts about Cuba. So Cuba is the biggest island, the Caribbean, also with tropical climates, it's a tourist destination. Also, our high education standard is true. So we are friendly people, so Cuba is also a safe place in America, but we are still living under the USA embargo. So it costs also limitation, like the ban which also restricted access to the information and internet, because some sites are blocking our Cuban IPs. So also the access to the infrastructure become problematic sometimes. An important remark, it's the Americans blocking their sites for Cuban access. It's not the Cuban government blocking the access for the Cuban population. It's the other way around. Many people think it's the other way around. Thank you for the clarification on that. This is Cuba. We are very close to USA Florida, so you can find us. I'll put them up. This pro idea started in 2015 when we started two small HPC clusters. One in Santa Clara, UCRB, and the other one in Oriental University in the eastern part of the country. But our goal is to provide access to the whole country. Now it's starting a new one in Havana in the University of Informatics Science. Maybe you have heard about the Linus Nova distribution that was developed there. But also in Cuba, you can take a walk, ride in nice old cars, and enjoy the Marcon during the evening. So in Santa Clara, you can find the Center University of La Fija with a very nice university campus in a very natural environment with a lot of streets. Also, you can find beautiful beaches in the north coast of the province, so in Cayo Santa Maria, where we have heard many workshops regarding ICT. In Oriente, it's also a nice place with very friendly people, where Virginians become Q-ants immediately. So it's difficult to recognize, to deter, but maybe because he is taller than Q-ants. So highlighted in red, the HPC closer in University of Oriente, also in University of Las Villas. What we are doing right now, so our plan now, immediately for 2017, is to insert two IBM clusters with the capacity of about 600 cores, one terabyte of RAM in Havana and Santa Clara. Also in Santa Clara, a big data cluster with 20 servers and two NVIDIA GPU cards. In Oriente University, a third cluster with similar capacity but using infinite band technology for the interconnectivity. So that is what we are talking about. One of these in Oriente was already a start, where we have already started, where either Joe or other specialists are working hard to make this possible. So this is our goal for this year. So maybe you can remark here that the hardware is donated by Gent University. It's impossible for CUM universities to buy IBM or Dell or HP hardware. It's illegal for Dell to export to Cuba. If a Dell salesperson sells to a CUM university, you lose this job. But thanks to the collaboration with the Flemish universities, we've been able to establish some infrastructure in Cuba. So everything we are using is free source work. So far, we have deviance operating system that is using Torqued, PBS, and MAUI for managing keywords and jobs. Also, it's in Ganglap for monitoring purposes. But we are exploring a new alternative for the new implementation, maybe using Centov and Slur, and maybe Orelsen. So that is what we are thinking about. We are exploring to do that. We have offered web portal to support users. So this is another example in the University of Oriente. Also, a wiki answering for the language, scoring for Q&A users. Also, building different ways to access to the cluster for monitoring purposes. For example, you can access using SSH for look the keywords state. You can do that. Also, using the web interface, the same for the job states, it's more comfortable for non-IT user to use the web access. And also, you are creating some templates for script to submit in jobs. And in the web, via web, also, it's possible. So now we can see how many institutions in Cuba are using our HPC facilities. Also, you can find a user from BUB. So I have no idea what he's doing in our cluster. But OK, he's here. Also, people from Brazil, from Brazilian University, because you are helping often, we are helping Brazilian guys. So we are happy with that. With the users, it's a lot of Cuba University in different provinces. So. Yes, let me allow you to remark there are a lot of universities in Cuba. That's a lot of very small universities. Almost every little town is its own university. So we have infrastructure in the three main, the biggest three main universities of the island. And as you can see, the infrastructure is working well already on a high level using nothing but free and open source solutions. And all kinds of institutions and research institutes are using already our infrastructure. And that's what this graph shows. OK, so some samples are different resorts. Cuba using our HPC, for example, for video encoding in the Cuban digital television broadcasting, for example. And also for sign up processing, users, network, network for health applications, also for the environment, and also for the chemistry application where they couldn't do the job with a very good laptop with 16-year-old, it was not possible, but in our close to the Oriental University in 15 minutes, it would solve. So other scientific papers. So it's just to have an idea that our scientists in Cuba are using what we are doing. So now we have a national strategy that includes capacity burden. So it would be important for us to train our HPC staff developing training or workshops. It could be in Cuba or in Belgium. Also, supporting the users and establishment of synergies with all institutions. So this is our national strategy. This is, it was our first workshop in Santa Clara. The participation of a HPC expert from University was sponsored by Blear. So we can see. Alvaro, you can see him here. So Alvaro is here. Everybody's experience of working with Cuba. So Alvaro could be a very good reference of Cuba, our job. Also Rudi is here and Dieter, of course. But also some people from Ecuador and Peru were trained because it had regional impacts also. And of course, for our national strategy we have to face some drawbacks. For example, the ban wish also, also the access policy because sometimes the SSH ports are broken. When you go from another institution to another, so we have to face that or some block house that we have. And of course the USA trade embargo is still present and nobody knows what we're trying to do in the future regarding Cuba. So we have to see. But now we are focused and to develop a centralized authentication system with a national front end. Also to improve our rates, our storage capacity, train our people and make it easier the user's installation by using some faceted that we will show now. This is our user management approach with national front end, with centralized authentication with redundancy, also the login nodes that are connected to the head of the cluster, and also access to scratch user in FiniBand to the home of the user or the data user in our FICI system. So actually the infrastructure is working at the separate universities and what we're working on now is like a layer on top of that to have national user accounts. So if you have a user account for HPC Cuba at your home university, you can access infrastructure at another university. Also for making it easier the installation for users, we are trying to use ECWID where we say that we already have a Cuban contribution to the ECWID community developing the JITs, for example. I know that Kenneth has been our guide and also a supervisor of our developers. So we are also working in helping something with the clip of Open4. So maybe Kenneth can explain much more. So it's a sample of the JIT installation for our staff in Cuba. So guys, we need HPC Esper light youth to join us in this trip to make it possible. So thank you very much for your attention. Thank you. Questions? The hardware we're using is... I was going to say you shouldn't look at that as a lesser generation. You should take some work on how to process it out. We're using the hardware that we can get our hands on. We don't really have the liberty to choose the hardware. We're just happy to receive it. These were good clusters working very well at Ghent University. They were decommissioned for all kinds of reasons. Instead of showing them into the furnace, we decided to export them to Cuba and donate them to CUNY University. If you have another cluster available for us, please help us. So just wondering, where can we find more information about the ongoing projects that you're asking for HPC experts to get involved? What are the opportunities available? Please send us an email so we have your email. And then we can see... In the future, we are always organizing trainings. Cubans are coming to Belgium. We are sending experts to Cuba every time we have funds. We get funding from the Flemish Inter-University Council. The deal is basically that it's volunteer work. But you get your ticket paid, your hotel service paid, you get it per DM. And what we see is people go there and then they stay for an extra week, extra two weeks to go on holiday. And it's just fine. We are right now applying for a call. So we can promise anything, but maybe for the next three years, we can have some budgets supporting our projects. So we need some Europe experts to support us, to teaching our staff. So our staff can come here and our trainers can go to Cuba also. So the question is about the electricity. Do we use UPS or do we generate our own electricity? Well, the experience is that the electricity in Cuba is unstable. It's very normal to have small outages. We use UPS mostly to stabilize the mains current. And we have generators also. But outages are quite common. So we use UPS, also some inverters with batteries. It's not as sad as UPS. Of course, also generators. Please. Yes? It seems like for you it's kind of a lot. You should get your hands on hardware because everything will move there. Everything is controlled by the U.S. in the environment. You look into open hardware solutions for now there are some major integrators, for instance, that are integrating ARM processors. So that's a field that is in the less control by the U.S. and actually put something like UCS across the world. Or also economically, it might be also difficult to have funding for, like, tiny hardware. Yes. The question is maybe we should focus on open hardware or open DC infrastructure instead of looking for U.S. hardware providers. Yeah, maybe we can look into it, but for right now, I think financially, it would be very difficult for us to start building or to start looking in that direction. From up until now, we've depended on free hardware. Free is always cheaper than maybe open can be. Yes? The HPC will be used only for scientific research or also for public services in the future. Okay, it could be used for anybody. So we are focused mainly now in this first step in scientific community because the people that have more needs about HPC. But for one of the samples that I showed regarding to the video encoding for the national Cuban digital television broadcasting, it's not for scientific purposes. It's just for storage. The materials in the format that they need also for broadcasting to the Cuban people. Any more questions? Thanks again to the people in the city.
 Next up is Hector and Dieter.  We're going to talk about the birth of HPC in Cuba.  Good morning.  My name is Hector, and I'm happy to be here to tell you  what we are doing for making HPC service available for  Cuban scientists.  The lowest you can see are institutions that are  supporting our projects.  The BLIR UELS, that is the Flemish University Council.  Also, BLRED.CU is a collaboration on projects  with Flemish University.  And six Cuban universities, where three of them are very  involved in this project that we call HPC Cuba.  We want to make HPC service available for all Cuban  scientists.  So our mission is to provide centralized access to HPC  infrastructure, also to offer training to our HPC staff,  and also to support our users belonging to the higher  education system, and also to our research institution in Cuba.  The HPC Cuba project that we can also call Cuba Center for  Academy Supercomputing.  It was sold with a roughly distribution of the HPC  infrastructure in three main points in Havana, Santa Clara,  and Santiago de Cuba.  In three universities, the University of Informatics  Science in Havana, the Central University of Las  Pillas in Santa Clara, University of Oriente in  Santiago de Cuba.  Some facts about Cuba.  So Cuba is the biggest island, the Caribbean, also with  tropical climates, it's a tourist destination.  Also, our high education standard is true.  So we are friendly people, so Cuba is also a safe place in  America, but we are still living under the USA embargo.  So it costs also limitation, like the ban which also  restricted access to the information and internet,  because some sites are blocking our Cuban IPs.  So also the access to the infrastructure become  problematic sometimes.  An important remark, it's the Americans blocking their  sites for Cuban access.  It's not the Cuban government blocking the access for the  Cuban population.  It's the other way around.  Many people think it's the other way around.  Thank you for the clarification on that.  This is Cuba.  We are very close to USA Florida, so you can  find us.  I'll put them up.  This pro idea started in 2015 when we started two small  HPC clusters.  One in Santa Clara, UCRB, and the other one in Oriental  University in the eastern part of the country.  But our goal is to provide access to the whole country.  Now it's starting a new one in Havana in the University of  Informatics Science.  Maybe you have heard about the Linus Nova distribution that  was developed there.  But also in Cuba, you can take a walk, ride in nice old  cars, and enjoy the Marcon during the evening.  So in Santa Clara, you can find the Center University of  La Fija with a very nice university campus in a very  natural environment with a lot of streets.  Also, you can find beautiful beaches in the north coast of  the province, so in Cayo Santa Maria, where we have heard  many workshops regarding ICT.  In Oriente, it's also a nice place with very friendly  people, where Virginians become Q-ants immediately.  So it's difficult to recognize, to deter, but maybe because  he is taller than Q-ants.  So highlighted in red, the HPC closer in University of  Oriente, also in University of Las Villas.  What we are doing right now, so our plan now, immediately for  2017, is to insert two IBM clusters with the capacity of  about 600 cores, one terabyte of RAM in Havana and Santa  Clara.  Also in Santa Clara, a big data cluster with 20 servers and  two NVIDIA GPU cards.  In Oriente University, a third cluster with similar capacity  but using infinite band technology for the  interconnectivity.  So that is what we are talking about.  One of these in Oriente was already a start, where we have  already started, where either Joe or other specialists are  working hard to make this possible.  So this is our goal for this year.  So maybe you can remark here that the hardware is donated by  Gent University.  It's impossible for CUM universities to buy IBM or Dell  or HP hardware.  It's illegal for Dell to export to Cuba.  If a Dell salesperson sells to a CUM university, you lose  this job.  But thanks to the collaboration with the Flemish  universities, we've been able to establish some  infrastructure in Cuba.  So everything we are using is free source work.  So far, we have deviance operating system that is using  Torqued, PBS, and MAUI for managing keywords and jobs.  Also, it's in Ganglap for monitoring purposes.  But we are exploring a new alternative for the new  implementation, maybe using Centov and Slur, and maybe  Orelsen.  So that is what we are thinking about.  We are exploring to do that.  We have offered web portal to support users.  So this is another example in the University of Oriente.  Also, a wiki answering for the language,  scoring for Q&A users.  Also, building different ways to access to the cluster  for monitoring purposes.  For example, you can access using SSH for look the  keywords state.  You can do that.  Also, using the web interface, the same for the job  states, it's more comfortable for non-IT user to use the  web access.  And also, you are creating some templates for script to  submit in jobs.  And in the web, via web, also, it's possible.  So now we can see how many institutions in Cuba are using  our HPC facilities.  Also, you can find a user from BUB.  So I have no idea what he's doing in our cluster.  But OK, he's here.  Also, people from Brazil, from Brazilian University,  because you are helping often, we are helping Brazilian  guys.  So we are happy with that.  With the users, it's a lot of Cuba University in  different provinces.  So.  Yes, let me allow you to remark there are a lot of  universities in Cuba.  That's a lot of very small universities.  Almost every little town is its own university.  So we have infrastructure in the three main, the biggest  three main universities of the island.  And as you can see, the infrastructure is working well  already on a high level using nothing but free and open  source solutions.  And all kinds of institutions and research institutes are  using already our infrastructure.  And that's what this graph shows.  OK, so some samples are different resorts.  Cuba using our HPC, for example, for video encoding in  the Cuban digital television broadcasting, for example.  And also for sign up processing, users, network,  network for health applications, also for the  environment, and also for the chemistry application where  they couldn't do the job with a very good laptop with  16-year-old, it was not possible, but in our close to the  Oriental University in 15 minutes, it would solve.  So other scientific papers.  So it's just to have an idea that our scientists in Cuba  are using what we are doing.  So now we have a national strategy that includes  capacity burden.  So it would be important for us to train our HPC staff  developing training or workshops.  It could be in Cuba or in Belgium.  Also, supporting the users and establishment of  synergies with all institutions.  So this is our national strategy.  This is, it was our first workshop in Santa Clara.  The participation of a HPC expert from  University was sponsored by Blear.  So we can see.  Alvaro, you can see him here.  So Alvaro is here.  Everybody's experience of working with Cuba.  So Alvaro could be a very good reference of Cuba,  our job.  Also Rudi is here and Dieter, of course.  But also some people from Ecuador and Peru were  trained because it had regional impacts also.  And of course, for our national strategy we have  to face some drawbacks.  For example, the ban wish also, also the access policy  because sometimes the SSH ports are broken.  When you go from another institution to another,  so we have to face that or some block house that we have.  And of course the USA trade embargo is still present  and nobody knows what we're trying to do in the future  regarding Cuba.  So we have to see.  But now we are focused and to develop a centralized  authentication system with a national front end.  Also to improve our rates, our storage capacity,  train our people and make it easier the user's installation  by using some faceted that we will show now.  This is our user management approach with national front end,  with centralized authentication with redundancy,  also the login nodes that are connected to the head  of the cluster, and also access to scratch user  in FiniBand to the home of the user or the data user  in our FICI system.  So actually the infrastructure is working  at the separate universities and what we're working on now  is like a layer on top of that to have national user accounts.  So if you have a user account for HPC Cuba at your home  university, you can access infrastructure at another university.  Also for making it easier the installation for users,  we are trying to use ECWID where we say that we already have  a Cuban contribution to the ECWID community developing the  JITs, for example.  I know that Kenneth has been our guide and also  a supervisor of our developers.  So we are also working in helping something with the clip  of Open4.  So maybe Kenneth can explain much more.  So it's a sample of the JIT installation for our staff in Cuba.  So guys, we need HPC Esper light youth to join us in this  trip to make it possible.  So thank you very much for your attention.  Thank you.  Questions?  The hardware we're using is...  I was going to say you shouldn't look at that as a lesser  generation.  You should take some work on how to process it out.  We're using the hardware that we can get our hands on.  We don't really have the liberty to choose the hardware.  We're just happy to receive it.  These were good clusters working very well at Ghent University.  They were decommissioned for all kinds of reasons.  Instead of showing them into the furnace, we decided to export them to Cuba  and donate them to CUNY University.  If you have another cluster available for us, please help us.  So just wondering, where can we find more information about the  ongoing projects that you're asking for HPC experts to get involved?  What are the opportunities available?  Please send us an email so we have your email.  And then we can see...  In the future, we are always organizing trainings.  Cubans are coming to Belgium.  We are sending experts to Cuba every time we have funds.  We get funding from the Flemish Inter-University Council.  The deal is basically that it's volunteer work.  But you get your ticket paid, your hotel service paid, you get it per DM.  And what we see is people go there and then they stay for an extra week,  extra two weeks to go on holiday.  And it's just fine.  We are right now applying for a call.  So we can promise anything, but maybe for the next three years,  we can have some budgets supporting our projects.  So we need some Europe experts to support us, to teaching our staff.  So our staff can come here and our trainers can go to Cuba also.  So the question is about the electricity.  Do we use UPS or do we generate our own electricity?  Well, the experience is that the electricity in Cuba is unstable.  It's very normal to have small outages.  We use UPS mostly to stabilize the mains current.  And we have generators also.  But outages are quite common.  So we use UPS, also some inverters with batteries.  It's not as sad as UPS.  Of course, also generators.  Please.  Yes?  It seems like for you it's kind of a lot.  You should get your hands on hardware because everything will move there.  Everything is controlled by the U.S. in the environment.  You look into open hardware solutions for now there are some major integrators,  for instance, that are integrating ARM processors.  So that's a field that is in the less control by the U.S.  and actually put something like UCS across the world.  Or also economically, it might be also difficult to have funding for, like, tiny hardware.  Yes.  The question is maybe we should focus on open hardware or open DC infrastructure  instead of looking for U.S. hardware providers.  Yeah, maybe we can look into it, but for right now, I think financially,  it would be very difficult for us to start building or to start looking in that direction.  From up until now, we've depended on free hardware.  Free is always cheaper than maybe open can be.  Yes?  The HPC will be used only for scientific research or also for public services in the future.  Okay, it could be used for anybody.  So we are focused mainly now in this first step in scientific community  because the people that have more needs about HPC.  But for one of the samples that I showed regarding to the video encoding  for the national Cuban digital television broadcasting,  it's not for scientific purposes.  It's just for storage.  The materials in the format that they need also for broadcasting to the Cuban people.  Any more questions?  Thanks again to the people in the city.
The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_0_83.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_84_131.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_132_1463.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_1464_5447.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_5448_6023.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_6024_7463.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_7464_8195.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_8196_8567.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_8568_8783.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_8784_9059.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_9060_11387.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_11388_11783.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_11784_11975.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_11976_12095.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_12096_12299.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_12300_12515.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_12516_14387.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_14388_14975.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_14976_19319.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_19320_20411.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_20412_22583.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_22584_22631.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_22632_22679.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_22680_24299.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_24300_24347.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_24348_30851.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_30852_30983.png The_birth_of_HPC_Cuba_How_supercomputing_is_being_made_available_to_all_Cuban_re_30984_31008.png