Storage issues

Maxim Samo of UBS details the challenges of managing not one, but two major data centre projects during an economic downturn.

It costs €4.4 million annually to power a 10,000sqm data centre

“These data centre projects are like huge ships, like the Titanic. You can't just totally turn them around from one second to the other”
-Maxim Samo, Director of IT at a major European bank

That data centre build project kicked off back in 2005, obviously when business was good and the economy was great. Everybody was looking to expand data centre capacity and space was at a premium. So back at that time we decided to build a new facility in the United States, a project build in two phases. The first phase is 4.5 megawatts of capacity and the second phase is another 4.5 for a site that would be able to finally have nine megawatts of capacity. July 2008 was our live date and we went live with about 2.2 megawatts of capacity. That was always the level we planned to start at, but now we’ll wait a little longer before ramping up to full phase one capacity. We just didn't put in the full mechanical and electrical, just because with the downturn in the economy the demand for all that capacity is no longer as high as we originally thought it would be.

That actually was one of the big challenges, the complete turn in the economy. These data centre projects are like huge ships, like the Titanic. You can't just totally turn them around from one second to the other.  So we basically had to go through the project and look at where we could defer as much cost as possible given the new circumstances. Not fully build out the site but just kind of put things in where it makes sense. Despite the circumstances, the project has been successful. It was on time and on budget and the businesses are happy. Since we have brought that site online we do have people using it, especially in migrations from rented collocation sites that we had.  We had to go into collocation rentals back in 2005 because of the capacity constraints. Now we have migrations out of those very expensive rental sites. If we bring them onto our site now it's a lot cheaper for us in the long term. While a lot of companies are looking to outsource things like data storage as a way of cutting costs, we look at things more case by case. We're actually working on a study looking at when we should we go into collocations and when we should build our own data centre? We're building this for our own company at the lowest cost possible. If you need a huge chunk of capacity and you know you're going fill that site, it actually makes sense for you to build it. However, things are different if you just need a tactical space – for example, at the time we went into the collocation agreement, we knew that this was going to be temporary. We knew that once we had the new site we could actually move out of it.

Of course, if the business situation improves, we can quickly scale up the centre’s capacity. The way it is now we can easily scale it up to the full phase one capacity of 4.5 megawatts. Once we see that we need to activate phase two, that would obviously be another major undertaking.

Any project like this now has to take environmental concerns into account. We have something called the platform design committee. In there, we look at energy efficiency for our servers and other technologies. However, we also looked at efficiency in our plant. We are using state of the art units with variable speed fans and other features like that so you don't have a data centre that uses a lot of energy for those servers in there. Even in the process of construction, together with the general contractor we looked at ensuring that resources were as local as possible. We tried to favour regional traders for all the contracts so that people didn’t have to drive across the country just to get to the job site.

Obviously we implemented heat exchanges that allow us to do free cooling during winter. We have carpets made from recycled materials. We even used recycled steel. Also we're utilising company credits from the utility providers that we get for energy efficient design. Alongside the environmental benefits we also have to ensure that these kind of efforts are financially viable. Sometimes it's hard to quantify. In addition to program managing these big constructions projects, I'm also the global head of the data centre design team. Something we're looking at is going into existing data centres and making the mechanical electrical plant more efficient and trying to come up with energy savings in our existing facilities.

The difficulty there is first of all you need a baseline, so you need to know exactly how much power you're already drawing. Especially in buildings that you share with office areas, there's quite an effort involved in just getting that baseline right. But we are actually running a pilot in Chicago where we have created our baseline. And what we did then is we started to go into the raised floor and start to do all the best practices for energy efficiency.  So we started to plug up all the holes we had in our raised floor and made sure we had blanket panels everywhere we could in the cabinets. We also looked at the CRAC units, their fan speeds, their humidification, the sub-points and all that kind of stuff. We essentially began to optimize the whole system.

We actually managed to create a business case and this is an initiative that we are looking into rolling out globally. It’s very hard and you need to put a lot of effort into it. But if you can correctly set a baseline, you can actually show that there is a business case there.  The trouble is that savings are very hard to quantify. 

They're actually very difficult to see. For example, you might start saving money on your utility bill, but at the same time you still have people installing new equipment. So maybe even in your existing data centre you will use more energy because you have growing amounts of equipment. So all your savings are basically gone because somebody installed a new piece of equipment, and it's using all that energy that you just saved. That's the difficulty. That's the challenge here. But there is a business case, even if in trying environments.

Room for improvement

After the work over in the US I’m now back in Europe working on the renovation of an existing data centre. I believe the building is somewhere between 15 and 20 years old, so it's got into its old age. It's a shared office and data centre. We decided to renovate it because otherwise we'd have to spend a lot of money either getting out of it and relocating elsewhere. The project is actually probably an even bigger challenge than the new data centre because this is open-heart surgery. We have to replace the electrical system in an existing data centre and be very careful while we do it. You really don't want to bring the centre down while you do that.

Fortunately we do have some extra space in the building available. So what we're doing is building the new electrical plant within the building while the old plant is running, and then there's going to be a switchover phase. That's going to be the critical moment when we switch over to the new electrical plant. Once we've done that we can rip out the old electrical plant.

What I learned on the building operation in the US has been very helpful in this new project. I'm originally a system administrator, and then I started to work in the applications space so it was my first real building project. First of all, it was a different culture, working with different types people. But I obviously did learn a lot. I learned a great deal about how construction projects work, what the difficulties are, how the accounting works, how project management works and what general contractors do. All of that will be of use for me in the renovation project.

We are still at a very early stage on this project and we’re planning to have it completed by 2012. This will be happening despite the ongoing economic crisis, though there will likely be some implications. It's not as if I have a top-down mandate where I’m told that I need to cut 10 percent of the budget.  However, we are revisiting what we are doing.  We're making sure that we're not gold plating anything that doesn't need to be gold plated. There probably will be some sacrifices once we actually get into the work. However, even in the difficult times, now more than ever, stability is absolutely the key to the endeavour. We cannot make any sacrifices where you would create a risk to the company. We can't do that.  So operational stability is our number one priority during this project.

Maxim Samo is Director of Information Technology Infrastructure at UBS. He joined Swiss Bank Corporation in 1994, after studying computer science at the Federal Institute of Technology (ETH) in Zurich.

As a Unix System Administrator Samo was involved in building a new investment banking centre outside of Zurich that also houses 450 trading positions. Leveraging his technical skills his responsibilities soon included database and application support focusing on foreign exchange derivatives.

In 1999 Swissbank and UBS merged to become the new UBS. Samo then had the Application project management responsibility for a 650 position trading floor move – at the time the largest trading floor in Europe. Shortly thereafter Samo stayed in IT management roles that included a short-term assignment in Singapore.

Other roles that were held by Samo were Chief of Staff IT Switzerland, IT production manager Switzerland, global network service manager for the Investment Bank.