MacResearch

This fall, I'm starting a new faculty position in the Department of Chemistry at the University of Pittsburgh.. I mix theoretical modeling and experimental materials chemistry. In setting up my lab, I'm aiming for a Mac-only lab. That will include a new Xserve cluster, my desktop (and trusty old PowerBook), and some lab machines for instruments and data analysis. The whole lab will share a server filesystem, making it easy to bring experimental data into theoretical models.

This series of articles is intended to be a narrative of that process. I'm hardly the only one to set up a Mac-only science lab. Still, I hope the series will turn out to be interesting and useful to others. I don't intend to solely focus on the positive side -- there have been bumps along the way, and undoubtedly will be in the future. I also intend to cover the switch from Tiger to Leopard on the server and desktop.

Furthermore, I'm hoping this will be a good forum for "switchers" and advice from the community. I'm open to suggestions about topics to include, as well as how to set things up.

Clustering

The first shipment, and perhaps the most important, was my cluster -- 16 Xserves for compute nodes, and a head server. For now, I skipped getting an Xserve RAID, although this is a likely upgrade in the future.

This week, I managed to rack all the servers myself, wire them up, and start setting up the head node.

Rack 'Em Up!

As much fun as it is to think about a nice cluster, there are plenty of practical physical issues. They take up space. They're heavy. They draw lots of power, and heat up a room like an oven.

In my case, this half-rack of servers is going into a dedicated server room with a large number of other clusters (8 at last count). So I knew that the weight was not an issue, the cooling unit has extra capacity, and there's enough power.

Ah, but when I was told I could use 120V power or 208V power, I thought there was an option. So I ordered rack-mount power distribution units (PDU) for regular American 120V power. When I arrived, I realized that all the other racks were plugged into 208V, 60A receptacles. A little math makes this obvious -- at 120V, an Xserve can consume 8A per box. So that's ~140A across my cluster. Ouch.

Fortunately, after talking with the in-house electronics shop and an order later, I have a new PDU and cables coming which will run on 208V power and solve everything. In retrospect, I should have been more proactive at ensuring the power needs for the cluster.

As far as heat, I know that the server room requires a "hot side" and a "cool side" of the server racks. So I made sure to order "blanking panels." These screen off the empty rack slots to prevent air from passing from the cool side (the front) to the hot side (the back).

The moral of the story is that you should check carefully with the server room before placing an order.

Wired In

I'm skipping over several hours of work unboxing and racking the servers. Suffice to say that I wish I had grad students or other helpers here, but it's finished. I was happy to have plenty of room to work, since my lab space is not yet a functioning chem lab.

Remember that I said I had 16 identical nodes, and one head node. Since they all came in one shipment, I don't actually know which node is the head beforehand. On the other hand, it has 3 drive modules and the rest only have 2. So as I unbox the nodes, I check to see if there's a drive in the right-hand bay. Murphy's law ensures it's actually the second-to-last box I unpack.

Once in the rack, I made sure to jot down the serial numbers for each server in order, since I'm performing remote administration. The default password for Apple's Server Admin tool is the first 8 characters of the serial number. I also want them in order in the rack, since node1 = rack position 1.

I've wired up all the Ethernet cables into my 48-port switch. The head server also gets an extra cable outside the rack -- it will serve as a DHCP (and other provider) for the internal network.

I also pull the head out of the rack, which I will set up first.

Just Starting

That's about it for this week -- I also set up my new desktop, filled out paperwork and other start-of-semester activities.

What I need to do right now is finish the setup of the head server. I mounted the server using a FireWire cable and Target Disk Mode. I've formatted the RAID and re-installed the server software.

However, to run the Server Assistant tool, I either need a small router (to put the Xserve and my Mac Pro on the same subnet with DHCP) or a VGA monitor to plug into the Xserve. I can't just plug the Xserve into a network jack -- the University only assigns static IP addresses. I also tried rebooting my Mac Pro from the server drive using Target Disk Mode, but that didn't work.

Any other ideas?