Thursday, September 29, 2016

IceCube Winterover Training

I have been in Madison, WI for training for almost three weeks.  I am training to be what we call a Winterover.  This name comes from the fact that I will be spending a full year at the South Pole.  Specifically, I will be spending the winter there.  During the Antarctic summer, planes go in and out of the South Pole regularly, and by regularly, I mean at least once a week or more.  During the Antarctic winter, however, planes rarely if every fly to the South Pole.  As I understand it, they have actually only ever flown in three times during the winter, and that was for big emergency situations.  In general, once the last plane leaves for the season, you are stuck there for 6 months with almost no hope of getting out!  For that reason, you have to go through all the medical and psychological evaluations that I talked about in a previous post.

All that being said, my training now, is mostly on the technical side.  Many people think of this as a scientific position, and to some degree it is.  But the work I am going to be doing at the pole is mostly network and system administration.  The IceCube Neutrino observatory is a large experiment that collects a great deal of data.  There is so much data that we can't even get it all to the North through our satellite connections.  Instead, we have a small cluster of servers located at the South Pole that collects and processes the data to select out only the most interesting parts.  All of the data are saved and eventually shipped to the North on an airplane, but the most interesting data are sent through a satellite connection.  While I am down there, I will be working to fix any problems that arise in the computers or network that collects the data and does the processing.

For the first couple of weeks, my training was largely software based.  There are many different custom applications that process the data collected by the detector.   Even the hardware that collects the data has to be watched and occasionally fixed or upgraded.  There is software that converts all the data collected to useful information, software that filters out the stuff we don't care about, software that does calibration, and even software that moves and saves information and files to the right places whether it is to the satellite or to disks to be sent north later.  As a winterover, I have to know how these systems work and how to fix them if things break.  This could be as simple as rebooting a server or as complicated as reading through tons of log files to try to figure out problems that don't make any sense!  Thankfully, the systems are fairly well designed so it is usually not too much work to fix problems, and I should always have some support from the IceCube members up in the North to help with any really big problems.  Though, in the end, it is the winterovers who have to implement most of these solutions, which is why there is so much training.

This past week I have also been doing some network and system admin training.  I have a background in network administration, so I am not seeing anything too far outside of my experience.  The systems on this network are more advanced than the systems I have used in the past and there is way more thought put into the network as a whole, but the general concepts are pretty much the same.  That being said, I think this would be very difficult and confusing if I didn't have at least somewhat of a networking background.

If anyone was interested in one day working as a winterover for IceCube in the future, I would strongly recommend spending time learning Linux and general network and system administration.  These are the skills that seem to be the most valuable so far.  Spend time breaking and fixing computers.  Installing all sorts of software that other Linux users and system admins use from day to day and try to learn how to use it in interesting ways.  It doesn't hurt to have a PhD in physics, but it really helps to have a solid understanding of bash! (If you don't know what bash is, that might be a good place to start!) I am excited to be a part of IceCube in this capacity and to  see how things work from this perspective, and I am really looking forward to my trip down to the South Pole!

Wednesday, September 21, 2016

Preparing for the South Pole

There are a surprising number of things that are involved in going to Antarctica, and the things you have to do can change based on what the purpose of your trip is and where you will be located.   There are a few things, however, that are similar for almost everyone.

The first hurdle that many people deal with is the medical tests.  The tests vary depending on where you are going to be based and for how long.  So someone spending a year at pole, like me, has to go through a number of blood tests and physical examinations and even a psychological evaluation.  If there is a problem with any given test, you may or may not be able to fix it.  The process also can be complicated by the large number of individuals all applying at near the same time.  There is really only one organization that handles the medical approval for all of the "big" US bases in Antarctica, and it can take a while to get all of your medical documents processed.  Suffice it to say that when everything is cleared and you are PQed (Physically Qualified), it is a bit of a relief.

Additionally, many people visiting the South Pole or other parts of Antarctica go through specialized training.  This can be anything from training on the work you will be doing to team building and fire and trauma training for those spending the winter.  My training started about a week and a half ago.  As I will be working on the IceCube Neutrino Observatory, I have been training at the University of Wisconsin with many other scientists and engineers who work on the detector every day up here in the North to learn how to fix problems and do regular maintenance on the detector.  I have learned more about how all of the servers at the South Pole are laid out and interconnected and how all the hardware works. I did my PhD research using data from IceCube, but this training has given me a new perspective.  I am now able to "look under the hood" to see much more of the detail that goes into collecting that data.  In a couple weeks I will be starting my team building training in Denver.  I will update more about that after I am actually involved.

The last thing that immediately comes to mind is packing.  This can be a little tricky.  I'm sure most people think you have to go shopping for a bunch of warm clothes and find cold weather gear, but honestly the big things are provided for you.  There is still some cold weather clothing you need to get, but it's really not as much as you'd think.  The tricky part is trying to pack for a year away.  There aren't many shopping opportunities at pole.  You can actually buy things on Amazon and have them shipped down, at least until the end of the antarctic summer.  After that, planes don't land for about six months.  Though, I did hear an amusing story about this from a colleague who spent time at pole in the past.  He said that a few Amazon sellers would cancel orders because they thought they were fake!  For some reason they didn't actually believe there was a research station at the South Pole.  There is also a small store on station, but it doesn't have too much.  I went to the store on a short visit I took to the South Pole a few years ago, but I really don't remember too much of what they had beyond souvenirs.  All that being said, you have to pack with the understanding that it may be difficult if not impossible to get the items you need or want while you are there.  You have to pack clothes, toiletries, and even things to keep you entertained, though there will be more discussion about that in a later post.  It turns out there are some nice facilities at pole to help keep you from getting too bored. On top of all that, you have a baggage weight limit!  So you have to pack carefully.  This has been difficult, but as I said before, you can ship things down so that helps.

All in all, there is a great deal of preparation that goes into a big trip like this, and anyone looking to undergo a trip to the South Pole has some work to do ahead of time!

Sunday, September 18, 2016

The South Pole Adventure Begins!

I recently received some very exciting news and have been waiting to post the details for various reasons, not the least of which being that I have been very busy dealing with many new responsibilities.  As I posted in the past, I applied for a year long position at the South Pole.  Originally, I was not selected as one of the two primary candidates.  A little less than two weeks ago, I found out that one of the primaries would be unable to take the position, so I was asked to take his place.  Since that time, I have left my job in Birmingham and moved most of my belongings into storage.   Less than a week later, I started my training in Madison Wisconsin!

As I have stated I the past this blog was started to fulfill many purposes.  One of those was to serve as a journal for my trip to the South Pole, if I ended up getting the job. Now that I have the job, I hope to share my adventures with friends and family and anyone else who is interested.  I plan to update a few times a month with pictures, updates about what all is involved in my work at the South Pole, the science that we and others are doing down there, and even some personal things here and there.  If anyone has questions feel free to post or email them to me. I will try to answer as I am able, especially over the next few weeks of training.  After I "deploy" to the South Pole, the Internet will be limited, but I will still have access and will continue to post and answer questions, but it may take a little more time.  In the mean time, the posts may be a little more frequent so I can get as much information out as possible.