December 2, 2010

PoD at CERN's lxplus

Anar here...

Today I've been testing the latest PoD nightly on different resource management systems and decided to test it at CERN as well. I have CERN's lxplus always in my list of thinks to test on, because of AFS - you know it has some restrictions, like pipes and so on. So, it is always worth to play with PoD at CERN.

I was quite surprised that PoD deployed for me 50 PROOF workers within 30 seconds! I was using standard LSF queue, named "1nh". Besides, the LSF looks quite busy these days... and you know why ;)
But think about it, these are just regular LSF machines at CERN and you get your private PROOF cluster just in two clicks (or two commands) in a matter of seconds - dynamically...  ;)

Usually I use PoD command line interface, but just for demonstration's sake I used its GUI:

I mean, look, I get in less than 30 seconds exactly the same number of workers as you can get on CAF (where a standard "static" PROOF cluster is installed). Taking into account, that CAF allows to run more that one worker per core, my PoD/PROOF cluster on LSF will be much faster. So these 50 on LSF (one worker per core), which PoD made for me will process my analysis faster and even so I can use CAF machines as storage elements ;)

Let's make another test: 100 workers. This can't be even compared to CAF, because regular users can't get more than 52 ;)
Starting PoD using its CLI and submitting 100 workers:

40 seconds later:


Of course I understand, that the start up time on LSF or any other RMS depends on many factors, but even without pre-configuring RMS, I will be and I am able to get my PROOF cluster fast and on demand.
Additionally, I've advantages by using PoD, which are:
- it is my PROOF cluster (I don't have anyone who administrates it for me),
- LSF (or other RMS) takes care of resource sharing, distributing, logging, accounting and so on...,
- I am free to use this cluster at anytime I want with any ROOT version I want,
- I can't misuse the cluster, since RMS accounts it,
- etc.

In case you want to try the latest PoD at CERN, use the latest nightly or stable release.  For users' convenience I also provide some environment settings for lxplus, check the "PoD at CERN's AFS" section.
I would of course recommend trying the latest nightly, even so it still has some bugs, but you can help to evaluate it ;)

No comments: