Anar here...
Today I've been testing the latest PoD nightly on different resource management systems and decided to test it at CERN as well. I have CERN's lxplus always in my list of thinks to test on, because of AFS - you know it has some restrictions, like pipes and so on. So, it is always worth to play with PoD at CERN.
I was quite surprised that PoD deployed for me 50 PROOF workers within 30 seconds! I was using standard LSF queue, named "1nh". Besides, the LSF looks quite busy these days... and you know why ;)
But think about it, these are just regular LSF machines at CERN and you get your private PROOF cluster just in two clicks (or two commands) in a matter of seconds - dynamically... ;)
Usually I use PoD command line interface, but just for demonstration's sake I used its GUI:
I mean, look, I get in less than 30 seconds exactly the same number of workers as you can get on CAF (where a standard "static" PROOF cluster is installed). Taking into account, that CAF allows to run more that one worker per core, my PoD/PROOF cluster on LSF will be much faster. So these 50 on LSF (one worker per core), which PoD made for me will process my analysis faster and even so I can use CAF machines as storage elements ;)
Let's make another test: 100 workers. This can't be even compared to CAF, because regular users can't get more than 52 ;)
Starting PoD using its CLI and submitting 100 workers:
40 seconds later:
:D
Of course I understand, that the start up time on LSF or any other RMS depends on many factors, but even without pre-configuring RMS, I will be and I am able to get my PROOF cluster fast and on demand.
Additionally, I've advantages by using PoD, which are:
- it is my PROOF cluster (I don't have anyone who administrates it for me),
- LSF (or other RMS) takes care of resource sharing, distributing, logging, accounting and so on...,
- I am free to use this cluster at anytime I want with any ROOT version I want,
- I can't misuse the cluster, since RMS accounts it,
- etc.
In case you want to try the latest PoD at CERN, use the latest nightly or stable release. For users' convenience I also provide some environment settings for lxplus, check the "PoD at CERN's AFS" section.
I would of course recommend trying the latest nightly, even so it still has some bugs, but you can help to evaluate it ;)
No comments:
Post a Comment