Download statistics. Part 2.

Anar here...

Download statistics. Part 1.

I got some free time on the holidays and slightly updated the download statistics page of PoD, in order to make it a bit more convenient.

A little bit of effort can make a beautiful combination of a docbook Website technology and Google Visualization API. I learned how to introduce custom docbook elements to docbook Website schemas and this helped to properly marry it with Google charts.

Stats are updated automatically and, thanks to docbook technology, the page is generated automatically as well. All is going via a Git repository. The whole process, including publishing of the site, is completely automated.

I am so much in love with DOCBOOK!!!

Now, concerning the stats. Since June, 2010, PoD was downloaded 179 times from 13 countries.

56% of downloads are made from different locations in Germany. Not a surprise, since PoD was started in terms of the D-Grid project and was developed in Germany and the first stable version was used in GSI (Germany), where, for the first time in the world ;), a production "static" PROOF cluster was staged out in favor of PROOF via PoD.

I will be not tired to repeat, that these are very good results for such a specific product like PoD is.

Check the stats page out yourself ;)

Here you have some screenshots:

Bugzilla

Anar here...


Due to a growing number of users I set up a PoD Bugzilla service, which will help to track bugs and new features.

From now on, all bug reports and support/feature requests must go via PoD Bugzilla. These stuff I will not accept via emails.
Anyway Feel Free to express yourself and show what a wonderful product PoD is ;) All the rest via Bugzilla.

direct link
or
on PoD Website: http://pod.gsi.de/support.html  ---> Bug Reports and Feature Requests.

Condor plug-in for PoD

Anar here...

As it was promised, I am very happy to release today an early beta of a Condor plug-in for PoD.

Before saying anything else I would like to show a screenshot-demo :)

In order to test the plug-in I have used a Condor cluster at CERN (special thanks to Wen Guan, who has helped me with that).

Now the demo. 

I will show PoD CLI, since the Condor plug-in doesn't support GUI yet.

First of all, same as always, PoD server needs to be started:

than I submitted 6 PoD jobs, using a Condor job manager (this is done via the plug-in). I submit only 6, because we were in hurry and there are only 2 worker nodes on which my account was enabled.

After a few seconds we can check whether there are already PROOF workers online and how many of them:

That's it! Our dynamic PROOF cluster is ready:

So, from now on PoD supports Condor resource management system as well. Which makes it 6.

Six supported RMS.

PoD supports: SSH, LSF, PBS (PBSPro/OpenPBS/Torque), Grid Engine (OGE/SGE), Condor, and gLite.

All most popular RMS are supported. If you need PROOF for your users, just get one of these RMSs and PoD will handle the rest ;)

No need to administrate, to setup and support any PROOF cluster. Each user can setup its own PROOF cluster on the fly using PoD just on demand.

The stable release of PoD, which will be shipped with the Condor plug-in is scheduled for the first week of January, 2011.

I think there is no reason to be hurry and to do the release before the upcoming Holidays ;) We rather can use this time to do more tests and improve User's manual...

But you shouldn't wait until the stable is out. Try the Condor plug-in right now and help us to improve the product. If you want to check this release out, please use the latest nightly build:

http://pod.gsi.de/releases/pod/nightly/

Contact us in case of any problem: http://pod.gsi.de/support.html

Downloads by Country

Anar here ... with a bit of propaganda right now... :)

I am very glad to see that PoD is being used or at least evaluated in many countries already!

The statistics below shows the downloads of PoD source tarball for the last 6 months. It is only 166 downloads by about 12 countries. 

"It's a very small number" - would you say.  But don't forget, that for such a young and specific product like PoD, which works on a very specific field - it is a great success. Every new user is a success, I think.

Since PoD supports a shared installation feature now, I expect the download counts to dramatically reduce. Due to the fact, that in most cases PoD is and will be downloaded only once for a site and shared between several users on that site.

BTW, I also expect more customers from the USA to join, as soon as a Condor plug-in for PoD is released. The plug-in now in development and expected in an early beta in several days.

Anyway the downloads count doesn't really matter to us. It is just interesting to see how our product slowly gets maturer.

What is really important to us is to create a product, which will help, simplify and populate a PROOF-like-analysis.

New beta of PoD 3.x series

A new PoD development (beta) release of upcoming PoD v3.1 was issued today...

Most of the work in this beta revision was concentrated on the shared installation feature, plug-ins compatibility, and tests on different RMSs (including SSH plug-in).

BTW, today I've also installed the first central PoD at my home institution - GSI. Now users at GSI can either use a centrally installed PoD by using a simple login script or install any PoD version privately in their desired locations.

If you want to check this release out, please use the latest nightly build:

http://pod.gsi.de/releases/pod/nightly/

PoD at CERN's lxplus

Anar here...

Today I've been testing the latest PoD nightly on different resource management systems and decided to test it at CERN as well. I have CERN's lxplus always in my list of thinks to test on, because of AFS - you know it has some restrictions, like pipes and so on. So, it is always worth to play with PoD at CERN.

I was quite surprised that PoD deployed for me 50 PROOF workers within 30 seconds! I was using standard LSF queue, named "1nh". Besides, the LSF looks quite busy these days... and you know why ;)

But think about it, these are just regular LSF machines at CERN and you get your private PROOF cluster just in two clicks (or two commands) in a matter of seconds - dynamically...  ;)

Usually I use PoD command line interface, but just for demonstration's sake I used its GUI:

I mean, look, I get in less than 30 seconds exactly the same number of workers as you can get on CAF (where a standard "static" PROOF cluster is installed). Taking into account, that CAF allows to run more that one worker per core, my PoD/PROOF cluster on LSF will be much faster. So these 50 on LSF (one worker per core), which PoD made for me will process my analysis faster and even so I can use CAF machines as storage elements ;)

Let's make another test: 100 workers. This can't be even compared to CAF, because regular users can't get more than 52 ;)

Starting PoD using its CLI and submitting 100 workers:

 

40 seconds later:

:D

Of course I understand, that the start up time on LSF or any other RMS depends on many factors, but even without pre-configuring RMS, I will be and I am able to get my PROOF cluster fast and on demand.

Additionally, I've advantages by using PoD, which are:

- it is my PROOF cluster (I don't have anyone who administrates it for me),

- LSF (or other RMS) takes care of resource sharing, distributing, logging, accounting and so on...,

- I am free to use this cluster at anytime I want with any ROOT version I want,

- I can't misuse the cluster, since RMS accounts it,

- etc.

In case you want to try the latest PoD at CERN, use the latest nightly or stable release.  For users' convenience I also provide some environment settings for lxplus, check the "PoD at CERN's AFS" section.

I would of course recommend trying the latest nightly, even so it still has some bugs, but you can help to evaluate it ;)

News on PoD Server

Anar here...

I want to continue to keep you informed on ongoing development of the upcoming stable release of PoD - v3.1.

This time I'm going to cover a new stuff in PoD server implementation, which were done recently.

First of all, the server now starts 3 times faster. A logic of start/stop/restart functions was changed and improved.
I have also improved a server processes discovery algorithm. This algorithms is responsible to find out which of server processes run( uid, pid, and tcp port it listens on). PoD server is a lot more smarter now when handling its processes, even third-party ones, like xproofd and its company. For example, the server implementation got a two-stage-shutdown machinery, which first tries to gracefully shutdown all its processes and only if a process ignores it, the server will send a non-ignorable kill signal to it.
The server now also smartly kills the children of xproofd (like proofserv). It kills not all of children, just by name and user id, but rather it kills only children processes produced by the xproofd, which were started by exactly this PoD server. This brings us one step further in PoD and AliEn integration as well, since it is now possible to start several PoD servers under the same user id - which is a requirement for AliEn Grid. On AliEn all processes run under the same user id.

What is also new in the latest releases is that there is a better support for shared installations. Many minor bugs were fixed. The xproofd configuration, which PoD creates on the fly, is now handled a bit differently. Namely, it stored in user's working directory and renewed every time user restart pod server. This helps to use several PoD versions with the same user's configuration.

If you want to check the new PoD server out, please use the latest nightly build:
http://pod.gsi.de/releases/pod/nightly/