PoD 2.1.1 on AFS

Yesterday, PoD has been tested at CERN.

We used PoD 2.1.1 and LSF plug-in. The LSF plug-in so-far only supports a shared file system on LSF, the installation therefore was done on AFS.

So, as it was expected ;) PoD worked out of the box! We submitted about 30 jobs. The start-up time for LSF jobs was very vast. I think we got all our workers within 20 seconds.

Now I can confirm, PoD (with LSF plug-in) has been officially tested at CERN.

The only problem we faced was the restriction of AFS.

New version of PoD creates a named-pipe - it is used in the latest thread pool algorithm of PoD. Unfortunately we found out that AFS doesn't support pipes:

Subject: 2.13  Can I create a fifo (aka named pipe) in /afs?

No. AFS does not support "mknod fifofile p".

It is actually very simple to fix in PoD. The patch is already in development and will be released very soon.

Have a nice day!

ALICE Offline week

Anar here ...

This week I gave a PoD presentation on the ALICE offline week.

My slides you can find here ;)

If everything goes as planned, PoD will take place of GSI's static PROOF cluster and will be used on daly basis as a default way of processing PROOF analysis. At least we are going in that direction currently.

Will keep you posting on this one...

PoD 2.1.1

Finally a new version of PoD has been released.

This time PoD got lots of improvements and most of all the pod-agent - a core module, has been upgraded and now made a use of the thread pool pattern. This gives a much smaller virtual memory footprint and also improves the performance by a huge factor in a network intensive operations (such as a file list validation when thousands of files are in the list and a merging) when many worker nods are used.

All users are kindly asked to stop using all prevues versions of PoD, including betas and move to v2.1.1.

The first reason is that this version is a factor of 10 faster in network intensive operations (like file validation and merging) and a virt. memory footprint of it is now reduced by a factor of 14 in compare to the prevues release + many many more improvements.

The second, since I am the only developer I can't give much support for older versions. So, if you have a problem with PoD the first question would be, "are you using the latest one?"... ;)

BTW, starting from this version, PoD generates a helper header file. The header should help you to automat a PROOF connection string in your scripts.

Read more in user's manual.

There are many more other features added and fixed. You can find more information in the Release Notes.

Find more details on PoD Trac.

PoD 2.0.10 is out

I've just released PoD 2.0.10.

This is a patch release, it addresses a hanging-on-merging issue (which could happen with PoD 2.0.8-2.0.9) and brings some performance improvements.

Find more details on PoD Trac.

PoD 2.0.8 has been released

Just a minute ago I have released version 2.0.8.

The main goal of this development was to reduce resource requirements PoD server need to handle its workers, in terms of a number of threads and memory. It looks like the task was successfully accomplished. Resource requirements were reduced by a factor of 3.

Now, since GSI ALICE group started to use PoD, PoD is tested with an ALICE PROOF-tutorial analysis (an example of a standard ALICE PROOF analysis) using 80 dynamic PROOF workers on LSF batch system.

Find more details on PoD Trac.

Release 2.0.8 2009-06-16

- Switch to $POD_LOCATION instead of $GLITE_PROOF_LOCATION.

- Using PROOFAgent's new configuration file.

- A better handling of multiple PoD workers on the same worker node started in the same time.

- Many minor fixes.

Submodules:

=== GAW v3.2.0

=== PROOFAgent v2.0.0 (2009-06-15) with the following changes:

   - Dramatically reduced a number of threads, which PA produces (on both server and client sides).

   - A slight redesign of the packet forwarder. PA now requires a factor of 3 less resources to handle each worker.

   - Removed the dependency on XERCES-C library.

   - The configuration file has been redesigned. Now using an INI style.

   - The build system has been switched from GNU Autotools to cmake.

   - Due to many bugs in FindBOOST of older versions of cmake we require cmake 2.6.4.

   - Switched from $GLITE_PROOF_LOCATION to $POD_LOCATION.

   - New configuration parameter - client.shutdown_if_idle_for_sec. Shut down a worker if its idle time is higher than this value.

   - Cosmetic changes.

=== PAConsole v1.0.7 (2009-06-15) with the following changes:

   - Source code cleaning.

   - Switched from $GLITE_PROOF_LOCATION to $POD_LOCATION.

   - Using new PROOFAgent's configuration file.

   - Fixed a case when several users run a PoD server and PAConsole couldn't recognize properly whether PoD runs or not.

   - LSF plug-in:

      * use LSF's queue information to limit the maximum number of workers in GUI.