Pre-compiled binaries on worker nodes

Today I continue to give an overview of upcoming PoD 2.4 ;)

In this post I will cover a new behavior of PoD in terms of how it handles its workers now.

What we had before is that each PoD worker during the startup was checking environment of the machine it runs on and according to OS and CPU architecture tried to download a pre-compiled binaries of the needed PoD modules.

The binaries were downloaded from the central PoD repo. The binaries are needed, since PoD could run on different type of workers with different architectures. Talking about binaries, these are  only two modules needed so-far, but they are very important ones. Namely, pod-agent (core module of PoD) and pod-user-dsefaults (handles PoD settings). 

As it was pointed out by one of PoD users, this schema has many disadvantages and security issues. Indeed, having each worker (in each PoD session)  to download "some" binaries from the outside of a site or an institution is very inconvenient and insecure. This schema also makes it very difficult for users to provide their own binaries, if, for example, some type of architecture is missing from PoD central repo, but user want's to support it.

This is actually very old schema and was introduced at the time when PoD was called gLitePROOF (designed for gLite Grid MW).

Now, since PoD 2.3.84, I've changed this behavior and introduced a new schema. 

PoD workers don't download pre-compiled bins from the PoD central repository anymore. Now only PoD UI downloads them if they are missing from the UI. When a user starts PoD server ("pod-server start") a package with pre-compiled  binaries will be downloaded. That actually means, it will only be downloaded once to one machine and users could also provide own binaries, instead of using the PoD central repository. If a user will provide binaries, than PoD will not download anything from the central repository.

Additionally, PoD workers now try to find out whether a shared home file system is present and whether they can share the binaries with PoD UI/Server. If a shared FS is present, than workers will try to use UI's bins directly and if bins fail to start, workers will use pre-compiled bins from the worker package.  The new schema has resolved security issues and made PoD much more flexible. BTW, the start up time of the workers is also slightly reduced.

I'll keep you updated, so stay tuned!