[RFC]Packaging MPI softwares

[Akane]

I'd like to package OpenMPI.

OpenMPI is one of MPI(Message Passing Interface:standard interface of parallel processing) software.
MPICH2, LAM-MPI, GridMPI and so on is a kind of it, too.
Each of these MPI software has mpicc, mpiexec, etc.(Same name, but different movement)

I think this may be problems.
For example, after loading openmpi.tcz, to load mpich2.tcz makes overwrote openmpi's mpicc.
Therefore, in this way, user must reboot to use openmpi's mpicc.

But IMHO this problem can be solved by packaging above MPI softwares as SCM, not TCZ.
As far as I know, SCM uses different install places(such as /apps/openmpi/bin) by every extension.
So, in above case, mpich2's mpicc don't overwrite openmpi's mpicc.

Is my idea right?
Requesting for comments.
Lastly, sorry for my non-native English.

Thanks for reading,
Akane.

[Rich]

Hi Akane
If I understand it correctly,  OpenMPI, MPICH2, LAM-MPI, and GridMPI  are different implementations of MPI
and probably would not normally be installed and run on the same machine. According to the OpenMPI
documentation located here:  http://www.open-mpi.org/faq/?category=running#simple-launch

Similar to many MPI implementations, Open MPI provides the commands mpirun and mpiexec to launch MPI jobs. Several of the questions in this FAQ category deal with using these commands.

Note, however, that these commands are exactly identical. Specifically, they are symbolic links to a common back-end launcher command named orterun (Open MPI's run-time environment interaction layer is named the Open Run-Time Environment, or ORTE -- hence orterun).

As such, the rest of this FAQ usually refers only to mpirun, even though the same discussions also apply to mpiexec and orterun (because they are all, in fact, the same command).

It sounds like commands like  mpiexec  are common to these different implementations to allow you to run a job
unmodified regardless of the MPI version you are using. If what I have said is correct, I would suggest you include
a sentence in the info file stating that only one version of MPI should be installed since the links will conflict. If
someone insists on installing multiple versions, they will simply have to correct the links as required.

[Rich]

Hi Akane
I think an SCM would have the same problem, except the links would be in  /apps/bin. See:
http://distro.ibiblio.org/tinycorelinux/arch_scm.html

[Akane]

Thank you for your kindness advice, Rich.

You understood me correctly.
Each of those MPI softwares has different libs, executables, headers, and so on.

If what I have said is correct, I would suggest you include
a sentence in the info file stating that only one version of MPI should be installed since the links will conflict.
If someone insists on installing multiple versions, they will simply have to correct the links as required.

But, I'd like to switch the MPI softwares by some commands; like Fedora's 'module' command.
According http://www.linuxcertif.com/man/1/module/370976/, the module command is to switch user environment(such as $PATH, /etc/ld.so.conf, and so forth).

module is a user interface to the Modules package. The Modules package provides for the dynamic modification of the user's environment via modulefiles.

Could you add a simplified module command (only changing $PATH and /etc/ld.so.conf) to Tiny Core?
In my cluster system, I'm using a shell script like the command which I made.

[Rich]

Hi Akane

Each of those MPI softwares has different libs, executables, headers, and so on.

Right, but they use the same names to run through links. I think the simplest solution is to set up a small script
for each version of MPI to restore the sym links, for example:
open-mpi-init

Code: [Select]

#!/bin/sh
ln -sf path/to/orterun /usr/local/bin/mpiexec
ln -sf path/to/orterun /usr/local/bin/mpirun
# Create all remaining links requiredThis should allow you to have multiple versions loaded, and switch between them by running a small  init  script
for the version you want to use.

[Akane]

That's nice script and system, Rich.

But, there still be problems.
MPICH2's and LAM-MPI's mpicc, mpiexec is normal executable file. It's not symlink.
So I think that such softwares should be installed to /usr/local/share/{progname}.
Such as /usr/local/share/mpich2-1.4.1p1, /usr/local/share/openmpi-1.6.1.
And add to their /usr/local/tce.installed like:

Code: [Select]

#!/bin/sh
# /usr/local/tce.installed/mpich2
EXTENSION_NAME=mpich2
EXTENSION_VER=1.4.1p1
#above environments can be in another file
echo /usr/local/share/$EXTENSION_NAME-$EXTENSION_VER >/tmp/mpi.pathls

then add to /etc/profile(or /etc/profile.d/mpipath.sh):

Code: [Select]

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
if [ -f /tmp/mpi.pathls ];then
    MPI=$(cat /tmp/mpi.pathls)
    PATH=$MPI/bin:$PATH
    if [ -d $MPI/sbin ];then
        PATH=$MPI/sbin:$PATH
    fi
fi

then create a script like this:

Code: [Select]

#!/bin/sh
# arrmpild.sh - arrange /etc/ld.so.conf from /tmp/mpi.pathls
if ! [ -f /tmp/mpi.pathls ];then
    echo "$0: /tmp/mpi.pathls not found." >&2
    exit 1
fi
. /etc/init.d/tc-functions
checkroot
cat <<END >/etc/ld.so.conf
/usr/local/lib
$(cat /tmp/mpi.pathls)
END
ldconfig


Now user can change the MPI softwares by

Code: [Select]

# whoami
root
# /usr/local/tce.installed/$MPINAME    #MPINAME can be openmpi, mpich2 etc.
# arrmpild.sh

How are my code?
I think that arrmpild.sh should be an extension which is depended by MPI softwares.

[Rich]

Hi Akane

But, there still be problems.
MPICH2's and LAM-MPI's mpicc, mpiexec is normal executable file. It's not symlink.
So I think that such softwares should be installed to /usr/local/share/{progname}.
Such as /usr/local/share/mpich2-1.4.1p1, /usr/local/share/openmpi-1.6.1.

And that's where the beauty of Tinycores architecture comes in. The tczs for each version would look like this:

Code: [Select]

/tmp/tcloop/openmpi-1.6.1/usr/local/bin/mpiexecand

Code: [Select]

/tmp/tcloop/mpich2-1.4.1p1/usr/local/bin/mpiexecAn openmpi script would be run to link  /tmp/tcloop/openmpi-1.6.1/usr/local/bin/mpiexec  to  /usr/local/bin/mpiexec
To switch to mpich2, the mpic2 script would be run and it would link  /tmp/tcloop/mpich2-1.4.1p1/usr/local/bin/mpiexec
to  /usr/local/bin/mpiexec  instead.

[Akane]

Thanks for your reply, Rich.
I am sorry that I did not reply to you sooner.

Well, I knew the beauty. That's functional!
So I wrote the scripts, arrmpild.sh which is in mpipathls.tcz.

Code: [Select]

#!/bin/sh
# arrmpils.sh
MPIPATHLS=/tmp/mpi.pathls

mpi_creat_symlinks{
    for f in $(find /tmp/tcloop/$1 -no -type d);do
        REALNAME=/$(echo $f | cut -d/ -f5-)
        mkdir -p $(dirname $REALNAME)
        ln -sf $f $REALNAME
    done
}

mpi_kill_symlinks{
    for f in $(find /tmp/tcloop/$1 -no -type d);do
        REALNAME=/$(echo $f | cut -d/ -f5-)
        rm $REALNAME
        if [ -z "$(dirname $REALNAME)/*" ];then
            rm -rf $(dirname $REALNAME)
        fi
    done
}

if ! [ -f $MPIPATHLS ];then
    echo "I work with a MPI software. Install it." >&2
    exit 1
fi

if [ -z "$1" ];then
    echo $(cat $MPIPATHLS) is available.
    exit 0
fi

mpi_kill_symlinks $(cat $MPIPATHLS)
mpi_creat_symlinks $1
echo $1 >$MPIPATHLS
sudo ldconfig

Also, each of MPI package should depend this extension.
And they should have /tmp/mpi.pathls which containts the name of package(e.g, openmpi).

Now, user can change the MPI package entering this command:

Code: [Select]

$ arrmpild.sh openmpi

How is my code?
Requesting for comments.

Regards,
Akane.

p.s.
Is it OK to use 'sudo'?

[Rich]

Hi Akane
I think you have the right idea, but I will offer a few comments.
Searching for $1 in /tmp/tcloop means you could match on any extension, not just the ones you are interested in. It
also means you wind up searching the whole tcloop directory tree unnecessarily.
For the openmpi extension, create a script called openmpi that contains a list of ln -sf commands to recreate the links
that conflict with other MPI extensions. Place that script under /tmp/MPIapps/ in the openmpi extension.
For the mpich2 extension, create a script called mpich2 that contains a list of ln -sf commands to recreate the links
that conflict with other MPI extensions. Place that script under /tmp/MPIapps/ in the mpich2 extension.
Now you can run your find command on the  /tmp/MPIapps/  directory, and if you find a match, just run that script.
Yes, it's OK to use  sudo  if you need to.

[Akane]

Thanks Rich.

I made some mistakes on the source code.
mpi_creat_symlinks and mpi_kill_symlinks are functions.

I think your thinkings are right.
But I think that typing "/tmp/MPIapps/openmpi" is bother than typing "arrmpild.sh openmpi".
So each of MPI packages should have "/tmp/MPIapps/$NAME" which containts only the package files.(not including "ln -sf")
E.g., /tmp/MPIapps/openmpi:

Code: [Select]

/usr/local/bin/mpicc
/usr/local/bin/mpic++
/usr/local/bin/mpiexec
/usr/local/bin/mpirun
...
/usr/local/lib/...
...
(Note that this file must not contents "/tmp/MPIapps/openmpi.)
Also, they should have a script under tce.installed to change the content of /tmp/mpi.pathls.
(It should not be symlink because the file under /tmp/tcloop is read-only.)

Now arrmpild.sh creates/removes some symlinks using the /tmp/MPIapps/$NAME.
And I think mpi_kill_symlinks is not needed, so deleted.

Here is the fixed code:

Code: [Select]

#!/bin/sh
# arrmpils.sh
MPIPATHLS=/tmp/mpi.pathls

mpi_creat_symlinks(){
    for f in $(cat /tmp/MPIapps/$1);do
        mkdir -p $(dirname /$f)
        ln -sf /tmp/tcloop/$1/$f /$f
    done
}

if ! [ -f $MPIPATHLS ];then
    echo "$0: No MPIs were installed." >&2
    exit 1
fi

CURRENTMPI=$(cat $MPIPATHLS)

if ! [ $1 ];then
    echo "$CURRENTMPI is available."
    exit 0
fi

if ! [ -f "/tmp/MPIapps/$1" ];then
    echo "$0: $1 is not tce-loaded." >&2
    exit 1
fi

if [ $CURRENTMPI = $1 ];then
    echo "$1 is already ready."
    exit 0
fi

mpi_creat_symlinks $1
echo $1 >$MPIPATHLS
sudo ldconfig

How are my code?
If it looks OK, then I'll package these systems sooner! (or later)

Thanks,
Akane.

[Rich]

Hi Akane

But I think that typing "/tmp/MPIapps/openmpi" is bother than typing "arrmpild.sh openmpi".

I was not suggesting to get rid of  arrmpild.sh, just some changes to it. By making  /tmp/MPIapps/openmpi
a script that creates the links, you eliminate the need for the  mpi_creat_symlinks  function. For example:
An extension called  openmpi-1.6.1.tcz
Has a script called  /tmp/MPIapps/openmpi
You make it the running version like this  arrmpild.sh openmpi
Now  arrmpild.sh  is simplified to:

Code: [Select]

#!/bin/sh
# arrmpils.sh
MPIPATHLS=/tmp/mpi.pathls

if ! [ -f $MPIPATHLS ];then
    echo "$0: No MPIs were installed." >&2
    exit 1
fi

CURRENTMPI=$(cat $MPIPATHLS)

if ! [ $1 ];then
    echo "$CURRENTMPI is available."
    exit 0
fi

if ! [ -f "/tmp/MPIapps/$1" ];then
    echo "$0: $1 is not tce-loaded." >&2
    exit 1
fi

/tmp/MPIapps/$1
sudo ldconfig
echo "$1 is ready"This is only a suggestion, there are many different ways of doing things.

[Akane]

Thanks Rich.

Your script is smarter than mine. Great!
But I think that writing /tmp/MPIapps/* like

Code: [Select]

/usr/local/bin/mpicc
/usr/local/bin/mpiexec
...
is simpler than

Code: [Select]

ln -sf /tmp/tcloop/openmpi/usr/local/bin/mpicc /usr/local/bin/mpicc
ln -sf /tmp/tcloop/openmpi/usr/local/bin/mpiexec /usr/local/bin/mpiexec
ln -sf ...
In this way, the code would be

Code: [Select]

#!/bin/sh
# arrmpils.sh
MPIPATHLS=/tmp/mpi.pathls

if ! [ -f $MPIPATHLS ];then
    echo "$0: No MPIs were installed." >&2
    exit 1
fi

CURRENTMPI=$(cat $MPIPATHLS)

if ! [ $1 ];then
    echo "$CURRENTMPI is available."
    exit 0
fi

if ! [ -f "/tmp/MPIapps/$1" ];then
    echo "$0: $1 is not tce-loaded." >&2
    exit 1
fi

for f in $(cat /tmp/MPIapps/$1);do
    sudo ln -sf /tmp/tcloop/$1/$f $f
done
sudo ldconfig
echo "Now $1 is ready."

I will make those extensions by this way.
Thank you for advising.

Regards,
Akane.

[Rich]

Hi Akane
Yes, that looks good too.