1. Port POP from Linux to Windows
For many HPC applications, porting from Linux to Windows is as straightforward as recompiling the application source code with a special toolchain. More complex applications may require a little more work, but even complex HPC applications (such as OpenFOAM) can be ported in this way.
This step of the tutorial will guide you through the process of recompiling POP with the PToolsWin development environment. PToolsWin generates native Windows code (no intermediate POSIX layer is required), so your application will perform as well as a native Windows application.
You will recompile recompile POP to produce a Windows executable and then copy the POP executable and run folder to Windows in preparation for uploading them to a Windows Azure Storage service.
Before you continue, make sure you have these software prerequisites:
1.2. Build POP as a Windows Executable
- Open a command line in your HPC Linux distro and load the PToolsWin module:
module load ptoolswin module list
- Create your POP directory:
setenv POPDIR $HOME/windows mkdir -p $POPDIR
Download and extract the POP source code from LANL to $POPDIR:
cd $POPDIR wget http://climate.lanl.gov/Models/POP/POP_2.0.1.tar.Z tar xvzf POP_2.0.1.tar.Z
- Execute the setup_run_dir script in the pop directory to create and populate a new POP run directory named “windows”:
cd $POPDIR/pop ./setup_run_dir windows
When we compiled NetCDF for Windows, we passed the --host=x86_64-w64-mingw32 flag to the configure script to indicate that the MinGW cross compilers should be used instead of the native compilers. This is the preferred method of cross compiling an application that uses the GNU Autoconf build system. Autoconf is very common among Linux applications, but there are some applications (like POP!) that do not use it, and hence do not have a configure script. POP uses a custom build system that gets its configuration from a file with a .gnu extension.
We have created a customized .gnu file for cross compilation, starting with linux.gnu as the template. Download ptoolswin.gnu to $POPDIR/pop/windows:
cd $POPDIR/pop/windows wget http://paratools.com/Azure/POP/Step2/ptoolswin.gnu -O ptoolswin.gnu
- If you compare ptoolswin.gnu with linux.gnu you can see what changes are required to switch from native compilation to cross compilation:
--- linux.gnu 2012-03-23 14:41:01.309869158 -0700 +++ ptoolswin.gnu 2012-03-23 08:36:37.000000000 -0700 @@ -1,5 +1,5 @@ # -# File: linux.gnu +# File: ptoolswin.gnu # # The commenting in this file is intended for occasional maintainers who # have better uses for their time than learning "make", "awk", etc. There @@ -8,7 +8,7 @@ # FC = mpif90 LD = mpif90 -CC = cc +CC = gcc Cp = /bin/cp Cpp = /lib/cpp -P AWK = /usr/bin/gawk @@ -21,8 +21,9 @@ # Adjust these to point to where netcdf is installed -NETCDFINC = -I/netcdf_include_path -NETCDFLIB = -L/netcdf_library_path +NETCDFDIR = $(HOME)/windows/netcdf-4.1.3 +NETCDFINC = -I$(NETCDFDIR)/include +NETCDFLIB = -L$(NETCDFDIR)/lib # Enable trapping and traceback of floating point exceptions, yes/no. # Note - Requires 'setenv TRAP_FPE "ALL=ABORT,TRACE"' for traceback. @@ -78,10 +79,10 @@ # #---------------------------------------------------------------------------- -LDFLAGS = $(ABI) -v +LDFLAGS = $(ABI) -v -Wl,--force-exe-suffix #LIBS = $(NETCDFLIB) -lnetcdf -lX11 -LIBS = $(NETCDFLIB) -lnetcdf +LIBS = $(NETCDFLIB) -lnetcdf $(NETCDFDIR)/lib/libnetcdf-7.dll $(NETCDFDIR)/lib/libnetcdff-5.dll ifeq ($(MPI),yes) LIBS := $(LIBS)
Notice that only a few changes are required. PToolsWin provides mpif90 and mpicc commands, so the only compiler change is to explicitly set the C compiler to the PToolsWin cross compiler. LDFLAGS has been updated to force a “.exe” suffix on the binary executable, and the NetCDF DLL files have been added to the linker command line arguments.
- Compile POP by setting the ARCHDIR environment variable and running “make”:
cd $POPDIR/pop/windows setenv ARCHDIR ptoolswin make
1.3. Copy POP and Required Libraries to Windows
Now that POP has compiled successfully, we need to gather together the POP executable and all its required files and transfer them to your Windows machine.
- Copy files from POP:
cd $POPDIR/pop/windows mkdir transfer cp pop.exe pop_in sample_* transfer
- Next, copy the NetCDF libraries and MinGW-w64 runtime libraries:
cp $NETCDFDIR/netcdf-4.1.3/lib/*.dll transfer cp /usr/local/pkgs/rts/* transfer
- POP depends on MPI, but we do not need to copy the Microsoft MPI libraries because they are already installed on the Windows host. All together, your transfer folder should look like this ($POPDIR is /home/livetau/windows):
[paratools07] 259 > pwd /home/livetau/windows/pop/windows/transfer [paratools07] 260 > ls -l total 26208 -rwxrwxr-x. 1 livetau livetau 594436 Mar 23 14:52 libgcc_s_sjlj-1.dll -rwxrwxr-x. 1 livetau livetau 9522373 Mar 23 14:52 libgfortran-3.dll -rwxr-xr-x. 1 livetau livetau 1823052 Mar 23 14:52 libnetcdf-7.dll -rwxr-xr-x. 1 livetau livetau 668674 Mar 23 14:52 libnetcdf_c++-4.dll -rwxr-xr-x. 1 livetau livetau 1239191 Mar 23 14:52 libnetcdff-5.dll -rwxrwxr-x. 1 livetau livetau 615374 Mar 23 14:52 libobjc-4.dll -rwxrwxr-x. 1 livetau livetau 1217969 Mar 23 14:52 libquadmath-0.dll -rwxrwxr-x. 1 livetau livetau 148957 Mar 23 14:52 libssp-0.dll -rwxrwxr-x. 1 livetau livetau 8454837 Mar 23 14:52 libstdc++-6.dll -rw-rw-r--. 1 livetau livetau 2357009 Mar 23 14:51 pop.exe -rw-r--r--. 1 livetau livetau 8474 Mar 23 14:51 pop_in -rwxrwxr-x. 1 livetau livetau 47616 Mar 23 14:52 pthreadGC2-w64.dll -rw-r--r--. 1 livetau livetau 41 Mar 23 14:51 sample_history_contents -rw-r--r--. 1 livetau livetau 54 Mar 23 14:51 sample_movie_contents -rw-r--r--. 1 livetau livetau 231 Mar 23 14:51 sample_tavg_contents -rw-r--r--. 1 livetau livetau 123 Mar 23 14:51 sample_transport_file -rwxrwxr-x. 1 livetau livetau 90112 Mar 23 14:52 zlib1.dll
- Create a zip file from the contents of the transfer folder:
cd $POPDIR/pop/windows/transfer zip -r $POPDIR/pop.zip *
Your $POPDIR/pop.zip file should be approximately 6.2M in size.
Verify that you have created pop.zip correctly by comparing your pop.zip file with ours. You can view the contents of our pop.zip file or you can download pop.zip and unpack it.
- If pop.zip looks correct, copy it to your local Windows installation. Don’t extract it yet. We will create a new Windows Azure service before we unpack pop.zip and upload it to a Windows Azure storage service.
You are now ready to proceed to Step 3: The Windows Azure HPC Scheduler.