1. Cross-compiling WRF for Windows Azure

Windows Azure is a powerful cloud computing solution that can significantly improve your access to high performance computing (HPC) resources and applications. This tutorial will help you fit Windows Azure into your workflow by guiding you through the straightforward process of migrating a parallel HPC application from Linux to Windows Azure. Welcome! Soon you'll be running parallel HPC applications in the cloud.

We will cross-compile the Weather Research and Forecasting (WRF) model for Windows Azure from an HPC Linux installation. WRF is a next-generation mesoscale numerical weather prediction system designed to serve both operational forecasting and atmospheric research needs. It is widely used by #researchers and policymakers to better understand many phenomena like hurricanes, wildfires, and climate change.

The Azure Weather Service uses WRF and NOAA data to calculate interactive forecasts for hundreds of locations around the world. This forecast shows water vapor, surface wind and temperature over Baltimore, Maryland USA for approximately 24 hours on 20 March 2012. After this tutorial, you will be able to run similar simulations on Windows Azure.

Water Vapor over Baltimore, MD (16 March 2012, 24hrs) Wind and Surface Temperature over Baltimore, MD (16 March 2012, 24hrs)

MSDPE Logo This work was sponsored by the Microsoft Developer and Platform Evangelism Team.

1.1. Overview and Navigation

Use the slideshow navigation controls next to the page title and the links at the bottom of the page to navigate through the tutorial. Click here if you prefer to view the whole tutorial as a single, printable document.

This tutorial will take between two and four hours to complete and consists of the following steps:

  1. Prepare all required libraries

  2. Compile WRF for Windows Azure

  3. Compile WPS for Windows Azure

  4. Create a Virtual Hard Drive (VHD) for WRF

  5. Deploy a new Windows Azure compute cluster service

  6. Run WRF on Windows Azure

  7. Tutorial Summary and Downloads

Continue to Step1: Prepare all required libraries when you are ready to begin.


MSDPE Logo


2. Obtain Required Libraries

WRF uses NetCDF to read and write data files, so we'll need NetCDF for Windows before we can build WRF for Windows. You can follow the instructions below to build NetCDF from scratch, or you can go to the downloads section of the NetCDF page and download pre-built binaries.

2.1. Build NetCDF as a Windows DLL

  1. ParaTools developed the PToolsWin development environment specifically for porting Linux applications to Windows. PToolsWin is distributed as part of HPC Linux. Download and install HPC Linux, either in a virtual machine or natively.

  2. Open a command line in your HPC Linux distro and load the PToolsWin module:
       1 module load ptoolswin
       2 module list
    
  3. Create your NetCDF folder:
       1 setenv NETCDFDIR ${HOME}/windows
       2 mkdir -p $NETCDFDIR
       3 cd $NETCDFDIR
    
  4. Download and extract the NetCDF 4.1.3 source code from Unidata to $NETCDFDIR:

       1 wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.1.3.tar.gz
       2 tar xvzf netcdf-4.1.3.tar.gz
    
  5. There's a bug in NetCDF that prevents it from cross compiling. Fortunately, we've developed a patch. Download and apply this patch to the NetCDF source:

       1 wget http://www.paratools.com/Azure/NetCDF/paratools-netcdf-4.1.3.patch -O paratools-netcdf-4.1.3.patch
       2 cd netcdf-4.1.3
       3 patch -p1 < ../paratools-netcdf-4.1.3.patch
    
  6. Move the NetCDF source code to a sub-folder:
       1 mkdir src
       2 mv * src
       3 cd src
    
  7. NetCDF includes a configuration script that will automatically generate the makefiles we need. Configure NetCDF as follows. Be sure to type the whole command as one line:
       1 ./configure --prefix=${NETCDFDIR}/netcdf-4.1.3 --host=x86_64-w64-mingw32 --enable-dll --enable-shared --disable-netcdf-4 LDFLAGS="-Wl,--export-all-symbols,--enable-auto-import" CPPFLAGS=-DgFortran
    

    Let's take a closer look at this command. We are executing the configure script according to the standard GNU build process. The --prefix flag tells the configure script where to install files after they are compiled. --host=x86_64-w64-mingw32 is very important because it tells the configure script to use the MinGW compiler instead of the default GNU compiler. Without this flag, NetCDF would build for Linux, not Windows. --enable-dll and --enable-shared tell configure that we want shared libraries and that those libraries should be Windows DLL files. We use the --disable-netcdf-4 flag to disable obsolete functionality. By setting the LDFLAGS environment variable we are passing flags directly to the program linker. -Wl,--export-all-symbols,--enable-auto-import tells the linker to automatically make all NetCDF functions and subroutines accessible to programs that link against the NetCDF DLLs. Without these flags, the DLL files will be created but your programs will not be able to link with them. Adding "-DgFortran" to CPPFLAGS is required when building with gfortran on Linux.

  8. Once the configure script is finished, build and install NetCDF. (!) If you have multiple cores in your machine, you can reduce compilation time by passing the '-j<ncpus>' flag to make (e.g. 'make -j4').

       1 make
       2 make install
    
  9. Some versions of GNU Libtool insert invalid -l flags on the linker command line. If your build fails with something like this:

    ... cannot find -l-L/usr/local/pkgs/mingw-w64-bin_x86_64-linux_20110831/...
    collect2: error: ld returned 1 exit status
    then you will need to fix the postdeps line in libtool.
    1. Open ${NETCDFDIR}/netcdf-4.1.3/src/libtool in a text editor.

    2. Scroll to the very bottom of the file and locate the line that begins with "postdeps="
    3. Remove any invalid -l flags from that line. For example, if you see -l -L/usr/local/..., change it to -L/usr/local/...

    4. Once you're satisfied, save the file and run 'make' and 'make install' again.
  10. If everything went well you should see the following message:
    +-------------------------------------------------------------+
    | Congratulations! You have successfully installed netCDF!    |
    |                                                             |
    | You can use script "nc-config" to find out the relevant     |
    | compiler options to build your application. Enter           |
    |                                                             |
    |     nc-config --help                                        |
    |                                                             |
    | for additional information.                                 |
    |                                                             |
    | CAUTION:                                                    |
    |                                                             |
    | If you have not already run "make check", then we strongly  |
    | recommend you do so. It does not take very long.            |
    |                                                             |
    | Before using netCDF to store important data, test your      |
    | build with "make check".                                    |
    |                                                             |
    | NetCDF is tested nightly on many platforms at Unidata       |
    | but your platform is probably different in some ways.       |
    |                                                             |
    | If any tests fail, please see the netCDF web site:          |
    | http://www.unidata.ucar.edu/software/netcdf/                |
    |                                                             |
    | NetCDF is developed and maintained at the Unidata Program   |
    | Center. Unidata provides a broad array of data and software |
    | tools for use in geoscience education and research.         |
    | http://www.unidata.ucar.edu                                 |
    +-------------------------------------------------------------+
    If you do not see this message, review your commands for errors and try again.
  11. The final step is to link the DLL files from the bin directory to the lib directory:
       1 cd $NETCDFDIR/netcdf-4.1.3/lib
       2 ln -s ../bin/*.dll .
    
    This is necessary because Linux makefiles and build scripts expect program libraries to be in the lib directory, but Windows expects them to be in the same path as the program executables. Making the files accessible at both locations keeps both ends of the cross-compilation process happy.
  12. Here is what the NetCDF bin and lib folders look like after a successful compilation ($NETCDFDIR is /home/livetau/windows):

    [paratools07] 579 > ls
    bin  include  lib  share  src
    [paratools07] 580 > pwd
    /home/livetau/windows/netcdf-4.1.3
    [paratools07] 581 > ls
    bin  include  lib  share  src
    [paratools07] 582 > ls -l bin lib
    bin:
    total 5600
    -rwxr-xr-x. 1 livetau livetau 1823052 Mar 23 13:53 libnetcdf-7.dll
    -rwxr-xr-x. 1 livetau livetau  668674 Mar 23 13:53 libnetcdf_c++-4.dll
    -rwxr-xr-x. 1 livetau livetau 1239191 Mar 23 13:53 libnetcdff-5.dll
    -rwxr-xr-x. 1 livetau livetau    4334 Mar 23 13:53 nc-config
    -rwxr-xr-x. 1 livetau livetau  243830 Mar 23 13:53 nccopy.exe
    -rwxr-xr-x. 1 livetau livetau  381728 Mar 23 13:53 ncdump.exe
    -rwxr-xr-x. 1 livetau livetau  464561 Mar 23 13:53 ncgen3.exe
    -rwxr-xr-x. 1 livetau livetau  885521 Mar 23 13:53 ncgen.exe
    
    lib:
    total 4984
    lrwxrwxrwx. 1 livetau livetau      22 Mar 23 13:56 libnetcdf-7.dll -> ../bin/libnetcdf-7.dll
    -rw-r--r--. 1 livetau livetau 1996680 Mar 23 13:53 libnetcdf.a
    lrwxrwxrwx. 1 livetau livetau      26 Mar 23 13:56 libnetcdf_c++-4.dll -> ../bin/libnetcdf_c++-4.dll
    -rw-r--r--. 1 livetau livetau  806660 Mar 23 13:53 libnetcdf_c++.a
    -rwxr-xr-x. 1 livetau livetau  315174 Mar 23 13:53 libnetcdf_c++.dll.a
    -rwxr-xr-x. 1 livetau livetau    1090 Mar 23 13:53 libnetcdf_c++.la
    -rwxr-xr-x. 1 livetau livetau  411564 Mar 23 13:53 libnetcdf.dll.a
    lrwxrwxrwx. 1 livetau livetau      23 Mar 23 13:56 libnetcdff-5.dll -> ../bin/libnetcdff-5.dll
    -rw-r--r--. 1 livetau livetau 1262872 Mar 23 13:53 libnetcdff.a
    -rwxr-xr-x. 1 livetau livetau  233966 Mar 23 13:53 libnetcdff.dll.a
    -rwxr-xr-x. 1 livetau livetau    1308 Mar 23 13:53 libnetcdff.la
    -rwxr-xr-x. 1 livetau livetau     930 Mar 23 13:53 libnetcdf.la
    -rw-rw-r--. 1 livetau livetau   16212 Mar 23 13:53 netcdf_c++dll.def
    -rw-rw-r--. 1 livetau livetau   16952 Mar 23 13:53 netcdfdll.def
    -rw-rw-r--. 1 livetau livetau   11907 Mar 23 13:53 netcdffdll.def
    drwxrwxr-x. 2 livetau livetau    4096 Mar 23 13:53 pkgconfig
    [paratools07] 583 >

You are now ready to proceed to Step 2: Cross-compile WRF for Windows Azure.


MSDPE Logo


3. Cross-compile WRF for Windows Azure

For many HPC applications, porting from Linux to Windows is as straightforward as recompiling the application source code with a special toolchain. More complex applications may require a little more work, but even complex HPC applications can be ported in this way. Visit the PToolsWin page for more information.

This step of the tutorial will guide you through the process of recompiling WRF 3.3.1 with the PToolsWin development environment. PToolsWin generates native Windows code (no intermediate POSIX layer is required), so your application will perform as well as a native Windows application.

3.1. Prerequisites

Before you continue, make sure you have these software prerequisites:

  1. An installation of HPC Linux with PToolsWin.

  2. NetCDF for Windows.

3.2. Build WRF as a Windows Executable

  1. Open a command line in your HPC Linux distro and load the PToolsWin module:
       1 module load ptoolswin
       2 module list
    
  2. Begin by creating a directory for WRF:
       1 setenv WRFDIR ${HOME}/windows/WRF
       2 mkdir -p $WRFDIR
       3 cd $WRFDIR
    
  3. Download and unpack the WRF source code:
       1 wget http://www.mmm.ucar.edu/wrf/src/WRFV3.3.1.TAR.gz
       2 tar xzf WRFV3.3.1.TAR.gz
    
  4. Apply this patch to the WRF source code to fix a few known problems and add a new build configuration for PToolsWin:

       1 wget http://paratools.com/Azure/WRF/Step2/paratools-WRFV3.3.1.patch -O paratools-WRFV3.3.1.patch
       2 cd WRFV3
       3 patch -p1 < ../paratools-WRFV3.3.1.patch
    
  5. Configure your environment. WRF relies on several environment variables to locate NetCDF and select the appropriate configuration files:
       1 setenv NETCDF ${HOME}/windows/netcdf-4.1.3
       2 setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1
       3 setenv WRF_EM_CORE 1 
       4 setenv WRF_OS Windows
    
  6. Configure WRF. Select Windows 64-bit, PToolsWin (dmpar) (Option 7) and basic nesting (Option 1):

       1 ./configure
    
  7. Compile WRF 1. There is a race condition in the build process, so it's a good idea to do a second, serial compilation after the first:

       1 ./compile em_real |& tee compile.log
       2 ./compile -j 1 em_real |& tee compile.log.2
    

    Important

    Verify that the ptoolswin module is loaded before you begin compiling. You can view loaded modules with the command module list

  8. You should now have several program executables in the main folder:

    [paratools07] 136 > echo $WRFDIR
    /home/livetau/windows/WRF
    [paratools07] 137 > pwd
    /home/livetau/windows/WRF/WRFV3
    [paratools07] 138 > ls main/*.exe
    main/ndown.exe  main/nup.exe  main/real.exe  main/tc.exe  main/wrf.exe
    [paratools07] 139 > 
    

    If any of these .exe files are missing from main, check configure.log.2 for errors ( (!) do a case-insensitive search for "Error") and try re-compiling. Otherwise, continue to Step 3.

3.3. Notes

  1. Check the Compile module of the WRF ARW Online Tutorial for more information on compiling WRF. (1)


MSDPE Logo


4. Cross-compile WPS for Windows Azure

This step of the tutorial will guide you through the process of recompiling WPS 3.3.1 with the PToolsWin development environment. /!\ WRF 3.3.1 is required to build WPS, so do not attempt this step until you have completed Step2.

4.1. Prerequisites

Before you continue, make sure you have these software prerequisites:

  1. An installation of HPC Linux with PToolsWin.

  2. NetCDF for Windows.

  3. WRF for Windows

4.2. Build WPS as a Windows Executable

  1. Open a command line in your HPC Linux distro and load the PToolsWin module:
       1 module load ptoolswin
       2 module list
    
  2. Navigate to the WRF directory you created in Step2:

       1 setenv WRFDIR ${HOME}/windows/WRF
       2 cd $WRFDIR
    
  3. Download and unpack the WPS source:
       1 wget http://www.mmm.ucar.edu/wrf/src/WPSV3.3.1.TAR.gz
       2 tar xzf WPSV3.3.1.TAR.gz
    
  4. Apply this patch to the WPS source code to add a new build configuration for PToolsWin:

       1 wget http://paratools.com/Azure/WRF/Step3/paratools-WPSV3.3.1.patch -O paratools-WPSV3.3.1.patch
       2 cd WPS
       3 patch -p1 < ../paratools-WPSV3.3.1.patch
    
  5. Configure your environment with the same values as when you compiled WRF:
       1 setenv NETCDF ${HOME}/windows/netcdf-4.1.3
       2 setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1
       3 setenv WRF_EM_CORE 1 
       4 setenv WRF_OS Windows
    
  6. Configure WPS. Select Windows x86_64, PToolsWin, DM PARALLEL, NO GRIB2 (Option 5) 1:

       1 ./configure
    
  7. Compile WPS 2:

       1 ./compile |& tee compile.log
    

    Important

    Verify that the ptoolswin module is loaded before you begin compiling. You can view loaded modules with the command module list

  8. If the compilation was successful, you should see three symlinks in the $WRFDIR/WPS folder:
    [paratools07] 142 > echo $WRFDIR
    /home/livetau/windows/WRF
    [paratools07] 143 > pwd
    /home/livetau/windows/WRF/WPS
    [paratools07] 144 > ls -l *.exe
    lrwxrwxrwx. 1 livetau livetau 23 Mar 25 14:09 geogrid.exe -> geogrid/src/geogrid.exe
    lrwxrwxrwx. 1 livetau livetau 23 Mar 25 14:10 metgrid.exe -> metgrid/src/metgrid.exe
    lrwxrwxrwx. 1 livetau livetau 21 Mar 25 14:10 ungrib.exe -> ungrib/src/ungrib.exe
    [paratools07] 145 > 
    

    If any of these symlinks are missing or broken, check configure.log for errors ( (!) do a case-insensitive search for "Error") and try re-compiling. Otherwise, continue to Step 4.

4.3. Notes

  1. If you want to work with GRIB2 data then you will also need JasPer, libPNG, and zlib for Windows. Source code for these three packages is available here. Porting these libraries is fairly simple and is left as an exercise for the reader ;) Be sure to add the include file and library file paths for these libraries to your configure.wps file before you compile WPS. (1)

  2. Check the Compile module of the WRF ARW Online Tutorial for more information on compiling WPS. (2)


MSDPE Logo


5. Create a WRF Virtual Hard Drive (VHD)

To demonstrate some of the features of Windows Azure storage accounts, we will create a Virtual Hard Drive (VHD) file containing our newly compiled WRF files and the supporting data files. Windows Azure service instances (e.g. the nodes in our compute cluster service) can mount VHD files directly from a Windows Azure storage account. This is a great way to provide persistent storage for your projects and share large static files among compute nodes.

5.1. Prerequisites

Before you continue, make sure you have these software prerequisites:

  1. An installation of HPC Linux with PToolsWin.

  2. NetCDF for Windows.

  3. WRF compiled for Windows.

  4. WPS compiled for Windows.

5.2. Create a 20GB NTFS-formated VHD File

  1. Open a command line in your HPC Linux distro and load the PToolsWin module:
       1 module load ptoolswin
       2 module list
    
  2. Navigate to the directory you created for WRF:
       1 setenv WRFDIR ${HOME}/windows/WRF
       2 cd $WRFDIR
    
  3. Use the dd command to create a new fixed-size raw disk image file:

       1 dd if=/dev/zero of=image.raw bs=1G count=20
    

    You can set "bs" to be any block size you like in kilobytes (K), megabytes (M), gigabytes (B), or terabytes (T). "count" sets the image file size in blocks, so "bs=1G count=1" would create a file of one gigabyte, or "bs=10M count=10" would create a file of 100 megabytes. (!) VHD files may not be larger that two terabytes.

  4. Create a new NTFS filesystem on the raw disk image:
       1 /sbin/mkntfs -F -f image.raw
    
    The '-F' flag forces the filesystem creation, even though we're not creating on a regular disk. The '-f' flag causes mkntfs to do a fast format.
  5. Use VirtualBox to convert image.raw to a VHD file named wrf.vhd:

       1 VBoxManage convertfromraw image.raw wrf.vhd --format VHD --variant Fixed
    
    If you see this error message:
    VBoxManage: error: Cannot create the disk image "test.vhd": VERR_INVALID_PARAMETER
    then first try creating the file as a standard VHD file and then re-create it as a fixed file:
       1 VBoxManage convertfromraw image.raw wrf.vhd --format VHD
       2 rm image.vhd
       3 VBoxManage convertfromraw image.raw wrf.vhd --format VHD --variant Fixed
    

    (!) The VHD file must be in fixed format to be mountable from a Windows Azure storage account.

  6. We're done with the raw image file, so you can remote it to save space:
       1 rm image.raw
    

5.3. Mount and Populate the VHD File

  1. Create a mountpoint folder and use FUSE to mount the VHD file:

       1 cd $WRFDIR
       2 mkdir mnt
       3 ntfs-3g -o windows_names wrf.vhd mnt
    

    The -o windows_names flag prevents files, directories and extended attributes from being created with a name not allowed by Windows, either because it contains some illegal character (e.g. " * / : < > ? \ |) or because the last character is a space or a dot.

  2. Now any files you create in $WRFDIR/mnt will actually be created on the VHD file. Copy WRF and WPS to the VHD file:

       1 cp -R WPS WRFV3 mnt
    
  3. Unmount the VHD file:
       1 fusermount -u mnt
    
  4. Copy your VHD file to a Windows Azure storage account. If you want to copy the VHD file anywhere else, we recommend compressing it first since it is mostly zeros and will compress very well.

You are now ready to proceed to Step 5: Deploy a new Windows Azure compute cluster service. If you'd like to know more about working with VHD files in Linux and Windows Azure, see the VHD Files How-To.


MSDPE Logo


6. Deploy the Windows Azure HPC Scheduler

Windows Azure is a powerful and versatile cloud platform with extensive features and capabilities. Everything from big data applications to mobile apps and gaming are supported, but we are most interested in Azure's parallel HPC capabilities.

If you already have a Windows Azure compute cluster service up and running then skip to the next step. Otherwise, follow these instructions to use Windows Azure HPC Schduler to deploy a parallel computing cluster of four nodes (eight cores) as a Windows Azure service. You will install two Microsoft SDKs to your local Windows installation, open the Windows Azure Management Portal, and execute a Powershell script to deploy the cluster service.

7. Prerequisites

We'll begin by configuring your local Windows installation with the Windows Azure SDK and the Windows Azure HPC Scheduler SDK and acquainting you with the Windows Azure Management Portal. Follow the instructions below to prepare your Windows installation.

7.1. Windows Azure SDK version 1.6

The Windows Azure SDK gives developers the necessary tools to interface local applications with Windows Azure services. The full SDK for Microsoft .NET is required to use the tools presented in this tutorial.

  1. Go to the Windows Azure Developer Downloads site and click the "install" button to download the full install of the Windows Azure SDK for .NET:

    azuresdk_install_snip1.png

  2. Execute the downloaded file. You will be prompted three times to allow the operation. Click Allow, Allow, Yes to begin the installation.

  3. The Web Platform Installer may also prompt you to select the authentication mode for SQL Server Express. If this happens, choose Mixed Mode Authentication and set a password that you will remember. The SDK will take a few minutes to download and install.

  4. Click Finish and Exit to complete the installation and exit Web Platform Installer.

7.2. Windows Azure HPC Scheduler SDK version 1.6

The Windows Azure HPC Scheduler will help you to launch and manage parallel HPC applications on a Windows Azure service. Its main jobs are to make it easy to deploy and use a parallel computing cluster as a Windows Azure service and to manage the HPC jobs you execute on the cluster.

  1. Download the Windows Azure HPC Scheduler SDK 64-bit installer from the Microsoft Download Center:

    azurehpcsched_install_snip1.png

  2. Execute the downloaded file. You will be prompted to allow the installation. Install the SDK in the default location.

7.3. The Windows Azure Management Portal

Windows Azure Management Portal is the seat of power for your Windows Azure subscriptions. From the portal, you can perform service deployment and management tasks and monitor the overall health of your deployments and accounts.

  1. Microsoft Silverlight is required to use the portal. Go to http://www.microsoft.com/silverlight to install Silverlight in your browser.

  2. Go to http://windows.azure.com to log on to the Windows Azure Management Portal. Be sure to use the same Windows Live ID and password that you used to create your Windows Azure subscription.

    Important

    Don't forget the dot '.' between "windows" and "azure" in the above URL. If you do, you will be taken to the Windows Azure Homepage, not the Windows Azure Management Portal.

  3. We'll be referring to the portal throughout this part of the tutorial, so stay logged in and keep the browser window open.

8. Deploy Windows Azure HPC Scheduler via Powershell

Deploying the Windows Azure HPC Scheduler involves creating Windows Azure components:

  • A Hosted Service containing six role instances:
    • One HeadNode instance to schedule, manage, and coordinate the parallel deployment,

    • Four ComputeNode instances to provide computational resources,

    • And one FrontEnd instance to provide web-based job submission and monitoring.

  • A Storage Account to hold state information for the cluster and provide a persistent shared storage location to all role instances,
  • And finally, a SQL Azure Database for maintaining scheduler configuration and state information.

Azure Deployment Diagram

All these components can be created easily with Microsoft Visual Studio Professional or via a Powershell script. We will use the Powershell approach because it has fewer prerequisites and is better suited to our needs in this tutorial.

  1. Download the PowerShell script package to your Windows installation and extract the zip file to create the WAHSPowershellDeployment folder.

  2. Double-click on Setup.cmd. Click Allow when you are prompted to allow Setup.cmd to make changes to your computer and you will be presented with a Powershell prompt in the WAHSPowershellDeployment\Code folder. We will refer to this window throughout this part of the tutorial.

  3. Execute the following command in the Powershell window:
    .\Deploy-HPCWindowsAzureScheduler.ps1
  4. Enter azuretutorial.csdef when you are prompted for InputCSDef and azuretutorial.cscfg when you are prompted for InputCSCfg:

    Supply values for the following parameters:
    InputCSDef: azuretutorial.csdef
    InputCSCfg: azuretutorial.cscfg
    We'll learn more about these two files later on.
  5. Enter your Subscription ID when you are prompted for SubscriptionID. To locate your Subscription ID, go to the Windows Azure Management Portal and click on Hosted Services, Storage Accounts & CDN, Hosted Services, and find the subscription you want to use. Your Subscription ID is in the properties pane on the right:

    azuresubscriptid_snip1.png

    Copy the Subscription ID by highlighting it with your mouse and pressing Ctrl+C. Return to the Powershell Prompt and paste it by right-clicking and selecting Paste. Press Enter to continue.

  6. Enter 0 when you are prompted for your Windows Azure management certificate to generate a new self-signed certificate, or, if you installed a certificate earlier, type the number of the certificate you would like to use.

  7. If you generated a new certificate you will need to install it. In the Management Portal, click on Hosted Services, Storage Accounts & CDN, Management Certificates, and find the subscription you wish to use. Click on Add Certificate.

    azureportal_snip2.png

    Tip

    Certificates and their use in Windows Azure is a detailed topic. See Overview of Certificates in Windows Azure to learn more.

  8. Check the Powershell window to find the location of the newly-generated self-signed certificate:

    azure_certpath_snip1.png

  9. In the Management Portal, browse to the certificate file and click OK. After a few seconds, your new certificate will appear on the list of management certificates. Return to the Powershell console and press Enter to continue.

  10. Enter 0 to create a new hosted service. Specify a name for the new hosted service and wait for the new hosted service to be created.

    The hosted service name must be less than 15 characters long

    The script will not check the length of your hosted service name, so be careful to not use more than 15 characters. If your hosted service name is too long, the script will fail later on.

  11. Enter Y when you are prompted to create a new certificate for password encryption and SSL. Wait a few seconds for the new certificate to be created.

  12. Enter 0 when you are prompted to select a storage account. Specify a name for your new storage account and wait for the new storage account to be created.

  13. Enter 0 when you are prompted to select a SQL Azure server. Enter a name for the SQL Azure server administrator (e.g. admin) and a password for the administrator account. The password must contain at least three of the following four classes:

    • Uppercase alphabetical (e.g. ABCDEFG...)
    • Lowercase alphabetical (e.g. abcdefg...)
    • Numeric (e.g. 123456789)
    • Non-alphanumeric (e.g. !@#$%.;[])
  14. Enter 0 when you are prompted to select a database. Specify the name of your database, and again enter the new administrator name and password. Wait while your database is initialized.

  15. The script will now begin the deployment process by uploading files to a Windows Azure storage container and creating role instances. This may take up to an hour to complete, so be patient. In the meantime, let's learn more about how we define the deployment topology.

    There are two files that work together to define our cluster: a service definition file with a .csdef extension and a service configuration file with a .cscfg extension. Earlier, we specified these files as azuretutorial.csdef and azuretutorial.cscfg. The service definition file defines the Windows Azure roles (e.g. HeadNode, ComputeNode, etc.) that our service will use. Here is a copy of azuretutorial.csdef:

       1 <?xml version="1.0" encoding="utf-8"?>
       2 <ServiceDefinition name="TestService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
       3   <WebRole name="FrontEnd" vmsize="Small">
       4     <Sites>
       5       <Site name="HPC">
       6         <VirtualApplication name="Portal" />
       7         <Bindings>
       8           <Binding name="HPCWebServiceHttps" endpointName="Microsoft.Hpc.Azure.Endpoint.HPCWebServiceHttps"/>
       9         </Bindings>
      10       </Site>
      11     </Sites>
      12     <Endpoints>
      13     </Endpoints>
      14     <Imports>
      15       <Import moduleName="Diagnostics" />
      16       <Import moduleName="RemoteAccess" />
      17       <Import moduleName="HpcWebFrontEnd" />
      18     </Imports>
      19     <ConfigurationSettings>
      20     </ConfigurationSettings>
      21     <LocalResources>
      22       <LocalStorage name="WFELocalStorage" cleanOnRoleRecycle="false" />
      23     </LocalResources>
      24     <Startup>
      25     </Startup>
      26   </WebRole>
      27   <WorkerRole name="HeadNode" vmsize="Medium">
      28     <Imports>
      29       <Import moduleName="Diagnostics" />
      30       <Import moduleName="RemoteAccess" />
      31       <Import moduleName="RemoteForwarder" />
      32       <Import moduleName="HpcHeadNode" />
      33     </Imports>
      34     <ConfigurationSettings>
      35     </ConfigurationSettings>
      36     <Endpoints>
      37     </Endpoints>
      38   </WorkerRole>
      39   <WorkerRole name="ComputeNode" vmsize="Medium">
      40     <Imports>
      41       <Import moduleName="Diagnostics" />
      42       <Import moduleName="RemoteAccess" />
      43       <Import moduleName="HpcComputeNode" />
      44     </Imports>
      45     <ConfigurationSettings>
      46     </ConfigurationSettings>
      47   </WorkerRole>
      48 </ServiceDefinition>
    

    As you can see, we've defined a small HpcWebFrontend role named "FrontEnd", a medium HpcHeadNode role named "HeadNode", and a medium HpcComputeNode role name "ComputeNode". The size of the node corresponds directly to the Windows Azure Virtual Machine Size and ranges from "extra small" to "extra large". Notice that the service configuration file does not say how many instances of each role we will create. Our cluster's configuration parameters are defined in azuretutorial.cscfg:

       1 <?xml version="1.0" encoding="utf-8"?>
       2 <ServiceConfiguration serviceName="TestService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*">
       3   <Role name="FrontEnd">
       4     <Instances count="1" />
       5     <ConfigurationSettings>
       6       <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
       7     </ConfigurationSettings>
       8   </Role>
       9   <Role name="HeadNode">
      10     <Instances count="1" />
      11     <ConfigurationSettings>
      12       <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
      13     </ConfigurationSettings>
      14   </Role>
      15   <Role name="ComputeNode">
      16     <Instances count="4" />
      17     <ConfigurationSettings>
      18       <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
      19     </ConfigurationSettings>
      20   </Role>
      21 </ServiceConfiguration>
    

    Here we've configured one instance of the FrontEnd role, one instance of the HeadNode role, and four instances of the ComputeNode role. The Powershell script parses the two configuration files and creates Windows Azure Nodegroups to correspond to each role. If in future you wish to change the topology of your cluster, simply edit these two files accordingly and re-run the Powershell script.

  16. Your Powershell console should look something like this when the deployment is complete:

    azure_complete_snip1.png

    Go to the Management Portal and click on Hosted Services, Storage Accounts & CDN. Select your subscription and find the Windows Azure HPC Scheduler Service to see all the node instances in your new compute cluster:

    azureportal_snip3.png

You are now ready to proceed to Step 6: Run WRF on Windows Azure


MSDPE Logo


9. Run POP on Windows Azure

You are ready to run WRF on Windows Azure! To do that, we will need to mount the VHD file from your Windows Azure storage account and submit a new compute job to the cluster. Jobs are submitted from a Command Prompt window on the cluster head node.

In this part of the tutorial, you will connect to the cluster's head node via Remote Desktop Connection (RDC). You will use hpcpack to mount the WRF VHD file as a drive, and configure the cluster's firewall for MPI communication.

9.1. Open a Remote Desktop Connection to the Head Node

  1. Go to http://windows.azure.com to log on to the Windows Azure Management Portal. Be sure to use the same Windows Live ID and password that you used to create your Windows Azure subscription. /!\ Be careful to include the dot '.' between windows and azure if you type the URL. If you omit the dot, you will go to the Windows Azure Homepage, not the Windows Azure Management Portal.

  2. In the Management Portal, click on Hosted Services, Storage Accounts & CDN, Hosted Services. In the main window, expand the Windows Azure HPC Scheduler Service and expand the HeadNode role. Select the HeadNode_IN_0 role instance and verify that the instance's status is set to Ready.

    Azure/HowToRDC/azureportal_snip4.png

  3. If the instance is not yet ready, wait until it becomes ready before continuing. It may take several minutes for the instance to transition to the ready state.
  4. With HeadNode_IN_0 selected and ready, click Connect on the ribbon bar. Click Open when you are prompted to download the .rdp file.

  5. The .rdp file will open in Remote Desktop Connection. Don't worry if you receive a warning that the remote connection cannot be identified, just click Connect to continue. Enter your administrator password when prompted and click OK. Wait for the connection to be established.

  6. Remote Desktop Connection may warn that the identity of the remote computer cannot be verified. If this happens, check the box next to Don't ask me again for connections to this computer and click Yes.

    Azure/HowToRDC/rdc_snip3.png

  7. The Remote Desktop Connection will open, presenting you with a view of the desktop on the head node. We'll be using this window throughout this part of the tutorial so keep it open.

9.2. Mount the WRF VHD File as a Drive

  1. In order to access the VHD file in your storage account, you will need your storage account access key. To locate your access key, return to the Windows Azure Management Console. Click on Hosted Services, Storage Accounts & CDN, Storage Accounts, and select your storage account. Click the View button in the Primary access key panel of the properties pane:

    azureportal_snip5.png

  2. Click the Copy to Clipboard button next to your storage account's primary key and click Close:

    storagekeys_snip1.png

    Tip

    For more information about Windows Azure storage keys, see How to View, Copy, and Regenerate Access Keys for a Windows Azure Storage Account.

  3. Return to the Command Prompt on the head node in the Remote Desktop Connection window. Type the following command on one line to mount the VHD file as a drive. Replace <account_name> and <storage_key> with your storage account name and key, respectively. Also replace <container> with the name of the blob container you placed wrf.vhd in. Since your storage account key has been copied to your clipboard, you can just right-click the Command Prompt window and select Paste when you need to enter your key.

       1 hpcpack mount wrf.vhd /account:<account_name> /key:<storage_key> /container:<container>
    

    If the command succeeds, you will be informed of which drive letter image.vhd has been mounted as. Drive letter "F" is usually chosen 1.

  4. The contents of image.vhd are now available at the specified drive letter on the head node. Only the head node as permission to both read and write files in the VHD file, but you can grant read-only access to all compute nodes. Execute this command to mount the VHD file as a read-only drive on all cluster nodes. Replace <account_name> and <storage_key> with your storage account name and key, respectively. Also replace <container> with the name of the blob container you placed wrf.vhd in:

       1 clusrun /nodegroup:computenode hpcpack mount wrf.vhd /account:<account_name> /key:<storage_key> /container:<container> /snapshot
    
    If the command succeeds, you will see output similar to this:

    Azure/HowToVHD/clusrun_hpcpack_mount.png

9.3. Configure Firewall Rules for MPI Communication

Before we can run WRF, we must open the firewall for communication between the compute nodes. These steps must be repeated for any application that communicates across nodes.

  1. Open a Command Prompt window on the cluster head node.
  2. Type the following commands to open the firewall to WRF and WPS on all compute nodes. F: is typically the drive letter assigned to your VHD file when it is mounted, but be sure to use the correct drive letter.

       1 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WRFV3\main\ndown.exe
       2 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WRFV3\main\nup.exe
       3 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WRFV3\main\real.exe
       4 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WRFV3\main\tc.exe
       5 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WRFV3\main\wrf.exe
       6 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WPS\geogrid.exe
       7 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WPS\metgrid.exe
       8 clusrun /nodegroup:computenode hpcfwutil register wrf.exe F:\WPS\ungrib.exe
    
  3. If you are prompted for a password, enter your administrator password and enter Y to remember the password.
  4. Wait for the commands to finish. All compute nodes should return 0 to indicate success:

9.4. Use Command Prompt to Submit a New Job

You now have everything you need to run WRF on your Windows Azure compute cluster service. Running WRF is a multi-step process, and the actions you take will depend on how you intend to run WRF. Please visit the official WRF tutoral for step-by-step instructions on running WRF.

When you are ready to execute WRF, use the job command to submit a new job to the cluster2:

   1 job submit /jobname:WRF /nodegroup:computenode /numcores:8 mpiexec -np 8 -wdir E:\approot F:\WRFV3\main\wrf.exe

If the job submission is successful your new job will be assigned a number.

9.5. Notes

  1. You can learn more about hpcpack and associated commands in the [[http://technet.microsoft.com/en-us/library/cc972841%28WS.10%29.aspx|Windows HPC Server 2008 R2 Technical Reference (1)

  2. Jobs can also be submitted via the Windows Azure HPC Scheduler Web Portal. To reach the portal, navigate to https://<service_name>.cloudapp.net/portal where <service_name> is the name of your Windows Azure Hosted Service. (2)


MSDPE Logo


10. Summary

Congratulations! You have completed the tutorial. In this tutorial you:

  1. Built NetCDF for Windows

  2. Used PToolsWin and HPC Linux to cross-compile WRF

  3. Used PToolsWin and HPC Linux to cross-compile WPS

  4. Created a Virtual Hard Drive (VHD) for WRF

  5. Deployed a new Windows Azure compute cluster service

  6. Ran WRF on Windows Azure

We encourage you to view our other Windows Azure tutorials. Please contact us if you have any questions.

10.1. Downloads

Here are the binary files created during this tutorial. Please feel free to use and redistribute this software according to the appropriate software licenses.





MSDPE Logo