LA-UR-
Approved for public release;
distribution is unlimited.
Title:
Author(s):
Submitted to:
Form 836 (8/00)
Los Alamos National Laboratory, an affirmative action/equal opportunity employer, is operated by the University of California for the U.S.
Department of Energy under contract W-7405-ENG-36. By acceptance of this article, the publisher recognizes that the U.S. Government
retains a nonexclusive, royalty-free license to publish or reproduce the published form of this contribution, or to allow others to do so, for U.S.
Government purposes. Los Alamos National Laboratory requests that the publisher identify this article as work performed under the
auspices of the U.S. Department of Energy. Los Alamos National Laboratory strongly supports academic freedom and a researcher’s right to
publish; as an institution, however, the Laboratory does not endorse the viewpoint of a publication or guarantee its technical correctness.
Nuclear Mathematical and Computational Sciences: A Century in Review, A Century Anew
Gatlinburg, Tennessee, April 6-11, 2003, on CD-ROM, American Nuclear Society, LaGrange Park, IL (2003)
MCNP 5TM IMPROVEMENTS FOR WINDOWS PCS
Tim Goorley, Forrest Brown, and Lawrence J. Cox
Los Alamos National Laboratory
X-Division, X-5, MS F663
Los Alamos, NM 87545
jgoorley@lanl.gov, fbrown@lanl.gov, ljcox@lanl.gov
ABSTRACT
With the release of MCNP 5, much more emphasis has been placed on improving its functionality
on PCs running Microsoft Windowsâ (9X/NT/2000/XP). Enhanced installation and build
systems, support for more Fortran compilers, integration with X11 graphics build files, and MPI
and PVM parallel capabilities have been implemented in MCNP 5 for Windows PCs. MCNP 5
can be installed with an InstallShield setup programs, similar to other Windows programs, for
users who only need to install executables and data libraries. For those users who need to compile
the source, the GNU make utility can used in conjunction with three supported Fortran compilers:
Compaq Visual Fortran (CVF), Lahey Fortran 95 (LF95) or Absoft Fortran 95 (AF95).
Alternatively, Compaq Developer Studio can be used to compile MCNP 5. The X-Windows
plotting capabilities have been improved, and all the appropriate open source X11R6 files for
compiling MCNP 5 are bundled with the MCNP 5 source code. X client software is still needed,
however, to display geometry, cross-section or tally plots. Parallel capabilities which exist on
other platforms have been extended to Windows PCs, allowing users to utilize dual CPU PCs,
clusters of homogeneous Windows PCs (preferably with MPI), or heterogeneous clusters
(preferably with PVM). Wall-clock runtimes show that MCNP 5 compiled with CVF runs 1.4
times faster than when it is compiled with LF95 or AF95. Wall-clock runtimes also show MCNP 5
with MPI and the balance keyword more effectively utilizes a dual-processor and a small
heterogeneous cluster Windows 2000 PC than MCNP 5 with PVM.
Key Words: MCNP 5, Microsoft Windows, Parallel, MPI, PVM
1. INTRODUCTION
MCNP Version 5[1,2,3]1, a general purpose radiation transport computer code, has been released
to the Radiation Safety Information Computational Center[4]. MCNP is developed and
supported by the Eolus team at Los Alamos National Laboratory (LANL) to support Accelerated
Strategic Computing Initiative (ASCI) projects. With this release, MCNP has been rewritten into
Fortran 90 and several new features have been added. These new features inc lude a new build
and installation process, color plotting enhancements, Doppler energy broadening for photons, a
neutral particle radiography imaging capability, improved source options, time importance
variance reduction, support for parallel communications with MPI, new random number
generators, dynamic memory on all platforms, and a new mesh tally capability[2].
1 MCNP is a trademark of the Regents of the University of California, Los Alamos National Laboratory.
T. Goorley, F. Brown, L.J. Cox
American Nuclear Society Topical Meeting in Mathematics & Computations, Gatlinburg, TN, 2003 2/16
Enhancements for Microsoft Windowsâ PCs have also been included with this release, since
desktop PCs increase their power and popularity for scientific computing applications. In
addition to the new code features which are available on all platforms, improvements for
Windows PCs include two new methods for installing and compiling the source, support for
three commercial FORTRAN compilers, and easier building of plotting versions with included
X11 graphics files. Support for parallel capability using Message Passing Interface MPI[5,6] or
Parallel Virtual Machine PVM[7,8] protocols, originally on other platforms[9], has been
extended to Windows PCs. These new parallel capabilities allow MCNP to be run on
homogeneous and heterogeneous clusters and/or dual processor Microsoft Windows PCs. These
PC enhancements are the subject of this paper. The following two sections describe how to
install and build MCNP 5 on a PC. The subsequent two sections discuss a comparison of the
wall clock runtimes for MCNP 5 on a dual processor desktop and a network of different CPU
speed laptops.
2. INSTALLING MCNP 5 ON WINDOWS PCS
There are two different ways to install MCNP 5 on a PC running a Windows operating system
(95/98/NT/2000/XP/ME). The simplest method is to use the InstallShieldâ setup programs,
similar to that of other Windows programs. The first setup program copies the MCNP
executables, source code, and test problems to a user-selected directory and then sets two
environmental variables. The MCNP Visual Editor and MCNP documentation are also installed.
The second setup program installs the data libraries, MAKXSF (a cross-section library
compression program), the files XSDIR and SPECS, and sets an environmental variable. The
user is then asked to logout and login and then run the test problems to verify that MCNP has
been installed correctly. The main advantage of using this method is that no compilers are
needed and no source code needs to be built. This option meets the needs of most users.
Alternatively the user can copy the MCNP directory tree to the desired location and use the
install script to build MCNP 5, MAKXSF, and/or run the test problem suite. The install script
can be used to build MCNP only if the appropriate compilers are already installed. The
advantage to this method is that the executables can be rebuilt to apply patches or modifications
to the source.
After installing the MCNP executable, additional software may need to be installed and
appropriate environmental variables may need to be set or changed to take advantage of X11
graphics or parallel communications capabilities in MCNP. X-windows client software, not
provided with MCNP 5, needs to be running to display geometry, tally, or cross-section plots.
X-Windows software is discussed in section 2.4 of this paper. To use the parallel versions of the
MCNP executables, parallel communications software will need to be installed prior to running
with this capability, and prior to building a parallel version of MCNP, if the InstallShield
executables are not used. Specific instructions on how to install MPI and PVM are given in
section 2.5. The environmental variables PATH and DATAPATH can be modified to make file
management easier. DISPLAY may need to be set to use the plotting capabilities of MCNP.
MCNP environmental variables are discussed in section 2.6.
MCNP 5 IMPROVEMENTS FOR WINDOWS PCS
American Nuclear Society Topical Meeting in Mathematics & Computations, Gatlinburg, TN, 2003 3/16
2.1. The InstallShield Setup Program
The InstallShield programs for MCNP 5 are similar to that of other windows applications.
Double clicking on the setup file will start the InstallShield program. After starting the MCNP 5
Executables installer, the initial setup window is displayed, then the next two windows present
the Copyright notice and Software License Agreement, and request user information. The
following window asks where MCNP should be installed. It does not need to be in the default
directory of /Program Files/LANL/MCNP5/. The installation package will then copy the
plotting-sequential and parallel executables, MCNP source code, documentation (including the
MCNP Manual), Visual Editor, and problem test suite into the chosen directory. The final screen
queries the user if it can change the appropriate environmental variables: PATH and DISPLAY.
Since PATH is already present, the MCNP5 directory path is appended to this variable. If these
environmental variables are changed, the user must log out and then log back in (or reboot for
some operating systems) before they will take effect. The user should be aware that if another
executable with the name MCNP5 is already present in the path, the first executable in the path,
i.e. the previously existing MCNP5, may be unintentionally used. The InstallShield setup
program cannot be used to build MCNP executables.
The second InstallShield setup program installs the data libraries, the MAKXSF program and the
XSDIR and SPECS files. The installer queries the user for a directory to place these files, which
may or may not be a subdirectory where MCNP5 was installed. After this directory is specified,
the environmental variable DATAPATH is set. Administrative privileges are also required, as
well as write permission to at least 2.5 Gigabytes of hard disk space. This large amount of space
is mostly used by the ASCII format (type 1) updated ENDF/B-VI cross-section libraries, which
can be compressed with MAKXSF and the SPECS file to ~800 Megabytes. For more
information about using MAKXSF, see section XIV of Appendix C in the MCNP 5 Manual.
After the InstallShield programs are completed, the user should run the test suite to verify that
the executable has been installed and operates correctly on the user’s specific operating system
and hardware. This testing procedure can be started by double clicking on the runprob.bat icon
located in the directory Installation, where MCNP5 was installed. After the test problems are
run, files that list the differences between the tally or output files generated and the expected
results are displayed. These difference files should be reviewed by the user to determine if the
differences are simple round-off errors or something more substantial, indicating incompatible
hardware or software and that MCNP may give incorrect results.
To uninstall either the MCNP 5 Executables or MCNP Data install packages, the user should
remove them via the Windows Control Panel, with the Add/Remove Programs function. This
will delete any files that were installed (files created while running the test problems will not be
deleted) and will modify the registry appropriately. The environmental variables will not be
removed, however, but these can be removed manually.
2.2. The Install Script
A second method to install MCNP 5 uses an install script, which interactively queries the user for
various build options and then executes the make utility to build MCNP, MAKSXF and/or run
T. Goorley, F. Brown, L.J. Cox
American Nuclear Society Topical Meeting in Mathematics & Computations, Gatlinburg, TN, 2003 4/16
the test problem suite. The various build options include which Fortran and C compilers should
be used, the location of appropriate X11 files, and the path to the xsdir file. The script also gives
the user the opportunity to only generate custom makefiles. These custom makefiles contain the
build options selected, and will be automatically used whenever the make utility is used. An
answer file, which lists the options chosen in the install script, is also created. The answer file
can only be used to set options in the install script, and only if it is specified on the install
command line. If MCNP is to be built, or the test suite is to be run, then install script will then
execute gmake, (the GNU version of make), and write most of the output to the file install.log.
This script, which uses the gmake utility to build the code, can only be used if a Unix based shell
is installed.
2.3. Installing a Unix shell - Cygwin
If the install script or gmake utility is used, a Unix command shell must be installed. Unix
command shells are not standard on Windows operating systems and must be installed. Cygwin,
a freeware port of a Unix command shell for Windows PCs, can be obtained at
http://www.cygwin.com or http://www.redhat.com/apps/download/. The setup program can be
downloaded or run from the Redhat website. This setup program will step users through the
Cygwin installation process, allowing them to select web installation or download installation
files to a local drive. The location where the cygwin software should be installed (a path without
spaces is recommended) and a temporary directory where files can be downloaded are specified
in the next two windows.
After selecting a website to download or install from, the user selects Cygwin packages to install.
In addition to the Cygwin packages that are selected by default, the gcc and make packages
(located under the Devel directory) will also need to be selected to build MCNP with the make
utility. The perl package (located under the Interpreters directory) is also recommended, and is
required if the Absoft or Lahey Fortran compilers are going to be used to compile MCNP.
2.4. Installing Plotting Software – X Windows Client
In order to display MCNP plots, an X11 windows client software package is needed. Several
commercial X-windows clients are available: Reflection X (http://www.wrq.com/products/),
Hummingbird’s Exceed_NT (http://www.hummingbird.com/products/nc/exceed/index.html), and
Starnet’s X-win32 (http://www.starnet.com/). No single commercial product is recommended. A
freeware X client is also available with Cygwin, X-Free86. It has also been tested with MCNP5.
These client software packages do not need to be the developer or professional versions, since
the X11 header and library files are included with the MCNP 5 distribution.
2.5. Installing Parallel Software – MPICH.NT or PVM
In order to use or build MCNP 5 with parallel capabilities, appropriate parallel communications
software must be installed. Either MPI or PVM communications protocols are supported. To
build a parallel version of MCNP 5 for Windows, see the section 3.6.
MCNP 5 IMPROVEMENTS FOR WINDOWS PCS
American Nuclear Society Topical Meeting in Mathematics & Computations, Gatlinburg, TN, 2003 5/16
The MPI port for Windows is MPICH.NT, developed at Argonne National Laboratory, and can
be downloaded from http://www-unix.mcs.anl.gov/~ashton/mpich.nt/. This website also offers
the helpful references MPICH.NT FAQ and MPICH Users Manual. If there is no need to rebuild
parallel MCNP 5, only the runtime dlls and mpirun package will be needed (mpich.nt.1.2.4.exe).
This package uses an InstallShield setup program which requires installation from an
administrative account on all PCs in the cluster. If MCNP 5 needs to be built, then the source
code (package mpich.nt.1.2.4.src.exe) should be downloaded and unzipped as well. The
program MPIConfig must be run on each computer after MPICH.NT installation. The local host
name must be added and the settings applied.
Once MPICH.NT is installed, a few additional steps are required. MPI enabled MCNP must be
copied to the same directory on all hosts. MCNP can be executed through either the Windows
MPIRun GUI or command line MPIRun. For the MPIRun GUI, hosts must be added by
selecting or typing their names in the hosts section. The DATAPATH may need to be set under
the advanced options. For the command line MPIRun, the hosts must be specified by the –hosts
option, which can be used to specify the number of processes started on each host. Typically the
number of processes is equal to the number of CPUs utilized plus one. The first process listed is
the master process, and does not run any histories. When “balance” is specified at the end of the
command line, MCNP will load balance its MPI processes. For example, the command to start
three MCNP MPI processes on CompA (a dual CPU machine) and one process on CompB is:
mpirun –hosts 2 CompA 3 CompB 1 mcnp5mpi inp=test balance
Alternatively, MCNP 5 can be built and run with PVM, developed at Oak Ridge National
Laboratory. The PVM port for Windows can be downloaded from
http://www.csm.ornl.gov/~sscott/PVM/Software/. The file ParallelVirtualMachine3.4.3.zip
contains the source and binaries needed to install and run PVM on a single computer. This
InstallShield program must be run on all Windows PCs in the cluster. PVM requires additional
communications software, a remote shell (RSH) client/server package, before it will run across a
cluster of Windows PCs. Two commercial RSH packages can be obtained from
http://www.winrshd.com/ and http://www.ataman.com/. The RSH package must be installed on
each computer in the cluster, and the permissions must be set to allow RSH or REXEC
connections for the desired user accounts. The Ataman RSH package was successfully tested.
Similar post- installation steps are required to run the PVM version of MCNP 5 on Windows PCs.
The MCNP executable must be copied into the %PVM_ROOT%/pvm3/bin/WIN32 directory on
all hosts. PVM must be started on a single computer before a PVM enabled MCNP can be
executed. After PVM is started, additional hosts can be added to the PVM cluster, with the “add
host” command from the PVM console prompt. PVM operability can be tested and verified with
the PVM example programs, such as hello. MCNP can then be started from a separate command
shell with the following command:
mcnp5pvm inp=test tasks n
where n is the number of slave processes. The number of tasks is usually the number of CPUs in
the cluster. A negative number entered for n causes MCNP to skip the initial load-balancing
T. Goorley, F. Brown, L.J. Cox
American Nuclear Society Topical Meeting in Mathematics & Computations, Gatlinburg, TN, 2003 6/16
feature, and is recommended for a homogeneous cluster. Additional information on how to
install MPI or PVM can be obtained in their respective user’s manuals. Additional information
on running MCNP in parallel can be found in Appendix C, section VIII of the MCNP 5 Manual.
2.6. Setting Environmental Variables
After installation, it may be necessary to change or add three Windows environmental variables.
They are set differently for Windows 95/NT/2000/XP/ME, but for each of these operating
systems the variables can be viewed the same way. The value of an environmental variable
PATH, for example, can be printed in a command shell window (i.e. a “DOS prompt”) with the
command echo %PATH%.
The first environmental variable that may need to be changed is PATH, a semicolon separated
list of directories used to find executables and dynamic link libraries. The directory where
MCNP is installed should be included, so that MCNP can be executed from any directory,
making file management more convenient. If several programs with the same name exist within
PATH, then the executable in the first occurring directory will be executed. Appending the
directory of the newly installed version of MCNP to the PATH may not change which MCNP is
executed if an older version with the same name is given earlier in the directory listing. Failure
to change PATH will mean that the MCNP executable must be located in the directory where the
input file is located. To use the plott