Submitting a Job to the HPC The HPC is accessed through a secure shell network (ssh) that exists at UF.

It is an intranet between computers that allows you to connect to a cluster of parallel processors (the HPC) and run MCNP. You must connect to the ssh to access the HPC. Files are moved around in a ssh by using the sftp, or the ssh file transfer protocol. The sftp provides file access, file transfer, and file management. In order to use the HPC, you must first have a familiarity with the ssh and sftp terminal commands. For users on Linux or OSX machines, this process is relatively simple, as these protocols are built into the terminal. On Windows, however, third-party add-ons are necessary. To connect to the SSH, download OpenSSH or PuTTY. For SFTP, WinSCP is a practical, windows-based client that is available for download. On the LINUX machine, you must use Dolphin. Click applications, then systems/tools, then select Dolphin. Here you should see a folder with a globe on it labeled HPC. Click on this folder and it will open your home directory for the HPC sftp. If this HPC folder does not exist, right click in Dolphin folder, open create new, and then click link to new location (URL). Enter the URL for sftp: sftp://submit.hpc.ufl.edu This will create a new folder that connects to the HPC sftp.

Steps to Submit a Job to the HPC

1. Open your command window
If you're using Windows and you've downloaded the aforementioned add-ons, you should open the MCNP command prompt.

2.

Log in to the HPC.

To do so write the following line and hit enter:

ssh username@submit.hpc.ufl.edu -----------username is the username given to you

3.

Enter your password.

The following will be output to the screen username@submit.hpc.ufl.edu's password: enter your password and hit enter -------- NOTE: if you want to change your password, use the command "passwd"

4.

Create the job submission script for your input file

In order to run an MCNP file, you must submit a job script to the HPC. The job script specifies certain things to the HPC (number of nodes, number of processors per node, the file name, etc.) Every time you want to submit a file, you must submit a job script. The job scripts as they are now will run fine; the only things you must change are the following lines: #PBS -N "your job name" ... mpiexec $MCNP output name" i="your file name" o="your file's

The first line is the name of your job in the HPC, so when you check the status of your job, this is what will appear. The second line is where you declare the name of your input file. So i="your file name" and what you want your output file to be named is o="your output file's name"

The output file name can be whatever you want, but it is customary to just add an o to the end of your input file name Example: your file name is booger #PBS -N booger ... mpiexec $MCNP i=booger o=boogero

5.

Move the job submission script and your input file into the scratch directory.
Unfortunately, the HPC doesn't run files out of your home directory. It runs them out of a different directory, the scratch directory. Therefore, you'll have to move any files that you put into the home directory to the scratch directory. On LINUX you must do this from the command prompt, unless you know how to sftp into scratch (which I could never do). The Windows interface is more convenient. You can easily move between the /home and /scratch folders with WinSCP.

----------NOTE: A good thing to do is to put your job submission script and your input file into one folder. This means that you only have to move a file from /home to /scratch once, and it keeps all your output, tally, srctpe, and runtpe files in one, organized location. To move the file (or folder) on LINUX, type the following line into the command prompt: mv "file name" /scratch/hpc/username This will move the file to your scratch directory. You could do the same in Windows, but it easier to move files with WinSCP

6.

Change directories
Now you'll have to move into the scratch directory to submit your job script. Type the following line into the command prompt: cd /scratch/hpc/username If you are using folders as mentioned above, you'll have to cd further into the folder. Type the following line: cd /scratch/hpc/username/foldername

7.

Submit your job

To submit your job to the HPC, type the following line qsub "job submission script name" To see the status of all of your jobs, use the following command: qstat -u "username"

8.

View your output

Once your job has completed, you'll have to move your file back to home directory to view it. On Windows you can easily access and move your files with the WinSCP app. On LINUX, use the mv command again: mv "file name" /home/username

---------NOTE: this is where using folders is really handy. When you use a folder, you'll only have to move one folder once instead of many files. If you don't, you'll have to type the mv command several times or click and drag repeatedly.

Some useful commands: ----If you want to view a file

more <filename>

----List all the files in your current directory

ls

----delete a file from a directory

rm <file name>

---A given property of a job can be changed with the command:

qalter <Job ID> <Job Property>

Further instructions on the use of this command can be found with a quick Google search. -----A job can be deleted from the queue with the command:
qdel <Job ID>

A job submission script that will work with MCNP is available on the HPC Wiki. It can be copied wholesale, but there is something important to note: MCNP5 is located in a separate folder as MCNPX and MCNP6b, and also requires loading a separate module. The script provided is for MCNP5, but, should you want to run MCNPX or MCNP6b, the script will have the following changes: MCNPX:
module load mcnpx/2.7.0 ... MCNP=/apps/mcnpx/2.7.0/bin/mcnpx.mpi $MCNP i=<input> o=<output>

MCNP6b:
module load mcnp/6b2 MCNP=/apps/mcnp/6b2/bin/mcnp6.mpi