http://www.staff.uni-marburg.de/~haimm/marc2practice

  http://www.staff.uni-marburg.de/~haimm/marc2practice
Using the MaRC2 HPC Cluster
Manuel Haim, 06/2013
Manuel Haim - Hochschulrechenzentrum
Using MaRC2???
Manuel Haim - Hochschulrechenzentrum
2
Using MaRC2 – Overview
• Get access rights and permissions
• Starting a terminal session (Linux, Windows, Mac)
• Intro to the BASH Shell (and available directories)
• Using the Environment Modules package
• Compiling your programs
• Submitting jobs to the SUN Grid Engine (SGE)
Manuel Haim - Hochschulrechenzentrum
3
Get access rights and permissions
Manuel Haim - Hochschulrechenzentrum
4
Get access rights and permissions
• Students / Staff account needed
• Ask your workgroup leader if MaRC2 is already being used
→ he/she must accept the terms of use
→ he/she nominates people who may tell us to grant you access
• Ask the nominated people at your workgroup to get access
• For further questions, terms of use etc., send us an email:
→ marc@hrz.uni-marburg.de
Manuel Haim - Hochschulrechenzentrum
5
Starting a terminal session
(Linux, Windows, Mac)
Manuel Haim - Hochschulrechenzentrum
6
Starting a terminal session (Linux, Mac)
• Start your favourite Terminal application
• Connect to MaRC2 via SSH (Secure Shell) by entering:
ssh
ssh -p223
-p223 username@marc2.hrz.uni-marburg.de
username@marc2.hrz.uni-marburg.de
→ replace username with your Students / Staff account name
→ remember that your account name is case-sensitive!
• You are then prompted for your account's password:
username@marc2.hrz.uni-marburg.de's
username@marc2.hrz.uni-marburg.de's password:
password:
Manuel Haim - Hochschulrechenzentrum
7
Starting a terminal session (Linux) – Example
Manuel Haim - Hochschulrechenzentrum
8
Starting a terminal session (Windows)
• Start an SSH client, e.g.
PuTTY (www.putty.org)
• Select “SSH”, then
insert host name and
port number
• Finally, click “open”
• A terminal window
opens where you will be
prompted for username
(case-sensitive!) and
password
Manuel Haim - Hochschulrechenzentrum
9
Some hints
• To exit the terminal session, just enter exit or press Ctrl+D
• By default, your login shell is set to /bin/bash .
You may change this setting globally (and choose another
login shell like csh, ksh or tcsh) for your account here:
Staff account:
https://admin.staff.uni-marburg.de/aendshell.html
Students account:
https://admin.students.uni-marburg.de/login_shell.html
Manuel Haim - Hochschulrechenzentrum
10
Introduction to the BASH Shell
(and available directories)
/bin/bash
Manuel Haim - Hochschulrechenzentrum
11
Introduction to the BASH Shell (Bourne-again shell)
• Right after login, the BASH Shell is started by default and
shows a command prompt like this:
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ ██
Input cursor
“$” for user, “#” for superuser (root)
Current directory name (“~” for home directory,
“/” for root directory)
Server name
Your username
• To exit, simply enter the command exit or press Ctrl+D
Manuel Haim - Hochschulrechenzentrum
12
First steps with BASH-builtin commands
• pwd – print current (working) directory:
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ pwd
pwd
/home/username
/home/username
• cd – change current directory:
cd .. – go to parent directory
cd <dirname> – go to subdirectory
[username@marc2-h1
[username@marc2-h1
[username@marc2-h1
[username@marc2-h1
/home
/home
[username@marc2-h1
[username@marc2-h1
[username@marc2-h1
[username@marc2-h1
/home/username
/home/username
~]$
~]$ cd
cd
home]$
home]$
..
..
pwd
pwd
home]$
home]$ cd
cd username
username
~]$
~]$ pwd
pwd
Manuel Haim - Hochschulrechenzentrum
13
Additional commands
• ls – list directory contents:
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ ls
ls
mydata
mydata readme
readme script.sh
script.sh
ls -l – list contents in long format
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ ls
ls -l
-l
drwxrwxr-x
drwxrwxr-x 22 username
username username
username 4096
4096 18.
18. Jun
Jun 11:12
11:12 mydata
mydata
-rw-rw-r-1
username
username
915
18.
Jun
11:37
readme
-rw-rw-r-- 1 username username 915 18. Jun 11:37 readme
-rwxr-xr-x
26
-rwxr-xr-x 11 username
username username
username
26 18.
18. Jun
Jun 11:35
11:35 script.sh
script.sh
permissions
(see later)
user
file type (d=directory)
Manuel Haim - Hochschulrechenzentrum
group
last change
file name
file size
(bytes)
14
Additional commands (2)
• ls -a – show all files (even hidden ones):
[username@marc2-h1
[username@marc2-h1
..
.bash_history
.bash_history
..
.bash_logout
..
.bash_logout
~]$
~]$ ls
ls -a
-a
.bash_profile
.bash_profile mydata
mydata script.sh
script.sh
.bashrc
readme
.bashrc
readme
(files starting with “.” are usually not shown)
Manuel Haim - Hochschulrechenzentrum
15
Additional commands (3)
• mkdir <dirname> – make subdirectory:
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ mkdir
mkdir temp
temp
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ cd
cd temp
temp
[username@marc2-h1
temp]$
pwd
[username@marc2-h1 temp]$ pwd
/home/username/temp
/home/username/temp
• rmdir <dirname> – remove directory (must be empty):
[username@marc2-h1
[username@marc2-h1
[username@marc2-h1
[username@marc2-h1
temp]$
temp]$ cd
cd
~]$
rmdir
~]$ rmdir
..
..
temp
temp
• rm <filename> – remove (delete) file
rm -R <dirname> – remove (delete) directory recursively!
Rm -Rf <dirname> – just as above, but “force”, don't ask ;-)
Manuel Haim - Hochschulrechenzentrum
16
Available directories on a typical Linux system
• The Unix/Linux filesystem is organized in a tree-like structure
• The top-most directory “/” is called the root directory
• All filesystem entries (including mounted filesystems) are
branches of this root.
/
/boot
/bin
executable boot
files
binaries
/dev
devices
Manuel Haim - Hochschulrechenzentrum
/lib
/usr ...
/home
/etc
UNIX
configuration home dirs libraries
system
files
ressources
/home/username
personal home directory “~”
17
Some special directories on MaRC2
/local
local storage (1.8TB)
on node001 – node088,
available to local processes
(not shared via network)
mkdir /local/username
/home/username
personal home directory “~”
/home/groupname
shared group directory
/home/software
shared software
/home
shared home dirs (33TB),
optimized for integrity
/scratch
shared directory (33TB),
optimized for speed
mkdir /scratch/username
Manuel Haim - Hochschulrechenzentrum
18
Getting help
• man <command> – Show man(ual) page for command:
[username@marc2-h1
[username@marc2-h1 ~]$
~]$ man
man ls
ls
→ Press space for next page, b for previous page
→ Arrow keys and PgUp/PgDn will also work nowadays
→ Press q to quit
• <command> --help or <command> -h – Show help
(not supported by all commands)
Manuel Haim - Hochschulrechenzentrum
19
Change file owner and permissions
• This is what ls -l shows:
-rwxr-xr-x
-rwxr-xr-x 11 username
username username
username
user other
group
permissions
user
26
26 18.
18. Jun
Jun 11:35
11:35 script.sh
script.sh
group
r = read
w = write
x = execute
• chmod <permissions> <file> – Change permissions
example: chmod u=rwx,g=rx,o=rx script.sh
• chown <user>:<group> <file> – Change owner and group
Manuel Haim - Hochschulrechenzentrum
20
Some tips and tricks
• Select text to copy, then paste with middlemouse button
(PuTTY: Use right mouse button)
• Tab auto-completes your input
• Shift+PgUp/PgDn scrolls the screen buffer up and down
(maximum scrollback buffer size can be set in your terminal options)
• ArrowUp and ArrowDown allows you to navigate through your
former input
• Ctrl-R allows you to reverse-i-search in your former input
• Edit your .bashrc and .bash_profile for personal settings
Manuel Haim - Hochschulrechenzentrum
21
Using the
Environment Modules package
Manuel Haim - Hochschulrechenzentrum
22
Using the Environment Modules package – why?
• Each piece of software typically needs a set of environment
variables to be set, in order to run properly
→ e.g. the $PATH variable which points to the bin directories
• On a HPC cluster like MaRC2, there may be many different
toolsets and toolset versions with the same purpose and/or
the same command names → Conflict!
• With the Environment Modules package, you can dynamically
modify your environment by loading so-called “modulefiles”
→ Use our predefined modulefiles, no need to create your own :-)
Manuel Haim - Hochschulrechenzentrum
23
Using the Environment Modules package – how?
• module avail – show available modules
[haimm@marc2-h1
[haimm@marc2-h1 ~]$
~]$ module
module avail
avail
----------------------------------------------- /usr/share/Modules/modulefiles
/usr/share/Modules/modulefiles ----------------------------------------------dot
module-cvs
module-info
modules
null
use.own
dot
module-cvs module-info modules
null
use.own
----------------------------------------------------- /usr/share/ModulesLocal
/usr/share/ModulesLocal ------------------------------------------------------acml/gfortran-5.1.0(default)
openmpi/pgi/1.4.3-qlc
acml/gfortran-5.1.0(default)
openmpi/pgi/1.4.3-qlc
acml/gfortran-5.2.0
openmpi-1.6.3/gcc/1.6.3
acml/gfortran-5.2.0
openmpi-1.6.3/gcc/1.6.3
acml/ifort-5.1.0
openmpi-1.6.3/icc/1.6.3
acml/ifort-5.1.0 (AMD Core Math Library) openmpi-1.6.3/icc/1.6.3
acml/pgi-5.1.0
openmpi-1.6.3/pgi/1.6.3
acml/pgi-5.1.0
openmpi-1.6.3/pgi/1.6.3
gcc/4.4.6
parastation/mpi2-gcc-5.0.27-1(default)
gcc/4.4.6
parastation/mpi2-gcc-5.0.27-1(default)
gcc/4.6.2(default)
parastation/mpi2-gcc-mt-5.0.27-1
gcc/4.6.2(default)
parastation/mpi2-gcc-mt-5.0.27-1
gcc/4.7.2
parastation/mpi2-intel-5.0.27-1
gcc/4.7.2
parastation/mpi2-intel-5.0.27-1
intel/intelPSXE2011SP1-32
parastation/mpi2-intel-mt-5.0.27-1
intel/intelPSXE2011SP1-32
parastation/mpi2-intel-mt-5.0.27-1
intel/intelPSXE2011SP1-64
parastation/mpi2-pgi-5.0.27-1
intel/intelPSXE2011SP1-64
parastation/mpi2-pgi-5.0.27-1
intel/intelPSXE2013SP1-64
parastation/mpi2-pgi-mt-5.0.27-1
intel/intelPSXE2013SP1-64
parastation/mpi2-pgi-mt-5.0.27-1
openmpi/gcc/1.4.3-qlc
pgi/12.2(default)
openmpi/gcc/1.4.3-qlc
pgi/12.2(default)
(Portland Group Inc. Compiler)
openmpi/intel/1.4.3-qlc
pgi/13.2
openmpi/intel/1.4.3-qlc
pgi/13.2
• module load <modulefile> – load module
→ You do not need to specify the full path in order to load the default, e.g.:
[haimm@marc2-h1
[haimm@marc2-h1 ~]$
~]$ module
module load
load gcc
gcc
Manuel Haim - Hochschulrechenzentrum
24
Using the Environment Modules package – how (2)
• module list – show loaded modules
[haimm@marc2-h1
[haimm@marc2-h1 ~]$
~]$ module
module list
list
Currently
Loaded
Modulefiles:
Currently Loaded Modulefiles:
1)
1) acml/gfortran-5.1.0
acml/gfortran-5.1.0
2)
gcc/4.6.2
2) gcc/4.6.2
3)
3) parastation/mpi2-gcc-5.0.27-1
parastation/mpi2-gcc-5.0.27-1
• module unload <modulefile> – unload module
→ Again, you do not need to specify the full path:
[haimm@marc2-h1
[haimm@marc2-h1 ~]$
~]$ module
module unload
unload gcc
gcc
• More information: man module
Manuel Haim - Hochschulrechenzentrum
25
Compiling your programs
Manuel Haim - Hochschulrechenzentrum
26
Compiling your programs – Choose a compiler
• Several compilers available:
→ gcc – GNU C Compiler
→ gfortran – GNU Fortran Compiler
→ icc – Intel C Compiler
→ ifort – Intel Fortran Compiler
→ pgcc – Portland Group Inc. (PGI) C Compiler
→ pgfortran – Portland Group Inc. (PGI) Fortran Compiler
• MPI Compilers (OpenMPI or Parastation MPI, GNU/Intel/PGI):
→ mpicc – MPI C Compiler
→ mpicxx – MPI C++ Compiler
→ mpif77 – MPI Fortran77 Compiler
Manuel Haim - Hochschulrechenzentrum
27
A simple C program
• Create a file hello-world.c with the following content:
#include
#include
#include
#include
<stdio.h>
<stdio.h>
<stdlib.h>
<stdlib.h>
int
int main()
main() {{
printf("Hello
printf("Hello World!\n");
World!\n");
exit(0);
exit(0);
}}
• Compile using gcc: (remember to have the module loaded first!)
[haimm@marc2-h1
[haimm@marc2-h1 ~]$
~]$ gcc
gcc hello-world.c
hello-world.c -o
-o hello-world
hello-world
• Then test locally: (don't do this with huge complex programs!)
[haimm@marc2-h1
[haimm@marc2-h1 ~]$
~]$ ./hello-world
./hello-world
Hello
World!
Hello World!
Manuel Haim - Hochschulrechenzentrum
28
We could also use a Makefile
• Create a file Makefile with the following content:
PROG=hello-world
PROG=hello-world
SRC=hello-world.c
SRC=hello-world.c
CC=gcc
CC=gcc
You must insert
a tab here!
all:
all: clean
clean build
build
clean:
clean:
build:
build:
rm
rm -f
-f $(PROG)
$(PROG)
$(CC)
$(CC) $(SRC)
$(SRC) -o
-o $(PROG)
$(PROG)
run:
run: all
all
./$(PROG)
./$(PROG)
• Now just enter make or make run to compile/run your code
Manuel Haim - Hochschulrechenzentrum
29
Submitting jobs to the
SUN Grid Engine (SGE)
Manuel Haim - Hochschulrechenzentrum
30
SUN Grid Engine???
• The SUN Grid Engine (SGE) implements a queueing system
• Users can submit jobs to the SGE queues
and define boundary conditions (like runtime, memory etc.)
• The SGE permanently reviews the available resources,
evaluates pending jobs and executes them on the compute
nodes
• The SGE is open source software with a free-to-use license
→ Its successor, the Oracle Grid Engine, is only available commercially
Manuel Haim - Hochschulrechenzentrum
31
How to submit a new job to the SGE
• First, write a start script for your software:
(e.g. hello-world.sh)
#!/bin/bash
#!/bin/bash
#$
#$
#$
#$
#$
#$
#$
#$
-S
-S
-e
-e
-o
-o
-l
-l
/bin/bash
/bin/bash
./stderr-file
./stderr-file
./stdout-file
./stdout-file
h_rt=10
h_rt=10
echo
echo "hello-world
"hello-world running
running on
on host
host $(hostname)"
$(hostname)"
./hello-world
./hello-world
exit
exit 00
• Then submit your start script: (-cwd: execute from current working dir)
$$ qsub
qsub -cwd
-cwd hello-world.sh
hello-world.sh
Your
job
144406
Your job 144406 ("hello-world.sh")
("hello-world.sh") has
has been
been submitted
submitted
Manuel Haim - Hochschulrechenzentrum
32
Get information on your submitted jobs
• qstat – List all your jobs
$$ qstat
qstat
job-ID
user
state
queue
slots
job-ID prior
prior name
name
user
state submit/start
submit/start at
at
queue
slots ja-task-ID
ja-task-ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------144407
06/19/2013
11
144407 0.00000
0.00000 start.sh
start.sh haimm
haimm qw
qw
06/19/2013 12:43:26
12:43:26
SGE may take a few seconds
q=queued, w=waiting,
to give your job a non-zero priority E=Error, r=running
1 CPU slot (core)
• qstat -u "*" – List all jobs from all users
• qstat -j <job-id> – Get information on a single job
$$ qstat
qstat -j
-j 144406
144406
==============================================================
==============================================================
job_number:
144406
job_number:
144406
exec_file:
job_scripts/144406
exec_file:
job_scripts/144406
submission_time:
Wed
submission_time:
Wed Jun
Jun 19
19 12:40:43
12:40:43 2013
2013
owner:
haimm
owner:
haimm
uid:
30376
uid:
30376
...
...
Manuel Haim - Hochschulrechenzentrum
33
Get information on your submitted jobs (2)
• qalter -w v <job-id> – Get scheduler information for job
(may be helpful if your job won't run)
$$ qalter
qalter -w
-w vv 144406
144406
Job
Job 144406
144406 queue
queue instance
instance "parallel_long@node028.marc2"
"parallel_long@node028.marc2" dropped
dropped because
because it
it is
is
temporarily
not
available
temporarily not available
Job
Job 144406
144406 queue
queue instance
instance "parallel_long@node032.marc2"
"parallel_long@node032.marc2" dropped
dropped because
because it
it is
is
temporarily
not
available
temporarily not available
Job
Job 144406
144406 queue
queue instance
instance "parallel_long@node043.marc2"
"parallel_long@node043.marc2" dropped
dropped because
because it
it is
is
temporarily
not
available
temporarily not available
...
...
• qdel <job-id> – Delete (terminate) a submitted job
Manuel Haim - Hochschulrechenzentrum
34
Hint for Java users (memory boundaries)
• Java usually takes more memory than you think it takes
• SGE's h_vmem value must be larger than Java's -Xmx value,
you may test it with a simple “Hello world” program like this:
#!/bin/bash
#!/bin/bash
#$
#$
#$
#$
#$
#$
#$
#$
-S
-S
-e
-e
-o
-o
-l
-l
/bin/bash
/bin/bash
./stderr-file
./stderr-file
./stdout-file
./stdout-file
h_vmem=9G,
h_vmem=9G, h_rt=60
h_rt=60
java
java -Xms5120m
-Xms5120m -Xmx5120m
-Xmx5120m -cp
-cp ./Hello.jar
./Hello.jar Hello
Hello
exit
exit 00
Manuel Haim - Hochschulrechenzentrum
35
Additional resources
• MaRC2 wiki:
http://marc2-doc.hrz.uni-marburg.de
(see Users Guide for information on the SGE, MPI and a usage tutorial)
• MaRC2 info page (German):
http://www.uni-marburg.de/hrz/infrastruktur/zserv/cluster
• Further questions?
→ Just ask :-)
→ or send us an email: marc@hrz.uni-marburg.de
References:
Several icons within this presentation are taken from “Crystal Project” (GNU LGPL).
Manuel Haim - Hochschulrechenzentrum
36
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising