kmimf tutorial - Jefferson Lab Technical Notes

JLAB­TN­07­049
kmimf 0.1
Kevin's minimization in FORTRAN
Dr. K.B.Beard,
Thomas Jefferson National Accelerator Facility
30 August 2007
Abstract
Often, it is convenient to have simple minimization performed on a complicated system, but the
programs used for the calculation lack the that ability. kmimf is intended to automatically run some
arbitrary program, process the output, change the input, and rerun the program while trying to minimize,
maximize, or set some quantity. While kmimf itself runs under UNIX, the program it runs can be under
some other operating system, such as Windows.
Introduction
After encountering subtle, difficult to circumnavigate, problems trying to use the gminuit1
program, this author wrote kmimf (Kevin's minimization in Fortran). It is functionally similar but
avoids the use of temporary script files and Tcl2. kmimf is mostly FORTRAN77 with a KWRAP3 C
wrapper and uses MINUIT4 to run the minimization. Rather then link to individual MINUIT files, the
program links to the CERN PACKLIB5 library. This version only considers the fitting of a single
number and not a data file like gminuit and tune6. The basic idea is very simple; use a very standard optimization routine to optimize the result of a
generic function. That generic function has an entire simulation & analysis process hidden inside; the
optimization routine only knows the final result. MINUIT is a very widely used minimization routine;
it sends numbers as input into some function FCN and adjusts those numbers to minimize the returned
value. kmimf just hides a whole script, program, and analysis inside FCN, leaving MINUIT ignorant
of the details. A detailed description of the code is in the Appendix and online7.
This simple approach has the advantage that kmimf will run on pretty much any UNIX, including
1
2
3
4
5
6
7
http://www.muonsinc.com/
http://en.wikipedia.org/wiki/Tcl
http://casa.jlab.org/internal/code_library/casa_lib/KWRAP/DOC/
http://project-mathlibs.web.cern.ch/project-mathlibs/minuit/home.html
http://cernlib.web.cern.ch/cernlib/packlib.html
http://www.muonsinc.com/
http://casa.jlab.org/internal/code_library/casa_lib/KWRAP/DOC/
JLAB­TN­07­049
the Cygwin8 emulation of UNIX running under other operating systems such as MacOS-X and Windows
XPpro. This allows the FCN function to call programs that are unavailable for UNIX, such as the current
LANL parmela9, which is only available for some WindowsOS's, freeing the user from manually going
through the optimization procedure. Figure 1. Nesting of kmimf; FCN can “talk” to both Cygwin and Windows.
FCN can communicate to kmimf, MINUIT, and the operating system(s); it can do anything a
shell script can. kmimf version 0.1 adds features to support nesting optimizations within
optimizations ­ a kmimf session can run another kmimf within a script to optimize some subsection,
and so forth. The final output ("­F file") is in a form that is convenient to send as input ("­pf file")
back into kmimf; it contains the values for that produced the best FCN value; not the last values tried. kmimf has a brief command line help: $> kmimf --help
kmimf - attempts to minimize the value returned from a shell command
kmimf {options}
-h
+h
-V
+V
-v
-i
-if
-M
-Mf
-p
-pf
-F
--help
++help
--version
++version
--verbose
--input
--inputfile
--MINUIT
--MINUITfile
--parameter
--parameterfile
--Final
--format
--hints
STRING
FILE
STRING
FILE
NAME:X{:dX{:lo:hi}} FILE
FILE
STRING
-
print short help and quit [default value]
print longer help and quit
print short version info and quit
print longer version info and quit
display informational messages
specify command string
read command string from a file
specify MINUIT commands (seperated by |)
read MINUIT commands from a file
set parameter start,step, & limits
set parameter settings from a file
write final best results to a file
specify a F77 insertion format [(f20.10)]
print additional guidance & quit
kmimf 0.1f7, 22mar2006
http://www.jlab.org/~beard/index.html#muons
and a more complete help: 8 http://www.cygwin.com/
9 http://laacg1.lanl.gov/laacg/services/serv_codes.phtml#parmela
JLAB­TN­07­049
$> kmimf ++help
kmimf - attempts to minimize the value returned from a shell command
kmimf {options}
-h
+h
-V
+V
-v
-i
-if
+i
-M
-Mf
+M
-p
-pf
+p
+pf
-o
+re
-F
+F
+FD
+SN
-SN
-m
+m
+n
-a
-d
--help
++help
--version
++version
--verbose
--input
STRING
--inputfile
FILE
++input
--MINUIT
STRING
--MINUITfile
FILE
++MINUIT
--parameter
NAME:X{:dX{:lo:hi}}
--parameterfile FILE
++parameter
++parmeterfile FILE
--output
FILE
++recordhunt
FILE
--Final
FILE
++Final
FILE
++FinalDelimiter STR
++SciNot
--SciNot
--test
--minimum
++maximum
++nearest
VALUE
--abort
N
--scan
N
--title
STRING
--delimiter
LR
--format
STRING
--hints
-
print short help and quit [default value]
print longer help and quit
print short version info and quit
print longer version info and quit
display informational messages
specify command string
read command string from a file
print current command string
specify MINUIT commands (seperated by |)
read MINUIT commands from a file
print current MINUIT command string
set parameter start,step, & limits
set parameter settings from a file
print current parameters
overwrite current parameters to a file
specify log file
specify hunt recording file
write final best results to a file
write final best results on a single line
set delimter to put between name and value [ ]
scientific notation in final report
no scientific notation in final report*
evaluate script string once & quit
seek minimum rather than minimum*
seek maximum rather than minimum
seek nearest rather than minimum
abruptly quit after N cycles
just scan +/-N steps in each parameter
specify a MINUIT title
specify a parameter delimiters [{}]
specify a F77 insertion format [(f20.10)]
print additional guidance & quit
kmimf 0.1f7, 22mar2006
http://www.jlab.org/~beard/index.html#muons
The heart of the matter is the function FCN; as far as MINUIT "knows" FCN is just a function
that returns a single REAL*8 output when given a set of REAL*8 inputs. Little dummy nominuit
routines allow for testing and development in the absence of the CERN libraries. Basically, one must tell kmimf a about a few things; what to do for FCN (the command script),
what parameters to use, where to record the results, and optional instructions on how to do the
minimization to pass to MINUIT.
MINUIT commands
Generally, the default MINUIT commands are adequate and need not be altered. The current
default MINUIT commands may be displayed using the +M option (the ­V just prevents further
processing): $> kmimf +M -V
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
set warnings
show fcnvalue
JLAB­TN­07­049
simplex
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
The use of "simplex" rather than "migrad" is due to the fact that the FCN subroutine knows
nothing about, and does not return, gradients. Substitute alternative MINUIT commands may be read from a file via ­Mf file or made from the
command line ­M expression. Use the "|" character to separate lines in a command string ­ for
example: $> kmimf -M "set warning|simplex" +M -V
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
set warning
simplex
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
kmimf 0.0e3, 6sep2005, K.B.Beard, TJNAF
parameters
The kmimf parameters include the standard MINUIT parameters and constants, but for
simplicity MINUIT is only “told” about the non­constant parameters that it is allowed to vary.
The parameter names are case sensitive and limited to KMIMF_MAXTAGNAME (currently 10) letters
and numbers and may not contain whitespace. If a parameter is not given a step size or it is given as 0,
the parameter is made a constant and not passed to MINUIT. Each parameter may have either both or
neither a lower and upper bound specified. Delimiters may be white space, "=", ":" or ",". Any
parameter whose name ends with a "_" is treated as an integer. If a parameter is specified more than
once, the most recent setting is always used.
-p parameter=value{:step,{lowlimit:highlimit}}
-pf filename
name:value
name=value
name=value:step
name value step
name:value:step,lowlimit:highlimit
name value step lowlimit highlimit
#
#
#
#
#
#
constant
constant
unbounded w/ initial step
unbounded w/ initial step
bounded
bounded
The file may have comments beginning with "#", but only one parameter/line. To list the current
parameters, use the +p option: $> cat km13.pf
# KM12 search starting points:
# name start step {low limit} {high limit}
t0a 2.2525894
0.15 #nS timeOffset
t0b 23.6972198 0.15 #nS timeOffset
t1a 28.2023987 0.15 #nS timeOffset
t1b 49.6470276 0.15 #nS timeOffset
t2a 54.1522095 0.15 #nS timeOffset
JLAB­TN­07­049
t2b 75.5968384 0.15 #nS timeOffset
t3a 80.1020142 0.15 #nS timeOffset
t3b 101.5466492 0.15 #nS timeOffset
j1 -2509.78
0
#A/mm2 in center coil
j2a 2919.675
0
#A/mm2 in leading coil
j2b 2919.675
0
#A/mm2 in trailing coil
$> kmimf -pf km13.pf -p t0a:3.0:0.5
+p -V
+++++++++++++++++++++++++++++++++++
1 t0a
0.300000E+01
0.500000E+00
2 t0b
0.233772E+02
0.150000E+00
3 t1a
0.278233E+02
0.150000E+00
4 t1b
0.489815E+02
0.150000E+00
5 t2a
0.534210E+02
0.150000E+00
6 t2b
0.745680E+02
0.150000E+00
7 t3a
0.790201E+02
0.150000E+00
8 t3b
0.100140E+03
0.150000E+00
9 j1
-0.251178E+04
10 j2a
0.291915E+04
11 j2b
0.291967E+04
+++++++++++++++++++++++++++++++++++
command script
The essential work is done by the command script. The script is just a single, possibly quite long,
string sent to the system shell that must do three things:
●
build the input file(s) for the simulation
●
run the simulation
●
process the results and write a single number to stdout as the FCN value.
kmimf substitutes the current parameter values in place of the named place holders (by default
"{name}") and sends that string to the system (usually Bourne, bsh, or Bourne Again, bash) shell for
evaluation. The script must result in a string being returned; that string must begin with a number that is
the result of the script; that number is the value returned by FCN. Extraneous characters are stripped off
if possible, so returned strings like "0.6376 cm" are fine. The script may be specified as a single string on the command line or as a file: ­i string
­if file
If read from a file, unquoted #'s begin comments and those comments are removed. Blank lines,
extraneous spaces, backslash \ continue character, and all linefeed and other control characters are
removed, so individual commands must be separated with a ";". For example: $> cat 3c.if
# SCAN4 script for kmimf minimization # passed to shell, must return a single #
export TWRK=`autoname -b 1 -f G%04d`;
mkdir $TWRK; cd $TWRK;
echo "g4beamline ../km14.in j1={j1} j2a={j2a} j2b={j2b} \
t0a={t0a} t0b={t0b} t1a={t1a} t1b={t1b} \
t2a={t2a} t2b={t2b} t3a={t3a} t3b={t3b}" > in.log;
g4beamline ../km14.in j1={j1} j2a={j2a} j2b={j2b} \
JLAB­TN­07­049
t0a={t0a} t0b={t0b} t1a={t1a} t1b={t1b} \
t2a={t2a} t2b={t2b} t3a={t3a} t3b={t3b} >& out.log;
txt2rms -NS +o -i g01.txt; txt2rms -NS +o -i g40.txt;
grep " Pz " g40.txt.rms | beshuffled -I "-#4 #t0a={t0a} $TWRK" | \
tail -1 | sdup -o g.out
After the aforementioned are removed and kmimf substitutions made, the string appears:
export TWRK=`autoname -b 1 -f G%04d`; ... j1=-2509.7800000000 ... -o g.out
In this case, the parameters j1, j2a, j2b, t0a, t0b, t1a, t1b, t2a, t2b, t3a,
t3b are all replaced with numbers by kmimf. It is important to understand that all the stdout from everything in the script is returned to
kmimf. The quantity of interest should always be 1st term in the 1st returned line; other terms may
follow. For example consider the shell command: $> today +f; pwd; echo "92.3 MeV RMS"
12Jan2006
/u/group/casa/acc_phys/6Dcooling/TOOLS/kmimf/0.0
92.3 MeV RMS
would be interpreted as 12.0 by kmimf, while $> today +f>d.d; pwd>>d.d; echo "92.3 MeV RMS"
92.3 MeV RMS
would be interpreted as 92.3 Example 1 – a trivial case
To illustrate the use of kmimf, let us just use it to minimize the function FCN = (x-2)*(x-2)
The parameter is x; let's give it a starting value of 3 and step size of 1, and tell it to quit after 10 passes,
log the search, be verbose, and when finished print the best result to stdout. The FCN function is echo '({x}-2)*({x}-2)' | bc -q -i
after substitution by kmimf becomes:
echo '(3.0000000000-2)*(3.0000000000-2)' | bc –q –i
which returns:
1.0000000000
Here, the input is just the string “(3.0000000000-2)*(3.0000000000-2)”and the simulation and analysis just a
calculator command “bc -q -i“. To work properly, both single and double quotes must be used to
protect the string from being interpreted by the shell, first when the kmimf command is given, and
again when the command string is sent to the shell to be evaluated:
$ kmimf -p x:3:1 -i "echo '({x}-2)*({x}-2)'|bc -q -i"
...
FCN: call#8 x=0.220000E+01
-a 10
-o x.log
-v
-F -
JLAB­TN­07­049
FCN: fcnval=> 0.400000E-01
FCN: call#9 x=0.260000E+01
FCN: fcnval=> 0.360000E+00
FCN: call#10 x=0.200000E+01
FCN: fcnval=> 0.000000E+00
SIMPLEX MINIMIZATION HAS CONVERGED.
x 1.9999999850988
# FCN:quit after cycle# 11
In general, it is good practice to use the --test option and to examine kmimf output to ensure
everything is working properly before starting a number of runs.
Example 2 ­ g4beamline
g4beamline10 runs under UNIX and reads a file describing a system to simulate and also
accepts values for various parameters within that file from the command line. In this beamline, muons
are sent through a series of solenoids. The currents in the two types solenoids can be varied, J1 and J2,
and we'd like to find the values to minimize the largest transverse beam size, RRMS, anywhere along the
path. The script file is g1.info:
rm -rf log; \
# just quietly forces the removal of a file
g4beamline 15aug05.in current1={J1} current2={J2} >& log; \ # {J1},{J2} are replaced w/ #s
retrack -F g4bl -i g*.txt -oP - | \
# converts G4 to KBB format
kbb2cyl -M mu -i - --RMS R --max --fmt f10.3 | \ #cylindrical coord & returns highest RRMS
tail -1
# keeps only the last line
While g4beamline performs the simulation, retrack11 analyzes its output, and kbb2cyl12
just converts retrack's output into cylindrical coordinates and returns the maximum value. The
typical kmimf command line and output would be:
$> kmimf -if g1.info +i +M --format f20.10 -p J1:-2708.517578:100 -p J2:2914.816:100 +p \
-o g1c.history -v
------------------------------rm -rf log; g4beamline 15aug05.in current1={J1} current2={J2} >& log; retrack -F g4bl -i g*.txt -oP - |
kbb2cyl -M mu -i - --RMS R --max --fmt f10.3 | tail -1
------------------------------$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
set warnings
show fcnvalue
simplex
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
+++++++++++++++++++++++++++++++++++
1 J1
-0.270852E+04
0.100000E+03
2 J2
0.291482E+04
0.100000E+03
+++++++++++++++++++++++++++++++++++
MINUIT RELEASE 96.03 INITIALIZED.
DIMENSIONS 100/ 50 EPSMAC= 0.89E-15
kmimf:MNINIT ok
kmimf:MINSETI("kmimf -if g1.info +i +M --format f20.10 -p J1:-270") ok
...
FCN: call#2 J1=-0.260852E+04 J2=0.291482E+04
10 http://www.muonsinc.com
11 file:/u/group/casa/acc_phys/6Dcooling/TOOLS/RETRACK/DOC/index.html
12 file:/u/group/casa/acc_phys/6Dcooling/TOOLS/KBB2CYL/DOC/index.html
JLAB­TN­07­049
FCN:
FCN:
FCN:
FCN:
FCN:
FCN:
FCN:
FCN:
FCN:
FCN:
FCN:
fcnval=> 0.900000E+01
call#3 J1=-0.274852E+04
fcnval=> 0.766300E+01
call#4 J1=-0.286852E+04
fcnval=> 0.889700E+01
call#5 J1=-0.274852E+04
fcnval=> 0.970000E+01
call#6 J1=-0.274852E+04
fcnval=> 0.799600E+01
call#7 J1=-0.274852E+04
fcnval=> 0.917600E+01
FCN=
7.663000
EXT PARAMETER
NO.
NAME
1
J1
2
J2
...
J2=0.291482E+04
J2=0.291482E+04
J2=0.301482E+04
J2=0.287482E+04
J2=0.275482E+04
FROM SIMPLEX
STATUS=PROGRESS
EDM= 0.71E+00
STRATEGY= 1
VALUE
-2748.5
2914.8
CURRENT GUESS
ERROR
100.00
100.00
6 CALLS
NO ERROR MATRIX
7 TOTAL
PHYSICAL LIMITS
NEGATIVE
POSITIVE
$> gnuplot
...
gnuplot> splot “g1c.history” u 2:3:4 w lp
Figure 2. kmimf history of FCN vs J1 vs J2.
The command string is sent to the shell and the response is evaluated by FCN. The +M and +p
just echo the MINUIT configuration commands and the initial parameter values, and the ­­format
f20.10 sets a convenient format for communicating to MINUIT (it apparently cannot read scientific
notation). Example 3 ­ LANL parmela
The LANL parmela only runs under Windows; to iteratively search for a solution using it
requires much patience, as each run may take many minutes. kmimf can be used to make this process
less tedious.
JLAB­TN­07­049
In this example, the goal is to adjust the two solenoids to reduce the longitudinal emittance of the
last bunch. To run the current LANL parmela with kmimf, I used a Windows­only machine,
phybeard3. For convenience, I used rdesktop13 to connect to it from a UNIX machine by:
$> rdesktop
-g 90%
phybeard3
rather than sit at the Windows machine, although that would work just as well. From there I clicked on
the Cygwin icon to begin; that started a simple UNIX emulation terminal window; to get a larger font, the
DISPLAY variable was set to the desktop and a larger xterm window started. Note that the bash shell is
in use:
$ export DISPLAY=phybeard3:0.0
$ alias vt100_grey
xterm -ls -fn 10x20 -cr red -132 +si -sb -ah +cb -sl 500 +t -geometry 80x24
grey08:58:43&
$ vt100_grey
-fg black -bg grey -title
Figure 3. Cygwin windows under WindowsXppro via rdesktop.
Since kmimf, axillary programs, and all required input files are all installed there, the kmimf
command script just changes the generic parmela input file by making a few substitutions:
cat 28acc.if | no_bs -s -SOL1- {sol1}
-s -SOL2- {sol2} -s -NPART- {Npart_} > INPUTFILE;
kmimf makes its own substitutions to the script before it is sent to the shell, so on the first pass the shell
13 http://www.rdesktop.org
JLAB­TN­07­049
is given:
cat 28acc.if | no_bs -s -SOL1- 0.9560000000
-s -SOL2- 1.0350000000 -s -NPART- 19 > INPUTFILE;
which changes 3 lines in the parmela input:
...
INPUT 9 -NPART- 5.1 0.5 5.5 11.0
INPUT 9 19 5.1 0.5 5.5 11.0
...
POISSON -24.7777 -SOL1- 1
POISSON -24.7777 0.9560000000 1
...
POISSON 0.0 -SOL2- 0.0 0.0
POISSON 0.0 1.0350000000 0.0 0.0
...
and then the script issues the command: $ parmela INPUTFILE >> EXISTINGLOGFILE;
The TAPE2.T2 output is converted by retrack14 to make a TJNAF parmela15 KBB format16 file
$> retrack -i TAPE2.T2 -oP KBBfile;
containing the normalized longitudinal emittance, emitZnorm, average, RMS, and other values. The
longitudinal emittance is extracted:
grep emittance "$WORKING".kbb | tail -1 | beshuffled -I "#10 um emitZnorm" | \
sdup +o $WORKING_PARMELA_LOG;
This number is the 1st thing sent to stdout and hence will be the FCN value returned to MINUIT.
Note that the parmela program opens its own windows; when complete, it closes and the other
programs run under Cygwin. Control is only passed back to MINUIT when the script completes its task,
and back to kmimf when MINUIT completes its task.
To start the process, kmimf is given a file containing the command string, another containing the
parameters, told to abort after 30 passes, told to keep a log file, and told to be verbose. All stdout from
kmimf is both logged and sent to the screen:
$ kmimf -if 28aug.if -pf 28aug.pf -a 30 -o d3b.log | sdup -o d3b.syslog
Since MINUIT often tends to “rattle around” a lot and often never really converges to an exact value, it
is wise to limit the the number of passes.
14 file:/u/group/casa/acc_phys/6Dcooling/TOOLS/RETRACK/DOC/index.html
15Parmela_fel0.2: FEL Injector Simulation, K. B. Beard, B. Yunn, C. H. Garcia, JLAB­TN­03­028
16 http://www.jlab.org/~beard/tracking_formats.html#Pfmt
JLAB­TN­07­049
Figure 4. Shortly after issuing the kmimf command, the parmela window opens.
# kmimf -if 28aug.if -pf 28aug.pf -a 30 -o d3b.log -v
# kmimf 0.0f3, 12jan2006
# 18:09:42 Aug 28, 2007
#MINUIT: set warnings|show fcnvalue|simplex
#SCRIPT-template: export WORKING=`autoname -f wrk%d`; export WORKING_PARMELA_LOG="$WORKING".log; export
WORKING_PARMELA_INPUT="$WORKING".acc; export WORKING_PARMELA_OUTPUT="$WORKING".out; rm -rf $WORKING
$WORKING_PARMELA_LOG $WORKING_PARMELA_INPUT $WORKING_PARMELA_OUTPUT; mkdir $WORKING; today -f "%d%m%y %
t" > $WORKING_PARMELA_LOG; cat 28aug.acc | no_bs -s -SOL1- {sol1} -s -SOL2- {sol2} -s -NPART- {Npart_} >
$WORKING_PARMELA_INPUT; diff 28aug.acc $WORKING_PARMELA_INPUT >> $WORKING_PARMELA_LOG; grep 00000
$WORKING_PARMELA_INPUT >> $WORKING_PARMELA_LOG; parmela $WORKING_PARMELA_INPUT >> $WORKING_PARMELA_LOG;
echo "#..............." >> $WORKING_PARMELA_LOG; today -f "#... %d%m%y %t" >> $WORKING_PARMELA_LOG;
retrack -i TAPE2.T2 -oP "$WORKING".kbb >> $WORKING_PARMELA_LOG; grep emittance "$WORKING".kbb | tail -1
| beshuffled -I "#10 um emitZnorm" | sdup +o $WORKING_PARMELA_LOG; echo "#..............." >>
$WORKING_PARMELA_LOG; mv "$WORKING".* TAPE3.T3 TAPE2.T2 OUTPAR.TXT $WORKING; rm -rf *.TBL *.tbl SAVECOR*;
#+++starting values: name start step {lowlimit highlimit}
# Npart_
0.199000E+03
# sol1
0.956000E+00
0.300000E+00
-0.400000E+01
0.400000E+01
# sol2
0.103500E+01
0.300000E+00
-0.400000E+01
0.400000E+01
#--# cycle Npart_ sol1 sol2 FCN
1 199.0000000000 0.9560000000 1.0350000000
0.337668E+02
# 0.337668E+02 um emitZnorm
2 199.0000000000 1.2528515794 1.0350000000
0.668428E+02
# 0.668428E+02 um emitZnorm
3 199.0000000000 0.8355628619 1.0350000000
0.517950E+02
# 0.517950E+02 um emitZnorm
4 199.0000000000 1.0039257786 1.0350000000
0.283109E+02
# 0.283109E+02 um emitZnorm
...
Of course, kmimf duly hunts for a solution – but here it found a solution that discards most of
the particles (fewer, more central, particles gives a smaller emittance). Again, be careful of what you
ask for; you might get it.
JLAB­TN­07­049
Figure 5. Initial (pass#0) and pass#10 solution when emitZnorm minimized without attention to the number of particles. What we want is to get all (or almost all) of the particles (N) to the end and minimize the
longitudinal emittance there. To reflect that goal, the FCN value must be constructed to reflect that. One
way to do that is to define FCN = emitZnorm – 100 * N
Since the longitudinal emittance is ~34 um, each particle weights about 3 times the emittance value.
To calculate that quantity, first, the mean & RMS info in the KBB file and emittance numbers are
extracted and stored in separate files:
grep RMS "$WORKING".kbb | sed -e "s/#mean,RMS://" > "$WORKING".kbb.rms;
grep emittance "$WORKING".kbb | sed -e "s/#emittance://" > "$WORKING".emit;
Then the value of emitZnorm is taken from the last line of the emittance file and the value for N from the
last line of the mean & RMS file. Both are put into an algebraic string to be evaluated:
# return emitZnorm - 100*#particles
echo "`tail -1 "$WORKING".emit|beshuffled -1 -I 10` - 100* \
`tail -1 "$WORKING".kbb.rms|beshuffled -1 -I 16`" | \
kbbmath -i | sdup +o $WORKING_PARMELA_LOG;
The output of the echo command looks like:
0.337668E+02 – 100*
200
The kbbmath program evaluates the expression and returns
-19997.662109
but in retrospect a better choice would have been to use the standard bc utility:
bc -q -i | tail -1
JLAB­TN­07­049
Rerunning kmimf with the new command string:
$ kmimf -if 28aug.if -pf 28aug.pf -a 100 -v -o d3c.log
Figure 6. Initial (pass#0) and pass#11 minimizing FCN.
Now all the particles make it to the end and the longitudinal emittance is smaller. Looking at
various variables as a function of pass:
Figure 7. History of search, showing N (number of surviving particles), KERMS, XRMS, YRMS, tRMS, and emitZnorm.
It is clear that MINUIT gets into a rut and never quite settles down, and that by pass#11 things
are about as good as they're going to get.
Plotting the history another way:
JLAB­TN­07­049
Figure 8. FCN vs. sol1 vs. sol2, blue dot marks pass#12.
MINUIT varies the parameters in the same order that they were specified to kmimf, so the order
in which they are defined is important. Also, MINUIT may be passed options to improve the
performance, but generally the user must look at the results to see if it all makes sense. JLAB­TN­07­049
Appendix 1. 28aug.acc – template for LANL parmela input Four additional files containing field maps are required for parmela in addition to its input
file, but the maps do not change between runs.
run 1 2 749.859 -.0076424 1.e-5
title
JLab Gun with accelerator
INPUT 9 -NPART- 5.1 0.5 5.5 11.0
DRIFT 0 5.0 1
cathode 0 .64 1 0.0 0.0
POISSON -24.7777 -SOL1- 1
JLABGE2A.PO7
drift 1.0 2.0 1
drift 1.0 2.0 1
drift 2.0 2.0 1
drift 2.0 2.0 1
drift 2.0 2.0 1
drift 2.0 2.0 1
drift 2.0 2.0 1
drift 2.0 2.0 1
drift 2.0 2.0 1
POISSON 0.0 -SOL2- 0.0 0.0
SOLLNEW3.PO7
;Next drift reg. is solenoid place holder, don't change
drift 15.0 2.0 1
drift 12.0 2.0 1
drift 11.0 2.0 1
cell 62.96668 5.0 1 -160.0 6.00 1 1.0 0 749.859
cfield 1
IC2&3D2.t7
drift 9.01666 2.0 1
drift 21.005554 2.0 1
cell 20.988893 2.0 1 -49.0 4.42 3 1.0 0 2250
cfield 3
HC3&3D02.t7
drift 21.005554 2.0 1
drift 9.01666 2.0 1
cell 62.96668 5.0 1 26.0 7.46 2 1.0 0 749.859
cfield 2
IC2&3D2.t7
drift 9.03332 2.0 1
cell 62.96668 5.0 1 95.0 7.46 2 1.0 0 749.859
drift 100.0 2.0 1
zout
OUTPUT 1
scheff 0.11 1.5 2.3 70 200 0 0 5 1.5 0 0.0
start 0.0 0.2 15000 1 100
save 1
end
JLAB­TN­07­049
Appendix 2. 28aug.if – kmimf command string
# little test script for the LANL parmela under KMIMF
export WORKING=`autoname -f wrk%d`;
#finds next wrk0, wrk1, ... that doesn't already exist
export WORKING_PARMELA_LOG="$WORKING".log;
#make files or form wrk0.log,...
export WORKING_PARMELA_INPUT="$WORKING".acc;
export WORKING_PARMELA_OUTPUT="$WORKING".out;
rm -rf $WORKING "$WORKING".*;
#clean start
mkdir $WORKING;
#make directory wrk0,...
today -f "%d%m%y %t" > $WORKING_PARMELA_LOG;
#record date & time #make substitutions to parmela input file:
cat 28aug.acc | \
no_bs -s -SOL1- {sol1} -s -SOL2- {sol2} \
-s -NPART- {Npart_} > $WORKING_PARMELA_INPUT;
#create simulation input file
diff 28aug.acc $WORKING_PARMELA_INPUT >> $WORKING_PARMELA_LOG;
#record changes
grep 00000 $WORKING_PARMELA_INPUT >> $WORKING_PARMELA_LOG;
#run parmela:
parmela $WORKING_PARMELA_INPUT >> $WORKING_PARMELA_LOG;
#run simulation, log any messages
echo "#..............." >> $WORKING_PARMELA_LOG;
today -f "#... %d%m%y %t" >> $WORKING_PARMELA_LOG;
#record date & time #find final normalized Z emittance
retrack -i TAPE2.T2 -oP "$WORKING".kbb >> $WORKING_PARMELA_LOG;
#create KBB file with analysis
grep RMS "$WORKING".kbb | sed -e "s/#mean,RMS://" > "$WORKING".kbb.rms;
#store mean,RMS values
grep emittance "$WORKING".kbb | sed -e "s/#emittance://" > "$WORKING".emit; #store emittance values
# return emitZnorm + 100*#particles
echo "`tail -1 "$WORKING".emit|beshuffled -1 -I 10` - 100* \
`tail -1 "$WORKING".kbb.rms|beshuffled -1 -I 16`" | \
kbbmath -i | sdup +o $WORKING_PARMELA_LOG;
#calculate and return FCN value
echo "#..............." >> $WORKING_PARMELA_LOG;
mv "$WORKING".* TAPE3.T3 TAPE2.T2 OUTPAR.TXT $WORKING;
#store everything import into directory wrk0,...
rm -rf *.TBL *.tbl SAVECOR*;
#cleanup
JLAB­TN­07­049
Appendix 3. 28aug.pf – kmimf parameter file
#
#
# name value {step {lowlimit highlimit}}
Npart_ 199
sol1
0.956 0.3
-4.
4.
sol2
1.035 0.3
-4.
4.
#
# change from initial
#[#macroparticles]
#[gauss]
#[gauss]
JLAB­TN­07­049
Appendix 4. kmimf internal structure
kmimf
main
KMIMF
KMIMF_HINTS
NO_UNQUOTED_COMMENTS
M2FCN
FCN
KMIMF_MAKEFINALR
EPORT
NEARESTF8I4
NEWCMD
NEARESTF8I4
FCN_LOG
HUNT_LOG
NEARESTF8I4
KMIMF_MAKEFINALREPORT
NEARESTF8I4
MNINIT
MNSETI
MNPARS
MNCOMD
MNPOUT
Key: [croutine] [FORTRANroutine] [recursive] ...linking: FORTRAN: XXXX(...) => C: xxxx_(...)
[ SRC_0.1f7/fcn.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] SUBROUTINE FCN( NPAR, GRAD, FCNVAL, XVAL, FLAG, FUTIL ) INTEGER NPAR ­ ! # of parameters REAL*8 GRAD(*) ­ ! returned gradients REAL*8 FCNVAL ­ ! return value of FCN REAL*8 XVAL(*) ­ ! current parameter values INTEGER FLAG ­ ! control flag EXTERNAL FUTIL ­ ! auxillary function *
* fills in the template using the current xval AS COUNTED
* BY MINUIT, returns the value of the function *
*
*
called by: M2FCN JLAB­TN­07­049
calls: KMIMF_MAKEFINALREPORT , NEARESTF8I4 , NEWCMD , FCN_LOG commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL [ SRC_0.1f7/fcn_log.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] SUBROUTINE FCN_LOG( CMND, RESULT ) CHARACTER*(*) CMND ­ !actual command sent to system REAL*8 RESULT ­ !result returned from system *
* Log the current parameters values and the
* FCN result (BEFORE max/min/nearest switch)
* if the logging channel is open. Update the
* parameter output file if open.
*
*
*
called by: FCN calls: HUNT_LOG , NEARESTF8I4 commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL [ SRC_0.1f7/hunt_log.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] SUBROUTINE HUNT_LOG( CMND, RESULT ) CHARACTER*(*) CMND ­ !actual command sent to system REAL*8 RESULT ­ !result returned from system *
* Log the current parameters values and the FCN reslt
* if the logging channel is open. Update the
* parameter output file if open.
*
*
*
called by: FCN_LOG commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL JLAB­TN­07­049
[ SRC_0.1f7/kmimf.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] INTEGER FUNCTION KMIMF( ) *
* attempt to minimize the output of a program
* or script ­ inspired by Tom Robert's tune
* program
*
* 8/17/05 KBB
*
called by: main calls: KMIMF_HINTS , NO_UNQUOTED_COMMENTS , M2FCN ,
KMIMF_MAKEFINALREPORT , NEARESTF8I4 , MNINIT , MNSETI , MNPARS , MNCOMD ,
MNPOUT commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL [ SRC_0.1f7/kmimf_hints.f ]
SUBROUTINE KMIMF_HINTS( O ) INTEGER O ­ !opened channel for writing *
* Just print some guidance on running kmimf
*
*
*
called by: KMIMF [ SRC_0.1f7/kmimf_makefinalreport.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] SUBROUTINE KMIMF_MAKEFINALREPORT( MESSAGE ) CHARACTER*(*) MESSAGE ­ ! comment to append to report *
* Make a brief final report.
*
called by: FCN , KMIMF commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL JLAB­TN­07­049
[ SRC_0.1f7/m2fcn.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] SUBROUTINE M2FCN( N, V, RSLT ) INTEGER*4 N ­ !total #of parameters including constants REAL*8 V(*) ­ !list of all parameters REAL*8 RSLT ­ !returned result *
* Wrapper that selects only those parameters to be
* passed to MINUIT
*
*
called by: KMIMF calls: FCN commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL [ SRC_0.1f7/mncomd.f ]
SUBROUTINE MNCOMD( FUNC, CMD, RETURNSTATUS, AUXFUNC ) EXTERNAL FUNC ­ !function that MINUIT should use CHARACTER*(*) CMD ­ !command string INTEGER RETURNSTATUS ­ !success=0, other failure EXTERNAL AUXFUNC ­ !auxillary function, 0=NONE *
* noMINUIT command
*
called by: KMIMF [ SRC_0.1f7/mninit.f ]
SUBROUTINE MNINIT( IOIN, IOOUT, IOSAV ) INTEGER IOIN ­ ! IO channel for read INTEGER IOOUT ­ ! IO channel for write INTEGER IOSAV ­ ! IO channel for file read *
* noMINUIT initialization
*
JLAB­TN­07­049
called by: KMIMF [ SRC_0.1f7/mnpars.f ]
SUBROUTINE MNPARS( COMMAND, RETURNSTATUS ) CHARACTER*(*) COMMAND ­ !form "id NAME start step {low high} INTEGER RETURNSTATUS ­ !success=0, other failure *
* noMINUIT set parameters
*
called by: KMIMF [ SRC_0.1f7/mnpout.f ]
SUBROUTINE MNPOUT( ITH, NAME, VAL, ERR, LO, HI, STATUS ) INTEGER ITH ­ !# of parameter as seen by FCN CHARACTER*(*) NAME ­ !name of parameter REAL*8 VAL ­ !value of parameter REAL*8 ERR ­ !error estimate on value REAL*8 LO ­ !lower limit, (zero if unused) REAL*8 HI ­ !upper limit, (zero if unused) INTEGER STATUS ­ !return 0=not calculated, 1=diagnal approx only, ful matrix, full covarient
matri* *
* noMINUIT return current parameter value
*
called by: KMIMF [ SRC_0.1f7/mnseti.f ]
SUBROUTINE MNSETI( TITLE ) CHARACTER*(*) TITLE ­ ! title line *
* noMINUIT set title
*
*
called by: KMIMF JLAB­TN­07­049
[ SRC_0.1f7/nearestf8i4.f ]
SUBROUTINE NEARESTF8I4( F8, I4 ) REAL*8 F8 ­ !input INTEGER*4 I4 ­ !output *
* Return the nearest integer i4 to the
* floating number f8. Does not consider
* cases where range of I*4 exceeded.
*
*
*
called by: FCN , FCN_LOG , KMIMF , NEWCMD [ SRC_0.1f7/newcmd.f ]
[ kmimf.par ] [ kmimf.cmn ] [ kmimf_pref.cmn ] SUBROUTINE NEWCMD( TEMPLATE, COMMAND ) CHARACTER*(*) TEMPLATE ­ !input template CHARACTER*(*) COMMAND ­ !output commandstring *
* Substitute current parameters into the command
* string replacing {TAG} with #
*
*
called by: FCN calls: NEARESTF8I4 commons: KM_2MN, KM_PARAMETERS, KM_RESULTS, KM_PREVIOUS, KM_BUFFERS,
KM_INFO, KM_SUBLIMITS, KMIMF_PREFS, KMIMF_QUIT, KMIMF_FINAL [ SRC_0.1f7/no_unquoted_comments.f ]
SUBROUTINE NO_UNQUOTED_COMMENTS( STR, COMMENTFLAG ) CHARACTER*(*) STR ­ !input/output string CHARACTER*(*) COMMENTFLAG ­ !beginning­of­comment delimiters *
* Removes everything after a commentflag character (typically #!)
* unless they are inside "" '' `` (possibly nested) quotes
* * JLAB­TN­07­049
*
*
called by: KMIMF [ kmimf_main.c]
int main( int argc, char *argv[] ) int argc ­ char *argv[] ­ Kevin's WRAPper
http://www.jlab.org/~beard
defined during compilation
calls: KMIMF COMMON blocks
COMMON
block name
group#1
routines
KMIMF_FINAL
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
LOGICAL*4 FINALSCIENTIFICNOTATION
LOGICAL*4 FINALREPORTONSINGLELINE
CHARACTER*(KMIMF_MAXFILENAME) FINALREPORTFILE
CHARACTER*(KMIMF_MAXTAGNAME) FINALDELIMITER
KMIMF_PREFS
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
LOGICAL*4 VERBOSE
LOGICAL*4 DEBUG
INTEGER IOSAV
INTEGER PFSAV
INTEGER HUSAV
CHARACTER*(KMIMF_MAXCOMMAND) R8FMT
KMIMF_QUIT
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
JLAB­TN­07­049
INTEGER*4 QUITAFTERSOMANYCYCLES
KM_2MN
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
CHARACTER*(KMIMF_MAXSCRIPT) CMD2MN
KM_BUFFERS
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
INTEGER*4 LSCRIPT_TEMPLATE
INTEGER*4 LSCRIPT_RESPONSE
CHARACTER*(KMIMF_MAXSCRIPT) SCRIPT_TEMPLATE
CHARACTER*(KMIMF_MAXREPLY) SCRIPT_RESPONSE
CHARACTER*(KMIMF_MAXSCRIPT) SCRIPT
CHARACTER*(KMIMF_MAXREPLY) SCRATCH
KM_INFO
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
CHARACTER*(KMIMF_MAXLINE) COMMANDLINE
CHARACTER*(KMIMF_MAXLINE) CURRENTTIME
CHARACTER*(KMIMF_MAXLINE) PROGRAM
CHARACTER*(KMIMF_MAXLINE) VERSIONINFO
LOGICAL*4 KMIMF_MAXIMUM
LOGICAL*4 KMIMF_NEAREST
REAL*8 KMIMF_NEARESTGOAL
CHARACTER*(KMIMF_MAXTITLE) TITLE
KM_PARAMETERS
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
CHARACTER*(KMIMF_MAXTAGNAME) PAR_NAME(KMIMF_MAXTAGS)
REAL*8 PAR_X(KMIMF_MAXTAGS)
REAL*8 PAR_DX(KMIMF_MAXTAGS)
REAL*8 PAR_LO(KMIMF_MAXTAGS)
REAL*8 PAR_HI(KMIMF_MAXTAGS)
LOGICAL*4 PAR_LIMIT(KMIMF_MAXTAGS)
LOGICAL*4 PAR_FIXED(KMIMF_MAXTAGS)
REAL*8 PAR_X_INITIAL(KMIMF_MAXTAGS)
REAL*8 PAR_DX_INITIAL(KMIMF_MAXTAGS)
INTEGER*4 PAR_MID(KMIMF_MAXTAGS)
INTEGER*4 N_PAR
INTEGER*4 M_PAR
INTEGER*4 PAR_M2K(KMIMF_MAXTAGS)
LOGICAL*4 PAR_ASI(KMIMF_MAXTAGS)
JLAB­TN­07­049
KM_PREVIOUS
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
REAL*8 PREV_PAR_X(KMIMF_MAXTAGS)
REAL*8 PREV_FCN
REAL*8 DRIFT_PAR_X(KMIMF_MAXTAGS)
KM_RESULTS
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
REAL*8 INIT_FCN
REAL*8 LAST_FCN
REAL*8 BEST_FCN
INTEGER*8 BEST_NTHCALL
REAL*8 PAR_X_BEST(KMIMF_MAXTAGS)
KM_SUBLIMITS
FCN FCN_LOG HUNT_LOG KMIMF KMIMF_MAKEFINALREPORT
M2FCN NEWCMD
CHARACTER*1 KMIMF_BEGINSUB
CHARACTER*1 KMIMF_ENDSUB
Produced using splitcf by Dr. K.B.Beard [splitcf v2.6f10 8/23/2005 Dr. K.B.Beard]