The tar.gz file is not printed

[stine_T]

Hi

I am having the same problem as last time, since I cannot get my tar.gz file (or molden.inp by -get). But my jobs run without any problems and I get the .out file.

I am writing the following:

/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton -t /scratch/\$SLURM_JOB_ID -N 8 -M 4000 -get molden.inp -d -o /polar/${kerne}.out /*u.dal /$molfil /${kerne}.guld

And I get the following error message in my *.err files:

....
1: Directories for basis set searches:
/kemi/stine/PhD/Northewestern/Dithienyl_small/junction_MO/E0/sikkerhedudenoverskriv:/users/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/basis


0: Directories for basis set searches:
/kemi/stine/PhD/Northewestern/Dithienyl_small/junction_MO/E0/sikkerhedudenoverskriv:/users/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/basis
/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton: line 502: /*u_/diethenyl_small.rot_/diethenyl_small.rot.guld.tar.gz: No such file or directory
/*u_/diethenyl_small.rot_/diethenyl_small.rot.guld.tar.gz has not been created and has thus not been copied to /kemi/stine/PhD/Northewestern/Dithienyl_small/junction_MO/E0/sikkerhedudenoverskriv
copying molden.inp back from scratch directory
cp: cannot create regular file `/kemi/stine/PhD/Northewestern/Dithienyl_small/junction_MO/E0/sikkerhedudenoverskriv//*u_/diethenyl_small.rot_/diethenyl_small.rot.guld.molden.inp': No such file or directory
Output is in /kemi/stine/PhD/Northewestern/Dithienyl_small/junction_MO/E0/sikkerhedudenoverskriv//polar/diethenyl_small.rot.out as requested in input.
grep: /polar/diethenyl_small.rot.out: No such file or directory
/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton: line 553: [: too many arguments
grep: /polar/diethenyl_small.rot.out: No such file or directory
/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton: line 567: [: -gt: unary operator expected
grep: /polar/diethenyl_small.rot.out: No such file or directory
/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton: line 575: [: -gt: unary operator expected



So it its something with my path, but I do not get how it can read my path for the input and output files with out any problems - and then failing when I want my molden.inp or my tar.gz file.
Any suggestions?

Best Regards
Stine

[taylor]

Once again, and I posted on this just two days ago, your failure to post your complete output file does not help. But you seem to have a command line that includes

Code: Select all

/polar/${kerne}.out

Do you really have permissions on your system that lets you create a top-level directory /polar ? (And if so, is this a good idea?)

Best regards
Pete

[stine_T]

I am not sure I understand you.
It is not a toplevel directory. Polar is just a directory I have in the same level where my input files is - thus it is just a folder collecting all my output files so they are separately collected from my inputfiles (and my .out files come without problems into this folder).

[taylor]

I am, in turn, even more confused. You specify a filename (note that a filename that begins with "/" is treated as an absolute pathname in every flavour of UNIX/Linux and in every shell underneath those systems)

Code: Select all

/polar/${kerne}.out

In any flavour of UNIX/Linux I know of, this will (try to) create a file in a directory called "/polar" under the top-level system directory "/", that is the root of the system filesystem. Do you have permission to create files under "/"? And if you do, is this really a good idea??

Best regards
Pete

[erik]

Hi Stine

it could be a simple syntax problem - try to use -get "molden.inp"

best regards
Erik

[taylor]

It seems to me on reflection that you want to create a file called <something-or-other.out> in a directory called "polar" underneath whatever directory you are trying to run a job from? In that case you need to start by removing leading "/" from your filenames! I repeat my posting of a moment ago: if you use a leading "/" any shell for UNIX/Linux will assume you are specifying an absolute pathname. The shell will not prepend (e.g.) /home/me to the filename, nor will it turn it into /scratch/petes_directory/my_dalton_job_scratch_dir/filename. Any filename that begins with a "/" will be treated as an explicit, complete filename: not just a file, but a complete filename including (the system will presume) a directory path as part of it. This has been the UNIX standard for more than 45 years. Which is not as old as me, but that's another story...

Best regards
Pete

[erik]

Another thing...
I am not sure that the -get command automatically direct files the same place as the outout, so if you direct the ouput to another folder i would not count on that -get is "aware" of this

with best regards
Erik

[stine_T]

Thanks for the replies.

Pete: Yes, I want to create a file called 'something-or-other.out' (my output file from my job) in a directory called "polar" underneath whatever directory I am trying to run my job from. And it is working fin - The problem is that Dalton is not printing the tar.gz file for my job - which should be default for Dalton according to the manual.

Since I am only interested in the MO-information I could live without the tar.gz file if I then could get the molden.inp file - Which I also fails to when I am using the option -get molden.inp

Erik: I have now set up and calculation where I specify where to put the molden.inp file, and I hope this will work. However, it does not explain why the tar.gz does not show up since it should be there pr default.

[kennethruud]

Looking at your command line, I am also concerned that you are directing the program to search for the *dal and *mol files in the root directory:
/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton -t /scratch/\$SLURM_JOB_ID -N 8 -M 4000 -get molden.inp -d -o /polar/${kerne}.out /*u.dal /$molfil /${kerne}.guld

You also use *u.dal: I do not know if this is meant to imply that you want to run a lot of dalton calculations with the same mol file but different dal-files? For sure the dalton run script will not be able to handle this, and you need to do the loop over multiple input files through a loop structure in your shell.

Anyway, the consequence of these things, is that I believe the program never starts (or rather, it starts, but finds no input files), thus aborts and no *tar.gz nor molden.inp file is created.

As Peter already indicated, helping you is a lot easier if you would allow us to see the output file from the calculation.


Kenneth

[stine_T]

Kenneth: I can give you the start and end of my output file (it is to long to be here):

.... ALL the standard of who made Dalton ......
....
Date and time (Linux) : Thu May 5 21:15:50 2016
Host name : node043.cluster

* Work memory size : 512000000 = 3.815 gigabytes.

* Directories for basis set searches:
1) /kemi/stine/PhD/Northewestern/Dithienyl_small/junction_MO/E0/sikkerhedudenoverskriv
2) /users/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/basis


Compilation information
-----------------------

Who compiled | stine
Host | fend05.hpc.ku.dk
System | Linux-2.6.18-348.3.1.el5
CMake generator | Unix Makefiles
Processor | x86_64
64-bit integers | OFF
MPI | ON
Fortran compiler | /kemi/stine/programs/openmpi/bin/mpif90
Fortran compiler version | ifort (IFORT) 13.0.1 20121010
C compiler | /kemi/stine/programs/openmpi/bin/mpicc
C compiler version | icc (ICC) 13.0.1 20121010
C++ compiler | /kemi/stine/programs/openmpi/bin/mpicxx
C++ compiler version | unknown
Static linking | OFF
Last Git revision | 575a5cf6d2c5c3776735f11fa026dbfca0382455
Configuration time | 2013-11-12 17:59:05.018703

* Parallel with MPI: Using 8 CPUs.


Content of the .dal input file
----------------------------------

**DALTON
.RUN RESPONSE
.DIRECT
**INTEGRALS
.DIPLEN
.NOSUP
.PRINT
2
*END OF HERMIT
**WAVE FUNCTIONS
.DFT
CAMB3LYP
*POPULATION ANALYSIS
.ALL
*SCF INP
.MAX DIIS
300
.PRINT
2
*OPTIMIZATION
.MAX MICRO
200
.MAX MACRO
100
**RESPONSE
*LINEAR
.DIPLEN
.FREQUE
1
0.0
**END OF


Content of the .mol file
----------------------------

ATOMBASIS
diethenyl_small.rot.xyz
Hej Mia
Atomtypes=3 Charge=0 Nosymmetry Angstrom
16.0000 4 Bas=cc-pVDZ
S 9.931534612 -0.020019809 -0.417721735
S 6.261153899 0.072104315 -0.412120799
S 12.892545681 0.000000000 0.000000000
S 3.300000000 0.000000000 0.000000000
6.0000 15 Bas=cc-pVDZ
C 8.826257313 -0.158327407 3.372791966
C 7.367310342 -0.177432404 3.371873106
C 6.677959168 -0.250595246 2.223028286
C 7.449885738 -0.460334910 0.915545195
C 8.742941828 0.373028824 0.958158684
C 9.514616178 0.029029893 2.236416264
C 10.939272422 0.032114888 2.012123528
C 11.287064949 0.010063898 0.715083060
C 5.253271355 -0.235347016 1.998590843
C 4.905580111 -0.080511957 0.710920353
C 7.690970124 -1.968187032 0.743731122
C 8.501998552 1.890460062 0.943516770
C 9.323743101 -0.320028529 4.789363449
C 6.883758831 -0.171983405 4.799214096
C 8.074353275 -0.777045154 5.566663409
1.0000 14 Bas=cc-pVDZ
H 11.660939525 0.025688239 2.819852699
H 4.531500786 -0.307846139 2.802982697
H 6.739734108 -2.494965165 0.819400992
H 8.134521725 -2.197645896 -0.223127038
H 8.355823809 -2.337037261 1.526042543
H 9.453217496 2.406441455 1.074200839
H 8.059030925 2.219612879 0.005622256
H 7.836645005 2.176141503 1.759661289
H 10.152337993 -1.026544415 4.875974658
H 9.685210472 0.640987686 5.172911676
H 6.699429424 0.855995636 5.133621672
H 5.957479896 -0.729804802 4.948721276
H 8.096965739 -0.481334523 6.616620176
H 8.004766442 -1.867551335 5.535600785


*******************************************************************
*********** Output from DALTON general input processing ***********
*******************************************************************

--------------------------------------------------------------------------------
Overall default print level: 0
Print level for DALTON.STAT: 1

Parallel calculation using MPI
AO-direct calculation (in sections where implemented)
HERMIT 1- and 2-electron integral sections will be executed
"Old" integral transformation used (limited to max 255 basis functions)
Wave function sections will be executed (SIRIUS module)
Dynamic molecular response properties section will be executed (RESPONSE module)
--------------------------------------------------------------------------------


****************************************************************************
*************** Output of molecule and basis set information ***************
****************************************************************************


The two title cards from your ".mol" input:
------------------------------------------------------------------------
1: diethenyl_small.rot.xyz
2: Hej Mia
------------------------------------------------------------------------

Coordinates are entered in Angstrom and converted to atomic units.
- Conversion factor : 1 bohr = 0.52917721 A

Atomic type no. 1
--------------------
Nuclear charge: 16.00000
Number of symmetry independent centers: 4
Number of basis sets to read; 2
The basis set is "cc-pVDZ" from the basis set library.
Basis set file used for this atomic type with Z = 16 :
"/users/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/basis/cc-pVDZ"

Atomic type no. 2
--------------------
Nuclear charge: 6.00000
Number of symmetry independent centers: 15
Number of basis sets to read; 2
The basis set is "cc-pVDZ" from the basis set library.
Basis set file used for this atomic type with Z = 6 :
"/users/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/basis/cc-pVDZ"

Atomic type no. 3
--------------------
Nuclear charge: 1.00000
Number of symmetry independent centers: 14
Number of basis sets to read; 2
The basis set is "cc-pVDZ" from the basis set library.
Basis set file used for this atomic type with Z = 1 :
"/users/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/basis/cc-pVDZ"


SYMGRP: Point group information

......... A LOT MORE ....................

...... Ending with:

*** THE REQUESTED 1 SOLUTION VECTORS CONVERGED

Convergence of RSP solution vectors, threshold = 1.00D-03
---------------------------------------------------------------
(dimension of paired reduced space: 16)
RSP solution vector no. 1; norm of residual 8.97D-04

*** RSPCTL MICROITERATIONS CONVERGED


Final output of second order properties from linear response
------------------------------------------------------------


@ Spin symmetry of operators: singlet

Note that minus the linear response function: - << A; B >>(omega) is printed.
The results are of quadratic accuracy using Sellers formula.

@ FREQUENCY INDEPENDENT SECOND ORDER PROPERTIES

@ -<< XDIPLEN ; XDIPLEN >> = 6.185705901948D+02
@ -<< XDIPLEN ; YDIPLEN >> = 2.083122822484D+00
@ -<< XDIPLEN ; ZDIPLEN >> = 7.942514189770D-01
@ -<< YDIPLEN ; YDIPLEN >> = 1.271010840095D+02
@ -<< YDIPLEN ; ZDIPLEN >> = -7.263007265385D+00
@ -<< ZDIPLEN ; ZDIPLEN >> = 2.481163288245D+02


Time used in linear response calculation is 6680.82 CPU seconds for symmetry 1

>>>> Total CPU time used in RESPONSE: 1 hour 51 minutes 21 seconds
>>>> Total wall time used in RESPONSE: 1 hour 51 minutes 21 seconds


.-------------------------------------------.
| End of Dynamic Property Section (RESPONS) |
`-------------------------------------------'

>>>> Total CPU time used in DALTON: 3 hours 23 minutes 35 seconds
>>>> Total wall time used in DALTON: 3 hours 23 minutes 36 seconds


Date and time (Linux) : Fri May 6 00:39:26 2016
Host name : node043.cluster




So as you can see I do not have any problems by running the job, and hence reading my inputfiles (here a file with my molecule (/$molfil) and a file with my metal clusters ( /${kerne}.guld)).
The reason I am writing *u.dal is that each folder I have, there is a lot of different molecules and their corresponding clusters (running QM/MM) and it is the same job-type I run for each system, hence the same *dal file (only one dal file in each folder)).

I do not if it would be easier to understand what I mean if I print the entire script that I am using to submit my job, in case, here it is:

#!/bin/bash

# to be executed from OPV1/E0/CAM-B3LYP/ or similar

navn=${PWD##*/stine/}
navn2=${PWD##*/stine/}

molfil=${1:?"du glemte angive mol fil"}
kerne=${molfil%.mol}
LLpol='polfil'


###### KIG HER #######
QUEUE=kemi4 # queue system
DAL_CORES=8 # antal cpuer
MEM=4 # memmory


# polarization calculation

cat > $LLpol <<!EOSLURM
#!/bin/bash
#SBATCH --job-name=${kerne}
#SBATCH -p $QUEUE
#SBATCH -c $DAL_CORES
#SBATCH -n 1
#SBATCH -N 1
#SBATCH --mem=$(( $MEM*1024 ))
#SBATCH --time=1336:00:00
#SBATCH -o ${kerne}-%j.err
$arraycmd



export SCR=/scratch/\$SLURM_JOB_ID


echo Running on \$HOSTNAME
echo JobID: \SLURM_JOB_ID
date


# EXPORT OPENMPI HERE
source /kemi/stine/.openmpi

# EXPORT INTEL STUFF HERE
source /software/kemi/intel/Compiler/composer_xe_2013/bin/compilervars.sh intel64

/kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton -t /scratch/\$SLURM_JOB_ID -N 8 -M 4000 -get molden.inp -d -o /polar/${kerne}.out /*u.dal /$molfil /${kerne}.guld

#cp \$SCR/molden.inp $navn/$geo/polar/${kerne}POP.inp
#cp \$SCR/DALTON.PROP $navn/$geo/polar/${kerne}POP.PROP
#cp \$SCR/SIRIUS.RST $navn/$geo/polar/${kerne}POP.rst


date
echo ========= Job finished ===================

!EOSLURM




sbatch $LLpol

rm -f $LLpol

[taylor]

It seems clear from what you have posted so far that your problem(s) have nothing to do with Dalton per se. They are problems of invoking (any) UNIX shell. I repeat my statement: you cannot create files into some directory you would like to by specifying files with filenames that begin with "/". I would also reinforce Kenneth's posting: the Dalton shellscript (dalton in the bin directory you built) is not designed to cope with wildcard characters (and indeed UNIX scripts that are so designed need careful programming as well as careful invocation). If you do not post the files that result from your failed attempts, it is difficult to see how to help you further. When you built the program did you run the test suite? And if so did the test runs work? If not, then there is a problem with your build. If they did, then there is a problem with how you are trying to run the program. Please post your output!

Best regards
Pete

[taylor]

And as many of us have said so many times that it is easy for us to get irritated: POST your output (that is, upload it)! Do not paste it inline! It is impossible to read conveniently in a browser!

[stine_T]

I am sorry that I did not get that I should upload - I am new in here and did not see that it was a possibility. It is attached now.

This,

diethenyl_small.rot.out

my out put file

(1.17 MiB) Downloaded 391 times

, is my output for my jobs which does not give me my tar.gz or molden.inp.

I cannot figure out to upload two files so I will upload the following error-message that I get in my corresponsing diethenyl_small.rot.err file.

I did not build the program myself since I am far from strong in programing/compeiling.

[stine_T]

and here is my error message.

It should be noted that it is printed as 'filename'.err and not as a .out as attached - I was just not allowed to upload it with is orginal name: diethenyl_small.rot-3502522.err .

err.out

(3.03 KiB) Downloaded 454 times

[taylor]

It is clear that the program exits after a completely successful calculation. Your problem must be how you are invoking the program, I'm afraid: you have some UNIX problems, not Dalton problems. But if you post your error file (which does not seem to be there, at least not yet?) one of us may be able to help, and certainly we will try.

All that said, I am somewhat confused. You are running Dalton2013 which is now quite an "old" version of the program? I do not say this is the problem: I already said above that it seems to be a UNIX rather than a Dalton problem. You (or whoever built the program) should update --- we cannot support old versions indefinitely. Further, your molecule seems to be oddly constructed: the centre of mass seems to very much displaced along the x-axis? And at least eyeballing the S and C coordinates it is far from obvious that the molecule has only C1 symmetry, so why turn symmetry off? Finally, you ask for Mulliken populations. The code provides these because every code does. But despite having had the privilege to know Mulliken personally and to respect his abilities and achievements tremendously, I have to say that his "population analysis" is useless. It may provide vaguely convincing and vaguely useful results in a minimal basis set, such as STO-3G. In anything larger the results are at best confusing/misleading and at worst downright nonsense. As a confirmation of this may I point that, for example, we can collapse any basis into a set of functions (possibly much larger than the original basis but that is irrelevant here) centred at an arbitrary point. A Mulliken analysis will happily then put all the population at that point. Doesn't seem physical...!

Best regards
Pete

[taylor]

It is always a problem with these forums that while one types a response something else comes in! I have now seen/read your error file. It is clear that the runscript (I mean here the dalton runscript) cannot cope with the wildcard characters and filenames you provide. Do not use wildcard characters like "*" or "?" in filenames, and do not try to create subdirectories by providing filenames that begin with "/". I think if you abide by these rules you should get the files back that you want.

Best regards
Pete

[taylor]

P.S. I will be travelling for the next two days (driving). I say this not so much because you might be interested --- of course you aren't! --- but because my communications access is going to be limited.

[taylor]

One clarification, when I said "don't use absolute pathnames" this is a bit over-the-top (what the late Jan Almloef used to call "a typical Taylor generalization"...). Because for instance I could use

Code: Select all

/home/pete/dalton_jobs/that_molecule/my.out

as a filename because I have write permissions on my own directories. I could also (assuming I'm submitting from my home directory) specify this output file as

Code: Select all

that_molecule/my.out

What I cannot do is specify it as

Code: Select all

/that_molecule/my.out

because this implies the file will be written into the directory "/that_molecule", to which I almost certainly have no access, because I cannot create a directory immediately under "/" unless I am root. And running as root is a Very Bad Idea...

Best regards
Pete

[kennethruud]

Peter has given basic information on how to reference files, and I can only encourage you to avoid wildcards and similar constructs. Looking at your error file, I can see that your scratch directory is set to /scratch/3502522, which is excellent, and which means that the working directory for your dalton calculation will be /scratch/3502522//*u_/diethenyl_small.rot (see the manual for why this is the case), and this is where things go wrong. The use of "/" in front of the names of files makes the script believe is should look into subdirectories instead of reading a specific file name.

You do not get problems with your output files, since you explicitly ask for the output file to be written in your home directory. I have to admit I have no idea where other scratch files are being written with the command line you have.

My suggestion for an invocation line: /kemi/stine/Dalton2013/DALTON-2013.0-Source/build/dalton -t /scratch/3502522 -N 8 -M 4000 -get molden.inp -d -o polar/diethenyl_small.rot.out u diethenyl_small.rot diethenyl_small.rot.guld

This requires that the directory "polar" exists in the directory from which you invoke the command. As already noted by Peter, wildcards etc. should not be used when invoking the script. If you need to run multiple calculations because of a large number of QM/MM snapshots, please script this look and invoke one Dalton calculation for each (see e.g. http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-7.html is you use bash). However, before doing so, make sure a single calculation runs fine.


Best regards,

Kenneth

[taylor]

Just a quick correction (and then we really are out the door!) you should continue to use

Code: Select all

-t /scratch/\$SLURM_JOB_ID

or something similar for your scratch directory. If you are running with SLURM the system is usually configured to give you a scratch directory with some name like that and usually the system creates it for you and you cannot always create it yourself. This is done to avoid users all trying to create directories with names like "/scratch/my_job" and overwriting one another's files.

(Believe it or not, on the earliest UNIX systems implemented on the Cray-2 everyone could access everyone else's files on the scratch file system. This meant that if you got fed up waiting for some idiot's job to finish and release some space, you could delete a couple of his/her files and their job would probably abort. Not that we chemists at NASA Ames ever did this, of course...)

Best regards
Pete

[stine_T]

Thank you for the many good suggestions. I am trying to do all the corrections and hopefully I will have my jobs done soon
Once again, thank you for your time and help.