[SOLVED] Cannot obtain hyperpolarizabilities with CC3

[pierrebeaujean]

EDIT: SOLVED, workaround and some debugging in this message !!

Hello,

I would like to report what seems to be a bug with CC3.

I try to reproduce some results from this paper, so I run this .dal on the geometry of water:

Code: Select all

**DALTON INPUT
.DIRECT
.RUN RESPONSE
**INTEGRAL
.DIPLEN
**WAVE FUNCTIONS
.CC
*CC INPUT   
.CC3
*CCFOP
.DIPMOM
*CCQR
.STATIC
.DIPOLE
**END OF DALTON INPUT

and I get this error in my .out file:

Code: Select all

 *******************************************************************
 *                                                                 *
 *---- OUTPUT FROM COUPLED CLUSTER RESPONSE  -------------*
 *                                                                 *
 *----------    CALCULATION OF FIRST ORDER PROPERTIES    >---------*
 *                                                                 *
 *******************************************************************



                         +-------------------------------+
                         ! Coupled Cluster model is: CC3 !
                         +-------------------------------+

 RPA: call cceq_str
 RPA: exit cceq_str
 CC_FOP> Norm of RHS vector:   0.0000000000000000     


                    Skipped solving for orbital relax. vector
                    -----------------------------------------


   Equations converged to residual less than:  0.000010
 CCFOP: call CC_D1AO to recalc the 1e Density
 I/O ERROR IN GETWA2:
 FILE:DPTAB
 UNIT:                    5
 IADR:                    1
 LEN :                    2
 IERR:                   -5

  --- SEVERE ERROR, PROGRAM WILL BE ABORTED ---
     Date and time (Linux)  : Thu Feb  4 09:24:36 2016
     Host name              : chim13-cptrx04.chimie.fundp.ac.be       

 Reason: I/O ERROR IN GETWA2.

  Total CPU  time used in DALTON:   0.06 seconds
  Total wall time used in DALTON:   0.16 seconds


 QTRACE dump of internal trace stack

 ========================
      level    module
 ========================
          6    CC_D1AO     
          5    CC_FOP      
          4    CC_DRV      
          3    CC          
          2    DALTON      
          1    DALTON main 
 ========================

(full output in attachement)

No problem to get the CC3 energy, but I'm unable to get any property. According to this topic, the solution is to use 64 bits integer, but this is the case (I recompile the code to activate them):

Code: Select all

Compilation information
-----------------------

 Who compiled             | pbeaujea
 Host                     | chim13-cptrx04.chimie.fundp.ac.be
 System                   | Linux-3.13.0-37-generic
 CMake generator          | Unix Makefiles
 Processor                | x86_64
 64-bit integers          | ON
 MPI                      | OFF
 Fortran compiler         | /usr/bin/gfortran
 Fortran compiler version | GNU Fortran (Ubuntu 4.8.4-2ubuntu1~14.04) 4.8.4
 C compiler               | /usr/bin/gcc
 C compiler version       | gcc (Ubuntu 4.8.4-2ubuntu1~14.04) 4.8.4
 C++ compiler             | /usr/bin/g++
 C++ compiler version     | g++ (Ubuntu 4.8.4-2ubuntu1~14.04) 4.8.4
 BLAS                     | /usr/lib/libblas.so
 LAPACK                   | /usr/lib/liblapack.so
 Static linking           | OFF
 Last Git revision        | 693735158eee69ff73df6846b9911e94803b8d7a
 Configuration time       | 2016-02-03 22:08:50.398132

No problem with CCS, CC2 or CCSD, on the other hand.

I reproduce this error with both Dalton 2015.1 and 2016, not compiled with MPI, and I also have this bug with the code I compile on my personal computer or the cluster of my university (I use gfortran in each case). Also, I try with different basis sets (here it is with STO-3G, so not a basis set effect, it seems).

Any idea on what is going on ? (I'm not very fluent in fortran, so I will try to read the source code of this "GETWA2"', but I don't promise that something will come out).

Thanks in advance,

P. Beaujean

[haettig]

Dear Pierre,

Christiansen, Gauss & Stanton implemented the CC3 quadratic response in ACES II (Mainz version)...
The implementation in Dalton was coded by Pawlowski and Jorgensen.

As work around you can try 2 things:
1) delete the .DIRECT keyword: as long as your examples are small enough too store the integrals on disk (max. 16 Gb per irrep) integral direct CC calculations are much slower than the non-direct version
2) delete the *CCFOP input group: maybe the bug is in section only needed for first-order properties (i.e. the one-electron density) but not for quadratic response

Christof

[pierrebeaujean]

Christiansen, Gauss & Stanton implemented the CC3 quadratic response in ACES II (Mainz version)...
The implementation in Dalton was coded by Pawlowski and Jorgensen.

Oups, my bad, I read to fast the "technical detail section". Thanks for the correction.

As work around you can try 2 things:
1) delete the .DIRECT keyword: as long as your examples are small enough too store the integrals on disk (max. 16 Gb per irrep) integral direct CC calculations are much slower than the non-direct version
2) delete the *CCFOP input group: maybe the bug is in section only needed for first-order properties (i.e. the one-electron density) but not for quadratic response

The first thing did nothing. On the other hand, deleting *CCFOP works perfectly, thanks !

[pierrebeaujean]

Oups, I speak to fast.

It works with the STO-3G basis set. But if I increase the size of the basis set (back to aug-cc-pVDZ), I get a SIGSEGV:

Code: Select all

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x7FC2289F0587
#1  0x7FC2289F0B9E
#2  0x7FC2255A617F
#3  0x7FC1D87BF31D
#4  0x7FC1D8DBB107
#5  0x7FC2283A3AF5
#6  0xD5DDAC in erisym_
#7  0xCB1272 in erint0_
#8  0xCB643A in odcpa2_
#9  0xCB6DEA in odcpar_
#10  0xCBD6E3 in odcdr1_
#11  0xCBF8EE in odcdrv_
#12  0xCA7B52 in eridi2_
#13  0x17681D7 in cc3_barint_
#14  0x291AEC3 in cc3_ft3b_
#15  0x18CFE7B in cc_fmatnew_
#16  0x18D4423 in cc_fmatrix_
#17  0x814FB7 in cc_solrsp_
#18  0x5F094C in cc_drv_
#19  0x45F54D in execc_
#20  0x463791 in dalton_exedrv_
#21  0x4564E5 in MAIN__ at dalton.F:?
/home/pbeaujea/DALTON2/dalton/dalton: line 722: 16879 Segmentation fault      $DALTON_EXE
Error in /home/pbeaujea/DALTON2/dalton/dalton.x, exit code 139

SERIOUS ERROR:
DALTON finished with non-zero exit code: 139

File list in scratch directory:

total 54612
-rw-r----- 1 pbeaujea pbeaujea  1386400 Feb  4 13:23 CCSDT_FBMAT1
-rw-r----- 1 pbeaujea pbeaujea   136576 Feb  4 13:23 CCSDT_FBMAT2
-rw-r--r-- 1 pbeaujea pbeaujea 17280288 Feb  4 13:23 AO2DIS00
-rw-r--r-- 1 pbeaujea pbeaujea       48 Feb  4 13:23 AOTWODIS
-rw-r----- 1 pbeaujea pbeaujea        0 Feb  4 13:23 CCSDT_FBMAT3
-rw-r----- 1 pbeaujea pbeaujea        0 Feb  4 13:23 CCSDT_FBMAT4
-rw-r----- 1 pbeaujea pbeaujea        0 Feb  4 13:23 CCSDT_FBMAT5
-rw-r----- 1 pbeaujea pbeaujea    49680 Feb  4 13:23 CCFM_BFI
-rw-r----- 1 pbeaujea pbeaujea   519328 Feb  4 13:23 CCFM_BZI
-rw-r----- 1 pbeaujea pbeaujea   473784 Feb  4 13:23 CCFM_CIM
-rw-r----- 1 pbeaujea pbeaujea   473784 Feb  4 13:23 CCFM_DIM
-rw-r----- 1 pbeaujea pbeaujea     2448 Feb  4 13:23 CCFM_FIM
-rw-r----- 1 pbeaujea pbeaujea     2184 Feb  4 13:23 CCFM_RHO
-rw-r----- 1 pbeaujea pbeaujea    22752 Feb  4 13:23 CCFM_RIM
-rw-r----- 1 pbeaujea pbeaujea      224 Feb  4 13:23 CCFM_XIM
-rw-r----- 1 pbeaujea pbeaujea    20208 Feb  4 13:23 CCFM_YIM
-rw-r----- 1 pbeaujea pbeaujea   519328 Feb  4 13:23 CCBFDENS
-rw-r----- 1 pbeaujea pbeaujea   432312 Feb  4 13:23 CCFM_CBA
-rw-r----- 1 pbeaujea pbeaujea   432312 Feb  4 13:23 CCFM_DBA
-rw-r----- 1 pbeaujea pbeaujea   302896 Feb  4 13:23 CCFMPQMO
-rw-r--r-- 1 pbeaujea pbeaujea    76205 Feb  4 13:23 CCR1_3___1
-rw-r--r-- 1 pbeaujea pbeaujea    57054 Feb  4 13:22 DALTON.OUT
-rw-r--r-- 1 pbeaujea pbeaujea    67637 Feb  4 13:22 CCR1_2___1
-rw-r--r-- 1 pbeaujea pbeaujea    79101 Feb  4 13:22 CCR1_1___1
-rw-r--r-- 1 pbeaujea pbeaujea   151061 Feb  4 13:21 CCX1_3___1
-rw-r--r-- 1 pbeaujea pbeaujea   151061 Feb  4 13:21 CCO1_3___1
-rw-r--r-- 1 pbeaujea pbeaujea   134261 Feb  4 13:21 CCX1_2___1
-rw-r--r-- 1 pbeaujea pbeaujea   134261 Feb  4 13:20 CCO1_2___1
-rw-r--r-- 1 pbeaujea pbeaujea   156613 Feb  4 13:20 CCX1_1___1
-rw-r--r-- 1 pbeaujea pbeaujea   156613 Feb  4 13:20 CCO1_1___1
-rw-r----- 1 pbeaujea pbeaujea        0 Feb  4 13:20 CC1DXFCK
-rw-r----- 1 pbeaujea pbeaujea        0 Feb  4 13:20 CCBFDEN1
-rw-r----- 1 pbeaujea pbeaujea        0 Feb  4 13:20 CCBFDEN2
-rw-r----- 1 pbeaujea pbeaujea     6856 Feb  4 13:20 DPTAB
-rw-r----- 1 pbeaujea pbeaujea      728 Feb  4 13:20 DPTIA
-rw-r----- 1 pbeaujea pbeaujea      728 Feb  4 13:20 DPTIA2
-rw-r----- 1 pbeaujea pbeaujea       88 Feb  4 13:20 DPTIJ
-rw-r--r-- 1 pbeaujea pbeaujea    79066 Feb  4 13:20 CCL0_1___1
-rw-r----- 1 pbeaujea pbeaujea  1566200 Feb  4 13:20 PTFOP2X
-rw-r----- 1 pbeaujea pbeaujea  1566200 Feb  4 13:20 DKBC3
-rw-r----- 1 pbeaujea pbeaujea  1566200 Feb  4 13:20 PTFOPX
-rw-r----- 1 pbeaujea pbeaujea   332864 Feb  4 13:20 CCPQIM
-rw-r--r-- 1 pbeaujea pbeaujea      328 Feb  4 13:19 CC_PRPC
-rw-r--r-- 1 pbeaujea pbeaujea      197 Feb  4 13:19 CC_RES
-rw-r--r-- 1 pbeaujea pbeaujea      164 Feb  4 13:19 DALTON.PROP
-rw-r--r-- 1 pbeaujea pbeaujea      854 Feb  4 13:19 DALTON.STAT
-rw-r--r-- 1 pbeaujea pbeaujea      164 Feb  4 13:19 midasifc.prop
-rw-r--r-- 1 pbeaujea pbeaujea    36664 Feb  4 13:19 molden_MOS.tmp
-rw-r--r-- 1 pbeaujea pbeaujea   116216 Feb  4 13:19 CC_BFIM
-rw-r--r-- 1 pbeaujea pbeaujea     6864 Feb  4 13:19 CC_E1IM
-rw-r--r-- 1 pbeaujea pbeaujea       96 Feb  4 13:19 CC_E2IM
-rw-r--r-- 1 pbeaujea pbeaujea     8408 Feb  4 13:19 CC_FCKH
-rw-r--r-- 1 pbeaujea pbeaujea      896 Feb  4 13:19 CC_GAMIM
-rw-r----- 1 pbeaujea pbeaujea   166432 Feb  4 13:19 PMAT_C
-rw-r----- 1 pbeaujea pbeaujea   166432 Feb  4 13:19 PMAT_D
-rw-r--r-- 1 pbeaujea pbeaujea    79066 Feb  4 13:19 CCR0_1___1
-rw-r----- 1 pbeaujea pbeaujea  1566200 Feb  4 13:19 DKBC
-rw-r----- 1 pbeaujea pbeaujea  1717648 Feb  4 13:19 CKDELD
-rw-r----- 1 pbeaujea pbeaujea  1717648 Feb  4 13:19 CC3_SORT
-rw-r----- 1 pbeaujea pbeaujea   166432 Feb  4 13:19 CKJDEL
-rw-r----- 1 pbeaujea pbeaujea  1717648 Feb  4 13:19 CC3_VI
-rw-r----- 1 pbeaujea pbeaujea  1717648 Feb  4 13:19 CC3_VI12
-rw-r--r-- 1 pbeaujea pbeaujea    76792 Feb  4 13:19 CCSD_IAJB
-rw-r----- 1 pbeaujea pbeaujea  1717648 Feb  4 13:19 PTFOP
-rw-r----- 1 pbeaujea pbeaujea  1717648 Feb  4 13:19 PTFOP2
-rw-r--r-- 1 pbeaujea pbeaujea     8408 Feb  4 13:19 CC_FCKREF
-rw-r----- 1 pbeaujea pbeaujea   166432 Feb  4 13:19 CCSDT_OC
-rw-r--r-- 1 pbeaujea pbeaujea    38448 Feb  4 13:19 SIRIFC
-rw-r--r-- 1 pbeaujea pbeaujea     8640 Feb  4 13:19 SIRIUS.RST
-rw-r--r-- 1 pbeaujea pbeaujea     5710 Feb  4 13:19 molden.inp
-rw-r--r-- 1 pbeaujea pbeaujea     3364 Feb  4 13:19 DALTON.CM
-rw-r--r-- 1 pbeaujea pbeaujea    14032 Feb  4 13:19 LUINDF
-rw-r--r-- 1 pbeaujea pbeaujea        0 Feb  4 13:19 MOTWOINT
-rw-r--r-- 1 pbeaujea pbeaujea   110840 Feb  4 13:19 AOONEINT
-rw-r--r-- 1 pbeaujea pbeaujea   212952 Feb  4 13:19 AOPROPER
-rw-r--r-- 1 pbeaujea pbeaujea     3827 Feb  4 13:19 AMFI_MNF.INP
-rw-r--r-- 1 pbeaujea pbeaujea     2723 Feb  4 13:19 AMFI_SYMINFO.TXT
-rw-r--r-- 1 pbeaujea pbeaujea     4222 Feb  4 13:19 DALTON.BAS
-rw-r--r-- 1 pbeaujea pbeaujea     5507 Feb  4 13:19 LUSH.BAS
-rw-r--r-- 1 pbeaujea pbeaujea      281 Feb  4 13:19 midasifc.cartrot
-rw-r--r-- 1 pbeaujea pbeaujea     1025 Feb  4 13:19 firstsym.wrl
-rw-r--r-- 1 pbeaujea pbeaujea      237 Feb  4 13:19 midasifc.cartorig
-rw------- 1 pbeaujea pbeaujea      255 Feb  4 13:19 MOLECULE.INP
-rw------- 1 pbeaujea pbeaujea      139 Feb  4 13:19 DALTON.INP
54M	.
beta_0_water.tar.gz has been copied to /home/pbeaujea/workdir/test
----------------------------------------------------------
WARNING for ERROR  (DALTON program stopped with exit code 139)
- THEREFORE scratch directory /scratch/DALTON_scratch_pbeaujea/beta_0_water_16855 is not deleted
  by this script so you may restart the calculation, if possible
  ( /scratch/DALTON_scratch_pbeaujea/beta_0_water_16855 may be deleted by your queueing system )
- REMEMBER to delete /scratch/DALTON_scratch_pbeaujea/beta_0_water_16855 yourself !!!


   Thu Feb  4 13:23:50 CET 2016
   ***************************************** 

I read this topic about this kind of error. 64 bit integers or not, its does not change anything. Also, i try to use MKL instead of OpenBLAS, and the result is the same. I also try to change gcc to ifort, but again, no improvement.

[xiongyan21]

Dear all
The calculation using aug-cc-pvdz gets CC3 results for the first hyperpolarizabilities when, according to Dr. Haettig, *CCFOP group is deleted from the input prepared by Dr. Beaujean if 64-bit integer is turned off, whether which is the cause for the pass is unknow to me, and the following is obtained

....

+--------------------------------------------------------------------------------------+
! FINAL CC3 RESULTS FOR THE FIRST HYPERPOLARIZABILITIES !
+--------------------------------------------------------------------------------------+


A operator B operator C operator property
------------------------------------------------------------------------------------------------------

XDIPLEN (unrel.) -0.0000 XDIPLEN (unrel.) 0.0000 ZDIPLEN (unrel.) 0.0000 -3.2121794


...

[pierrebeaujean]

TL;DR:

• add .NONREL to the *CCFOP input group
• do not use .DIRECT.

-------------------

Dear Christof, dear xiongyan21,

So I made some tests on my own (and with the help of a colleague to go trough the source code). There is two things going on there ...

First, there is this issue with the first-order properties (*CCFOP input group): from the code source, it appears that this is related to a file named "LUPTAB" (and "LUPTIJ"), which is only filled when orbitals are not relaxed (line 4680 of cc/cc_foppt.F of the 2016 source code, IF (.NOT. RELORB) THEN ... PUTWA2(LUPTAB ...)). But when it comes to first order properties, then this file is read no mater if orbitals are relaxed or not (line 372 of cc/cc_d1ao.F of the 2016 source code, IF (CC3) THEN ... GETWA2(LUPTAB, ...). So of course, Dalton try to access to something that does not exists and crash. We did try to add some (.NOT. RELORB) at some position, but the code was definitely not designed to take into account relaxation and this is an infinite chain of trial and errors. Therefore, one can add .NONREL to *CCFOP input group, and everything is good, you can get your dipole moment in CC3 (I need it in order to calculate β_||).

On the other hand, there is this issue with the size of the basis set (or the size of the molecule, so the number of orbitals): you can calculate your hyperpolarizability with STO-3G, but not with d-aug-cc-pVDZ ( xiongyan21, you forget the doubly augmented). Like I report previously, the trouble happen inERISYM, and more precisely, in a matrix multiplication of this function (to be exact, line 109 of eri/eri2sym.F of the 2016 code, CALL DGEMM(...)). So the errors involve a call to the BLAS library, but the problem is on the dalton side, because no matter what mathematic library I use (MKL. BLAS, OPENBLAS), the error happen at the same place. I'm not very familiar with BLAS related stuffs, but this looks like a memory access with a index larger than what it should. There is probably something to change in cc/cc3_barint.F to change to make thing right, but I don't know what because I must read the entire functions (and related papers) to get exactly what's is going on with those matrices. But this is symmetry related, so one just have to put nosymmetry and everything is good, you can finally get your hyperpolarizability !
So, for now, everything is OK! (even if this nosymmetry keywords kills all performances).

EDIT: thanks to xiongyan21, I find out that it also works if you remove .DIRECT

Also, if someone know the code better than I do, there is some possibility of correction. For the first problem, according to the original paper about first order properties in CC3, there is no inclusion of relaxation in the equations, so a "quick and dirty" fix is simply to disallow those kind of calculations. For the second one, more investigations are needed.

With best regards,

Pierre

[xiongyan21]

Dear Dr. Beaujean
Although it seems to me that you only mentioned aug-cc-pvdz previously, there are not any problems for DALTON 2016.0 to finish the calculation employing d-aug-cc-pvdz with the following result
...

+--------------------------------------------------------------------------------------+
! FINAL CC3 RESULTS FOR THE FIRST HYPERPOLARIZABILITIES !
+--------------------------------------------------------------------------------------+


A operator B operator C operator property
------------------------------------------------------------------------------------------------------

XDIPLEN (unrel.) -0.0000 XDIPLEN (unrel.) 0.0000 ZDIPLEN (unrel.) 0.0000 -3.7240273

...

The inclusion of *CCFOP group with the addition of .NONREL makes the calculation pass without the keyword nosymmetry, and the same FOP results seem to be maintained
...
+--------------------------------------------------------------------------------------+
! FINAL CC3 RESULTS FOR THE FIRST HYPERPOLARIZABILITIES !
+--------------------------------------------------------------------------------------+


A operator B operator C operator property
------------------------------------------------------------------------------------------------------

XDIPLEN (unrel.) -0.0000 XDIPLEN (unrel.) 0.0000 ZDIPLEN (unrel.) 0.0000 -3.7240273


...

[pierrebeaujean]

Lucky you ! (I check, and you get the same results, but I have a lot of zero ones without the symmetry)

But there is probably something behind it: what is the operating systems on which Dalton runs ? which math library do you use ? How many memory do you give to your job ?

Sorry for all those questions, but I really try to understand what cause the bug and how to fix it

[pierrebeaujean]

Just to be sure we are on the same page, here is the two files that work for sure: (notice the "nosymmetry")

[xiongyan21]

.dal file
**DALTON INPUT
.RUN RESPONSE
**INTEGRAL
.DIPLEN
**WAVE FUNCTIONS
.CC
*CC INPUT
.CC3
*CCFOP (optional)
.DIPMOM
.NONREL
*CCQR
.STATIC
.DIPOLE
**END OF DALTON INPUT

Please note there is no nosymmetry added in the following.

.mol file
BASIS
d-aug-cc-pvdz
Water

Atomtypes=2 Angstrom
Charge=8.0 Atoms=1
O 0.000000000 0.000000000 -0.115095289
Charge=1.0 Atoms=2
H 0.000000000 0.767462907 0.460381156
H 0.000000000 -0.767462907 0.460381156

The operating system is OS X EI Capitan 10.11.2 and 12 GB memory is used.
If something is preferred to be called a bug here, perhaps it is later, after the end of the coupled cluster section, the dynamic property section is invoked and the following appears in the output where the TDHF result is left blank, which maybe caused by the somewhat improper input preparation.
.------------------------------------------------.
| Starting in Dynamic Property Section (RESPONS) |
`------------------------------------------------'


------------------------------------------------------------------------------
RESPONSE - an MCSCF, MC-srDFT, DFT, and SOPPA response property program
------------------------------------------------------------------------------


-------- OUTPUT FROM RESPONSE INPUT PROCESSING --------

...

***************************************
*** RHF response calculation (TDHF) ***
***************************************


Total CPU time used in RESPONSE: 0.00 seconds
Total wall time used in RESPONSE: 0.00 seconds


.-------------------------------------------.
| End of Dynamic Property Section (RESPONS) |
`-------------------------------------------'

[pierrebeaujean]

Dear xiongyan21,

I'm confuse. I try on the cluster of my university (on the 2016 version) with 12Gio of memory, and I still get an error o_O ! I suppose that you use the default Blas/Lapack of your Mac OSX, do you ?

[pierrebeaujean]

Oh, ok, I get it ... It only happen in direct mode. I always run my calculations with .DIRECT, which is not the case for you, xiongyan21, and infact, in non-direct mode, it's working.

Even if the performance are worst in non-direct mode, it's better than with no symmetry

[xiongyan21]

I think this maybe caused by a different basis set here from that used in literature, and a calculation with DALTON2016.0 employing aug-cc-pv5z has given fine beta parallel values, suggesting if programmable and hardware permitting, FCI and CBS will get the best results.