ROHF response problems

If you think you found a bug this is where you can report it
Post Reply
taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

ROHF response problems

Post by taylor » 26 Aug 2014, 04:27

Having had difficulties with ROHF linear response calculation of excitation energies in a large and quite complicated system, I have gradually worked my way down to a small and hopefully tractable test system. Namely, the ground (triplet Sigma - g) state of O2, in a VDZ basis. If I run this system using MCSCF, constrained to a two electrons in two active orbital space, and hence a single determinant, the energy calculation completes and then I get (in this case --- it's just a pretty random choice of roots to go for) four excited states with sensible excitation energies and the appropriate symmetries. Note that this calculation is exactly equivalent to single configuration ROHF+response: I'm just doing it, more expensively, by using the MCSCF code.

Now, I construct a calculation that is exactly the same except that instead of doing it via MCSCF I do it using single configuration ROHF. The energy I obtain from the wave function analysis agrees with the MCSCF number to within 1E-12. But the response calculation is clearly completely broken. The iterative solution of the response equations stalls and produces the same residuals and eigenvalue estimates, iteration after iteration, with no change or improvement, until eventually the solver gives up because it cannot add a new trial vector that is linearly independent to the current set. And the "excitation energies" printed, not surprisingly, are completely unphysical and bear no relation to the values calculated using "MCSCF" response.

At present this is a serious issue for me, because for my "real" problem, if I use ROHF I can run direct, whereas if I use MCSCF I cannot. And this is the difference between a big, but just feasible, calculation, and in the latter case something that is out of the question.

Olav Vahtras has confirmed that it is expected that ROHF response should work for both linear and quadratic response. I have investigated and found that the problems I encounter here, which for the files uploaded were obtained with the 2014 release candidate, are also observed not only for 2013, but also 2011. And if this is not a bug but a blunder on my part, I'm willing to live with the embarrassment...

Attached
o2rhf.dal
o2mcscf.dal
expt.mol
O2.out (ROHF output)
O2MC.out (MCSCF output)

Best regards
Pete
Attachments
O2MC.out
(65.45 KiB) Downloaded 415 times
O2.out
(73.08 KiB) Downloaded 435 times
expt.mol
(188 Bytes) Downloaded 396 times
o2mcscf.dal
(342 Bytes) Downloaded 408 times
o2rhf.dal
(215 Bytes) Downloaded 425 times

rinkevic
Posts: 12
Joined: 04 Sep 2013, 14:22
First name(s): Zilvinas
Last name(s): Rinkevicius

Re: ROHF response problems

Post by rinkevic » 26 Aug 2014, 06:28

Hello Pete,

I could reproduce your MCSCF results with older version of dalton and little modification of o2rhf (high spin response should always run via direct route). Files attached. However, I get same problem as you with Dalton 2013 release version.

Regards, Zilvinas.
Attachments
o2rhf_expt.out
ref response with .direct
(79.52 KiB) Downloaded 415 times

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: ROHF response problems

Post by taylor » 28 Aug 2014, 08:53

Thanks for this, but it does not match my experience. I have what is as far as I know a standard copy of Dalton2011, and when I run (after adding the .DIRECT option: surely this was not always necessary, in the past like Dalton2,0?), the response calculation still stops with the diagnostic
*** MICROITERATIONS STOPPED DUE TO LINEAR DEPENDENCE BETWEEN NEW TRIAL VECTORS
for each response calculation, and the residual is not only far from converged, it does not change at all during the last few iterations.

That said, your ROHF calculation with Dalton2011 agrees essentially perfectly with my "MCSCF" calculation, which as I said in my original posting is really a single configuration (and which gives identical results for both Dalton2011 and the current 2014 release candidate, by the way). However, I am extremely dubious about some of the actual results from these tests --- the calculations that appear to work, I mean. If you look at the calculation for the case
*** @ Excit. operator sym 8 & ref. state sym 4 => excited state symmetry 5 ***
then you will see that the result for this state derives from exciting from pi_uX to pi_gY and pi_uY to pi_gX. This would give overall either Sigma^+_u or (one component of a Delta_u state. This I can buy. However, the predicted excitation energy is only 0.14 eV! (From both your ROHF and my MCSCF.) This is completely unphysical. There is no state of such symmetry within about 4.5 eV of the ground state. I do not see how this part of the calculation, at least, can be meaningful?

Best regards
Pete

olav
Posts: 143
Joined: 28 Aug 2013, 06:20
First name(s): Olav
Last name(s): Vahtras

Re: ROHF response problems

Post by olav » 28 Aug 2014, 09:09

Yes, you are right. The bug was introduced before the 2011 release. For a March 2011 version from the master branch it works.
/Olav

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: ROHF response problems

Post by taylor » 28 Aug 2014, 09:47

OK, thanks for this, but as both Zilvinas and I posted this appears to be broken in the 2013 releases? And it seems to be broken in the current 2014 release candidate (of course, I should reflect that I got the latter from Thomas Kjaergaard to try to attack some LSDalton problems, so in fact the files in the 2014rc for Dalton might be just the 2013.4 files at this point?).

So, is it the case that the fix for 2011 might not have been carried forward into the 2013 release? And can someone provide an old-school style patch for this? I don't do git and so on --- I don't have time to learn this. It's not that I don't want to (I bought a book...) I just don't have the time these days.

By the way, Olav (and Zilvinas), any thoughts about my concerns about the anomalously low excitation energy for the excited state in symmetry 5?

Best regards
Pete

olav
Posts: 143
Joined: 28 Aug 2013, 06:20
First name(s): Olav
Last name(s): Vahtras

Re: ROHF response problems

Post by olav » 28 Aug 2014, 10:12

The 2011, 2013 releases and the current 2014 candidate are all broken with respect to this example. There is no fix, first we must identify what went wrong in the pre-2011 versions.

So the first concern is to get consistent results from MCSCF and ROHF so we can trust the code. I observe that the linear response solver shows convergence issues such as complex eigenvalues for that symmetry. I would need to look at it in more detail.

Olav

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: ROHF response problems

Post by taylor » 28 Aug 2014, 11:46

Well, wait a bit. If there's a version (March 2011, I think you said) that works --- and Zilvinas certainly has one --- can we not make a start by diffing the files in that version with the actual release, to see what got changed? I have no idea whether this would yield hundreds+ of differences, which would make it extremely difficult, of course, or perhaps a tractable number. If someone (Zilvinas?) wants to send me a source tree that yields a working ROHF response code, I'd be willing to at least have a crack at it. But at the moment I'd be flying blind because I don't have a version that works.

Note that most cases in which the response solver "stalls" and eventually cannot find a linearly independent update vector there are no warnings about complex eigenvalues: these do sometimes appear, but usually not. It seems to me there is a definite problem --- the solver stalling --- and a worry, and possibly a second problem, in that some solutions found even by the MCSCF response code appear to be bogus.

Best regards
Pete

bast
Posts: 1197
Joined: 26 Aug 2013, 13:22
First name(s): Radovan
Last name(s): Bast
Affiliation: none
Country: Germany

Re: ROHF response problems

Post by bast » 28 Aug 2014, 11:48

this can be done semi- or even fully automatically with git bisect where you bisect
your way until you find the commit that broke it. with some luck the commit is relatively small
and then it will hopefully become clear why it broke.
radovan

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: ROHF response problems

Post by taylor » 28 Aug 2014, 12:12

Sounds great but I hope it wasn't directed at me: see above posting where I admit I know nothing about git and no clue what to actually do...

Best regards
Pete

olav
Posts: 143
Joined: 28 Aug 2013, 06:20
First name(s): Olav
Last name(s): Vahtras

Re: ROHF response problems

Post by olav » 28 Aug 2014, 14:32

git bisect sounds neat. Does is work when the tests not in the repo as this is a new test?

I made some brute force scanning and the last commit that works for me is ba66fee2e00612ece2d210dd07d5c46af1250aef from 2011-05-15
I hope that's a start for someone willing to have a look.

To remember about the configure version of that time: in order to make it compile you have to create a modules directory by hand

i.e.
at the top level

$ git checkout ba66fee
$ ./configure -q
$ mkdir modules
$ make -j 4

The test of this bug report is in DALTON/test/rsp_rohf_lr of the branch Dalton2014-issue-605

Cheers,
Olav

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: ROHF response problems

Post by taylor » 28 Aug 2014, 16:29

I repeat: I'm willing to look at differences between two source trees, but I need to be provided with a reference one that works. I am simply not in a position to do anything with git.

Best regards
Pete

rinkevic
Posts: 12
Joined: 04 Sep 2013, 14:22
First name(s): Zilvinas
Last name(s): Rinkevicius

Re: ROHF response problems

Post by rinkevic » 28 Aug 2014, 18:10

Dear Pete,

I will provide you with working version of code as soon as I man back from summer school i.e in the beginning of next week.

Regards, Zilvinas.

bast
Posts: 1197
Joined: 26 Aug 2013, 13:22
First name(s): Radovan
Last name(s): Bast
Affiliation: none
Country: Germany

Re: ROHF response problems

Post by bast » 28 Aug 2014, 22:04

olav wrote:git bisect sounds neat. Does is work when the tests not in the repo as this is a new test?

I made some brute force scanning and the last commit that works for me is ba66fee2e00612ece2d210dd07d5c46af1250aef from 2011-05-15
I hope that's a start for someone willing to have a look.

To remember about the configure version of that time: in order to make it compile you have to create a modules directory by hand

i.e.
at the top level

$ git checkout ba66fee
$ ./configure -q
$ mkdir modules
$ make -j 4

The test of this bug report is in DALTON/test/rsp_rohf_lr of the branch Dalton2014-issue-605

Cheers,
Olav
git bisect is the way to go.
the way i would do it is to keep the test run outside the repo (cp)
and symlink the binary into it. this way if we go back in time we change
the binary but not the testing setup.

you can even do a git bisect run [script]. the script can be anything you like.
you define in there what you consider as good or bad. inside there you can
compile the code, run some tests, go to some other directory, full freedom.

all that counts to git is the return code: 0 as success, non-0 as failure.
whatever that means to you.

so then you can write a bash script that will test whether ROHF works and returns
0/non-0, you start it, you go for lunch and when you are back it tells you which
commit broke it.

that was in ideal life. in real life it probably won't work because we have
revisions that do not compile at all and if git bisect run hits them, you are out of luck.
that's why it's so important to have every single revision compile on master.

if you run git bisect manually you can "git bisect skip" noncompiling revisions.

with git bisect you typically find the commit that broke something within less
than 10 compilations.

radovan

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: ROHF response problems

Post by taylor » 29 Aug 2014, 09:34

It would probably be useful to take the "git bisect" discussion out of this thread. As the initiator of this as a genuine bug report, I think having asserted \emph{more than once} I am not competent to execute a git-based investigation, and therefore am unable to do so, it is an exercise in pointless in the context of my bug report to keep selling the virtues of git features. I don't need the description of them, because I can't use them and have no opportunity at all to acquire those skills on a meaningful timescale. I just want a version of the code that works (which Zilvinas will send me) and will then proceed with the tools I know. Since I don't know \emph{anything} about git (have I mentioned that in my previous postings...?) I do not understand why the thread has been torqued into a discussion about how to use it! This is of no use to me whatsoever. If anyone wants to volunteer to attack this bug using the git bisect approach, well, I got plenty of stuff they pay me to do in my day job, and I'm happy to stand aside and let volunteers tackle the Dalton bug instead. But I'm trying to do my bit and help identify the problem, and I can only do this with the tools that I know, as I have now posted at least three times...

\emph{Please}, no more git about this on this thread...

Best regards
Pete

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest