Interoperability: API/library model for integrals using HDF5

Post here if you would like to suggest new features for Dalton
Post Reply
jvtrudel
Posts: 3
Joined: 20 Mar 2015, 10:16
First name(s): Jérémy
Last name(s): Viau-Trudel
Affiliation: Université Laval (Québec) and Université Paris-Sud (France)
Country: Canada

Interoperability: API/library model for integrals using HDF5

Post by jvtrudel » 20 Mar 2015, 12:49

Could it be possible to document your Hermite module and its interfaces. It could be useful to allow users to use the low level routine of integrals in their own code like it is possible to do with BLAS and LAPACK.

Also, it would be nice to implement exportation of integrals (and other data) in HDF5 format. An interesting proposal of this idea have been made recently (DOI: 10.1002/jcc.23492) but, unfortunately, this project seems dead and I have failed to obtain wrappers for any Quantum Chemistry (QC) code.

Personally, I would like to use molecular integrals and wave functions as building blocks to study multielectron dynamics driven by intense and ultra-short laser field. Many of the integrals available in your code are not computed by other QCcode and many users could be interested by use Dalton to get these data.

Actually, interoperability between QC codes is very hard, especially for a PhD student or a postdoc who may waste a lot of time on this issue. Although, plenty of examples of QC code interfacing may be found - for example Columbus uses Hermite to compute molecular integrals - there is still a lot of progress to do to achieve functional interoperability of QC codes.

I would like to hear about any experience on QC code interoperability. I would also like to know if the Dalton team would accept contributions for exporting HDF5 format.

Thanks al lot.Interoperability: API/library model for integrals using HDF5

kennethruud
Posts: 241
Joined: 27 Aug 2013, 16:42
First name(s): Kenneth
Last name(s): Ruud
Affiliation: UiT The Arctic University of Norway
Country: Norway

Re: Interoperability: API/library model for integrals using

Post by kennethruud » 20 Mar 2015, 13:05

Hi!

Thanks for your interest in Dalton and interfacing parts of the Dalton functionality. Let me start by noting that I very much agree with your strategy of modularizing and interfacing functionalities, and we are in our research group very much working in this direction, although there are many things we learn on the way in terms of complications. Nevertheless, various modules have been developed and will be gradually released. We would like to interoperate with at least three different QC codes ourselves (Dalton, LSDalton and Dirac, and also ideally ReSpect), so whatever modules we developed gets to test a lot of the complications of interoperability and clean interfaces.

In terms of Dalton and Q5Cost, we actually have a fairly complete interface ready, but it never really made it back to the release branch. We will look into whether this can be revived. Perhaps send me/us a PM.

As your main interest appears to be integrals at this moment in time, you may want to look into Gen1Int (https://repo.ctcc.no/projects/gen1int) which is designed for modularity and better interfaces.


Best regards,

Kenneth

jvtrudel
Posts: 3
Joined: 20 Mar 2015, 10:16
First name(s): Jérémy
Last name(s): Viau-Trudel
Affiliation: Université Laval (Québec) and Université Paris-Sud (France)
Country: Canada

Re: Interoperability: API/library model for integrals using

Post by jvtrudel » 20 Mar 2015, 15:00

It's great to ear about these forthcoming projects. I'll follow these developments.

Gen1int seems great indeed, but I also need 2 electron integrals...

taylor
Posts: 525
Joined: 15 Oct 2013, 05:37
First name(s): Peter
Middle name(s): Robert
Last name(s): Taylor
Affiliation: Tianjin University
Country: China

Re: Interoperability: API/library model for integrals using

Post by taylor » 21 Mar 2015, 04:01

I make an offer here that I suggest we take offline (i.e., send me a PM if you're interested) because I don't want to make what I'm about to describe generally available at this point --- I can't spare the time to support it.

Several years ago I wrote a free-standing two-electron integral transformation routine that reads Dalton integrals in the symmetry-orbital basis and transforms them to e.g. MO basis. The (I think) cute thing about this routine is that it runs entirely in memory. There are no sorting steps, no need for any disk storage beyond what is needed for the original integrals and the final integrals: the typical I/O-bound sorting steps of a conventional two-electron integral transformation are completely absent. The constraint is of course that a single set of integrals of the same symmetry type have to fit into available memory. Especially for low-symmetry cases this can be a significant constraint, especially because for the transformation the index permutational symmetry ij <-> kl is not used, so in the ultimate case of no symmetry the storage required in the transformation step is therefore N^4/4, not N^4/8. One can do the maths here: in (say) 16GB one can handle 2 billion memory locations (i.e., integral values), which in no symmetry would mean about 300 basis functions. But the code, which is in essence two sets of matrix multiplications, a giant matrix transposition, and two more sets of matrix multiplications, really flies. We are hoping to have this incorporated in a future Dalton release.

The reason I introduce all this here is that the code, which I like to think is relatively clean and straightforward Fortran90, is fairly well documented as to what it is doing reading a Dalton two-electron integral file, and looking at it could probably answer a number of your questions about interoperability. If you are interested, PM me, as I said, and we can talk about getting you the code. Given that most of our possessions are somewhere on a boat between Melbourne and Aarhus, I might have to dig through some old archives to get you what you might want!

Two substantial caveats about what you are talking about doing (and these are quite independent of whether or not you look at my transformation program). First, it is important to be aware that (unless one explicitly turns symmetry off, and I have very strong feelings, given my upbringing by at least one card-carrying group theorist [A. C. Hurley], that one should not do this, and that one should use whatever symmetry is available within the program) the different two-electron operators for which Dalton can calculate integrals have different symmetry properties. The Coulomb operator is for example a totally symmetric operator (as a scalar operator it transforms as a basis function for the totally symmetric representation of the molecular point group). If we label (for D2h and its subgroups) the "symmetries", that is, irreducible representations of the point group, by I, J, K, and L, then there are four possible combinations of symmetries: IIII, IIJJ, IJIJ, and IJKL, where in the latter case the direct product of I, J, K, and L has to be totally symmetric. However, there are as you observe many other property operators. For example, there are those whose spatial parts have a denominator of the form (r_12)^-3 and a numerator like x_12. For most symmetries this operator will not be totally symmetric, and so the symmetry structure of the resulting blocks will be different: the non-vanishing blocks may include IIJK or IIIJ, and the blocks IIII, IIJJ, and IJIJ will be vanishing. (Incidentally, my transformation programs knows all this and knows how to deal with it.) Another "symmetry" property is index permutation. A two-electron repulsion integral (ij|kl) is invariant to the permutations i <->j , k <-> l, and ij <-> kl, which form a subgroup of order 8 of the full permutation group (order 24) on 4 objects. But there are operators that do not possess the same permutational symmetry: for spin-orbit operators the integral will be symmetric under one interchange (say, i <-> j) and antisymmetric under the other (k <-> l in this case). All this has to be accounted for if you want to use the integrals outside Dalton.

Second, interoperability with codes outside Dalton/LSDalton/Dirac and their descendants requires considerable care in attention to issues like phase and normalization. For example, I am aware of (at least) one code that uses a different phase convention for some (but not all!) spherical harmonics for l values larger than 2 (i.e., for f-functions and beyond). Trying to mix and match between such codes is a very fraught business and needs to be approached with caution and detailed analysis. Another issue is whether codes maintain a phase convention (say, the largest orbital coefficient in an MO is always taken positive) or not. This again is a trap for the unwary. I do not seek to discourage you from using quantities, like integrals, from the Dalton program in other contexts and programs. But this is a very much more complicated situation than it might naively seem (this by the way has nothing to do with it being Dalton, one would encounter the same problems with e.g. Molcas or Molpro or GAMESS) because unfortunately there is no universal agreement about how basis functions should be normalized, how index permutational symmetry should be handled, how MOs should be phased, etc.!

If after this series of stern warnings you are still interested to learn more about how to extract information from Dalton integral files, contact me separately and I'll try to find, off the limited amount of hardware I still have on hand here, the in-core transformation program. Worst case, I won't have it, and you'll have to wait until we get set up in Aarhus and have our home compute cluster up and running again, which will probably not be until May. But I might have the code somewhere local in Melbourne where I can get at it. In fact, now I think about it I sent it Radovan Bast last year, so worst case he might be able to send it back to me!

Best regards
Pete

bast
Posts: 1197
Joined: 26 Aug 2013, 13:22
First name(s): Radovan
Last name(s): Bast
Affiliation: none
Country: Germany

Re: Interoperability: API/library model for integrals using

Post by bast » 21 Mar 2015, 12:14

taylor wrote:In fact, now I think about it I sent it Radovan Bast last year, so worst case he might be able to send it back to me!
indeed i have it in the mailbox. sending it now to Pete :-)

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest