gen1int and CMakeLists.txt.

Problems with Dalton installation? Find answers or ask for help here
Post Reply
jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

gen1int and CMakeLists.txt.

Post by jfct_sussex » 13 Feb 2020, 18:10

I have tried to install Dalton 2018 after downloading the tar.gz file from the github site. I have intel mkl installed on a fedora 30 linux system. My .setup command was

./setup --fc=gfortran --cc=gcc --cxx=g++ --int64 --mkl=parallel --gpu

which was successful, giving me the message

"configure step is done
now you need to compile the sources:
$ cd build
$ make"

on executing

[ build] make -j

the process works successfully until

[ 2%] Performing configure step for 'gen1int'
CMake Error: The source directory "/~/dalton/external/gen1int" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
make[2]: *** [gen1int/CMakeFiles/gen1int.dir/build.make:107: external/gen1int-stamp/gen1int-configure] Error 1
make[1]: *** [CMakeFiles/Makefile2:1233: gen1int/CMakeFiles/gen1int.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 2%] Built target peter_utils_blocks
[ 2%] Linking Fortran executable labread
[ 2%] Built target labread
[ 2%] Linking Fortran executable FChk2HES
[ 2%] Built target FChk2HES
make: *** [Makefile:141: all] Error 2

at which point, the installation falls over apparently.
Google seems not to have the answer and I really need to get this done asap - any help would be very gratefully received.

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 13 Feb 2020, 22:18

Unfortunately, downloading the source code from the GitLab repository does not work (hopefully this will change for the next release). Instead you need to clone it using git. The following commands should work:

Code: Select all

git clone --recursive https://gitlab.com/dalton/dalton.git
cd dalton
git checkout Dalton2018.2
git submodule update
./setup --fc=gfortran --cc=gcc --cxx=g++ --int64 --mkl=parallel
cd build
make
Note, that "--gpu" does nothing for Dalton and it is therefore not included above. Also note that 64-bit integers (--int64) are rarely needed and break some functionality in the current release. As far as I know, the "--mkl=parallel" is only really beneficial for the parts of Dalton that are not parallelized, e.g., the coupled cluster code, since threading is not used by Dalton itself. Some parts of Dalton are MPI parallelized and for those parts you will gain more from MPI than using threaded MKL.

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 14 Feb 2020, 18:21

Hi Magnus
Thank you so much. I ditched the --mkl=parallel as it crashed the first make attempt but I now have a dalton.x target sitting on my hard drive. I'll test it and then report back if there are any failures.

I think it's important to change the installation instructions - it's completely erroneous and is so straightforward via your method :)

Thank you again - very grateful
J

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 14 Feb 2020, 21:15

Glad I could help! Yes, we unfortunately have some outdated information in some places that we would like to update, but since this is mostly volunteer work, it is sometimes difficult to find time. Can I ask which instructions you used originally?

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 17 Feb 2020, 20:47

Hi Magnus - I downloaded the tar.gz file from the Dalton download page and followed the instructions from there. I ran ctest and all of them failed but I wonder whether I've been following the instructions from the installation manual and none of it is right so I'm going to run again with the git commands and then run ctest immediately

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 17 Feb 2020, 20:48

Also - my home machine has a large number of processors - is there any way to exploit this using Dalton to speed things up?

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 17 Feb 2020, 20:53

Yes, compile with MPI. To do this add the "--mpi" option when you run the setup script. Note, however, that the coupled cluster code will not be able to use MPI.

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 18 Feb 2020, 18:59

Hi Magnus

I've finally managed to get it to go at the most basic level, without --mkl=parallel and without --mpi - both failed and gave errors. I used

./setup --fc=gfortran --cc=gcc --cxx=g++
cd build
make
and now the tests are passing though slowly. I'll post the test times when they're done.

Next step is to get the most out of the machine, which has a lot of RAM and a large numbers of processors; top shows me that there is one process running, whereas when I run GAMESS, there's a very simple way to run in parallel and dictate the number of processors. This doesn't seem to be obvious in dalton from the documentation and the only way seems to be bugging you on the forum - apologies.

I'm also writing up my experiences in a document for my students and I'll punt it to you - it would be good to have a simple way to install dalton, thereby increasing its adoption :)

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 19 Feb 2020, 09:37

Don't worry about asking for help on the forum. That's why it is here :)

I recommend that you try:

Code: Select all

./setup  --mpi --mkl=sequential
cd build
make
This will let the build system find the compilers and allow you to use MKL together with MPI. The output from the setup step will let you know which compilers it found and some more information. Note that you can build faster by using "-j [jobs]" to compile faster.

If there is a problem during setup or make, then feel free to attach the output from the setup and compile steps, and we can try to help you identify the issues.

If the build successfully completes, then you can set the number of processes to use, e.g., as follows:

Code: Select all

export DALTON_LAUNCHER="mpirun -n [num_procs]"
And then run:

Code: Select all

ctest -L dalton --output-on-failure
where the "-L dalton" skips some very long tests, and "--output-on-failure" will print any errors if a test fails.

Sounds good to have such a document :) I'm also hoping that we'll get some time to improve our documentation.

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 19 Feb 2020, 11:11

OK. I ran
./setup --fc=gfortran --cc=gcc --cxx=g++ --mpi --mkl=sequential | tee setup.out

which gave me

FC=gfortran CC=gcc CXX=g++ cmake -DENABLE_MPI=ON -DENABLE_SGI_MPT=OFF -DENABLE_OPENMP=OFF -DENABLE_64BIT_INTEGERS=OFF -DENABLE_GPU=OFF -DENABLE_CUBLAS=OFF -DENABLE_REAL_SP=OFF -DENABLE_STATIC_LINKING=OFF -DENABLE_SCALAPACK=OFF -DENABLE_AUTO_BLAS=ON -DENABLE_AUTO_LAPACK=ON -DMKL_FLAG="-mkl=sequential" -DENABLE_AUTO_BLAS=OFF -DENABLE_AUTO_LAPACK=OFF -DCMAKE_BUILD_TYPE=release ~/dalton

-- The Fortran compiler identification is GNU 9.2.1
-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS)
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
-- Could NOT find MPI_Fortran (missing: MPI_Fortran_LIB_NAMES MPI_Fortran_F77_HEADER_DIR MPI_Fortran_MODULE_DIR MPI_Fortran_WORKS)
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND MPI_Fortran_FOUND)
-- Configuring incomplete, errors occurred!
See also "~/dalton/build/CMakeFiles/CMakeOutput.log".
See also "~/dalton/build/CMakeFiles/CMakeError.log".

In terms of the tests on the previous build, which had no mpi and no specification wrt mkl, ctest returned

98% tests passed, 8 tests failed out of 496

Label Time Summary:
aosoppa = 57.89 sec*proc (18 tests)
benchmark = 9990.53 sec*proc (10 tests)
cc = 1740.52 sec*proc (87 tests)
cc3 = 1606.66 sec*proc (31 tests)
ccr12 = 3186.29 sec*proc (68 tests)
cholesky = 812.25 sec*proc (10 tests)
dalton = 7927.37 sec*proc (475 tests)
dft = 990.58 sec*proc (45 tests)
dpt = 11.37 sec*proc (8 tests)
energy = 86.60 sec*proc (19 tests)
essential = 74.20 sec*proc (117 tests)
fde = 360.92 sec*proc (2 tests)
gen1int = 21.23 sec*proc (2 tests)
geo = 411.83 sec*proc (29 tests)
long = 2274.18 sec*proc (18 tests)
mcscf = 23.20 sec*proc (3 tests)
medium = 2414.77 sec*proc (75 tests)
mp2r12 = 366.56 sec*proc (17 tests)
multistep = 211.47 sec*proc (9 tests)
numder = 6.25 sec*proc (5 tests)
pcm = 595.37 sec*proc (17 tests)
peqm = 39.36 sec*proc (24 tests)
prop = 325.92 sec*proc (44 tests)
qfit = 1.89 sec*proc (3 tests)
qm3 = 299.67 sec*proc (25 tests)
qmmm = 156.14 sec*proc (8 tests)
rsp = 409.86 sec*proc (79 tests)
runtest = 3588.02 sec*proc (204 tests)
short = 1943.49 sec*proc (373 tests)
soppa = 136.93 sec*proc (18 tests)
unknown = 1.69 sec*proc (1 test)
verylong = 2884.22 sec*proc (18 tests)
walk = 102.48 sec*proc (8 tests)
weekly = 11652.83 sec*proc (21 tests)

Total Test time (real) = 19580.56 sec

The following tests FAILED:
291 - cc_rsp_mcdbterm (Failed)
487 - benchmark_eri_adz (Timeout)
488 - benchmark_eri_adzs (Timeout)
489 - benchmark_eri_atzs (Timeout)
491 - benchmark_eri_r12xl (Timeout)
492 - benchmark_her_adz (Timeout)
493 - benchmark_her_adzs (Timeout)
494 - benchmark_her_atzs (Timeout)
Errors while running CTest

so we're getting there... :)

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 19 Feb 2020, 11:18

Ok, so the problem is that there are no MPI compiler wrappers available on your machine (at least not that the build system can find). Do you know that they are installed on your system?

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 19 Feb 2020, 11:19

Btw. the failed "benchmark_*" tests are ok since they just timeout. These tests are skipped if you run "ctest -L dalton". The last one is most likely also ok.

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 19 Feb 2020, 15:32

I have Intel impi installed and also openmpi, both done through dnf from the appropriate repo.

locate <name> gives me

/opt/intel/impi/2019.6.166 and a bunch of other places, all in /opt

and

/etc/openmpi-x86_64

/usr/include/openmpi-x86_64

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 19 Feb 2020, 15:42

Ok, sounds like you just need to export the environment variables. For OpenMPI when installed through dnf, e.g., on Fedora, then you need to load it by running "module load mpi/openmpi-x86_64" before running the Dalton setup script.

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 19 Feb 2020, 15:50

Thank you - I did

module load mpi/openmpi-x86_64

and then setup which gave

FC=gfortran CC=gcc CXX=g++ cmake -DENABLE_MPI=ON -DENABLE_SGI_MPT=OFF -DENABLE_OPENMP=OFF -DENABLE_64BIT_INTEGERS=OFF -DENABLE_GPU=OFF -DENABLE_CUBLAS=OFF -DENABLE_REAL_SP=OFF -DENABLE_STATIC_LINKING=OFF -DENABLE_SCALAPACK=OFF -DENABLE_AUTO_BLAS=ON -DENABLE_AUTO_LAPACK=ON -DMKL_FLAG="-mkl=sequential" -DENABLE_AUTO_BLAS=OFF -DENABLE_AUTO_LAPACK=OFF -DCMAKE_BUILD_TYPE=release /home/jturner/dalton

-- The Fortran compiler identification is GNU 9.2.1
-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /usr/lib64/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/lib64/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI_Fortran: /usr/lib64/openmpi/lib/libmpi_usempif08.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Performing Test MPI_COMPATIBLE
-- Performing Test MPI_COMPATIBLE - Failed
-- Performing Test MPI_F90_I4
-- Performing Test MPI_F90_I4 - Failed
-- Performing Test MPI_F90_I8
-- Performing Test MPI_F90_I8 - Failed
-- Performing Test ENABLE_MPI3_FEATURES
-- Performing Test ENABLE_MPI3_FEATURES - Failed
-- Found Git: /usr/bin/git
-- Polarizable Continuum Model via PCMSolver DISABLED
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jturner/dalton/build

configure step is done
now you need to compile the sources:
$ cd build
$ make

and now the make is proceeding straightforwardly so far. V. v. appreciative of your help :)

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 19 Feb 2020, 15:54

Great! Except I just realized that the "--mkl" option will not work since this is intended only for Intel compilers. So you'll need to either let CMake find suitable math libraries and hopefully it will find the MKL. Otherwise, you may need to set MATH_ROOT to point to the MKL installation directory first.

jfct_sussex
Posts: 9
Joined: 12 Feb 2020, 22:26
First name(s): John
Last name(s): Turner
Affiliation: University of Sussex
Country: United Kingdom

Re: gen1int and CMakeLists.txt.

Post by jfct_sussex » 19 Feb 2020, 17:36

setup still fails with the --mpi flag and without the --mkl=sequential flag.

[~/dalton]$ module list
Currently Loaded Modulefiles:
1) mpi/openmpi-x86_64

but
./setup --fc=gfortran --cc=gcc --cxx=g++ --mpi | tee setup.out

works fine except for

-- Performing Test MPI_COMPATIBLE
-- Performing Test MPI_COMPATIBLE - Failed
-- Performing Test MPI_F90_I4
-- Performing Test MPI_F90_I4 - Failed
-- Performing Test MPI_F90_I8
-- Performing Test MPI_F90_I8 - Failed
-- Performing Test ENABLE_MPI3_FEATURES
-- Performing Test ENABLE_MPI3_FEATURES - Failed
which seems to suggest that the mpi has fallen over somehow. ctest then fails everything...

User avatar
magnus
Posts: 499
Joined: 27 Jun 2013, 16:32
First name(s): Jógvan Magnus
Middle name(s): Haugaard
Last name(s): Olsen
Affiliation: Hylleraas Centre, UiT The Arctic University of Norway
Country: Norway

Re: gen1int and CMakeLists.txt.

Post by magnus » 19 Feb 2020, 20:32

That could be because of a discrepancy between the compiler used to compile OpenMPI and the compiler used here. What is the output you get from "mpif90 --version"? You could try:

Code: Select all

./setup --fc=mpif90 --cc=mpicc --cxx=mpicxx | tee setup.out

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests