Problems with parallel builds (make -j)

[Nicolas]

When using parallel builds (make -j3), sometimes linking of crypt_prog or boinc_cmd fails saying "libboinc.a: no such file or directory", because the library wasn't created yet. Both executables should have an explicit dependency to libboinc.a, so that make doesn't attempt to link them until the library is ready.

[Nicolas]

Eric is working on the Linux build system.

I think we should split the "Client - Build" Trac component into platforms.

[icode4u]

This is definitely a problem. Which Makefile needs the dependencies added? I'm not very good with makefiles, but I can take a look. Should just be a couple of lines, right?

[Nicolas]

Since I filed this bug, there were major changes to the build system that caused new parallel build issues, apart from the ones in lib. Also, boinccmd was moved to client, making the particular problem I first reported go away.

[icode4u]

If I get the latest http://boinc.berkeley.edu/svn/trunk/boinc source code, this problem goes away (using "make -j 4"). Is that the latest code? If so, this ticket can be closed when that source code goes to release.

[Nicolas]

I just tried make clean; make -j4. The build system apparently tried to link libboinc.a before libtool was done writing one of the object files:

make[3]: Entering directory `/home/nicolas/src/boinc/lib'
/bin/bash ../libtool --tag=CXX   --mode=compile ccache g++ -DHAVE_CONFIG_H -I. -I..  -I../lib -I../api -I../db -I../client -I../tools -I../sched -I../lib/mac -pthread   -fPIC -DPIC -pthread -g -O2 -MT libboinc_la-util.lo -MD -MP -MF .deps/libboinc_la-util.Tpo -c -o libboinc_la-util.lo `test -f 'util.cpp' || echo './'`util.cpp
mv -f .deps/libboinc_la-util.Tpo .deps/libboinc_la-util.Plo
/bin/bash ../libtool --tag=CXX   --mode=link ccache g++  -fPIC -DPIC -pthread -g -O2 -L/usr/local/lib -rpath /usr/local/lib -version-number 6:7:4  -o libboinc.la -rpath /usr/local/lib libboinc_la-app_ipc.lo libboinc_la-base64.lo libboinc_la-cert_sig.lo libboinc_la-coproc.lo libboinc_la-crypt.lo libboinc_la-diagnostics.lo libboinc_la-filesys.lo libboinc_la-gui_rpc_client.lo libboinc_la-gui_rpc_client_ops.lo libboinc_la-gui_rpc_client_print.lo libboinc_la-hostinfo.lo libboinc_la-mac_address.lo libboinc_la-md5.lo libboinc_la-md5_file.lo libboinc_la-mem_usage.lo libboinc_la-mfile.lo libboinc_la-miofile.lo libboinc_la-msg_log.lo libboinc_la-msg_queue.lo libboinc_la-network.lo libboinc_la-parse.lo libboinc_la-prefs.lo libboinc_la-procinfo_unix.lo libboinc_la-proxy_info.lo libboinc_la-shmem.lo libboinc_la-str_util.lo libboinc_la-synch.lo libboinc_la-unix_util.lo libboinc_la-util.lo
ccache g++ -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/4.2.4/../../../../lib/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.2.4/crtbeginS.o  .libs/libboinc_la-app_ipc.o .libs/libboinc_la-base64.o .libs/libboinc_la-cert_sig.o .libs/libboinc_la-coproc.o .libs/libboinc_la-crypt.o .libs/libboinc_la-diagnostics.o .libs/libboinc_la-filesys.o .libs/libboinc_la-gui_rpc_client.o .libs/libboinc_la-gui_rpc_client_ops.o .libs/libboinc_la-gui_rpc_client_print.o .libs/libboinc_la-hostinfo.o .libs/libboinc_la-mac_address.o .libs/libboinc_la-md5.o .libs/libboinc_la-md5_file.o .libs/libboinc_la-mem_usage.o .libs/libboinc_la-mfile.o .libs/libboinc_la-miofile.o .libs/libboinc_la-msg_log.o .libs/libboinc_la-msg_queue.o .libs/libboinc_la-network.o .libs/libboinc_la-parse.o .libs/libboinc_la-prefs.o .libs/libboinc_la-procinfo_unix.o .libs/libboinc_la-proxy_info.o .libs/libboinc_la-shmem.o .libs/libboinc_la-str_util.o .libs/libboinc_la-synch.o .libs/libboinc_la-unix_util.o .libs/libboinc_la-util.o  -L/usr/local/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.2.4 -L/usr/lib/gcc/x86_64-linux-gnu/4.2.4/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.2.4/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/4.2.4/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/4.2.4/../../../../lib/crtn.o  -pthread -Wl,-soname -Wl,libboinc.so.6 -o .libs/libboinc.so.6.7.4
g++: .libs/libboinc_la-util.o: No such file or directory
make[2]: *** [libboinc.la] Error 1
make[2]: *** Waiting for unfinished jobs....
ccache g++ -DHAVE_CONFIG_H -I. -I.. -I../lib -I../api -I../db -I../client -I../tools -I../sched -I../lib/mac -pthread -fPIC -DPIC -pthread -g -O2 -MT libboinc_la-util.lo -MD -MP -MF .deps/libboinc_la-util.Tpo -c util.cpp  -fPIC -DPIC -o .libs/libboinc_la-util.o
mv -f .deps/crypt_prog-crypt_prog.Tpo .deps/crypt_prog-crypt_prog.Po
ccache g++ -DHAVE_CONFIG_H -I. -I.. -I../lib -I../api -I../db -I../client -I../tools -I../sched -I../lib/mac -pthread -fPIC -DPIC -pthread -g -O2 -MT libboinc_la-util.lo -MD -MP -MF .deps/libboinc_la-util.Tpo -c util.cpp -o libboinc_la-util.o >/dev/null 2>&1
mv -f .deps/libboinc_la-util.Tpo .deps/libboinc_la-util.Plo

[mjakubicek]

Would be nice to have this resolved as it significantly speeds up the build...