wiki:GPUApp

Version 15 (modified by TuanLe, 14 years ago) (diff)

--

Building BOINC applications with Cuda and OpenCL

  1. Building BOINC applications with Cuda and OpenCL
    1. Windows
      1. NVIDIA Cuda
      2. NVIDIA OpenCL
      3. ATI Stream OpenCL
    2. Linux
      1. NVIDIA Cuda
      2. NVIDIA OpenCL
      3. ATI Stream OpenCL
    3. Mac OS X
      1. NVIDIA Cuda

We have built sample BOINC-Cuda and BOINC-OpenCL applications as templates for making your own app. The source files are in the BOINC source tree? in samples/nvcuda, samples/nvopencl, and samples/atiopencl.

Windows

The 32-bit Windows compile under Visual Studio

NVIDIA Cuda

The VS project file, example_app_nvcuda.vcproj, is in "boinc/win_build/". Note that NVIDIA Cuda SDK 3.0 or older supports device emulation mode. If your machine doesn't have a CUDA-enabled GPU, then you should build the sample app in the device emulation mode. Do so by setting "Build -> Configuration Manger -> Configuration" to either Emudebug or Emurelease.

The executable file can be found at "boinc/win_build/Build/Win32/build_mode/" (build_mode could be release, debug, emurelease or emudebug).

NVIDIA OpenCL

The project file, example_app_nvopencl.vcproj, is in "boinc/win_build/". OpenCL doesn't have emulation mode, but it could run on CPU alone by specifying CL_DEVICE_TYPE_CPU or CL_DEVICE_TYPE_ALL when creating an OpenCL context.

ATI Stream OpenCL

The project file, example_app_atiopencl.vcproj, is in "boinc/win_build/". It is identical to NVIDIA OpenCL.

Linux

NVIDIA Cuda

Before running the Makefile, you will need to install gcc 4.3 and g++ 4.3. This is because the NVIDIA Cuda SDK 3.0 has not yet worked with gcc 4.0 and g++ 4.0. There should be no issue compiling cuda files with gcc 4.3 and g++ 4.3 on newer NVIDIA Cuda SDK versions. For a successful compilation, please follow these steps:

1) Install gcc-4.3 and g++-4.3: 

$ sudo aptitude install gcc-4.3 g++-4.3

2) Go to SDK source directory: 

$ cd ~/NVIDIA_GPU_Computing_SDK/C

3) Create a directory and create symlinks to gcc-4.3/g++-4.3 

$ mkdir mygcc
$ cd mygcc
$ ln -s $(which g++-4.3) g++
$ ln -s $(which gcc-4.3) gcc

The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location "$(ROOT)/home/USER_NAME/NVIDIA_GPU_Computing_SDK/". On my Linux machine, for example, it would be "$(ROOT)/home/tuanle/NVIDIA_GPU_Computing_SDK/". Since the absolute path is used in Makefile, you will probably need to edit file "common.mk" before compiling the sample app. Do so by looking for "/home/tuanle/NVIDIA_GPU_Computing_SDK" in file "common.mk" and replace this path with an appropriate path on your machine.

You will also need to build the necessary Cuda SDK libraries by running makefile in "~/NVIDIA_GPU_Computing_SDK/C/". The resulting libraries such as "libcutil_i386" are created in "~/NVIDIA_GPU_Computing_SDK/C/lib/". Notice that if these libraries are not built, running nvcuda-sample-app makefile could result in the following error: 

/usr/bin/ld: cannot find -lcutil_i386

If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine. To make an executable file in emulation mode, simply type "make emu=1".

Also, note that you will need to define some environment variables before running the executable file. 

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH

This will resolve the following error: 

./example_app_nvcuda: error while loading shared libraries: libcudart.so.3: cannot open shared object file: No such file or directory

NVIDIA OpenCL

It appears that NVIDIA OpenCL SDK is missing shared library file libOpenCL.so which is needed at compile time. This library file comes with the kernel driver package which can be installed only when your machine has an NVIDIA Cuda-enabled GPU. Thus, any attempt to compile OpenCL sample codes on a non-cuda-enabled GPU machine might result in the following error: 

/usr/bin/ld: cannot find -lOpenCL

The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location "$(ROOT)/home/USER_NAME/NVIDIA_GPU_Computing_SDK/". On my Linux machine, for example, it would be "$(ROOT)/home/tuanle/NVIDIA_GPU_Computing_SDK/". Since the absolute path is used in Makefile, you will probably need to edit file "common.mk" before compiling the sample app. Do so by looking for "/home/tuanle/NVIDIA_GPU_Computing_SDK" in file "common.mk" and replace this path with an appropriate path on your machine.

ATI Stream OpenCL

The Makefile for Linux is made with the assumption that the ATI Stream OpenCL SDK version 2.1 for 32-bit Linux and BOINC are installed in the same parent directory, i.e. "ati-stream-sdk-v.2.1-lnx32" directory and "boinc" directory are both installed at "$(ROOT)/home/" directory. If you are using the SDK v2.1 for 64-bit Linux, or if BOINC and the OpenCL SDK are not installed in the same parent directory, then Makefile needs to be edited prior to any compilation. Do so by looking for 

CXXFLAGS2 = -g \
     ...
     -L ../../../ati-stream-sdk-v2.1-lnx32/lib/x86 \
     -L ../../../ati-stream-sdk-v2.1-lnx32/TempSDKUtil/lib/x86 \

atiopencl.o: atiopencl.cpp atiopencl.hpp
     ...
     -I ../../../ati-stream-sdk-v2.1-lnx32/include \
     -I ../../../ati-stream-sdk-v2.1-lnx32/samples/opencl/SDKUtil

in file "Makefile" and replace these paths with appropriate paths on your machine.

Most problems related to running the ATI Stream OpenCL executable files can be found in ATI_Stream_SDK_Installation_Notes.pdf section 2.2. Here are the two most common run-time errors:

1) ./atiopencl: error while loading shared libraries: libOpenCL.so: cannot open shared object file: No such file or directory -> Solution: Set the environment variable ATISTREAMSDKROOT 

$ export ATISTREAMSDKROOT=<location where Stream SDK is extracted>

Set the library path LD_LIBRARY_PATH

  • For 32-bit systems: 
    $ export LD_LIBRARY_PATH=$ATISTREAMSDKROOT/lib/x86:$LD_LIBRARY_PATH
  • For 64-bit systems: 
    $ export LD_LIBRARY_PATH=$ATISTREAMSDKROOT/lib/x86_64:$LD_LIBRARY_PATH
  • For the 32-bit SDK to run on 64-bit systems: 
    $ export LD_LIBRARY_PATH=$ATISTREAMSDKROOT/lib/x86:$ATISTREAMSDKROOT/lib/x86_64:$LD_LIBRARY_PATH

2) ./atiopencl: error CL_PLATFORM_NOT_FOUND ->

Solution: Register the OpenCL ICD by following these steps:

  • Download icd-registration.tgz at ​http://developer.amd.com/Downloads/icd-registration.tgz
  • In the command shell, type "cd /"
  • Type "ls", you will see a directory named "etc". The icd-registration.tgz needs to be unzipped to that directory.
  • Now copy icd-registration.tgz to "cd /" by typing "sudo cp -r /home/tuanle/Downloads/icd-registration.tgz ."
  • You will need to replace the specified path above with the appropriate path on your machine, and don't forget "sudo". Otherwise you will get a "Permission denied".
  • Unzip by typing "sudo tar xzf icd-registration.tgz"
  • Now you should be able to find an "OpenCL" directory in "etc". Check etc/OpenCL/vendors/, make sure there are "atiocl32.icd" and "atiocl64.icd" there.
  • Done!

Mac OS X

The Mac versions are built using gcc/make (not xcode).

NVIDIA Cuda

Unlike Windows and Linux, the 'nvcc' cuda compiler for Mac has some trouble compiling .cu files that contain both BOINC and Cuda code. Any attempt to compile such .cu files might result in errors like these: 

/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(55): error: identifier "__builtin_ia32_emms" is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(68): error: identifier "__builtin_ia32_vec_init_v2si" is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(111): error: identifier "__builtin_ia32_vec_ext_v2si" is undefined
...

The solution that we came up with is to compile BOINC and Cuda separately, meaning that compile .c files that have BOINC codes with gcc and compile .u files that have Cuda codes such as kernel definitions with nvcc (This can simply be done by putting .cu and .c file names at corresponding "CUFILES := " and "CCFILES := " entries in Makefile_mac). To handle the results computed by the kernels, you will need to write external functions. The function definitions that make kernel calls will be put in .cu file while the function headers with "extern" prefix are put in .c file and can be called in main function or elsewhere.

If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine. To make an executable file in emulation mode, simply type "make -f Makefile_mac emu=1".

Also, note that you will need to define some environment variables before running the executable file. 

export PATH=/usr/local/cuda/bin:$PATH
export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH

One last thing to mention, the makefile for Mac is made with the assumption that the NVIDIA Cuda SDK is installed at the default location: "$ROOT/Developer/GPU Computing/" on Mac. If it is installed at a different location on your machine, then you will need to edit file "common_mac.mk". Do so by looking for "ROOTDIR ?= /Developer/GPU\ Computing" in file "common_mac.mk" and replace this path with an appropriate path on your machine.