Compilation error due to the absence of some symbols

[jjerphan]

After having adapted the setup by:

• Creating a python3.9 conda environment
• Installing packaging, setuptools, distro, scikit-build, dpctl from Intel channel
• Symlinking icpx so that's in PATH
• Specifying IntelDPCPP explicitly (which was not found by CMake)
• Commenting the call to SYCL_FEATURE_TEST_EXTRACT(${test_output}) in IntelDPCPPConfig.cmake

My setup launches the C++ compilation, but this then fails due to the absence of some symbols.

Must I use a specific version of dpctl?

Details

CXX=icpx python setup.py develop -- -DIntelDPCPP_DIR=/opt/intel/oneapi/compiler/latest/linux/IntelDPCPP -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir)

/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(


--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 12.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib64/ccache/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is IntelLLVM 2022.2.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/jjerphan/.scripts/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jjerphan/dev/kmeans_dpcpp/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - success
--------------------------------------------------------------------------------

Configuring Project
  Working directory:
    /home/jjerphan/dev/kmeans_dpcpp/_skbuild/linux-x86_64-3.9/cmake-build
  Command:
    cmake /home/jjerphan/dev/kmeans_dpcpp -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/jjerphan/dev/kmeans_dpcpp/_skbuild/linux-x86_64-3.9/cmake-install -DPYTHON_VERSION_STRING:STRING=3.9.15 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/include/python3.9 -DPYTHON_LIBRARY:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/lib/libpython3.9.so -DPython_EXECUTABLE:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/bin/python -DPython_ROOT_DIR:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel -DPython_INCLUDE_DIR:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/include/python3.9 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_NumPy_INCLUDE_DIRS:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/bin/python -DPython3_ROOT_DIR:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel -DPython3_INCLUDE_DIR:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/include/python3.9 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_NumPy_INCLUDE_DIRS:PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/numpy/core/include -DIntelDPCPP_DIR=/opt/intel/oneapi/compiler/latest/linux/IntelDPCPP -DDPCTL_MODULE_PATH=/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/dpctl/resources/cmake -DCMAKE_BUILD_TYPE:STRING=Release

SYCL feature test compile failed!
compile output is:
-- The SYCL compiler is /home/jjerphan/.scripts/icpx
-- The SYCL Flags are --dpcpp -fsycl
-- The SYCL Language Version is
CMake Warning (dev) at /home/jjerphan/.local/share/miniconda3/envs/intel/share/cmake-3.25/Modules/FetchContent.cmake:1279 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  CMakeLists.txt:22 (FetchContent_Declare)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- pybind11 v2.10.1
-- Found PythonInterp: /home/jjerphan/.local/share/miniconda3/envs/intel/bin/python (found suitable version "3.9.15", minimum required is "3.6")
-- Found PythonInterp: /home/jjerphan/.local/share/miniconda3/envs/intel/bin/python (found version "3.9.15")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jjerphan/dev/kmeans_dpcpp/_skbuild/linux-x86_64-3.9/cmake-build
[1/3] Building CXX object CMakeFiles/_kmeans_dpcpp.dir/python_api/_kmeans_lloyd.cpp.o
FAILED: CMakeFiles/_kmeans_dpcpp.dir/python_api/_kmeans_lloyd.cpp.o
/home/jjerphan/.scripts/icpx -D_kmeans_dpcpp_EXPORTS -I/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/dpctl/include -I/home/jjerphan/dev/kmeans_dpcpp/src -isystem /home/jjerphan/dev/kmeans_dpcpp/_skbuild/linux-x86_64-3.9/cmake-build/_deps/pybind11-src/include -isystem /home/jjerphan/.local/share/miniconda3/envs/intel/include/python3.9 --dpcpp -fsycl  -O3 -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++17 -MD -MT CMakeFiles/_kmeans_dpcpp.dir/python_api/_kmeans_lloyd.cpp.o -MF CMakeFiles/_kmeans_dpcpp.dir/python_api/_kmeans_lloyd.cpp.o.d -o CMakeFiles/_kmeans_dpcpp.dir/python_api/_kmeans_lloyd.cpp.o -c /home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:23:40: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
  if (X_ndim != 2 || y_ndim != 1 || !X.is_c_contiguous() || !y.is_c_contiguous()) {
                                     ~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:23:64: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
  if (X_ndim != 2 || y_ndim != 1 || !X.is_c_contiguous() || !y.is_c_contiguous()) {
                                                             ~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:31:22: error: no member named 'queues_are_compatible' in namespace 'dpctl::utils'
  if (!dpctl::utils::queues_are_compatible(q, {X.get_queue(), y.get_queue()})) {
       ~~~~~~~~~~~~~~^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:40:30: error: no member named 'dpctl_capi' in namespace 'dpctl::detail'; did you mean 'dpctl_api'?
  auto &api = dpctl::detail::dpctl_capi::get();
              ~~~~~~~~~~~~~~~^~~~~~~~~~
                             dpctl_api
/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/dpctl/include/dpctl4pybind11.hpp:218:8: note: 'dpctl_api' declared here
struct dpctl_api
       ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:43:24: error: no member named 'UAR_FLOAT_' in 'dpctl::detail::dpctl_api'
  if (y_typenum == api.UAR_FLOAT_) {
                   ~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:47:31: error: no member named 'UAR_DOUBLE_' in 'dpctl::detail::dpctl_api'
  } else if (y_typenum == api.UAR_DOUBLE_) {
                          ~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:70:40: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
  if (X_ndim != 2 || y_ndim != 1 || !X.is_c_contiguous() || !y.is_c_contiguous()) {
                                     ~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:70:64: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
  if (X_ndim != 2 || y_ndim != 1 || !X.is_c_contiguous() || !y.is_c_contiguous()) {
                                                             ~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:78:22: error: no member named 'queues_are_compatible' in namespace 'dpctl::utils'
  if (!dpctl::utils::queues_are_compatible(q, {X.get_queue(), y.get_queue()})) {
       ~~~~~~~~~~~~~~^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:87:30: error: no member named 'dpctl_capi' in namespace 'dpctl::detail'; did you mean 'dpctl_api'?
  auto &api = dpctl::detail::dpctl_capi::get();
              ~~~~~~~~~~~~~~~^~~~~~~~~~
                             dpctl_api
/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/dpctl/include/dpctl4pybind11.hpp:218:8: note: 'dpctl_api' declared here
struct dpctl_api
       ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:90:24: error: no member named 'UAR_FLOAT_' in 'dpctl::detail::dpctl_api'
  if (y_typenum == api.UAR_FLOAT_) {
                   ~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:94:31: error: no member named 'UAR_DOUBLE_' in 'dpctl::detail::dpctl_api'
  } else if (y_typenum == api.UAR_DOUBLE_) {
                          ~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:153:37: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
  if (!cluster_sizes_private_copies.is_c_contiguous() ||
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:154:35: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
      !centroids_t_private_copies.is_c_contiguous() ||
       ~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:155:26: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
      !out_cluster_sizes.is_c_contiguous() ||
       ~~~~~~~~~~~~~~~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:156:24: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
      !out_centroids_t.is_c_contiguous() ||
       ~~~~~~~~~~~~~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:157:32: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
      !out_empty_clusters_list.is_c_contiguous() ||
       ~~~~~~~~~~~~~~~~~~~~~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:158:29: error: no member named 'is_c_contiguous' in 'dpctl::tensor::usm_ndarray'
      !out_n_empty_clusters.is_c_contiguous())
       ~~~~~~~~~~~~~~~~~~~~ ^
/home/jjerphan/dev/kmeans_dpcpp/python_api/_kmeans_lloyd.cpp:163:25: error: no member named 'queues_are_compatible' in namespace 'dpctl::utils'
  if (! ::dpctl::utils::queues_are_compatible(q, {
        ~~~~~~~~~~~~~~~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 640, in setup
    cmkr.make(make_args, install_target=cmake_install_target, env=env)
  File "/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/skbuild/cmaker.py", line 672, in make
    self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  File "/home/jjerphan/.local/share/miniconda3/envs/intel/lib/python3.9/site-packages/skbuild/cmaker.py", line 703, in make_impl
    raise SKBuildError(

An error occurred while building with CMake.
  Command:
    cmake --build . --target install --config Release --
  Install target:
    install
  Source directory:
    /home/jjerphan/dev/kmeans_dpcpp
  Working directory:
    /home/jjerphan/dev/kmeans_dpcpp/_skbuild/linux-x86_64-3.9/cmake-build
Please check the install target is valid and see CMake's output for more information.

[oleksandr-pavlyk]

Sorry about scarce details. Here is what I did:

1. Create Python 3.9 conda environment conda create -n env_name python=3.9 cmake ninja
2. Activate
3. Install packages from pip install --no-cache-dir packaging setuptools distro scikit-build pytest numpy.
4. Activate oneAPI compiler and TBB:

$ source /opt/intel/oneapi/compiler/latest/env/vars.sh
$ source /opt/intel/oneapi/tbb/latest/env/vars.sh

5. Clone dpctl trunk: git clone https://github.com/IntelPython/dpctl (you can also install dpctl from dppy/label/dev, but you need to ensure that oneAPI compiler is the matching version to dpcpp-cpp-rt conda package installed as dependency of dpctl conda package).
6. Build it using

$ pushd dpctl
$ python scripts/build_locally.py
$ popd

7. Clone this repo (no source modification should be necessary)
8. Building using the following steps:

$ cd kmeans_dpcpp
$ python setup.py develop -- -DCMAKE_CXX_COMPILER:PATH=$(which icpx) -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir)
$ python -m pytest test

Notice that -DCMAKE_CXX_COMPILER setting is necessary for find_package(IntelDPCPP REQUIRED) to work. One can alternatively use CXX environment variable to specify the compiler:

$ CXX=icpx python setup.py develop -- -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir)

Notice that your CMake output shows that GCC 12 has been chosen which does not support SYCL, so find_package(IntelDPCPP REQUIRED) fails.

[jjerphan]

Thanks! I have been able to install the project. 👍