[mlir][nvgpu] Build error with "mgpuTensorMapEncodeTiled"

[yifeihe007]

I encountered a build error for the latest main branch llvm code(20230808) when building the mlir target with CUDA enabled.

OS: Ubuntu 22.04
CPU: Intel
CUDA Version: 11.8

Build flags:

cmake -G Ninja ../llvm -DLLVM_ENABLE_PROJECTS=mlir -DLLVM_BUILD_EXAMPLES=ON -DLLVM_TARGETS_TO_BUILD="X86;NVPTX;AMDGPU" -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_ASSERTIONS=ON -DMLIR_ENABLE_CUDA_RUNNER=ON -DMLIR_ENABLE_BINDINGS_PYTHON=ON -DPython3_EXECUTABLE="/usr/bin/python3"

cmake --build . --target check-mlir

Error:
FAILED: tools/mlir/lib/ExecutionEngine/CMakeFiles/mlir_cuda_runtime.dir/CudaRuntimeWrappers.cpp.o
/usr/bin/c++ -DBUILD_EXAMPLES -DGTEST_HAS_RTTI=0 -DMLIR_CUDA_CONVERSIONS_ENABLED=1 -DMLIR_ROCM_CONVERSIONS_ENABLED=1 -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_GNU_SOURCE -D_LIBCPP_ENABLE_HARDENED_MODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/hdd/yifei/workSpace/mlir/0808/llvm-project/build/tools/mlir/lib/ExecutionEngine -I/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine -I/hdd/yifei/workSpace/mlir/0808/llvm-project/build/include -I/hdd/yifei/workSpace/mlir/0808/llvm-project/llvm/include -I/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/include -I/hdd/yifei/workSpace/mlir/0808/llvm-project/build/tools/mlir/include -I/usr/local/cuda-11.8/targets/x86_64-linux/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -fno-lifetime-dse -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-nonnull -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG -fPIC -fno-exceptions -funwind-tables -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/mlir/lib/ExecutionEngine/CMakeFiles/mlir_cuda_runtime.dir/CudaRuntimeWrappers.cpp.o -MF tools/mlir/lib/ExecutionEngine/CMakeFiles/mlir_cuda_runtime.dir/CudaRuntimeWrappers.cpp.o.d -o tools/mlir/lib/ExecutionEngine/CMakeFiles/mlir_cuda_runtime.dir/CudaRuntimeWrappers.cpp.o -c /hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:299:43: error: variable or field ‘mgpuTensorMapEncodeTiled’ declared void
299 | extern "C" MLIR_CUDA_WRAPPERS_EXPORT void mgpuTensorMapEncodeTiled(
| ^~~~~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:300:5: error: ‘CUtensorMap’ was not declared in this scope
300 | CUtensorMap *tensorMap, // Tensor map object
| ^~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:300:18: error: ‘tensorMap’ was not declared in this scope
300 | CUtensorMap *tensorMap, // Tensor map object
| ^~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:301:5: error: ‘CUtensorMapDataType’ was not declared in this scope
301 | CUtensorMapDataType tensorDataType, // Tensor data type
| ^~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:302:16: error: expected primary-expression before ‘tensorRank’
302 | cuuint32_t tensorRank, // Dimensionality of tensor
| ^~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:303:5: error: expected primary-expression before ‘void’
303 | void *globalAddress, // Starting address
| ^~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:304:5: error: expected primary-expression before ‘const’
304 | const cuuint64_t *globalDim, // Tensor size (number of elements)
| ^~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:305:5: error: expected primary-expression before ‘const’
305 | const cuuint64_t globalStrides, // Stride size (in bytes)
| ^~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:306:5: error: expected primary-expression before ‘const’
306 | const cuuint32_t boxDim, // Traversal box (number of elments)
| ^~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:307:5: error: expected primary-expression before ‘const’
307 | const cuuint32_t elementStrides, // Traversal stride
| ^~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:308:5: error: ‘CUtensorMapInterleave’ was not declared in this scope
308 | CUtensorMapInterleave interleave, // Type of interleaved layout
| ^~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:309:5: error: ‘CUtensorMapSwizzle’ was not declared in this scope
309 | CUtensorMapSwizzle swizzle, // Bank swizzling pattern
| ^~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:310:5: error: ‘CUtensorMapL2promotion’ was not declared in this scope
310 | CUtensorMapL2promotion l2Promotion, // L2 promotion size
| ^~~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:311:5: error: ‘CUtensorMapFloatOOBfill’ was not declared in this scope
311 | CUtensorMapFloatOOBfill oobFill // Padding zfill or NaN fill
| ^~~~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:341:11: error: ‘CUtensorMapDataType’ does not name a type
341 | const CUtensorMapDataType tensorDataType, // Stride size (in bytes)
| ^~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:342:5: error: ‘CUtensorMapInterleave’ has not been declared
342 | CUtensorMapInterleave interleave, // Type of interleaved layout
| ^~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:343:5: error: ‘CUtensorMapSwizzle’ has not been declared
343 | CUtensorMapSwizzle swizzle, // Bank swizzling pattern
| ^~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:344:5: error: ‘CUtensorMapL2promotion’ has not been declared
344 | CUtensorMapL2promotion l2Promotion, // L2 promotion size
| ^~~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:345:5: error: ‘CUtensorMapFloatOOBfill’ has not been declared
345 | CUtensorMapFloatOOBfill oobFill, // Padding zfill or NaN fill
| ^~~~~~~~~~~~~~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp: In function ‘void mgpuTensorMapEncodeTiledMemref(int64_t, StridedMemRefType<char, 1>, int, int, int, int, int, int64_t)’:
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:348:3: error: ‘CUtensorMap’ was not declared in this scope
348 | CUtensorMap tensorMap;
| ^~~~~~~~~~~
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:369:29: error: ‘tensorMap’ was not declared in this scope; did you mean ‘tensorRank’?
369 | mgpuTensorMapEncodeTiled(&tensorMap, tensorDataType, tensorRank32,
| ^~~~~~~~~
| tensorRank
/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:369:3: error: ‘mgpuTensorMapEncodeTiled’ was not declared in this scope; did you mean ‘mgpuTensorMapEncodeTiledMemref’?
369 | mgpuTensorMapEncodeTiled(&tensorMap, tensorDataType, tensorRank32,
| ^~~~~~~~~~~~~~~~~~~~~~~~
| mgpuTensorMapEncodeTiledMemref
[23/658] Building CXX object tools/mlir/lib/CAP...Files/obj.MLIRCAPILinalg.dir/LinalgPasses.cpp.o
ninja: build stopped: subcommand failed.

[llvmbot]

@llvm/issue-subscribers-mlir-gpu

[neozhang307]

Same issue for CUDA Version: 11.5

[joker-eph]

/hdd/yifei/workSpace/mlir/0808/llvm-project/mlir/lib/ExecutionEngine/CudaRuntimeWrappers.cpp:300:5: error: ‘CUtensorMap’ was not declared in this scope

The CUtensorMap API was introduced in Cuda 12.
We should either guards our source with ifdef if we want to support Cuda 11, or have CMake require Cuda 12.

[grypp]

I can take care of this. I added CUtensorMap api recently. I am going to guard it with #if (CUDA_VERSION >= 12000)

I also need to check the driver version on the runtime as well.