IPC Mempool Serialization and multiprocessing Module Support (#1020)

* Restructures IPC mempool tests into a subdirectory.

* Simplify the IPC interface, adding create_ipc_channel and import_/export methods.

* Simply the interface to IPCBufferTestHelper.

* Adds more tests.

* Removes sequence forms of certain function (exception behavior was unclear). Added a test for an error case.

* Changes channel methods export/import_ to send_buffer/receive_buffer, for clarity.

* Implement serialization methods for Device, Buffer, and DeviceMemoryResource. Add tests for buffer IPC through serialization.

* Protects serialization where needed to avoid resource leaks. Adds a registry from imported memory resources so that buffers can be serialized using an mr key. Test updates.

* Add tests for leaked file descriptors and fix leaks.

* Eliminates IPCChannel.

* Changes DeviceMemoryResource remote_id to uuid.

* Embeds the memory resource UUID into allocation handles.

* Minor changes to address feedback.

* Removes obsolte tests. Moves imports to nested contexts.

* Removes pickling for Device objects. Registers the pickle method with multiprocessing instead.

* Updates register function to return registered object. Avoids possible early deregistration.

* Renames Buffer import_/export methods.

* Moves AllocationHandle serialization to a registration with multiprocessing, since it depends on DupFd.

* Use DeviceMemoryResourceOptions throughout tests.

* Merged tests/memory_ipc/conftest.py into tests/conftest.py because certain configurations could not resolve it consistently.

* Makes the psutil module an optional dependency for testing.

* Bump the child timeout for IPC tests.

* Add docstrings. Change is_imported to is_mapped. Register DeviceMemoryResource reduction with multiprocessing. Add a quick exit to from_allocation_handle. Simplify the worker pool tests based on the new reduction method.

* Remove call to set_current in Device reconstruction. Add device set-up to tests.

* fix docstring rendering

---------

Co-authored-by: Leo Fang <leof@nvidia.com>

Author: Andy-Jost

cuda_core/cuda/core/experimental/__init__.py

@@ -17,7 +17,7 @@
 from cuda.core.experimental._memory import (
     Buffer,
     DeviceMemoryResource,
-    IPCChannel,
+    DeviceMemoryResourceOptions,
     LegacyPinnedMemoryResource,
     MemoryResource,
 )

cuda_core/cuda/core/experimental/_device.py

@@ -1160,6 +1160,9 @@ def __int__(self):
     def __repr__(self):
         return f"<Device {self._id} ({self.name})>"
 
+    def __reduce__(self):
+        return Device, (self.device_id,)
+
     def set_current(self, ctx: Context = None) -> Union[Context, None]:
         """Set device to be used for GPU executions.