Backends

Backend Manager

The Backend Manager is a singleton class that manages backend selection and initialization. It is designed to allow only one backend to be active at a time. Users can select backends at runtime, during the initialization phase. By default, the library will try to initialize the OpenCL backend. If initialization fails, it will attempt to initialize the CUDA backend. If both fail, the library will throw an exception.

Warning

Switching between backends during runtime is not supported and may lead to undefined behavior.

class BackendManager

BackendManager class This class is a singleton that holds the current backend and provides a way to change it.

Public Functions

auto setBackend(const std::string &backend = "opencl") -> void

Set the backend.

Parameters:

backend – The backend to be set, default is “opencl”

auto getBackend() const -> const Backend&

Get the backend.

Returns:

const Backend&

~BackendManager() = default

Destroy the BackendManager object.

BackendManager(const BackendManager&) = delete

Copy constructor (delete)

auto operator=(const BackendManager&) -> BackendManager& = delete

Copy assignment operator (delete)

Public Static Functions

static auto getInstance() -> BackendManager&

Get the singleton instance of the BackendManager.

static auto getBackendsList() -> std::vector<std::string>

Get the list of available backends.

static auto cudaEnabled() -> bool

Check if CUDA is enabled.

Returns:

bool True if CUDA is enabled, False otherwise

static auto openCLEnabled() -> bool

Check if OpenCL is enabled.

Returns:

bool True if OpenCL is enabled, False otherwise

Friends

inline friend auto operator<<(std::ostream &out, const BackendManager &backend_manager) -> std::ostream&

Operator << to print the backend manager.

Backend Class

The Backend class is an abstract class defining the interface for the different hardware backends supported by CLIc. Inherited classes implement the necessary low-level functions to operate the hardware. This abstraction enforces a design pattern allowing switching between different hardware backends without changing the high-level code. Currently, the library supports OpenCL and CUDA backends, with the possibility of adding more in the future.

Note

This class operates closest to the hardware, and casual developers should not need to interact with it.

Warning

The CUDABackend class is operational but not yet fully released.

class Backend

Backend class.

This class holds low-level device operations. It is used to allocate memory, copy data, execute kernels, etc. It is standardized to operate with different hardware API (CUDA, OpenCL, etc).

Subclassed by cle::CUDABackend, cle::OpenCLBackend

Public Functions

Backend() = default

Construct a new Backend object.

Backend(const Backend&) = default

Construct by copy a new Backend object.

Backend(Backend&&) = default

Construct by move a new Backend object.

virtual ~Backend() = default

Destroy the Backend object.

auto operator=(const Backend&) -> Backend& = default

Copy assignment operator.

auto operator=(Backend&&) -> Backend& = default

Move assignment operator.

virtual auto getType() const -> Backend::Type = 0

Get the type of the backend.

Returns:

Backend::Type

virtual auto getDevicesList(const std::string &type) const -> std::vector<std::string> = 0

Get the list of devices names of a specific type.

Parameters:

type –

Returns:

std::vector<std::string>

virtual auto getDevices(const std::string &type) const -> std::vector<Device::Pointer> = 0

Get the list of devices of a specific type.

Parameters:

type –

Returns:

std::vector<Device::Pointer>

virtual auto getDevice(const std::string &name, const std::string &type) const -> Device::Pointer = 0

Get a device by name and type.

Parameters:

  • name –

  • type –

Returns:

Device::Pointer

virtual auto getDeviceFromIndex(size_t index, const std::string &type) const -> Device::Pointer = 0

Get a device by index and type.

Parameters:

  • index –

  • type –

Returns:

Device::Pointer

virtual auto getPreamble() const -> std::string = 0

Get the preamble of the backend The preamble is a string that contains the necessary code to be included in the kernel source code before compiling it.

Returns:

std::string

virtual auto getRefCount(void *data_ptr) const -> int = 0

Get number of reference count.

Returns:

Cache::Pointer

virtual auto allocateMemory(const Device::Pointer &device, const std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, std::shared_ptr<void> &data_ptr) const -> void = 0

Allocate a memory space in the device.

Parameters:

  • device –

  • size –

  • data_ptr –

virtual auto freeMemory(const Device::Pointer &device, const mType &mtype, const std::shared_ptr<void> &data_ptr) const -> void = 0

Free a memory space in the device.

Parameters:

  • device –

  • mtype –

  • data_ptr –

virtual auto writeMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, const void *host_ptr) const -> void = 0

Write data from host to device buffer.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • host_ptr –

virtual auto readMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, void *host_ptr) const -> void = 0

Read data from device buffer to host.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • host_ptr –

virtual auto copyMemoryBufferToBuffer(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void = 0

Copy data from device buffer to device buffer.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryImageToBuffer(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void = 0

Copy data from device image to device buffer.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryBufferToImage(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void = 0

Copy data from device buffer to device image.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryImageToImage(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void = 0

Copy data from device image to device image.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto setMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, const float &value) const -> void = 0

Set a memory space to a specific value.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • dtype –

  • mtype –

  • value –

virtual auto buildKernel(const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, std::shared_ptr<void> &kernel) const -> void = 0

Build a kernel from source code.

Parameters:

  • device –

  • kernel_source –

  • kernel_name –

  • kernel –

virtual auto executeKernel(const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, const std::array<size_t, 3> &global_size, const std::vector<std::shared_ptr<void>> &args, const std::vector<size_t> &sizes) const -> void = 0

Execute a kernel.

Parameters:

  • device –

  • kernel_source –

  • kernel_name –

  • global_size –

  • args –

  • sizes –

Friends

inline friend auto operator<<(std::ostream &out, const Backend::Type &backend_type) -> std::ostream&

Operator << to print the backend type.

inline friend auto operator<<(std::ostream &out, const Backend &backend) -> std::ostream&

Operator << to print the backend.

OpenCL Backend

class OpenCLBackend : public cle::Backend

OpenCL backend class This class holds low-level device operations for OpenCL devices.

Public Functions

virtual auto getDevices(const std::string &type) const -> std::vector<Device::Pointer> override

Get the list of devices of a specific type.

Parameters:

type –

Returns:

std::vector<Device::Pointer>

virtual auto getDevice(const std::string &name, const std::string &type) const -> Device::Pointer override

Get a device by name and type.

Parameters:

  • name –

  • type –

Returns:

Device::Pointer

virtual auto getDeviceFromIndex(size_t index, const std::string &type) const -> Device::Pointer override

Get a device by index and type.

Parameters:

  • index –

  • type –

Returns:

Device::Pointer

virtual auto getDevicesList(const std::string &type) const -> std::vector<std::string> override

Get the list of devices names of a specific type.

Parameters:

type –

Returns:

std::vector<std::string>

virtual auto getType() const -> Backend::Type override

Get the type of the backend.

Returns:

Backend::Type

virtual auto allocateMemory(const Device::Pointer &device, const std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, std::shared_ptr<void> &data_ptr) const -> void override

Allocate a memory space in the device.

Parameters:

  • device –

  • size –

  • data_ptr –

virtual auto freeMemory(const Device::Pointer &device, const mType &mtype, const std::shared_ptr<void> &data_ptr) const -> void override

Free a memory space in the device.

Parameters:

  • device –

  • mtype –

  • data_ptr –

virtual auto writeMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, const void *host_ptr) const -> void override

Write data from host to device buffer.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • host_ptr –

virtual auto readMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, void *host_ptr) const -> void override

Read data from device buffer to host.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • host_ptr –

virtual auto copyMemoryBufferToBuffer(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device buffer to device buffer.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryImageToBuffer(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device image to device buffer.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryBufferToImage(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device buffer to device image.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryImageToImage(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device image to device image.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto setMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, const float &value) const -> void override

Set a memory space to a specific value.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • dtype –

  • mtype –

  • value –

virtual auto buildKernel(const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, std::shared_ptr<void> &kernel) const -> void override

Build a kernel from source code.

Parameters:

  • device –

  • kernel_source –

  • kernel_name –

  • kernel –

virtual auto executeKernel(const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, const std::array<size_t, 3> &global_size, const std::vector<std::shared_ptr<void>> &args, const std::vector<size_t> &sizes) const -> void override

Execute a kernel.

Parameters:

  • device –

  • kernel_source –

  • kernel_name –

  • global_size –

  • args –

  • sizes –

virtual auto getPreamble() const -> std::string override

Get the preamble of the backend The preamble is a string that contains the necessary code to be included in the kernel source code before compiling it.

Returns:

std::string

virtual auto getRefCount(void *data_ptr) const -> int override

Get number of reference count.

Returns:

Cache::Pointer

CUDA Backend

class CUDABackend : public cle::Backend

CUDA backend class This class holds low-level device operations for CUDA devices.

Public Functions

virtual auto getDevices(const std::string &type) const -> std::vector<Device::Pointer> override

Get the list of devices of a specific type.

Parameters:

type –

Returns:

std::vector<Device::Pointer>

virtual auto getDevice(const std::string &name, const std::string &type) const -> Device::Pointer override

Get a device by name and type.

Parameters:

  • name –

  • type –

Returns:

Device::Pointer

virtual auto getDeviceFromIndex(size_t index, const std::string &type) const -> Device::Pointer override

Get a device by index and type.

Parameters:

  • index –

  • type –

Returns:

Device::Pointer

virtual auto getDevicesList(const std::string &type) const -> std::vector<std::string> override

Get the list of devices names of a specific type.

Parameters:

type –

Returns:

std::vector<std::string>

virtual auto getType() const -> Backend::Type override

Get the type of the backend.

Returns:

Backend::Type

virtual auto allocateMemory(const Device::Pointer &device, const std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, std::shared_ptr<void> &data_ptr) const -> void override

Allocate a memory space in the device.

Parameters:

  • device –

  • size –

  • data_ptr –

virtual auto freeMemory(const Device::Pointer &device, const mType &mtype, const std::shared_ptr<void> &data_ptr) const -> void override

Free a memory space in the device.

Parameters:

  • device –

  • mtype –

  • data_ptr –

virtual auto writeMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, const void *host_ptr) const -> void override

Write data from host to device buffer.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • host_ptr –

virtual auto readMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, void *host_ptr) const -> void override

Read data from device buffer to host.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • host_ptr –

virtual auto copyMemoryBufferToBuffer(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device buffer to device buffer.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryImageToBuffer(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device image to device buffer.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryBufferToImage(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device buffer to device image.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto copyMemoryImageToImage(const Device::Pointer &device, const std::shared_ptr<void> &src_ptr, std::array<size_t, 3> &src_origin, std::array<size_t, 3> &src_shape, const std::shared_ptr<void> &dst_ptr, std::array<size_t, 3> &dst_origin, std::array<size_t, 3> &dst_shape, std::array<size_t, 3> &region, const size_t &bytes) const -> void override

Copy data from device image to device image.

Parameters:

  • device –

  • src_ptr –

  • src_origin –

  • src_shape –

  • dst_ptr –

  • dst_origin –

  • dst_shape –

  • region –

  • bytes –

virtual auto setMemory(const Device::Pointer &device, const std::shared_ptr<void> &buffer_ptr, std::array<size_t, 3> &buffer_shape, std::array<size_t, 3> &buffer_origin, std::array<size_t, 3> &region, const dType &dtype, const mType &mtype, const float &value) const -> void override

Set a memory space to a specific value.

Parameters:

  • device –

  • buffer_ptr –

  • buffer_shape –

  • buffer_origin –

  • region –

  • dtype –

  • mtype –

  • value –

virtual auto buildKernel(const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, std::shared_ptr<void> &kernel) const -> void override

Build a kernel from source code.

Parameters:

  • device –

  • kernel_source –

  • kernel_name –

  • kernel –

virtual auto executeKernel(const Device::Pointer &device, const std::string &kernel_source, const std::string &kernel_name, const std::array<size_t, 3> &global_size, const std::vector<std::shared_ptr<void>> &args, const std::vector<size_t> &sizes) const -> void override

Execute a kernel.

Parameters:

  • device –

  • kernel_source –

  • kernel_name –

  • global_size –

  • args –

  • sizes –

virtual auto getPreamble() const -> std::string override

Get the preamble of the backend The preamble is a string that contains the necessary code to be included in the kernel source code before compiling it.

Returns:

std::string

virtual auto getRefCount(void *data_ptr) const -> int override

Get number of reference count.

Returns:

Cache::Pointer