fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2025-12-20 03:00:11 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

delete clover

Browse source 
there comes a time when a project has to be declared unfit to remain
in the tree

this frontend hasn't seen actual development in about 6 years

if someone has a pressing need to continue development, there's no
blocker to un-deleting it, but unless that happens, there's now a
more featureful, more conformant, more active CL frontend in the tree

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19385>
This commit is contained in:
Mike Blumenkrantz 2022-10-28 14:10:57 -04:00
parent 555821ff93
commit 5844dea316
83 changed files with 6 additions and 15266 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
.gitlab-ci/build/gitlab-ci.yml
View file

@ -138,7 +138,6 @@ debian-testing-asan:
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-rusticl=false
-D gallium-nine=false
-D gallium-drivers=
@ -177,7 +176,6 @@ debian-testing-msan:
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=panfrost
-D vulkan-drivers=
-D video-codecs=
@ -213,7 +211,6 @@ debian-testing-ubsan:
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-rusticl=false
-D gallium-drivers=
-D vulkan-drivers=
@ -419,7 +416,6 @@ debian-android:
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=panfrost
-D vulkan-drivers=
-D video-codecs=
@ -498,7 +494,6 @@ debian-arm32:
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=panfrost
-D vulkan-drivers=
-D video-codecs=
@ -791,7 +786,6 @@ debian-x86_32:
HOST_BUILD_OPTIONS: >
-D build-tests=false
-D enable-glcpp-tests=false
-D gallium-opencl=disabled
-D gallium-drivers=
-D vulkan-drivers=
-D video-codecs=

1
.gitlab-ci/meson/build.sh
View file

@ -177,7 +177,6 @@ meson setup _build \
-D libunwind=${UNWIND} \
${DRI_LOADERS} \
${GALLIUM_ST} \
-D gallium-opencl=disabled \
-D gallium-drivers=${GALLIUM_DRIVERS:-[]} \
-D vulkan-drivers=${VULKAN_DRIVERS:-[]} \
-D video-codecs=all \

1
.gitlab-ci/windows/mesa_build.ps1
View file

@ -53,7 +53,6 @@ meson setup `
-Dvideo-codecs="all" `
-Dgles1=enabled `
-Dgles2=enabled `
-Dgallium-opencl=icd `
-Dgallium-rusticl=false `
-Dmicrosoft-clc=enabled `
-Dstatic-libclc=all `

48
docs/envvars.rst
View file

@ -1135,54 +1135,6 @@ Gallium environment variables
``sse4.1``
``avx``
Clover environment variables
----------------------------
.. envvar:: CLOVER_DEVICE_TYPE
allows to overwrite the device type of devices. Possible values are
``accelerator``, ``cpu``, ``custom`` and ``gpu``
.. envvar:: CLOVER_DEVICE_VERSION_OVERRIDE
overwrites the auto detected OpenCL version of a device. Possible values:
``1.0``
``1.1``
``1.2``
``2.0``
``2.1``
``2.2``
``3.0``
.. envvar:: CLOVER_DEVICE_CLC_VERSION_OVERRIDE
overwrites the auto detected CLC version. Possible values:
``1.0``
``1.1``
``1.2``
``2.0``
``2.1``
``2.2``
``3.0``
.. envvar:: CLOVER_EXTRA_BUILD_OPTIONS
allows specifying additional compiler and linker options. Specified
options are appended after the options set by the OpenCL program in
``clBuildProgram``.
.. envvar:: CLOVER_EXTRA_COMPILE_OPTIONS
allows specifying additional compiler options. Specified options are
appended after the options set by the OpenCL program in
``clCompileProgram``.
.. envvar:: CLOVER_EXTRA_LINK_OPTIONS
allows specifying additional linker options. Specified options are
appended after the options set by the OpenCL program in
``clLinkProgram``.
.. _rusticl-env-var:
Rusticl environment variables

154
docs/features.txt
View file

@ -697,160 +697,6 @@ Khronos extensions that are not part of any Vulkan version:
VK_QCOM_fragment_density_map_offset DONE (tu)
Clover OpenCL 1.0 -- all DONE:
Image support in progress
- Optional image formats in progress
Clover OpenCL 1.1 -- all DONE:
Additional queries for clGetDeviceInfo DONE (nvc0, r600, radeonsi)
CL_CONTEXT_NUM_DEVICES for clGetContextInfo DONE
New optional image formats not started
- CL_Rx not started
- CL_RGx not started
- CL_RGBx not started
clCreateSubBuffer DONE
Read from, write to, copy rectangular regions DONE
clSetMemObjectDestructorCallback DONE
Control OpenCL C version when building DONE
Query for preferred work-group size multiple DONE (nvc0, r600, radeonsi)
Support user events DONE
clSetEventCallback DONE
Minimum requirement changes for clGetDeviceInfo DONE (nvc0, r600, radeonsi)
Arg prerequisite change for clEnqueueNDRangeKernel DONE ()
OpenCL C 1.1 DONE (nvc0, r600, radeonsi)
- 3-component vector data types DONE (nvc0, r600, radeonsi)
- cl_khr_byte_addressable_store DONE (nvc0, r600, radeonsi)
- cl_khr_global_int32_base_atomics DONE (nvc0, r600, radeonsi)
- cl_khr_global_int32_extended_atomics DONE (nvc0, r600, radeonsi)
- cl_khr_local_int32_base_atomics DONE (nvc0, r600, radeonsi)
- cl_khr_local_int32_extended_atomics DONE (nvc0, r600, radeonsi)
Clover OpenCL 1.2 -- all DONE:
Custom devices DONE
Built-in kernels in progress
Device partitioning not started
Separate compilation and linking of programs DONE
Extend cl_mem_flags DONE
clEnqueueFillBuffer, clEnqueueFillImage DONE
Add CL_MAP_WRITE_INVALIDATE_REGION to cl_map_flags DONE
New image types not started
clCreateImage DONE
clEnqueueMigrateMemObjects DONE
Retrieve kernels information from a program DONE
clGetKernelArgInfo DONE
clEnqueueMarkerWithWaitList DONE
clEnqueueBarrierWithWaitList DONE
clUnloadPlatformCompiler DONE
cl_khr_fp64 DONE (nvc0, r600, radeonsi)
printf DONE (nvc0)
CL_KERNEL_ATTRIBUTES for clGetKernelInfo DONE
OpenCL C 1.2 DONE
Clover OpenCL 2.0 -- all DONE:
Shared virtual memory DONE (nvc0, llvmpipe)
Device queues not started
- cl_khr_create_command_queue not started
- Additional queries for clGetDeviceInfo not started
Pipes not started
Extended 2D images creation in progress
- CL_ABGR DONE
- cl_khr_image2d_from_buffer not started
- cl_khr_depth_images not started
- from sRGB images not started
clCreateSamplerWithProperties not started
Non-uniform work-group sizes not started
cl_khr_3d_image_writes not started
OpenCL C 2.0 in progress
- Work-group Collective Functions not started
- Generic address space in progress
Clover OpenCL 2.1 -- all DONE:
Sub groups not started
- cl_khr_subgroups not started
cl_khr_il_program DONE (nvc0)
Device and host timer synchronization not started
clEnqueueSVMMigrateMem not started
clCloneKernel not started
Default device command queue not started
CL_UNORM_INT_101010_2 DONE
Clover OpenCL 2.2 -- all DONE:
clSetProgramSpecializationConstant not started
clSetProgramReleaseCallback not started
Initialization and clean-up kernels not started
CL_MAX_SIZE_RESTRICTION_EXCEEDED for clSetKernelArg not started
Support SPIR-V 1.1 and 1.2 not started
Clover OpenCL 3.0 -- all DONE:
Optional device capabilities queries in progress
cl_khr_extended_versioning DONE
clSetContextDestructorCallback DONE
clCreateBufferWithProperties DONE
clCreateImageWithProperties DONE
Query properties arrays in progress
Supported OpenCL C versions and features queries DONE
CL_COMMAND_SVM_MIGRATE_MEM for clGetEventInfo not started
OpenCL C 3.0 DONE
Latest conformance version passed for devices not started
Clover extensions that are not part of any OpenCL version:
cl_khr_async_copy_fence not started
cl_khr_async_work_group_copy_fence not started
cl_khr_device_enqueue_local_arg_types not started
cl_khr_device_uuid not started
cl_khr_egl_event not started
cl_khr_egl_image not started
cl_khr_expect_assume not started
cl_khr_extended_async_copies not started
cl_khr_extended_bit_ops not started
cl_khr_fp16 DONE ()
cl_khr_gl_depth_images not started
cl_khr_gl_msaa_sharing not started
cl_khr_gl_sharing not started
cl_khr_icd DONE
cl_khr_initialize_memory not started
cl_khr_int64_base_atomics DONE ()
cl_khr_int64_extended_atomics DONE ()
cl_khr_integer_dot_product not started
cl_khr_mipmap_image not started
cl_khr_pci_bus_info not started
cl_khr_priority_hints not started
cl_khr_spirv_extended_debug_info not started
cl_khr_spirv_linkonce_odr not started
cl_khr_spirv_no_integer_wrap_decoration not started
cl_khr_srgb_image_writes not started
cl_khr_subgroup_ballot not started
cl_khr_subgroup_clustered_reduce not started
cl_khr_subgroup_extended_types not started
cl_khr_subgroup_named_barrier not started
cl_khr_subgroup_non_uniform_arithmetic not started
cl_khr_subgroup_non_uniform_vote not started
cl_khr_subgroup_rotate not started
cl_khr_subgroup_shuffle not started
cl_khr_subgroup_shuffle_relative not started
cl_khr_suggested_local_work_size not started
cl_khr_terminate_context not started
cl_khr_throttle_hints not started
cl_khr_work_group_uniform_arithmetic not started
cl_arm_non_uniform_work_group_size not started
cl_arm_shared_virtual_memory DONE (nvc0)
cl_intel_unified_shared_memory not started
Rusticl OpenCL 1.0 -- all DONE:
Image support DONE

5
docs/gallium/distro.rst
View file

@ -76,11 +76,6 @@ Wrapper driver. Trace dumps an XML record of the calls made to the
Gallium frontends
-----------------
Clover
^^^^^^
Tracker that implements the Khronos OpenCL standard.
.. _dri:
Direct Rendering Infrastructure

1
docs/relnotes/new_features.txt
View file

@ -0,0 +1 @@
removed clover frontend

1
docs/sourcetree.rst
View file

@ -132,7 +132,6 @@ each directory.
- **frontends** - These implement various libraries using the
device drivers
- **clover** - OpenCL frontend
- **d3d10umd** - D3D10 frontend for Windows only. It's similar to Microsoft WARP, but using LLVMpipe/Softpipe.
- **dri** - Meta frontend for DRI drivers, see mesa/state_tracker
- **glx** - Meta frontend for GLX

10
include/meson.build
View file

@ -89,7 +89,7 @@ if with_gallium_st_nine
subdir : 'd3dadapter',
)
endif
opencl_headers = files(
'CL/cl.h',
'CL/cl.hpp',
@ -113,11 +113,3 @@ opencl_headers = files(
'CL/opencl.h',
'CL/opencl.hpp',
)
# Only install the headers if we are building a stand alone implementation and
# not an ICD enabled implementation
if with_gallium_clover and not with_opencl_icd
install_headers(
opencl_headers,
subdir: 'CL'
)
endif

41
meson.build
View file

@ -743,24 +743,7 @@ if get_option('vmware-mks-stats')
pre_args += '-DVMX86_STATS=1'
endif
_opencl = get_option('gallium-opencl')
_rtti = get_option('cpp_rtti')
if _opencl != 'disabled'
warning('Clover will be removed in Mesa 25.2')
if not with_gallium
error('OpenCL Clover implementation requires at least one gallium driver.')
endif
if not _rtti
error('The Clover OpenCL state tracker requires rtti')
endif
with_gallium_clover = true
with_opencl_icd = _opencl == 'icd'
else
with_gallium_clover = false
with_opencl_icd = false
endif
with_gallium_rusticl = get_option('gallium-rusticl')
if with_gallium_rusticl
@ -829,7 +812,7 @@ else
endif
dep_clc = null_dep
if with_gallium_clover or with_clc
if with_clc
dep_clc = dependency('libclc')
endif
@ -1716,15 +1699,6 @@ if with_amd_vk or with_gallium_radeonsi or with_gallium_r600
llvm_modules += 'asmparser'
endif
endif
if with_gallium_clover
llvm_modules += [
'linker', 'coverage', 'instrumentation', 'ipo', 'irreader',
'lto', 'option', 'objcarcopts', 'profiledata'
]
# all-targets is needed to support static linking LLVM build with multiple targets
# windowsdriver is needded with LLVM>=15, but we don't know what LLVM verrsion we are using yet
llvm_optional_modules += ['all-targets', 'frontendopenmp', 'windowsdriver']
endif
if with_clc
llvm_modules += ['coverage', 'target', 'linker', 'irreader', 'option', 'libdriver', 'lto']
# all-targets is needed to support static linking LLVM build with multiple targets.
@ -1750,8 +1724,6 @@ if with_amd_vk or with_gallium_radeonsi
_llvm_version = '>= 18.0.0'
elif with_clc or llvm_with_orcjit
_llvm_version = '>= 15.0.0'
elif with_gallium_clover
_llvm_version = '>= 11.0.0'
else
_llvm_version = '>= 5.0.0'
endif
@ -1771,7 +1743,7 @@ if _llvm.allowed()
modules : llvm_modules,
optional_modules : llvm_optional_modules,
required : (
with_amd_vk or with_gallium_radeonsi or with_gallium_clover or with_clc
with_amd_vk or with_gallium_radeonsi or with_clc
or _llvm.enabled()
),
static : not _shared_llvm,
@ -1825,8 +1797,6 @@ elif with_swrast_vk
error('lavapipe requires LLVM and is enabled, but LLVM is disabled.')
elif with_any_llvmpipe
error('llvmpipe requires LLVM and is enabled, but LLVM is disabled.')
elif with_gallium_clover
error('The OpenCL "Clover" state tracker requires LLVM, but LLVM is disabled.')
elif with_clc
error('The CLC compiler requires LLVM, but LLVM is disabled.')
else
@ -1876,7 +1846,7 @@ if dep_spirv_tools.found()
endif
dep_clang = null_dep
if with_clc or with_gallium_clover
if with_clc
llvm_libdir = dep_llvm.get_variable(cmake : 'LLVM_LIBRARY_DIR', configtool: 'libdir')
dep_clang = cpp.find_library('clang-cpp', dirs : llvm_libdir, required : false)
@ -1891,7 +1861,7 @@ if with_clc or with_gallium_clover
if dep_llvm.version().version_compare('>= 15.0')
clang_modules += 'clangSupport'
endif
if dep_llvm.version().version_compare('>= 16.0') or with_gallium_clover
if dep_llvm.version().version_compare('>= 16.0')
clang_modules += 'clangASTMatchers'
endif
if dep_llvm.version().version_compare('>= 18.0')
@ -2414,9 +2384,6 @@ if with_gallium
if with_gallium_st_nine
gallium_frontends += 'nine'
endif
if with_gallium_clover
gallium_frontends += 'clover'
endif
if with_gallium_rusticl
gallium_frontends += 'rusticl'
endif

9
meson.options
View file

@ -147,15 +147,6 @@ option(
description : 'build gallium D3D10 WDDM UMD frontend.',
)
option(
'gallium-opencl',
type : 'combo',
choices : ['icd', 'standalone', 'disabled'],
value : 'disabled',
description : 'build gallium "clover" OpenCL frontend.',
deprecated: true,
)
option(
'gallium-rusticl',
type : 'boolean',

7
src/gallium/drivers/r600/meson.build
View file

@ -144,13 +144,6 @@ egd_tables_h = custom_target(
)
r600_c_args = []
if with_gallium_clover
if dep_elf.found()
r600_c_args += '-DHAVE_OPENCL'
else
warning('r600 requires libelf to support opencl.')
endif
endif
r600_cpp_args = []
if cpp.has_type('std::pmr::monotonic_buffer_resource',

162
src/gallium/frontends/clover/api/context.cpp
View file

@ -1,162 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/context.hpp"
#include "core/platform.hpp"
using namespace clover;
CLOVER_API cl_context
clCreateContext(const cl_context_properties *d_props, cl_uint num_devs,
const cl_device_id *d_devs,
void (CL_CALLBACK *pfn_notify)(const char *, const void *,
size_t, void *),
void *user_data, cl_int *r_errcode) try {
auto props = obj<property_list_tag>(d_props);
auto devs = objs(d_devs, num_devs);
if (!pfn_notify && user_data)
throw error(CL_INVALID_VALUE);
for (auto &prop : props) {
if (prop.first == CL_CONTEXT_PLATFORM)
find_platform(prop.second.as<cl_platform_id>());
else
throw error(CL_INVALID_PROPERTY);
}
const auto notify = (!pfn_notify ? context::notify_action() :
[=](const char *s) {
pfn_notify(s, NULL, 0, user_data);
});
ret_error(r_errcode, CL_SUCCESS);
return desc(new context(props, devs, notify));
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_context
clCreateContextFromType(const cl_context_properties *d_props,
cl_device_type type,
void (CL_CALLBACK *pfn_notify)(
const char *, const void *, size_t, void *),
void *user_data, cl_int *r_errcode) try {
cl_platform_id d_platform;
cl_uint num_platforms;
cl_int ret;
std::vector<cl_device_id> devs;
cl_uint num_devices;
ret = clGetPlatformIDs(1, &d_platform, &num_platforms);
if (ret || !num_platforms)
throw error(CL_INVALID_PLATFORM);
ret = clGetDeviceIDs(d_platform, type, 0, NULL, &num_devices);
if (ret)
throw error(CL_DEVICE_NOT_FOUND);
devs.resize(num_devices);
ret = clGetDeviceIDs(d_platform, type, num_devices, devs.data(), 0);
if (ret)
throw error(CL_DEVICE_NOT_FOUND);
return clCreateContext(d_props, num_devices, devs.data(), pfn_notify,
user_data, r_errcode);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clRetainContext(cl_context d_ctx) try {
obj(d_ctx).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseContext(cl_context d_ctx) try {
if (obj(d_ctx).release())
delete pobj(d_ctx);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetContextInfo(cl_context d_ctx, cl_context_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &ctx = obj(d_ctx);
switch (param) {
case CL_CONTEXT_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = ctx.ref_count();
break;
case CL_CONTEXT_NUM_DEVICES:
buf.as_scalar<cl_uint>() = ctx.devices().size();
break;
case CL_CONTEXT_DEVICES:
buf.as_vector<cl_device_id>() = descs(ctx.devices());
break;
case CL_CONTEXT_PROPERTIES:
buf.as_vector<cl_context_properties>() = desc(ctx.properties());
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clSetContextDestructorCallback(cl_context d_ctx,
void (CL_CALLBACK *pfn_notify)(cl_context, void *),
void *user_data) try {
CLOVER_NOT_SUPPORTED_UNTIL("3.0");
auto &ctx = obj(d_ctx);
if (!pfn_notify)
return CL_INVALID_VALUE;
ctx.destroy_notify([=]{ pfn_notify(d_ctx, user_data); });
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}

525
src/gallium/frontends/clover/api/device.cpp
View file

@ -1,525 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/platform.hpp"
#include "core/device.hpp"
#include "git_sha1.h"
using namespace clover;
namespace {
std::string
supported_il_versions_as_string(const device &dev) {
std::string il_versions_string;
for (const auto &il_version : dev.supported_il_versions()) {
if (!il_versions_string.empty())
il_versions_string += " ";
il_versions_string += std::string(il_version.name) + "_" +
std::to_string(CL_VERSION_MAJOR(il_version.version)) + "." +
std::to_string(CL_VERSION_MINOR(il_version.version));
}
return il_versions_string;
}
}
CLOVER_API cl_int
clGetDeviceIDs(cl_platform_id d_platform, cl_device_type device_type,
cl_uint num_entries, cl_device_id *rd_devices,
cl_uint *rnum_devices) try {
auto &platform = obj(d_platform);
std::vector<cl_device_id> d_devs;
if ((!num_entries && rd_devices) ||
(!rnum_devices && !rd_devices))
throw error(CL_INVALID_VALUE);
// Collect matching devices
for (device &dev : platform) {
if (((device_type & CL_DEVICE_TYPE_DEFAULT) &&
dev == platform.front()) ||
(device_type & dev.type()))
d_devs.push_back(desc(dev));
}
if (d_devs.empty())
throw error(CL_DEVICE_NOT_FOUND);
// ...and return the requested data.
if (rnum_devices)
*rnum_devices = d_devs.size();
if (rd_devices)
copy(range(d_devs.begin(),
std::min((unsigned)d_devs.size(), num_entries)),
rd_devices);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clCreateSubDevices(cl_device_id d_dev,
const cl_device_partition_property *props,
cl_uint num_devs, cl_device_id *rd_devs,
cl_uint *rnum_devs) {
// There are no currently supported partitioning schemes.
return CL_INVALID_VALUE;
}
CLOVER_API cl_int
clRetainDevice(cl_device_id d_dev) try {
obj(d_dev);
// The reference count doesn't change for root devices.
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseDevice(cl_device_id d_dev) try {
obj(d_dev);
// The reference count doesn't change for root devices.
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetDeviceInfo(cl_device_id d_dev, cl_device_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &dev = obj(d_dev);
switch (param) {
case CL_DEVICE_TYPE:
buf.as_scalar<cl_device_type>() = dev.type();
break;
case CL_DEVICE_VENDOR_ID:
buf.as_scalar<cl_uint>() = dev.vendor_id();
break;
case CL_DEVICE_MAX_COMPUTE_UNITS:
buf.as_scalar<cl_uint>() = dev.max_compute_units();
break;
case CL_DEVICE_MAX_WORK_ITEM_DIMENSIONS:
buf.as_scalar<cl_uint>() = dev.max_block_size().size();
break;
case CL_DEVICE_MAX_WORK_ITEM_SIZES:
buf.as_vector<size_t>() = dev.max_block_size();
break;
case CL_DEVICE_MAX_WORK_GROUP_SIZE:
buf.as_scalar<size_t>() = dev.max_threads_per_block();
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_CHAR:
buf.as_scalar<cl_uint>() = 16;
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_SHORT:
buf.as_scalar<cl_uint>() = 8;
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_INT:
buf.as_scalar<cl_uint>() = 4;
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_LONG:
buf.as_scalar<cl_uint>() = 2;
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_FLOAT:
buf.as_scalar<cl_uint>() = 4;
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_DOUBLE:
buf.as_scalar<cl_uint>() = dev.has_doubles() ? 2 : 0;
break;
case CL_DEVICE_PREFERRED_VECTOR_WIDTH_HALF:
buf.as_scalar<cl_uint>() = dev.has_halves() ? 8 : 0;
break;
case CL_DEVICE_MAX_CLOCK_FREQUENCY:
buf.as_scalar<cl_uint>() = dev.max_clock_frequency();
break;
case CL_DEVICE_ADDRESS_BITS:
buf.as_scalar<cl_uint>() = dev.address_bits();
break;
case CL_DEVICE_MAX_READ_IMAGE_ARGS:
buf.as_scalar<cl_uint>() = dev.max_images_read();
break;
case CL_DEVICE_MAX_WRITE_IMAGE_ARGS:
buf.as_scalar<cl_uint>() = dev.max_images_write();
break;
case CL_DEVICE_MAX_MEM_ALLOC_SIZE:
buf.as_scalar<cl_ulong>() = dev.max_mem_alloc_size();
break;
case CL_DEVICE_IMAGE2D_MAX_WIDTH:
case CL_DEVICE_IMAGE2D_MAX_HEIGHT:
buf.as_scalar<size_t>() = dev.max_image_size();
break;
case CL_DEVICE_IMAGE3D_MAX_WIDTH:
case CL_DEVICE_IMAGE3D_MAX_HEIGHT:
case CL_DEVICE_IMAGE3D_MAX_DEPTH:
buf.as_scalar<size_t>() = dev.max_image_size_3d();
break;
case CL_DEVICE_IMAGE_MAX_BUFFER_SIZE:
buf.as_scalar<size_t>() = dev.max_image_buffer_size();
break;
case CL_DEVICE_IMAGE_MAX_ARRAY_SIZE:
buf.as_scalar<size_t>() = dev.max_image_array_number();
break;
case CL_DEVICE_IMAGE_SUPPORT:
buf.as_scalar<cl_bool>() = dev.image_support();
break;
case CL_DEVICE_MAX_PARAMETER_SIZE:
buf.as_scalar<size_t>() = dev.max_mem_input();
break;
case CL_DEVICE_MAX_SAMPLERS:
buf.as_scalar<cl_uint>() = dev.max_samplers();
break;
case CL_DEVICE_MEM_BASE_ADDR_ALIGN:
buf.as_scalar<cl_uint>() = 8 * dev.mem_base_addr_align();
break;
case CL_DEVICE_MIN_DATA_TYPE_ALIGN_SIZE:
buf.as_scalar<cl_uint>() = 128;
break;
case CL_DEVICE_HALF_FP_CONFIG:
// This is the "mandated minimum half precision floating-point
// capability" for OpenCL 1.x.
buf.as_scalar<cl_device_fp_config>() =
CL_FP_INF_NAN | CL_FP_ROUND_TO_NEAREST;
break;
case CL_DEVICE_SINGLE_FP_CONFIG:
// This is the "mandated minimum single precision floating-point
// capability" for OpenCL 1.1. In OpenCL 1.2, nothing is required for
// custom devices.
buf.as_scalar<cl_device_fp_config>() =
CL_FP_INF_NAN | CL_FP_ROUND_TO_NEAREST;
break;
case CL_DEVICE_DOUBLE_FP_CONFIG:
if (dev.has_doubles())
// This is the "mandated minimum double precision floating-point
// capability"
buf.as_scalar<cl_device_fp_config>() =
CL_FP_FMA
| CL_FP_ROUND_TO_NEAREST
| CL_FP_ROUND_TO_ZERO
| CL_FP_ROUND_TO_INF
| CL_FP_INF_NAN
| CL_FP_DENORM;
else
buf.as_scalar<cl_device_fp_config>() = 0;
break;
case CL_DEVICE_GLOBAL_MEM_CACHE_TYPE:
buf.as_scalar<cl_device_mem_cache_type>() = CL_NONE;
break;
case CL_DEVICE_GLOBAL_MEM_CACHELINE_SIZE:
buf.as_scalar<cl_uint>() = 0;
break;
case CL_DEVICE_GLOBAL_MEM_CACHE_SIZE:
buf.as_scalar<cl_ulong>() = 0;
break;
case CL_DEVICE_GLOBAL_MEM_SIZE:
buf.as_scalar<cl_ulong>() = dev.max_mem_global();
break;
case CL_DEVICE_MAX_CONSTANT_BUFFER_SIZE:
buf.as_scalar<cl_ulong>() = dev.max_const_buffer_size();
break;
case CL_DEVICE_MAX_CONSTANT_ARGS:
buf.as_scalar<cl_uint>() = dev.max_const_buffers();
break;
case CL_DEVICE_LOCAL_MEM_TYPE:
buf.as_scalar<cl_device_local_mem_type>() = CL_LOCAL;
break;
case CL_DEVICE_LOCAL_MEM_SIZE:
buf.as_scalar<cl_ulong>() = dev.max_mem_local();
break;
case CL_DEVICE_ERROR_CORRECTION_SUPPORT:
buf.as_scalar<cl_bool>() = CL_FALSE;
break;
case CL_DEVICE_PROFILING_TIMER_RESOLUTION:
buf.as_scalar<size_t>() = 0;
break;
case CL_DEVICE_ENDIAN_LITTLE:
buf.as_scalar<cl_bool>() = (dev.endianness() == PIPE_ENDIAN_LITTLE);
break;
case CL_DEVICE_AVAILABLE:
case CL_DEVICE_COMPILER_AVAILABLE:
case CL_DEVICE_LINKER_AVAILABLE:
buf.as_scalar<cl_bool>() = CL_TRUE;
break;
case CL_DEVICE_EXECUTION_CAPABILITIES:
buf.as_scalar<cl_device_exec_capabilities>() = CL_EXEC_KERNEL;
break;
case CL_DEVICE_QUEUE_PROPERTIES:
buf.as_scalar<cl_command_queue_properties>() = CL_QUEUE_PROFILING_ENABLE;
break;
case CL_DEVICE_BUILT_IN_KERNELS:
buf.as_string() = "";
break;
case CL_DEVICE_NAME:
buf.as_string() = dev.device_name();
break;
case CL_DEVICE_VENDOR:
buf.as_string() = dev.vendor_name();
break;
case CL_DRIVER_VERSION:
buf.as_string() = PACKAGE_VERSION;
break;
case CL_DEVICE_PROFILE:
buf.as_string() = "FULL_PROFILE";
break;
case CL_DEVICE_VERSION:
buf.as_string() = "OpenCL " + dev.device_version_as_string() + " Mesa " PACKAGE_VERSION MESA_GIT_SHA1;
break;
case CL_DEVICE_EXTENSIONS:
buf.as_string() = dev.supported_extensions_as_string();
break;
case CL_DEVICE_PLATFORM:
buf.as_scalar<cl_platform_id>() = desc(dev.platform);
break;
case CL_DEVICE_HOST_UNIFIED_MEMORY:
buf.as_scalar<cl_bool>() = dev.has_unified_memory();
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_CHAR:
buf.as_scalar<cl_uint>() = 16;
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_SHORT:
buf.as_scalar<cl_uint>() = 8;
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_INT:
buf.as_scalar<cl_uint>() = 4;
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_LONG:
buf.as_scalar<cl_uint>() = 2;
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_FLOAT:
buf.as_scalar<cl_uint>() = 4;
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_DOUBLE:
buf.as_scalar<cl_uint>() = dev.has_doubles() ? 2 : 0;
break;
case CL_DEVICE_NATIVE_VECTOR_WIDTH_HALF:
buf.as_scalar<cl_uint>() = dev.has_halves() ? 8 : 0;
break;
case CL_DEVICE_OPENCL_C_VERSION:
buf.as_string() = "OpenCL C " + dev.device_clc_version_as_string() + " ";
break;
case CL_DEVICE_PRINTF_BUFFER_SIZE:
buf.as_scalar<size_t>() = dev.max_printf_buffer_size();
break;
case CL_DEVICE_PREFERRED_INTEROP_USER_SYNC:
buf.as_scalar<cl_bool>() = CL_TRUE;
break;
case CL_DEVICE_PARENT_DEVICE:
buf.as_scalar<cl_device_id>() = NULL;
break;
case CL_DEVICE_PARTITION_MAX_SUB_DEVICES:
buf.as_scalar<cl_uint>() = 0;
break;
case CL_DEVICE_PARTITION_PROPERTIES:
buf.as_vector<cl_device_partition_property>() =
desc(property_list<cl_device_partition_property>());
break;
case CL_DEVICE_PARTITION_AFFINITY_DOMAIN:
buf.as_scalar<cl_device_affinity_domain>() = 0;
break;
case CL_DEVICE_PARTITION_TYPE:
buf.as_vector<cl_device_partition_property>() =
desc(property_list<cl_device_partition_property>());
break;
case CL_DEVICE_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = 1;
break;
case CL_DEVICE_SVM_CAPABILITIES:
case CL_DEVICE_SVM_CAPABILITIES_ARM:
buf.as_scalar<cl_device_svm_capabilities>() = dev.svm_support();
break;
case CL_DEVICE_NUMERIC_VERSION:
buf.as_scalar<cl_version>() = dev.device_version();
break;
case CL_DEVICE_OPENCL_C_NUMERIC_VERSION_KHR:
buf.as_scalar<cl_version>() = dev.device_clc_version(true);
break;
case CL_DEVICE_OPENCL_C_ALL_VERSIONS:
buf.as_vector<cl_name_version>() = dev.opencl_c_all_versions();
break;
case CL_DEVICE_EXTENSIONS_WITH_VERSION:
buf.as_vector<cl_name_version>() = dev.supported_extensions();
break;
case CL_DEVICE_OPENCL_C_FEATURES:
buf.as_vector<cl_name_version>() = dev.opencl_c_features();
break;
case CL_DEVICE_IL_VERSION:
if (dev.supported_extensions_as_string().find("cl_khr_il_program") == std::string::npos)
throw error(CL_INVALID_VALUE);
buf.as_string() = supported_il_versions_as_string(dev);
break;
case CL_DEVICE_ILS_WITH_VERSION:
buf.as_vector<cl_name_version>() = dev.supported_il_versions();
break;
case CL_DEVICE_BUILT_IN_KERNELS_WITH_VERSION:
buf.as_vector<cl_name_version>() = std::vector<cl_name_version>{};
break;
case CL_DEVICE_MAX_READ_WRITE_IMAGE_ARGS:
case CL_DEVICE_IMAGE_PITCH_ALIGNMENT:
case CL_DEVICE_IMAGE_BASE_ADDRESS_ALIGNMENT:
case CL_DEVICE_PREFERRED_PLATFORM_ATOMIC_ALIGNMENT:
case CL_DEVICE_PREFERRED_GLOBAL_ATOMIC_ALIGNMENT:
case CL_DEVICE_PREFERRED_LOCAL_ATOMIC_ALIGNMENT:
case CL_DEVICE_MAX_NUM_SUB_GROUPS:
case CL_DEVICE_QUEUE_ON_DEVICE_PREFERRED_SIZE:
case CL_DEVICE_QUEUE_ON_DEVICE_MAX_SIZE:
case CL_DEVICE_MAX_ON_DEVICE_QUEUES:
case CL_DEVICE_MAX_ON_DEVICE_EVENTS:
case CL_DEVICE_MAX_PIPE_ARGS:
case CL_DEVICE_PIPE_MAX_ACTIVE_RESERVATIONS:
case CL_DEVICE_PIPE_MAX_PACKET_SIZE:
buf.as_scalar<cl_uint>() = 0;
break;
case CL_DEVICE_MAX_GLOBAL_VARIABLE_SIZE:
case CL_DEVICE_GLOBAL_VARIABLE_PREFERRED_TOTAL_SIZE:
buf.as_scalar<size_t>() = 0;
break;
case CL_DEVICE_SUB_GROUP_INDEPENDENT_FORWARD_PROGRESS:
case CL_DEVICE_NON_UNIFORM_WORK_GROUP_SUPPORT:
case CL_DEVICE_WORK_GROUP_COLLECTIVE_FUNCTIONS_SUPPORT:
case CL_DEVICE_GENERIC_ADDRESS_SPACE_SUPPORT:
case CL_DEVICE_PIPE_SUPPORT:
buf.as_scalar<cl_bool>() = CL_FALSE;
break;
case CL_DEVICE_QUEUE_ON_DEVICE_PROPERTIES:
buf.as_scalar<cl_command_queue_properties>() = 0;
break;
case CL_DEVICE_ATOMIC_MEMORY_CAPABILITIES:
buf.as_scalar<cl_device_atomic_capabilities>() = (CL_DEVICE_ATOMIC_ORDER_RELAXED |
CL_DEVICE_ATOMIC_SCOPE_WORK_GROUP);
break;
case CL_DEVICE_ATOMIC_FENCE_CAPABILITIES:
buf.as_scalar<cl_device_atomic_capabilities>() = (CL_DEVICE_ATOMIC_ORDER_RELAXED |
CL_DEVICE_ATOMIC_ORDER_ACQ_REL |
CL_DEVICE_ATOMIC_SCOPE_WORK_GROUP);
break;
case CL_DEVICE_DEVICE_ENQUEUE_CAPABILITIES:
buf.as_scalar<cl_device_device_enqueue_capabilities>() = 0;
break;
case CL_DEVICE_PREFERRED_WORK_GROUP_SIZE_MULTIPLE:
buf.as_scalar<size_t>() = 1;
break;
case CL_DEVICE_LATEST_CONFORMANCE_VERSION_PASSED:
buf.as_string() = "";
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}

204
src/gallium/frontends/clover/api/dispatch.cpp
View file

@ -1,204 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/dispatch.hpp"
namespace clover {
const cl_icd_dispatch _dispatch = {
// OpenCL 1.0
clGetPlatformIDs,
GetPlatformInfo,
clGetDeviceIDs,
clGetDeviceInfo,
clCreateContext,
clCreateContextFromType,
clRetainContext,
clReleaseContext,
clGetContextInfo,
clCreateCommandQueue,
clRetainCommandQueue,
clReleaseCommandQueue,
clGetCommandQueueInfo,
NULL, // clSetCommandQueueProperty
clCreateBuffer,
clCreateImage2D,
clCreateImage3D,
clRetainMemObject,
clReleaseMemObject,
clGetSupportedImageFormats,
clGetMemObjectInfo,
clGetImageInfo,
clCreateSampler,
clRetainSampler,
clReleaseSampler,
clGetSamplerInfo,
clCreateProgramWithSource,
clCreateProgramWithBinary,
clRetainProgram,
clReleaseProgram,
clBuildProgram,
clUnloadCompiler,
clGetProgramInfo,
clGetProgramBuildInfo,
clCreateKernel,
clCreateKernelsInProgram,
clRetainKernel,
clReleaseKernel,
clSetKernelArg,
clGetKernelInfo,
clGetKernelWorkGroupInfo,
clWaitForEvents,
clGetEventInfo,
clRetainEvent,
clReleaseEvent,
clGetEventProfilingInfo,
clFlush,
clFinish,
clEnqueueReadBuffer,
clEnqueueWriteBuffer,
clEnqueueCopyBuffer,
clEnqueueReadImage,
clEnqueueWriteImage,
clEnqueueCopyImage,
clEnqueueCopyImageToBuffer,
clEnqueueCopyBufferToImage,
clEnqueueMapBuffer,
clEnqueueMapImage,
clEnqueueUnmapMemObject,
clEnqueueNDRangeKernel,
clEnqueueTask,
clEnqueueNativeKernel,
clEnqueueMarker,
clEnqueueWaitForEvents,
clEnqueueBarrier,
GetExtensionFunctionAddress,
NULL, // clCreateFromGLBuffer
NULL, // clCreateFromGLTexture2D
NULL, // clCreateFromGLTexture3D
NULL, // clCreateFromGLRenderbuffer
NULL, // clGetGLObjectInfo
NULL, // clGetGLTextureInfo
NULL, // clEnqueueAcquireGLObjects
NULL, // clEnqueueReleaseGLObjects
// cl_khr_d3d10_sharing
NULL, // clGetGLContextInfoKHR
NULL, // clGetDeviceIDsFromD3D10KHR
NULL, // clCreateFromD3D10BufferKHR
NULL, // clCreateFromD3D10Texture2DKHR
NULL, // clCreateFromD3D10Texture3DKHR
NULL, // clEnqueueAcquireD3D10ObjectsKHR
NULL, // clEnqueueReleaseD3D10ObjectsKHR
// OpenCL 1.1
clSetEventCallback,
clCreateSubBuffer,
clSetMemObjectDestructorCallback,
clCreateUserEvent,
clSetUserEventStatus,
clEnqueueReadBufferRect,
clEnqueueWriteBufferRect,
clEnqueueCopyBufferRect,
// cl_ext_device_fission
NULL, // clCreateSubDevicesEXT
NULL, // clRetainDeviceEXT
NULL, // clReleaseDeviceEXT
// cl_khr_gl_event
NULL, // clCreateEventFromGLsyncKHR
// OpenCL 1.2
clCreateSubDevices,
clRetainDevice,
clReleaseDevice,
clCreateImage,
clCreateProgramWithBuiltInKernels,
clCompileProgram,
clLinkProgram,
clUnloadPlatformCompiler,
clGetKernelArgInfo,
clEnqueueFillBuffer,
clEnqueueFillImage,
clEnqueueMigrateMemObjects,
clEnqueueMarkerWithWaitList,
clEnqueueBarrierWithWaitList,
GetExtensionFunctionAddressForPlatform,
NULL, // clCreateFromGLTexture
// cl_khr_d3d11_sharing
NULL, // clGetDeviceIDsFromD3D11KHR
NULL, // clCreateFromD3D11BufferKHR
NULL, // clCreateFromD3D11Texture2DKHR
NULL, // clCreateFromD3D11Texture3DKHR
NULL, // clCreateFromDX9MediaSurfaceKHR
NULL, // clEnqueueAcquireD3D11ObjectsKHR
NULL, // clEnqueueReleaseD3D11ObjectsKHR
// cl_khr_dx9_media_sharing
NULL, // clGetDeviceIDsFromDX9MediaAdapterKHR
NULL, // clEnqueueAcquireDX9MediaSurfacesKHR
NULL, // clEnqueueReleaseDX9MediaSurfacesKHR
// cl_khr_egl_image
NULL, // clCreateFromEGLImageKHR
NULL, // clEnqueueAcquireEGLObjectsKHR
NULL, // clEnqueueReleaseEGLObjectsKHR
// cl_khr_egl_event
NULL, // clCreateEventFromEGLSyncKHR
// OpenCL 2.0
clCreateCommandQueueWithProperties,
clCreatePipe,
clGetPipeInfo,
clSVMAlloc,
clSVMFree,
clEnqueueSVMFree,
clEnqueueSVMMemcpy,
clEnqueueSVMMemFill,
clEnqueueSVMMap,
clEnqueueSVMUnmap,
NULL, // clCreateSamplerWithProperties
clSetKernelArgSVMPointer,
clSetKernelExecInfo,
// cl_khr_sub_groups
NULL, // clGetKernelSubGroupInfoKHR
// OpenCL 2.1
NULL, // clCloneKernel
clCreateProgramWithIL,
clEnqueueSVMMigrateMem,
clGetDeviceAndHostTimer,
clGetHostTimer,
clGetKernelSubGroupInfo,
clSetDefaultDeviceCommandQueue,
// OpenCL 2.2
clSetProgramReleaseCallback,
clSetProgramSpecializationConstant,
clCreateBufferWithProperties,
clCreateImageWithProperties,
clSetContextDestructorCallback
};
}

109
src/gallium/frontends/clover/api/dispatch.hpp
View file

@ -1,109 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef API_DISPATCH_HPP
#define API_DISPATCH_HPP
#include "CL/cl.h"
#include "CL/cl_ext.h"
#include "CL/cl_egl.h"
#include "CL/cl_gl.h"
#include "CL/cl_icd.h"
namespace clover {
extern const cl_icd_dispatch _dispatch;
cl_int CL_API_CALL
GetPlatformInfo(cl_platform_id d_platform, cl_platform_info param,
size_t size, void *r_buf, size_t *r_size);
void * CL_API_CALL
GetExtensionFunctionAddress(const char *p_name);
void * CL_API_CALL
GetExtensionFunctionAddressForPlatform(cl_platform_id d_platform,
const char *p_name);
cl_int CL_API_CALL
IcdGetPlatformIDsKHR(cl_uint num_entries, cl_platform_id *rd_platforms,
cl_uint *rnum_platforms);
cl_int CL_API_CALL
EnqueueSVMFree(cl_command_queue command_queue,
cl_uint num_svm_pointers,
void *svm_pointers[],
void (CL_CALLBACK *pfn_free_func) (
cl_command_queue queue, cl_uint num_svm_pointers,
void *svm_pointers[], void *user_data),
void *user_data,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event,
cl_int cmd);
cl_int CL_API_CALL
EnqueueSVMMemcpy(cl_command_queue command_queue,
cl_bool blocking_copy,
void *dst_ptr,
const void *src_ptr,
size_t size,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event,
cl_int cmd);
cl_int CL_API_CALL
EnqueueSVMMap(cl_command_queue command_queue,
cl_bool blocking_map,
cl_map_flags map_flags,
void *svm_ptr,
size_t size,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event,
cl_int cmd);
cl_int CL_API_CALL
EnqueueSVMMemFill(cl_command_queue command_queue,
void *svm_ptr,
const void *pattern,
size_t pattern_size,
size_t size,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event,
cl_int cmd);
cl_int CL_API_CALL
EnqueueSVMUnmap(cl_command_queue command_queue,
void *svm_ptr,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event,
cl_int cmd);
cl_program CL_API_CALL
CreateProgramWithILKHR(cl_context d_ctx, const void *il,
size_t length, cl_int *r_errcode);
}
#endif

310
src/gallium/frontends/clover/api/event.cpp
View file

@ -1,310 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/event.hpp"
using namespace clover;
CLOVER_API cl_event
clCreateUserEvent(cl_context d_ctx, cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
ret_error(r_errcode, CL_SUCCESS);
return desc(new soft_event(ctx, {}, false));
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clSetUserEventStatus(cl_event d_ev, cl_int status) try {
auto &sev = obj<soft_event>(d_ev);
if (status > 0)
return CL_INVALID_VALUE;
if (sev.status() <= 0)
return CL_INVALID_OPERATION;
if (status)
sev.abort(status);
else
sev.trigger();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clWaitForEvents(cl_uint num_evs, const cl_event *d_evs) try {
auto evs = objs(d_evs, num_evs);
for (auto &ev : evs) {
if (ev.context() != evs.front().context())
throw error(CL_INVALID_CONTEXT);
if (ev.status() < 0)
throw error(CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST);
}
// Create a temporary soft event that depends on all the events in
// the wait list
auto sev = create<soft_event>(evs.front().context(), evs, true);
// ...and wait on it.
sev().wait();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetEventInfo(cl_event d_ev, cl_event_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &ev = obj(d_ev);
switch (param) {
case CL_EVENT_COMMAND_QUEUE:
buf.as_scalar<cl_command_queue>() = desc(ev.queue());
break;
case CL_EVENT_CONTEXT:
buf.as_scalar<cl_context>() = desc(ev.context());
break;
case CL_EVENT_COMMAND_TYPE:
buf.as_scalar<cl_command_type>() = ev.command();
break;
case CL_EVENT_COMMAND_EXECUTION_STATUS:
buf.as_scalar<cl_int>() = ev.status();
break;
case CL_EVENT_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = ev.ref_count();
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clSetEventCallback(cl_event d_ev, cl_int type,
void (CL_CALLBACK *pfn_notify)(cl_event, cl_int, void *),
void *user_data) try {
auto &ev = obj(d_ev);
if (!pfn_notify ||
(type != CL_COMPLETE && type != CL_SUBMITTED && type != CL_RUNNING))
throw error(CL_INVALID_VALUE);
// Create a temporary soft event that depends on ev, with
// pfn_notify as completion action.
create<soft_event>(ev.context(), ref_vector<event> { ev }, true,
[=, &ev](event &) {
ev.wait();
pfn_notify(desc(ev), ev.status(), user_data);
});
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clRetainEvent(cl_event d_ev) try {
obj(d_ev).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseEvent(cl_event d_ev) try {
if (obj(d_ev).release())
delete pobj(d_ev);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueMarker(cl_command_queue d_q, cl_event *rd_ev) try {
auto &q = obj(d_q);
if (!rd_ev)
throw error(CL_INVALID_VALUE);
*rd_ev = desc(new hard_event(q, CL_COMMAND_MARKER, {}));
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueMarkerWithWaitList(cl_command_queue d_q, cl_uint num_deps,
const cl_event *d_deps, cl_event *rd_ev) try {
auto &q = obj(d_q);
auto deps = objs<wait_list_tag>(d_deps, num_deps);
for (auto &ev : deps) {
if (ev.context() != q.context())
throw error(CL_INVALID_CONTEXT);
}
// Create a hard event that depends on the events in the wait list:
// previous commands in the same queue are implicitly serialized
// with respect to it -- hard events always are.
auto hev = create<hard_event>(q, CL_COMMAND_MARKER, deps);
ret_object(rd_ev, hev);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueBarrier(cl_command_queue d_q) try {
obj(d_q);
// No need to do anything, q preserves data ordering strictly.
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueBarrierWithWaitList(cl_command_queue d_q, cl_uint num_deps,
const cl_event *d_deps, cl_event *rd_ev) try {
auto &q = obj(d_q);
auto deps = objs<wait_list_tag>(d_deps, num_deps);
for (auto &ev : deps) {
if (ev.context() != q.context())
throw error(CL_INVALID_CONTEXT);
}
// Create a hard event that depends on the events in the wait list:
// subsequent commands in the same queue will be implicitly
// serialized with respect to it -- hard events always are.
auto hev = create<hard_event>(q, CL_COMMAND_BARRIER, deps);
ret_object(rd_ev, hev);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueWaitForEvents(cl_command_queue d_q, cl_uint num_evs,
const cl_event *d_evs) try {
// The wait list is mandatory for clEnqueueWaitForEvents().
objs(d_evs, num_evs);
return clEnqueueBarrierWithWaitList(d_q, num_evs, d_evs, NULL);
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetEventProfilingInfo(cl_event d_ev, cl_profiling_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
hard_event &hev = dynamic_cast<hard_event &>(obj(d_ev));
if (hev.status() != CL_COMPLETE)
throw error(CL_PROFILING_INFO_NOT_AVAILABLE);
switch (param) {
case CL_PROFILING_COMMAND_QUEUED:
buf.as_scalar<cl_ulong>() = hev.time_queued();
break;
case CL_PROFILING_COMMAND_SUBMIT:
buf.as_scalar<cl_ulong>() = hev.time_submit();
break;
case CL_PROFILING_COMMAND_START:
buf.as_scalar<cl_ulong>() = hev.time_start();
break;
case CL_PROFILING_COMMAND_END:
case CL_PROFILING_COMMAND_COMPLETE:
buf.as_scalar<cl_ulong>() = hev.time_end();
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (std::bad_cast &) {
return CL_PROFILING_INFO_NOT_AVAILABLE;
} catch (lazy<cl_ulong>::undefined_error &) {
return CL_PROFILING_INFO_NOT_AVAILABLE;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clFinish(cl_command_queue d_q) try {
auto &q = obj(d_q);
// Create a temporary hard event -- it implicitly depends on all
// the previously queued hard events.
auto hev = create<hard_event>(q, 0, ref_vector<event> {});
// And wait on it.
hev().wait();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}

69
src/gallium/frontends/clover/api/interop.cpp
View file

@ -1,69 +0,0 @@
//
// Copyright 2015 Advanced Micro Devices, Inc.
// All Rights Reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/event.hpp"
#include "api/util.hpp"
using namespace clover;
extern "C" {
PUBLIC bool
opencl_dri_event_add_ref(cl_event event)
{
/* This should fail if the event hasn't been created by
* clEnqueueReleaseGLObjects or clEnqueueReleaseEGLObjects.
*
* TODO: implement the CL functions
*/
return false; /*return clRetainEvent(event) == CL_SUCCESS;*/
}
PUBLIC bool
opencl_dri_event_release(cl_event event)
{
return clReleaseEvent(event) == CL_SUCCESS;
}
PUBLIC bool
opencl_dri_event_wait(cl_event event, uint64_t timeout) try {
if (!timeout) {
return obj(event).status() == CL_COMPLETE;
}
obj(event).wait();
return true;
} catch (error &) {
return false;
}
PUBLIC struct pipe_fence_handle *
opencl_dri_event_get_fence(cl_event event) try {
return obj(event).fence();
} catch (error &) {
return NULL;
}
}

100
src/gallium/frontends/clover/api/invalid.cpp
View file

@ -1,100 +0,0 @@
//
// Copyright 2020 Red Hat
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/context.hpp"
#include "core/platform.hpp"
using namespace clover;
// This contains all the CL 2.x API entrypoints that return INVALID_OPERATON
// on CL 3.0. If these are implemented they should be moved out of this file.
CLOVER_API cl_mem
clCreatePipe(cl_context d_ctx,
cl_mem_flags flags,
cl_uint pipe_packet_size,
cl_uint pipe_max_packets,
const cl_pipe_properties *properties,
cl_int *r_errorcode) {
*r_errorcode = CL_INVALID_OPERATION;
return nullptr;
}
CLOVER_API cl_int
clGetPipeInfo(cl_mem pipe,
cl_pipe_info param_name,
size_t param_value_size,
void *param_value,
size_t *param_value_size_ret) {
return CL_INVALID_MEM_OBJECT;
}
CLOVER_API cl_int
clGetDeviceAndHostTimer(cl_device_id device,
cl_ulong *device_timestamp,
cl_ulong *host_timestamp) {
return CL_INVALID_OPERATION;
}
CLOVER_API cl_int
clGetHostTimer(cl_device_id device,
cl_ulong *host_timestamp) {
return CL_INVALID_OPERATION;
}
CLOVER_API cl_int
clGetKernelSubGroupInfo(cl_kernel d_kern,
cl_device_id device,
cl_kernel_sub_group_info param_name,
size_t input_value_size,
const void *input_value,
size_t param_size_value,
void *param_value,
size_t *param_value_size_ret) {
return CL_INVALID_OPERATION;
}
CLOVER_API cl_int
clSetDefaultDeviceCommandQueue(cl_context context,
cl_device_id device,
cl_command_queue command_queue) {
return CL_INVALID_OPERATION;
}
CLOVER_API cl_int
clSetProgramReleaseCallback(cl_program d_prog,
void (CL_CALLBACK *pfn_notify)(cl_program program, void *user_data),
void *user_data) {
return CL_INVALID_OPERATION;
}
CLOVER_API cl_int
clSetProgramSpecializationConstant(cl_program program,
cl_uint spec_id,
size_t spec_size,
const void* spec_value) {
return CL_INVALID_OPERATION;
}

436
src/gallium/frontends/clover/api/kernel.cpp
View file

@ -1,436 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/kernel.hpp"
#include "core/event.hpp"
using namespace clover;
CLOVER_API cl_kernel
clCreateKernel(cl_program d_prog, const char *name, cl_int *r_errcode) try {
auto &prog = obj(d_prog);
if (!name)
throw error(CL_INVALID_VALUE);
auto &sym = find(name_equals(name), prog.symbols());
ret_error(r_errcode, CL_SUCCESS);
return new kernel(prog, name, range(sym.args));
} catch (std::out_of_range &) {
ret_error(r_errcode, CL_INVALID_KERNEL_NAME);
return NULL;
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clCreateKernelsInProgram(cl_program d_prog, cl_uint count,
cl_kernel *rd_kerns, cl_uint *r_count) try {
auto &prog = obj(d_prog);
auto &syms = prog.symbols();
if (rd_kerns && count < syms.size())
throw error(CL_INVALID_VALUE);
if (rd_kerns)
copy(map([&](const binary::symbol &sym) {
return desc(new kernel(prog,
std::string(sym.name.begin(),
sym.name.end()),
range(sym.args)));
}, syms),
rd_kerns);
if (r_count)
*r_count = syms.size();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clRetainKernel(cl_kernel d_kern) try {
obj(d_kern).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseKernel(cl_kernel d_kern) try {
if (obj(d_kern).release())
delete pobj(d_kern);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clSetKernelArg(cl_kernel d_kern, cl_uint idx, size_t size,
const void *value) try {
obj(d_kern).args().at(idx).set(size, value);
return CL_SUCCESS;
} catch (std::out_of_range &) {
return CL_INVALID_ARG_INDEX;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetKernelInfo(cl_kernel d_kern, cl_kernel_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &kern = obj(d_kern);
switch (param) {
case CL_KERNEL_FUNCTION_NAME:
buf.as_string() = kern.name();
break;
case CL_KERNEL_NUM_ARGS:
buf.as_scalar<cl_uint>() = kern.args().size();
break;
case CL_KERNEL_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = kern.ref_count();
break;
case CL_KERNEL_CONTEXT:
buf.as_scalar<cl_context>() = desc(kern.program().context());
break;
case CL_KERNEL_PROGRAM:
buf.as_scalar<cl_program>() = desc(kern.program());
break;
case CL_KERNEL_ATTRIBUTES:
buf.as_string() = find(name_equals(kern.name()), kern.program().symbols()).attributes;
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetKernelWorkGroupInfo(cl_kernel d_kern, cl_device_id d_dev,
cl_kernel_work_group_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &kern = obj(d_kern);
auto &dev = (d_dev ? *pobj(d_dev) : unique(kern.program().devices()));
if (!count(dev, kern.program().devices()))
throw error(CL_INVALID_DEVICE);
switch (param) {
case CL_KERNEL_WORK_GROUP_SIZE:
buf.as_scalar<size_t>() = dev.max_threads_per_block();
break;
case CL_KERNEL_COMPILE_WORK_GROUP_SIZE:
buf.as_vector<size_t>() = kern.required_block_size();
break;
case CL_KERNEL_LOCAL_MEM_SIZE:
buf.as_scalar<cl_ulong>() = kern.mem_local();
break;
case CL_KERNEL_PREFERRED_WORK_GROUP_SIZE_MULTIPLE:
buf.as_scalar<size_t>() = dev.subgroup_size();
break;
case CL_KERNEL_PRIVATE_MEM_SIZE:
buf.as_scalar<cl_ulong>() = kern.mem_private();
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
} catch (std::out_of_range &) {
return CL_INVALID_DEVICE;
}
CLOVER_API cl_int
clGetKernelArgInfo(cl_kernel d_kern,
cl_uint idx, cl_kernel_arg_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto info = obj(d_kern).args_infos().at(idx);
if (info.arg_name.empty())
return CL_KERNEL_ARG_INFO_NOT_AVAILABLE;
switch (param) {
case CL_KERNEL_ARG_ADDRESS_QUALIFIER:
buf.as_scalar<cl_kernel_arg_address_qualifier>() = info.address_qualifier;
break;
case CL_KERNEL_ARG_ACCESS_QUALIFIER:
buf.as_scalar<cl_kernel_arg_access_qualifier>() = info.access_qualifier;
break;
case CL_KERNEL_ARG_TYPE_NAME:
buf.as_string() = info.type_name;
break;
case CL_KERNEL_ARG_TYPE_QUALIFIER:
buf.as_scalar<cl_kernel_arg_type_qualifier>() = info.type_qualifier;
break;
case CL_KERNEL_ARG_NAME:
buf.as_string() = info.arg_name;
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (std::out_of_range &) {
return CL_INVALID_ARG_INDEX;
} catch (error &e) {
return e.get();
}
namespace {
///
/// Common argument checking shared by kernel invocation commands.
///
void
validate_common(const command_queue &q, kernel &kern,
const ref_vector<event> &deps) {
if (kern.program().context() != q.context() ||
any_of([&](const event &ev) {
return ev.context() != q.context();
}, deps))
throw error(CL_INVALID_CONTEXT);
if (any_of([](kernel::argument &arg) {
return !arg.set();
}, kern.args()))
throw error(CL_INVALID_KERNEL_ARGS);
// If the command queue's device is not associated to the program, we get
// a binary, with no sections, which will also fail the following test.
auto &b = kern.program().build(q.device()).bin;
if (!any_of(type_equals(binary::section::text_executable), b.secs))
throw error(CL_INVALID_PROGRAM_EXECUTABLE);
}
std::vector<size_t>
validate_grid_size(const command_queue &q, cl_uint dims,
const size_t *d_grid_size) {
auto grid_size = range(d_grid_size, dims);
if (dims < 1 || dims > q.device().max_block_size().size())
throw error(CL_INVALID_WORK_DIMENSION);
return grid_size;
}
std::vector<size_t>
validate_grid_offset(const command_queue &q, cl_uint dims,
const size_t *d_grid_offset) {
if (d_grid_offset)
return range(d_grid_offset, dims);
else
return std::vector<size_t>(dims, 0);
}
std::vector<size_t>
validate_block_size(const command_queue &q, const kernel &kern,
cl_uint dims, const size_t *d_grid_size,
const size_t *d_block_size) {
auto grid_size = range(d_grid_size, dims);
if (d_block_size) {
auto block_size = range(d_block_size, dims);
if (any_of(is_zero(), block_size) ||
any_of(greater(), block_size, q.device().max_block_size()))
throw error(CL_INVALID_WORK_ITEM_SIZE);
if (any_of(modulus(), grid_size, block_size))
throw error(CL_INVALID_WORK_GROUP_SIZE);
if (fold(multiplies(), 1u, block_size) >
q.device().max_threads_per_block())
throw error(CL_INVALID_WORK_GROUP_SIZE);
return block_size;
} else {
return kern.optimal_block_size(q, grid_size);
}
}
}
CLOVER_API cl_int
clEnqueueNDRangeKernel(cl_command_queue d_q, cl_kernel d_kern,
cl_uint dims, const size_t *d_grid_offset,
const size_t *d_grid_size, const size_t *d_block_size,
cl_uint num_deps, const cl_event *d_deps,
cl_event *rd_ev) try {
auto &q = obj(d_q);
auto &kern = obj(d_kern);
auto deps = objs<wait_list_tag>(d_deps, num_deps);
auto grid_size = validate_grid_size(q, dims, d_grid_size);
auto grid_offset = validate_grid_offset(q, dims, d_grid_offset);
auto block_size = validate_block_size(q, kern, dims,
d_grid_size, d_block_size);
validate_common(q, kern, deps);
auto hev = create<hard_event>(
q, CL_COMMAND_NDRANGE_KERNEL, deps,
[=, &kern, &q](event &) {
kern.launch(q, grid_offset, grid_size, block_size);
});
ret_object(rd_ev, hev);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueTask(cl_command_queue d_q, cl_kernel d_kern,
cl_uint num_deps, const cl_event *d_deps,
cl_event *rd_ev) try {
auto &q = obj(d_q);
auto &kern = obj(d_kern);
auto deps = objs<wait_list_tag>(d_deps, num_deps);
validate_common(q, kern, deps);
auto hev = create<hard_event>(
q, CL_COMMAND_TASK, deps,
[=, &kern, &q](event &) {
kern.launch(q, { 0 }, { 1 }, { 1 });
});
ret_object(rd_ev, hev);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clEnqueueNativeKernel(cl_command_queue d_q,
void (CL_CALLBACK * func)(void *),
void *args, size_t args_size,
cl_uint num_mems, const cl_mem *d_mems,
const void **mem_handles, cl_uint num_deps,
const cl_event *d_deps, cl_event *rd_ev) {
return CL_INVALID_OPERATION;
}
CLOVER_API cl_int
clSetKernelArgSVMPointer(cl_kernel d_kern,
cl_uint arg_index,
const void *arg_value) try {
if (!any_of(std::mem_fn(&device::svm_support), obj(d_kern).program().devices()))
return CL_INVALID_OPERATION;
obj(d_kern).args().at(arg_index).set_svm(arg_value);
return CL_SUCCESS;
} catch (std::out_of_range &) {
return CL_INVALID_ARG_INDEX;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clSetKernelExecInfo(cl_kernel d_kern,
cl_kernel_exec_info param_name,
size_t param_value_size,
const void *param_value) try {
if (!any_of(std::mem_fn(&device::svm_support), obj(d_kern).program().devices()))
return CL_INVALID_OPERATION;
auto &kern = obj(d_kern);
const bool has_system_svm = all_of(std::mem_fn(&device::has_system_svm),
kern.program().context().devices());
if (!param_value)
return CL_INVALID_VALUE;
switch (param_name) {
case CL_KERNEL_EXEC_INFO_SVM_FINE_GRAIN_SYSTEM:
case CL_KERNEL_EXEC_INFO_SVM_FINE_GRAIN_SYSTEM_ARM: {
if (param_value_size != sizeof(cl_bool))
return CL_INVALID_VALUE;
cl_bool val = *static_cast<const cl_bool*>(param_value);
if (val == CL_TRUE && !has_system_svm)
return CL_INVALID_OPERATION;
else
return CL_SUCCESS;
}
case CL_KERNEL_EXEC_INFO_SVM_PTRS:
case CL_KERNEL_EXEC_INFO_SVM_PTRS_ARM:
if (has_system_svm)
return CL_SUCCESS;
CLOVER_NOT_SUPPORTED_UNTIL("2.0");
return CL_INVALID_VALUE;
default:
return CL_INVALID_VALUE;
}
} catch (error &e) {
return e.get();
}

648
src/gallium/frontends/clover/api/memory.cpp
View file

@ -1,648 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "util/format/u_format.h"
#include "util/u_math.h"
#include "api/util.hpp"
#include "core/memory.hpp"
#include "core/format.hpp"
using namespace clover;
namespace {
cl_mem_flags
validate_flags(cl_mem d_parent, cl_mem_flags d_flags, bool svm) {
const cl_mem_flags dev_access_flags =
CL_MEM_READ_WRITE | CL_MEM_WRITE_ONLY | CL_MEM_READ_ONLY;
const cl_mem_flags host_ptr_flags =
CL_MEM_USE_HOST_PTR | CL_MEM_ALLOC_HOST_PTR | CL_MEM_COPY_HOST_PTR;
const cl_mem_flags host_access_flags =
CL_MEM_HOST_WRITE_ONLY | CL_MEM_HOST_READ_ONLY | CL_MEM_HOST_NO_ACCESS;
const cl_mem_flags svm_flags =
CL_MEM_SVM_FINE_GRAIN_BUFFER | CL_MEM_SVM_ATOMICS;
const cl_mem_flags valid_flags =
dev_access_flags
| (svm || d_parent ? 0 : host_ptr_flags)
| (svm ? svm_flags : host_access_flags);
if ((d_flags & ~valid_flags) ||
util_bitcount(d_flags & dev_access_flags) > 1 ||
util_bitcount(d_flags & host_access_flags) > 1)
throw error(CL_INVALID_VALUE);
if ((d_flags & CL_MEM_USE_HOST_PTR) &&
(d_flags & (CL_MEM_COPY_HOST_PTR | CL_MEM_ALLOC_HOST_PTR)))
throw error(CL_INVALID_VALUE);
if ((d_flags & CL_MEM_SVM_ATOMICS) &&
!(d_flags & CL_MEM_SVM_FINE_GRAIN_BUFFER))
throw error(CL_INVALID_VALUE);
if (d_parent) {
const auto &parent = obj(d_parent);
const cl_mem_flags flags = (d_flags |
(d_flags & dev_access_flags ? 0 :
parent.flags() & dev_access_flags) |
(d_flags & host_access_flags ? 0 :
parent.flags() & host_access_flags) |
(parent.flags() & host_ptr_flags));
if (~flags & parent.flags() & (dev_access_flags & ~CL_MEM_READ_WRITE))
throw error(CL_INVALID_VALUE);
// Check if new host access flags cause a mismatch between
// host-read/write-only.
if (!(flags & CL_MEM_HOST_NO_ACCESS) &&
(~flags & parent.flags() & host_access_flags))
throw error(CL_INVALID_VALUE);
return flags;
} else {
return d_flags | (d_flags & dev_access_flags ? 0 : CL_MEM_READ_WRITE);
}
}
std::vector<cl_mem_properties>
fill_properties(const cl_mem_properties *d_properties) {
std::vector<cl_mem_properties> properties;
if (d_properties) {
while (*d_properties) {
if (*d_properties != 0)
throw error(CL_INVALID_PROPERTY);
properties.push_back(*d_properties);
d_properties++;
};
properties.push_back(0);
}
return properties;
}
}
CLOVER_API cl_mem
clCreateBufferWithProperties(cl_context d_ctx,
const cl_mem_properties *d_properties,
cl_mem_flags d_flags, size_t size,
void *host_ptr, cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
const cl_mem_flags flags = validate_flags(NULL, d_flags, false);
std::vector<cl_mem_properties> properties = fill_properties(d_properties);
if (bool(host_ptr) != bool(flags & (CL_MEM_USE_HOST_PTR |
CL_MEM_COPY_HOST_PTR)))
throw error(CL_INVALID_HOST_PTR);
if (!size ||
size > fold(maximum(), cl_ulong(0),
map(std::mem_fn(&device::max_mem_alloc_size), ctx.devices())
))
throw error(CL_INVALID_BUFFER_SIZE);
ret_error(r_errcode, CL_SUCCESS);
return new root_buffer(ctx, properties, flags, size, host_ptr);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_mem
clCreateBuffer(cl_context d_ctx, cl_mem_flags d_flags, size_t size,
void *host_ptr, cl_int *r_errcode) {
return clCreateBufferWithProperties(d_ctx, NULL, d_flags, size,
host_ptr, r_errcode);
}
CLOVER_API cl_mem
clCreateSubBuffer(cl_mem d_mem, cl_mem_flags d_flags,
cl_buffer_create_type op,
const void *op_info, cl_int *r_errcode) try {
auto &parent = obj<root_buffer>(d_mem);
const cl_mem_flags flags = validate_flags(d_mem, d_flags, false);
if (op == CL_BUFFER_CREATE_TYPE_REGION) {
auto reg = reinterpret_cast<const cl_buffer_region *>(op_info);
if (!reg ||
reg->origin > parent.size() ||
reg->origin + reg->size > parent.size())
throw error(CL_INVALID_VALUE);
if (!reg->size)
throw error(CL_INVALID_BUFFER_SIZE);
ret_error(r_errcode, CL_SUCCESS);
return new sub_buffer(parent, flags, reg->origin, reg->size);
} else {
throw error(CL_INVALID_VALUE);
}
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_mem
clCreateImageWithProperties(cl_context d_ctx,
const cl_mem_properties *d_properties,
cl_mem_flags d_flags,
const cl_image_format *format,
const cl_image_desc *desc,
void *host_ptr, cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
if (!any_of(std::mem_fn(&device::image_support), ctx.devices()))
throw error(CL_INVALID_OPERATION);
if (!format)
throw error(CL_INVALID_IMAGE_FORMAT_DESCRIPTOR);
if (!desc)
throw error(CL_INVALID_IMAGE_DESCRIPTOR);
if (desc->image_array_size == 0 &&
(desc->image_type == CL_MEM_OBJECT_IMAGE1D_ARRAY ||
desc->image_type == CL_MEM_OBJECT_IMAGE2D_ARRAY))
throw error(CL_INVALID_IMAGE_DESCRIPTOR);
if (!host_ptr &&
(desc->image_row_pitch || desc->image_slice_pitch))
throw error(CL_INVALID_IMAGE_DESCRIPTOR);
if (desc->num_mip_levels || desc->num_samples)
throw error(CL_INVALID_IMAGE_DESCRIPTOR);
if (bool(desc->buffer) != (desc->image_type == CL_MEM_OBJECT_IMAGE1D_BUFFER))
throw error(CL_INVALID_IMAGE_DESCRIPTOR);
if (bool(host_ptr) != bool(d_flags & (CL_MEM_USE_HOST_PTR |
CL_MEM_COPY_HOST_PTR)))
throw error(CL_INVALID_HOST_PTR);
const cl_mem_flags flags = validate_flags(desc->buffer, d_flags, false);
if (!supported_formats(ctx, desc->image_type, d_flags).count(*format))
throw error(CL_IMAGE_FORMAT_NOT_SUPPORTED);
std::vector<cl_mem_properties> properties = fill_properties(d_properties);
ret_error(r_errcode, CL_SUCCESS);
const size_t row_pitch = desc->image_row_pitch ? desc->image_row_pitch :
util_format_get_blocksize(translate_format(*format)) * desc->image_width;
switch (desc->image_type) {
case CL_MEM_OBJECT_IMAGE1D:
if (!desc->image_width)
throw error(CL_INVALID_IMAGE_SIZE);
if (all_of([=](const device &dev) {
const size_t max = dev.max_image_size();
return (desc->image_width > max);
}, ctx.devices()))
throw error(CL_INVALID_IMAGE_SIZE);
return new image1d(ctx, properties, flags, format,
desc->image_width,
row_pitch, host_ptr);
case CL_MEM_OBJECT_IMAGE1D_BUFFER:
if (!desc->image_width)
throw error(CL_INVALID_IMAGE_SIZE);
if (all_of([=](const device &dev) {
const size_t max = dev.max_image_buffer_size();
return (desc->image_width > max);
}, ctx.devices()))
throw error(CL_INVALID_IMAGE_SIZE);
return new image1d_buffer(ctx, properties, flags, format,
desc->image_width,
row_pitch, host_ptr, desc->buffer);
case CL_MEM_OBJECT_IMAGE1D_ARRAY: {
if (!desc->image_width)
throw error(CL_INVALID_IMAGE_SIZE);
if (all_of([=](const device &dev) {
const size_t max = dev.max_image_size();
const size_t amax = dev.max_image_array_number();
return (desc->image_width > max ||
desc->image_array_size > amax);
}, ctx.devices()))
throw error(CL_INVALID_IMAGE_SIZE);
const size_t slice_pitch = desc->image_slice_pitch ?
desc->image_slice_pitch : row_pitch;
return new image1d_array(ctx, properties, flags, format,
desc->image_width,
desc->image_array_size, slice_pitch,
host_ptr);
}
case CL_MEM_OBJECT_IMAGE2D:
if (!desc->image_width || !desc->image_height)
throw error(CL_INVALID_IMAGE_SIZE);
if (all_of([=](const device &dev) {
const size_t max = dev.max_image_size();
return (desc->image_width > max ||
desc->image_height > max);
}, ctx.devices()))
throw error(CL_INVALID_IMAGE_SIZE);
return new image2d(ctx, properties, flags, format,
desc->image_width, desc->image_height,
row_pitch, host_ptr);
case CL_MEM_OBJECT_IMAGE2D_ARRAY: {
if (!desc->image_width || !desc->image_height || !desc->image_array_size)
throw error(CL_INVALID_IMAGE_SIZE);
if (all_of([=](const device &dev) {
const size_t max = dev.max_image_size();
const size_t amax = dev.max_image_array_number();
return (desc->image_width > max ||
desc->image_height > max ||
desc->image_array_size > amax);
}, ctx.devices()))
throw error(CL_INVALID_IMAGE_SIZE);
const size_t slice_pitch = desc->image_slice_pitch ?
desc->image_slice_pitch : row_pitch * desc->image_height;
return new image2d_array(ctx, properties, flags, format,
desc->image_width, desc->image_height,
desc->image_array_size, row_pitch,
slice_pitch, host_ptr);
}
case CL_MEM_OBJECT_IMAGE3D: {
if (!desc->image_width || !desc->image_height || !desc->image_depth)
throw error(CL_INVALID_IMAGE_SIZE);
if (all_of([=](const device &dev) {
const size_t max = dev.max_image_size_3d();
return (desc->image_width > max ||
desc->image_height > max ||
desc->image_depth > max);
}, ctx.devices()))
throw error(CL_INVALID_IMAGE_SIZE);
const size_t slice_pitch = desc->image_slice_pitch ?
desc->image_slice_pitch : row_pitch * desc->image_height;
return new image3d(ctx, properties, flags, format,
desc->image_width, desc->image_height,
desc->image_depth, row_pitch,
slice_pitch, host_ptr);
}
default:
throw error(CL_INVALID_IMAGE_DESCRIPTOR);
}
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_mem
clCreateImage(cl_context d_ctx,
cl_mem_flags d_flags,
const cl_image_format *format,
const cl_image_desc *desc,
void *host_ptr, cl_int *r_errcode) {
return clCreateImageWithProperties(d_ctx, NULL, d_flags, format, desc, host_ptr, r_errcode);
}
CLOVER_API cl_mem
clCreateImage2D(cl_context d_ctx, cl_mem_flags d_flags,
const cl_image_format *format,
size_t width, size_t height, size_t row_pitch,
void *host_ptr, cl_int *r_errcode) {
const cl_image_desc desc = { CL_MEM_OBJECT_IMAGE2D, width, height, 0, 0,
row_pitch, 0, 0, 0, { NULL } };
return clCreateImageWithProperties(d_ctx, NULL, d_flags, format, &desc, host_ptr, r_errcode);
}
CLOVER_API cl_mem
clCreateImage3D(cl_context d_ctx, cl_mem_flags d_flags,
const cl_image_format *format,
size_t width, size_t height, size_t depth,
size_t row_pitch, size_t slice_pitch,
void *host_ptr, cl_int *r_errcode) {
const cl_image_desc desc = { CL_MEM_OBJECT_IMAGE3D, width, height, depth, 0,
row_pitch, slice_pitch, 0, 0, { NULL } };
return clCreateImageWithProperties(d_ctx, NULL, d_flags, format, &desc, host_ptr, r_errcode);
}
CLOVER_API cl_int
clGetSupportedImageFormats(cl_context d_ctx, cl_mem_flags flags,
cl_mem_object_type type, cl_uint count,
cl_image_format *r_buf, cl_uint *r_count) try {
auto &ctx = obj(d_ctx);
auto formats = supported_formats(ctx, type, flags);
if (flags & CL_MEM_KERNEL_READ_AND_WRITE) {
if (r_count)
*r_count = 0;
return CL_SUCCESS;
}
if (flags & (CL_MEM_WRITE_ONLY | CL_MEM_READ_WRITE) &&
type == CL_MEM_OBJECT_IMAGE3D) {
if (r_count)
*r_count = 0;
return CL_SUCCESS;
}
validate_flags(NULL, flags, false);
if (r_buf && !count)
throw error(CL_INVALID_VALUE);
if (r_buf)
std::copy_n(formats.begin(),
std::min((cl_uint)formats.size(), count),
r_buf);
if (r_count)
*r_count = formats.size();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetMemObjectInfo(cl_mem d_mem, cl_mem_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &mem = obj(d_mem);
switch (param) {
case CL_MEM_TYPE:
buf.as_scalar<cl_mem_object_type>() = mem.type();
break;
case CL_MEM_FLAGS:
buf.as_scalar<cl_mem_flags>() = mem.flags();
break;
case CL_MEM_SIZE:
buf.as_scalar<size_t>() = mem.size();
break;
case CL_MEM_HOST_PTR:
buf.as_scalar<void *>() = mem.host_ptr();
break;
case CL_MEM_MAP_COUNT:
buf.as_scalar<cl_uint>() = 0;
break;
case CL_MEM_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = mem.ref_count();
break;
case CL_MEM_CONTEXT:
buf.as_scalar<cl_context>() = desc(mem.context());
break;
case CL_MEM_ASSOCIATED_MEMOBJECT: {
sub_buffer *sub = dynamic_cast<sub_buffer *>(&mem);
if (sub) {
buf.as_scalar<cl_mem>() = desc(sub->parent());
break;
}
image *img = dynamic_cast<image *>(&mem);
if (img) {
buf.as_scalar<cl_mem>() = desc(img->buffer());
break;
}
buf.as_scalar<cl_mem>() = NULL;
break;
}
case CL_MEM_OFFSET: {
sub_buffer *sub = dynamic_cast<sub_buffer *>(&mem);
buf.as_scalar<size_t>() = (sub ? sub->offset() : 0);
break;
}
case CL_MEM_USES_SVM_POINTER:
case CL_MEM_USES_SVM_POINTER_ARM: {
// with system SVM all host ptrs are SVM pointers
// TODO: once we support devices with lower levels of SVM, we have to
// check the ptr in more detail
const bool system_svm = all_of(std::mem_fn(&device::has_system_svm),
mem.context().devices());
buf.as_scalar<cl_bool>() = mem.host_ptr() && system_svm;
break;
}
case CL_MEM_PROPERTIES:
buf.as_vector<cl_mem_properties>() = mem.properties();
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetImageInfo(cl_mem d_mem, cl_image_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &img = obj<image>(d_mem);
switch (param) {
case CL_IMAGE_FORMAT:
buf.as_scalar<cl_image_format>() = img.format();
break;
case CL_IMAGE_ELEMENT_SIZE:
buf.as_scalar<size_t>() = img.pixel_size();
break;
case CL_IMAGE_ROW_PITCH:
buf.as_scalar<size_t>() = img.row_pitch();
break;
case CL_IMAGE_SLICE_PITCH:
buf.as_scalar<size_t>() = img.slice_pitch();
break;
case CL_IMAGE_WIDTH:
buf.as_scalar<size_t>() = img.width();
break;
case CL_IMAGE_HEIGHT:
buf.as_scalar<size_t>() = img.dimensions() > 1 ? img.height() : 0;
break;
case CL_IMAGE_DEPTH:
buf.as_scalar<size_t>() = img.dimensions() > 2 ? img.depth() : 0;
break;
case CL_IMAGE_ARRAY_SIZE:
buf.as_scalar<size_t>() = img.array_size();
break;
case CL_IMAGE_BUFFER:
buf.as_scalar<cl_mem>() = img.buffer();
break;
case CL_IMAGE_NUM_MIP_LEVELS:
buf.as_scalar<cl_uint>() = 0;
break;
case CL_IMAGE_NUM_SAMPLES:
buf.as_scalar<cl_uint>() = 0;
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clRetainMemObject(cl_mem d_mem) try {
obj(d_mem).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseMemObject(cl_mem d_mem) try {
if (obj(d_mem).release())
delete pobj(d_mem);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clSetMemObjectDestructorCallback(cl_mem d_mem,
void (CL_CALLBACK *pfn_notify)(cl_mem, void *),
void *user_data) try {
auto &mem = obj(d_mem);
if (!pfn_notify)
return CL_INVALID_VALUE;
mem.destroy_notify([=]{ pfn_notify(d_mem, user_data); });
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API void *
clSVMAlloc(cl_context d_ctx,
cl_svm_mem_flags flags,
size_t size,
unsigned int alignment) try {
auto &ctx = obj(d_ctx);
if (!any_of(std::mem_fn(&device::svm_support), ctx.devices()))
return NULL;
validate_flags(NULL, flags, true);
if (!size ||
size > fold(minimum(), cl_ulong(ULONG_MAX),
map(std::mem_fn(&device::max_mem_alloc_size), ctx.devices())))
return nullptr;
if (!util_is_power_of_two_or_zero(alignment))
return nullptr;
if (!alignment)
alignment = 0x80; // sizeof(long16)
#if defined(HAVE_POSIX_MEMALIGN)
bool can_emulate = all_of(std::mem_fn(&device::has_system_svm), ctx.devices());
if (can_emulate) {
// we can ignore all the flags as it's not required to honor them.
void *ptr = nullptr;
if (alignment < sizeof(void*))
alignment = sizeof(void*);
int ret = posix_memalign(&ptr, alignment, size);
if (ret)
return nullptr;
if (ptr)
ctx.add_svm_allocation(ptr, size);
return ptr;
}
#endif
CLOVER_NOT_SUPPORTED_UNTIL("2.0");
return nullptr;
} catch (error &) {
return nullptr;
}
CLOVER_API void
clSVMFree(cl_context d_ctx,
void *svm_pointer) try {
auto &ctx = obj(d_ctx);
if (!any_of(std::mem_fn(&device::svm_support), ctx.devices()))
return;
bool can_emulate = all_of(std::mem_fn(&device::has_system_svm), ctx.devices());
if (can_emulate) {
ctx.remove_svm_allocation(svm_pointer);
return free(svm_pointer);
}
CLOVER_NOT_SUPPORTED_UNTIL("2.0");
} catch (error &) {
}

256
src/gallium/frontends/clover/api/platform.cpp
View file

@ -1,256 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include <unordered_map>
#include "api/dispatch.hpp"
#include "api/util.hpp"
#include "core/platform.hpp"
#include "git_sha1.h"
#include "util/u_debug.h"
using namespace clover;
namespace {
platform _clover_platform;
}
CLOVER_API cl_int
clGetPlatformIDs(cl_uint num_entries, cl_platform_id *rd_platforms,
cl_uint *rnum_platforms) {
if ((!num_entries && rd_platforms) ||
(!rnum_platforms && !rd_platforms))
return CL_INVALID_VALUE;
if (rnum_platforms)
*rnum_platforms = 1;
if (rd_platforms)
*rd_platforms = desc(_clover_platform);
return CL_SUCCESS;
}
platform &clover::find_platform(cl_platform_id d_platform)
{
/* this error is only added in CL2.0 */
if (d_platform != desc(_clover_platform))
throw error(CL_INVALID_PLATFORM);
return obj(d_platform);
}
cl_int
clover::GetPlatformInfo(cl_platform_id d_platform, cl_platform_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &platform = find_platform(d_platform);
switch (param) {
case CL_PLATFORM_PROFILE:
buf.as_string() = "FULL_PROFILE";
break;
case CL_PLATFORM_VERSION: {
buf.as_string() = "OpenCL " + platform.platform_version_as_string() + " Mesa " PACKAGE_VERSION MESA_GIT_SHA1;
break;
}
case CL_PLATFORM_NAME:
buf.as_string() = "Clover";
break;
case CL_PLATFORM_VENDOR:
buf.as_string() = "Mesa";
break;
case CL_PLATFORM_EXTENSIONS:
buf.as_string() = platform.supported_extensions_as_string();
break;
case CL_PLATFORM_ICD_SUFFIX_KHR:
buf.as_string() = "MESA";
break;
case CL_PLATFORM_NUMERIC_VERSION: {
buf.as_scalar<cl_version>() = platform.platform_version();
break;
}
case CL_PLATFORM_EXTENSIONS_WITH_VERSION:
buf.as_vector<cl_name_version>() = platform.supported_extensions();
break;
case CL_PLATFORM_HOST_TIMER_RESOLUTION:
buf.as_scalar<cl_ulong>() = 0;
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
void *
clover::GetExtensionFunctionAddressForPlatform(cl_platform_id d_platform,
const char *p_name) try {
obj(d_platform);
return GetExtensionFunctionAddress(p_name);
} catch (error &) {
return NULL;
}
namespace {
cl_int
enqueueSVMFreeARM(cl_command_queue command_queue,
cl_uint num_svm_pointers,
void *svm_pointers[],
void (CL_CALLBACK *pfn_free_func) (
cl_command_queue queue, cl_uint num_svm_pointers,
void *svm_pointers[], void *user_data),
void *user_data,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event) {
return EnqueueSVMFree(command_queue, num_svm_pointers, svm_pointers,
pfn_free_func, user_data, num_events_in_wait_list,
event_wait_list, event, CL_COMMAND_SVM_FREE_ARM);
}
cl_int
enqueueSVMMapARM(cl_command_queue command_queue,
cl_bool blocking_map,
cl_map_flags map_flags,
void *svm_ptr,
size_t size,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event) {
return EnqueueSVMMap(command_queue, blocking_map, map_flags, svm_ptr, size,
num_events_in_wait_list, event_wait_list, event,
CL_COMMAND_SVM_MAP_ARM);
}
cl_int
enqueueSVMMemcpyARM(cl_command_queue command_queue,
cl_bool blocking_copy,
void *dst_ptr,
const void *src_ptr,
size_t size,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event) {
return EnqueueSVMMemcpy(command_queue, blocking_copy, dst_ptr, src_ptr,
size, num_events_in_wait_list, event_wait_list,
event, CL_COMMAND_SVM_MEMCPY_ARM);
}
cl_int
enqueueSVMMemFillARM(cl_command_queue command_queue,
void *svm_ptr,
const void *pattern,
size_t pattern_size,
size_t size,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event) {
return EnqueueSVMMemFill(command_queue, svm_ptr, pattern, pattern_size,
size, num_events_in_wait_list, event_wait_list,
event, CL_COMMAND_SVM_MEMFILL_ARM);
}
cl_int
enqueueSVMUnmapARM(cl_command_queue command_queue,
void *svm_ptr,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event) {
return EnqueueSVMUnmap(command_queue, svm_ptr, num_events_in_wait_list,
event_wait_list, event, CL_COMMAND_SVM_UNMAP_ARM);
}
const std::unordered_map<std::string, void *>
ext_funcs = {
// cl_arm_shared_virtual_memory
{ "clEnqueueSVMFreeARM", reinterpret_cast<void *>(enqueueSVMFreeARM) },
{ "clEnqueueSVMMapARM", reinterpret_cast<void *>(enqueueSVMMapARM) },
{ "clEnqueueSVMMemcpyARM", reinterpret_cast<void *>(enqueueSVMMemcpyARM) },
{ "clEnqueueSVMMemFillARM", reinterpret_cast<void *>(enqueueSVMMemFillARM) },
{ "clEnqueueSVMUnmapARM", reinterpret_cast<void *>(enqueueSVMUnmapARM) },
{ "clSetKernelArgSVMPointerARM", reinterpret_cast<void *>(clSetKernelArgSVMPointer) },
{ "clSetKernelExecInfoARM", reinterpret_cast<void *>(clSetKernelExecInfo) },
{ "clSVMAllocARM", reinterpret_cast<void *>(clSVMAlloc) },
{ "clSVMFreeARM", reinterpret_cast<void *>(clSVMFree) },
// cl_khr_icd
{ "clIcdGetPlatformIDsKHR", reinterpret_cast<void *>(IcdGetPlatformIDsKHR) },
// cl_khr_il_program
{ "clCreateProgramWithILKHR", reinterpret_cast<void *>(CreateProgramWithILKHR) },
};
} // anonymous namespace
void *
clover::GetExtensionFunctionAddress(const char *p_name) try {
return ext_funcs.at(p_name);
} catch (...) {
return nullptr;
}
cl_int
clover::IcdGetPlatformIDsKHR(cl_uint num_entries, cl_platform_id *rd_platforms,
cl_uint *rnum_platforms) {
return clGetPlatformIDs(num_entries, rd_platforms, rnum_platforms);
}
CLOVER_ICD_API cl_int
clGetPlatformInfo(cl_platform_id d_platform, cl_platform_info param,
size_t size, void *r_buf, size_t *r_size) {
return GetPlatformInfo(d_platform, param, size, r_buf, r_size);
}
CLOVER_ICD_API void *
clGetExtensionFunctionAddress(const char *p_name) {
return GetExtensionFunctionAddress(p_name);
}
CLOVER_ICD_API void *
clGetExtensionFunctionAddressForPlatform(cl_platform_id d_platform,
const char *p_name) {
return GetExtensionFunctionAddressForPlatform(d_platform, p_name);
}
CLOVER_ICD_API cl_int
clIcdGetPlatformIDsKHR(cl_uint num_entries, cl_platform_id *rd_platforms,
cl_uint *rnum_platforms) {
return IcdGetPlatformIDsKHR(num_entries, rd_platforms, rnum_platforms);
}

580
src/gallium/frontends/clover/api/program.cpp
View file

@ -1,580 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/program.hpp"
#include "core/platform.hpp"
#include "util/u_debug.h"
#include <limits>
#include <sstream>
using namespace clover;
namespace {
std::string
build_options(const char *p_opts, const char *p_debug) {
auto opts = std::string(p_opts ? p_opts : "");
std::string extra_opts = debug_get_option(p_debug, "");
return detokenize(std::vector<std::string>{opts, extra_opts}, " ");
}
class build_notifier {
public:
build_notifier(cl_program prog,
void (CL_CALLBACK * notifer)(cl_program, void *), void *data) :
prog_(prog), notifer(notifer), data_(data) { }
~build_notifier() {
if (notifer)
notifer(prog_, data_);
}
private:
cl_program prog_;
void (CL_CALLBACK * notifer)(cl_program, void *);
void *data_;
};
void
validate_build_common(const program &prog, cl_uint num_devs,
const cl_device_id *d_devs,
void (CL_CALLBACK * pfn_notify)(cl_program, void *),
void *user_data) {
if (!pfn_notify && user_data)
throw error(CL_INVALID_VALUE);
if (prog.kernel_ref_count())
throw error(CL_INVALID_OPERATION);
if (any_of([&](const device &dev) {
return !count(dev, prog.devices());
}, objs<allow_empty_tag>(d_devs, num_devs)))
throw error(CL_INVALID_DEVICE);
}
enum program::il_type
identify_and_validate_il(const std::string &il,
const cl_version opencl_version,
const context::notify_action &notify) {
return program::il_type::none;
}
}
CLOVER_API cl_program
clCreateProgramWithSource(cl_context d_ctx, cl_uint count,
const char **strings, const size_t *lengths,
cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
std::string source;
if (!count || !strings ||
any_of(is_zero(), range(strings, count)))
throw error(CL_INVALID_VALUE);
// Concatenate all the provided fragments together
for (unsigned i = 0; i < count; ++i)
source += (lengths && lengths[i] ?
std::string(strings[i], strings[i] + lengths[i]) :
std::string(strings[i]));
// ...and create a program object for them.
ret_error(r_errcode, CL_SUCCESS);
return new program(ctx, std::move(source), program::il_type::source);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_program
clCreateProgramWithBinary(cl_context d_ctx, cl_uint n,
const cl_device_id *d_devs,
const size_t *lengths,
const unsigned char **binaries,
cl_int *r_status, cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
auto devs = objs(d_devs, n);
if (!lengths || !binaries)
throw error(CL_INVALID_VALUE);
if (any_of([&](const device &dev) {
return !count(dev, ctx.devices());
}, devs))
throw error(CL_INVALID_DEVICE);
// Deserialize the provided binaries,
std::vector<std::pair<cl_int, binary>> result = map(
[](const unsigned char *p, size_t l) -> std::pair<cl_int, binary> {
if (!p || !l)
return { CL_INVALID_VALUE, {} };
try {
std::stringbuf bin( std::string{ (char*)p, l } );
std::istream s(&bin);
return { CL_SUCCESS, binary::deserialize(s) };
} catch (std::istream::failure &) {
return { CL_INVALID_BINARY, {} };
}
},
range(binaries, n),
range(lengths, n));
// update the status array,
if (r_status)
copy(map(keys(), result), r_status);
if (any_of(key_equals(CL_INVALID_VALUE), result))
throw error(CL_INVALID_VALUE);
if (any_of(key_equals(CL_INVALID_BINARY), result))
throw error(CL_INVALID_BINARY);
// initialize a program object with them.
ret_error(r_errcode, CL_SUCCESS);
return new program(ctx, devs, map(values(), result));
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
cl_program
clover::CreateProgramWithILKHR(cl_context d_ctx, const void *il,
size_t length, cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
if (!il || !length)
throw error(CL_INVALID_VALUE);
// Compute the highest OpenCL version supported by all devices associated to
// the context. That is the version used for validating the SPIR-V binary.
cl_version min_opencl_version = std::numeric_limits<uint32_t>::max();
for (const device &dev : ctx.devices()) {
const cl_version opencl_version = dev.device_version();
min_opencl_version = std::min(opencl_version, min_opencl_version);
}
const char *stream = reinterpret_cast<const char *>(il);
std::string binary(stream, stream + length);
const enum program::il_type il_type = identify_and_validate_il(binary,
min_opencl_version,
ctx.notify);
if (il_type == program::il_type::none)
throw error(CL_INVALID_VALUE);
// Initialize a program object with it.
ret_error(r_errcode, CL_SUCCESS);
return new program(ctx, std::move(binary), il_type);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_program
clCreateProgramWithIL(cl_context d_ctx,
const void *il,
size_t length,
cl_int *r_errcode) {
return CreateProgramWithILKHR(d_ctx, il, length, r_errcode);
}
CLOVER_API cl_program
clCreateProgramWithBuiltInKernels(cl_context d_ctx, cl_uint n,
const cl_device_id *d_devs,
const char *kernel_names,
cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
auto devs = objs(d_devs, n);
if (any_of([&](const device &dev) {
return !count(dev, ctx.devices());
}, devs))
throw error(CL_INVALID_DEVICE);
// No currently supported built-in kernels.
throw error(CL_INVALID_VALUE);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clRetainProgram(cl_program d_prog) try {
obj(d_prog).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseProgram(cl_program d_prog) try {
if (obj(d_prog).release())
delete pobj(d_prog);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clBuildProgram(cl_program d_prog, cl_uint num_devs,
const cl_device_id *d_devs, const char *p_opts,
void (CL_CALLBACK * pfn_notify)(cl_program, void *),
void *user_data) try {
auto &prog = obj(d_prog);
auto devs =
(d_devs ? objs(d_devs, num_devs) : ref_vector<device>(prog.devices()));
const auto opts = build_options(p_opts, "CLOVER_EXTRA_BUILD_OPTIONS");
validate_build_common(prog, num_devs, d_devs, pfn_notify, user_data);
auto notifier = build_notifier(d_prog, pfn_notify, user_data);
if (prog.il_type() != program::il_type::none) {
prog.compile(devs, opts);
prog.link(devs, opts, { prog });
} else if (any_of([&](const device &dev){
return prog.build(dev).binary_type() != CL_PROGRAM_BINARY_TYPE_EXECUTABLE;
}, devs)) {
// According to the OpenCL 1.2 specification, “if program is created
// with clCreateProgramWithBinary, then the program binary must be an
// executable binary (not a compiled binary or library).”
throw error(CL_INVALID_BINARY);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clCompileProgram(cl_program d_prog, cl_uint num_devs,
const cl_device_id *d_devs, const char *p_opts,
cl_uint num_headers, const cl_program *d_header_progs,
const char **header_names,
void (CL_CALLBACK * pfn_notify)(cl_program, void *),
void *user_data) try {
auto &prog = obj(d_prog);
auto devs =
(d_devs ? objs(d_devs, num_devs) : ref_vector<device>(prog.devices()));
const auto opts = build_options(p_opts, "CLOVER_EXTRA_COMPILE_OPTIONS");
header_map headers;
validate_build_common(prog, num_devs, d_devs, pfn_notify, user_data);
auto notifier = build_notifier(d_prog, pfn_notify, user_data);
if (bool(num_headers) != bool(header_names))
throw error(CL_INVALID_VALUE);
if (prog.il_type() == program::il_type::none)
throw error(CL_INVALID_OPERATION);
for_each([&](const char *name, const program &header) {
if (header.il_type() == program::il_type::none)
throw error(CL_INVALID_OPERATION);
if (!any_of(key_equals(name), headers))
headers.push_back(std::pair<std::string, std::string>(
name, header.source()));
},
range(header_names, num_headers),
objs<allow_empty_tag>(d_header_progs, num_headers));
prog.compile(devs, opts, headers);
return CL_SUCCESS;
} catch (invalid_build_options_error &) {
return CL_INVALID_COMPILER_OPTIONS;
} catch (build_error &) {
return CL_COMPILE_PROGRAM_FAILURE;
} catch (error &e) {
return e.get();
}
namespace {
ref_vector<device>
validate_link_devices(const ref_vector<program> &progs,
const ref_vector<device> &all_devs,
const std::string &opts) {
std::vector<device *> devs;
const bool create_library =
opts.find("-create-library") != std::string::npos;
const bool enable_link_options =
opts.find("-enable-link-options") != std::string::npos;
const bool has_link_options =
opts.find("-cl-denorms-are-zero") != std::string::npos ||
opts.find("-cl-no-signed-zeroes") != std::string::npos ||
opts.find("-cl-unsafe-math-optimizations") != std::string::npos ||
opts.find("-cl-finite-math-only") != std::string::npos ||
opts.find("-cl-fast-relaxed-math") != std::string::npos ||
opts.find("-cl-no-subgroup-ifp") != std::string::npos;
// According to the OpenCL 1.2 specification, "[the
// -enable-link-options] option must be specified with the
// create-library option".
if (enable_link_options && !create_library)
throw error(CL_INVALID_LINKER_OPTIONS);
// According to the OpenCL 1.2 specification, "the
// [program linking options] can be specified when linking a program
// executable".
if (has_link_options && create_library)
throw error(CL_INVALID_LINKER_OPTIONS);
for (auto &dev : all_devs) {
const auto has_binary = [&](const program &prog) {
const auto t = prog.build(dev).binary_type();
return t == CL_PROGRAM_BINARY_TYPE_COMPILED_OBJECT ||
t == CL_PROGRAM_BINARY_TYPE_LIBRARY;
};
// According to the OpenCL 1.2 specification, a library is made of
// “compiled binaries specified in input_programs argument to
// clLinkProgram“; compiled binaries does not refer to libraries:
// “input_programs is an array of program objects that are compiled
// binaries or libraries that are to be linked to create the program
// executable”.
if (create_library && any_of([&](const program &prog) {
const auto t = prog.build(dev).binary_type();
return t != CL_PROGRAM_BINARY_TYPE_COMPILED_OBJECT;
}, progs))
throw error(CL_INVALID_OPERATION);
// According to the CL 1.2 spec, when "all programs specified [..]
// contain a compiled binary or library for the device [..] a link is
// performed",
else if (all_of(has_binary, progs))
devs.push_back(&dev);
// otherwise if "none of the programs contain a compiled binary or
// library for that device [..] no link is performed. All other
// cases will return a CL_INVALID_OPERATION error."
else if (any_of(has_binary, progs))
throw error(CL_INVALID_OPERATION);
// According to the OpenCL 1.2 specification, "[t]he linker may apply
// [program linking options] to all compiled program objects
// specified to clLinkProgram. The linker may apply these options
// only to libraries which were created with the
// -enable-link-option."
else if (has_link_options && any_of([&](const program &prog) {
const auto t = prog.build(dev).binary_type();
return !(t == CL_PROGRAM_BINARY_TYPE_COMPILED_OBJECT ||
(t == CL_PROGRAM_BINARY_TYPE_LIBRARY &&
prog.build(dev).opts.find("-enable-link-options") !=
std::string::npos));
}, progs))
throw error(CL_INVALID_LINKER_OPTIONS);
}
return map(derefs(), devs);
}
}
CLOVER_API cl_program
clLinkProgram(cl_context d_ctx, cl_uint num_devs, const cl_device_id *d_devs,
const char *p_opts, cl_uint num_progs, const cl_program *d_progs,
void (CL_CALLBACK * pfn_notify) (cl_program, void *), void *user_data,
cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
const auto opts = build_options(p_opts, "CLOVER_EXTRA_LINK_OPTIONS");
auto progs = objs(d_progs, num_progs);
auto all_devs =
(d_devs ? objs(d_devs, num_devs) : ref_vector<device>(ctx.devices()));
auto prog = create<program>(ctx, all_devs);
auto r_prog = ret_object(prog);
auto notifier = build_notifier(r_prog, pfn_notify, user_data);
auto devs = validate_link_devices(progs, all_devs, opts);
validate_build_common(prog, num_devs, d_devs, pfn_notify, user_data);
try {
prog().link(devs, opts, progs);
ret_error(r_errcode, CL_SUCCESS);
} catch (build_error &) {
ret_error(r_errcode, CL_LINK_PROGRAM_FAILURE);
}
return r_prog;
} catch (invalid_build_options_error &) {
ret_error(r_errcode, CL_INVALID_LINKER_OPTIONS);
return NULL;
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clUnloadCompiler() {
return CL_SUCCESS;
}
CLOVER_API cl_int
clUnloadPlatformCompiler(cl_platform_id d_platform) try {
find_platform(d_platform);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetProgramInfo(cl_program d_prog, cl_program_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &prog = obj(d_prog);
switch (param) {
case CL_PROGRAM_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = prog.ref_count();
break;
case CL_PROGRAM_CONTEXT:
buf.as_scalar<cl_context>() = desc(prog.context());
break;
case CL_PROGRAM_NUM_DEVICES:
buf.as_scalar<cl_uint>() = (prog.devices().size() ?
prog.devices().size() :
prog.context().devices().size());
break;
case CL_PROGRAM_DEVICES:
buf.as_vector<cl_device_id>() = (prog.devices().size() ?
descs(prog.devices()) :
descs(prog.context().devices()));
break;
case CL_PROGRAM_SOURCE:
buf.as_string() = prog.source();
break;
case CL_PROGRAM_BINARY_SIZES:
buf.as_vector<size_t>() = map([&](const device &dev) {
return prog.build(dev).bin.size();
},
prog.devices());
break;
case CL_PROGRAM_BINARIES:
buf.as_matrix<unsigned char>() = map([&](const device &dev) {
std::stringbuf bin;
std::ostream s(&bin);
prog.build(dev).bin.serialize(s);
return bin.str();
},
prog.devices());
break;
case CL_PROGRAM_NUM_KERNELS:
buf.as_scalar<cl_uint>() = prog.symbols().size();
break;
case CL_PROGRAM_KERNEL_NAMES:
buf.as_string() = fold([](const std::string &a, const binary::symbol &s) {
return ((a.empty() ? "" : a + ";") + s.name);
}, std::string(), prog.symbols());
break;
case CL_PROGRAM_SCOPE_GLOBAL_CTORS_PRESENT:
case CL_PROGRAM_SCOPE_GLOBAL_DTORS_PRESENT:
buf.as_scalar<cl_bool>() = CL_FALSE;
break;
case CL_PROGRAM_IL:
if (prog.il_type() == program::il_type::spirv)
buf.as_vector<char>() = prog.source();
else if (r_size)
*r_size = 0u;
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetProgramBuildInfo(cl_program d_prog, cl_device_id d_dev,
cl_program_build_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &prog = obj(d_prog);
auto &dev = obj(d_dev);
if (!count(dev, prog.context().devices()))
return CL_INVALID_DEVICE;
switch (param) {
case CL_PROGRAM_BUILD_STATUS:
buf.as_scalar<cl_build_status>() = prog.build(dev).status();
break;
case CL_PROGRAM_BUILD_OPTIONS:
buf.as_string() = prog.build(dev).opts;
break;
case CL_PROGRAM_BUILD_LOG:
buf.as_string() = prog.build(dev).log;
break;
case CL_PROGRAM_BINARY_TYPE:
buf.as_scalar<cl_program_binary_type>() = prog.build(dev).binary_type();
break;
case CL_PROGRAM_BUILD_GLOBAL_VARIABLE_TOTAL_SIZE:
buf.as_scalar<size_t>() = 0;
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}

155
src/gallium/frontends/clover/api/queue.cpp
View file

@ -1,155 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/queue.hpp"
using namespace clover;
CLOVER_API cl_command_queue
clCreateCommandQueue(cl_context d_ctx, cl_device_id d_dev,
cl_command_queue_properties props,
cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
auto &dev = obj(d_dev);
if (!count(dev, ctx.devices()))
throw error(CL_INVALID_DEVICE);
if (props & ~(CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE |
CL_QUEUE_PROFILING_ENABLE))
throw error(CL_INVALID_VALUE);
ret_error(r_errcode, CL_SUCCESS);
return new command_queue(ctx, dev, props);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clRetainCommandQueue(cl_command_queue d_q) try {
obj(d_q).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseCommandQueue(cl_command_queue d_q) try {
auto &q = obj(d_q);
q.flush();
if (q.release())
delete pobj(d_q);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetCommandQueueInfo(cl_command_queue d_q, cl_command_queue_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &q = obj(d_q);
switch (param) {
case CL_QUEUE_CONTEXT:
buf.as_scalar<cl_context>() = desc(q.context());
break;
case CL_QUEUE_DEVICE:
buf.as_scalar<cl_device_id>() = desc(q.device());
break;
case CL_QUEUE_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = q.ref_count();
break;
case CL_QUEUE_PROPERTIES:
buf.as_scalar<cl_command_queue_properties>() = q.props();
break;
case CL_QUEUE_PROPERTIES_ARRAY:
buf.as_vector<cl_queue_properties>() = q.properties();
break;
case CL_QUEUE_DEVICE_DEFAULT:
if (r_size)
*r_size = 0;
break;
case CL_QUEUE_SIZE:
throw error(CL_INVALID_COMMAND_QUEUE);
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clFlush(cl_command_queue d_q) try {
obj(d_q).flush();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_command_queue
clCreateCommandQueueWithProperties(cl_context d_ctx, cl_device_id d_dev,
const cl_queue_properties *d_properties,
cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
auto &dev = obj(d_dev);
if (!count(dev, ctx.devices()))
throw error(CL_INVALID_DEVICE);
ret_error(r_errcode, CL_SUCCESS);
std::vector<cl_queue_properties> properties;
if (d_properties) {
int idx = -1;
/* these come in pairs, bail if the first is 0 */
do {
idx++;
properties.push_back(d_properties[idx]);
} while (d_properties[idx & ~1]);
}
return new command_queue(ctx, dev, properties);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}

100
src/gallium/frontends/clover/api/sampler.cpp
View file

@ -1,100 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "api/util.hpp"
#include "core/sampler.hpp"
using namespace clover;
CLOVER_API cl_sampler
clCreateSampler(cl_context d_ctx, cl_bool norm_mode,
cl_addressing_mode addr_mode, cl_filter_mode filter_mode,
cl_int *r_errcode) try {
auto &ctx = obj(d_ctx);
if (!any_of(std::mem_fn(&device::image_support), ctx.devices()))
throw error(CL_INVALID_OPERATION);
ret_error(r_errcode, CL_SUCCESS);
return new sampler(ctx, norm_mode, addr_mode, filter_mode);
} catch (error &e) {
ret_error(r_errcode, e);
return NULL;
}
CLOVER_API cl_int
clRetainSampler(cl_sampler d_s) try {
obj(d_s).retain();
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clReleaseSampler(cl_sampler d_s) try {
if (obj(d_s).release())
delete pobj(d_s);
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}
CLOVER_API cl_int
clGetSamplerInfo(cl_sampler d_s, cl_sampler_info param,
size_t size, void *r_buf, size_t *r_size) try {
property_buffer buf { r_buf, size, r_size };
auto &s = obj(d_s);
switch (param) {
case CL_SAMPLER_REFERENCE_COUNT:
buf.as_scalar<cl_uint>() = s.ref_count();
break;
case CL_SAMPLER_CONTEXT:
buf.as_scalar<cl_context>() = desc(s.context());
break;
case CL_SAMPLER_NORMALIZED_COORDS:
buf.as_scalar<cl_bool>() = s.norm_mode();
break;
case CL_SAMPLER_ADDRESSING_MODE:
buf.as_scalar<cl_addressing_mode>() = s.addr_mode();
break;
case CL_SAMPLER_FILTER_MODE:
buf.as_scalar<cl_filter_mode>() = s.filter_mode();
break;
default:
throw error(CL_INVALID_VALUE);
}
return CL_SUCCESS;
} catch (error &e) {
return e.get();
}

1326
src/gallium/frontends/clover/api/transfer.cpp
View file

File diff suppressed because it is too large Load diff

88
src/gallium/frontends/clover/api/util.hpp
View file

@ -1,88 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_API_UTIL_HPP
#define CLOVER_API_UTIL_HPP
#include <cassert>
#include <iostream>
#include "core/error.hpp"
#include "core/property.hpp"
#include "util/algorithm.hpp"
#include "util/detect_os.h"
#if DETECT_OS_WINDOWS
#define CLOVER_API
#define CLOVER_ICD_API
#elif HAVE_CLOVER_ICD
#define CLOVER_API
#define CLOVER_ICD_API PUBLIC
#else
#define CLOVER_API PUBLIC
#define CLOVER_ICD_API PUBLIC
#endif
#define CLOVER_NOT_SUPPORTED_UNTIL(version) \
do { \
std::cerr << "CL user error: " << __func__ \
<< "() requires OpenCL version " << (version) \
<< " or greater." << std::endl; \
} while (0)
namespace clover {
///
/// Return an error code in \a p if non-zero.
///
inline void
ret_error(cl_int *p, const clover::error &e) {
if (p)
*p = e.get();
}
///
/// Return a clover object in \a p if non-zero incrementing the
/// reference count of the object.
///
template<typename T>
void
ret_object(typename T::descriptor_type **p,
const intrusive_ref<T> &v) {
if (p) {
v().retain();
*p = desc(v());
}
}
///
/// Return an API object from an intrusive reference to a Clover object,
/// incrementing the reference count of the object.
///
template<typename T>
typename T::descriptor_type *
ret_object(const intrusive_ref<T> &v) {
v().retain();
return desc(v());
}
}
#endif

243
src/gallium/frontends/clover/core/binary.cpp
View file

@ -1,243 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include <type_traits>
#include <iostream>
#include "core/binary.hpp"
using namespace clover;
namespace {
template<typename T, typename = void>
struct _serializer;
/// Serialize the specified object.
template<typename T>
void
_proc(std::ostream &os, const T &x) {
_serializer<T>::proc(os, x);
}
/// Deserialize the specified object.
template<typename T>
void
_proc(std::istream &is, T &x) {
_serializer<T>::proc(is, x);
}
template<typename T>
T
_proc(std::istream &is) {
T x;
_serializer<T>::proc(is, x);
return x;
}
/// Calculate the size of the specified object.
template<typename T>
void
_proc(binary::size_t &sz, const T &x) {
_serializer<T>::proc(sz, x);
}
/// (De)serialize a scalar value.
template<typename T>
struct _serializer<T, typename std::enable_if<
std::is_scalar<T>::value>::type> {
static void
proc(std::ostream &os, const T &x) {
os.write(reinterpret_cast<const char *>(&x), sizeof(x));
}
static void
proc(std::istream &is, T &x) {
is.read(reinterpret_cast<char *>(&x), sizeof(x));
}
static void
proc(binary::size_t &sz, const T &x) {
sz += sizeof(x);
}
};
/// (De)serialize a vector.
template<typename T>
struct _serializer<std::vector<T>,
typename std::enable_if<
!std::is_scalar<T>::value>::type> {
static void
proc(std::ostream &os, const std::vector<T> &v) {
_proc<uint32_t>(os, v.size());
for (size_t i = 0; i < v.size(); i++)
_proc<T>(os, v[i]);
}
static void
proc(std::istream &is, std::vector<T> &v) {
v.resize(_proc<uint32_t>(is));
for (size_t i = 0; i < v.size(); i++)
new(&v[i]) T(_proc<T>(is));
}
static void
proc(binary::size_t &sz, const std::vector<T> &v) {
sz += sizeof(uint32_t);
for (size_t i = 0; i < v.size(); i++)
_proc<T>(sz, v[i]);
}
};
template<typename T>
struct _serializer<std::vector<T>,
typename std::enable_if<
std::is_scalar<T>::value>::type> {
static void
proc(std::ostream &os, const std::vector<T> &v) {
_proc<uint32_t>(os, v.size());
os.write(reinterpret_cast<const char *>(&v[0]),
v.size() * sizeof(T));
}
static void
proc(std::istream &is, std::vector<T> &v) {
v.resize(_proc<uint32_t>(is));
is.read(reinterpret_cast<char *>(&v[0]),
v.size() * sizeof(T));
}
static void
proc(binary::size_t &sz, const std::vector<T> &v) {
sz += sizeof(uint32_t) + sizeof(T) * v.size();
}
};
/// (De)serialize a string.
template<>
struct _serializer<std::string> {
static void
proc(std::ostream &os, const std::string &s) {
_proc<uint32_t>(os, s.size());
os.write(&s[0], s.size() * sizeof(std::string::value_type));
}
static void
proc(std::istream &is, std::string &s) {
s.resize(_proc<uint32_t>(is));
is.read(&s[0], s.size() * sizeof(std::string::value_type));
}
static void
proc(binary::size_t &sz, const std::string &s) {
sz += sizeof(uint32_t) + sizeof(std::string::value_type) * s.size();
}
};
/// (De)serialize a printf format
template<>
struct _serializer<binary::printf_info> {
template<typename S, typename QT>
static void
proc(S & s, QT &x) {
_proc(s, x.arg_sizes);
_proc(s, x.strings);
}
};
/// (De)serialize a binary::section.
template<>
struct _serializer<binary::section> {
template<typename S, typename QT>
static void
proc(S &s, QT &x) {
_proc(s, x.id);
_proc(s, x.type);
_proc(s, x.size);
_proc(s, x.data);
}
};
/// (De)serialize a binary::argument.
template<>
struct _serializer<binary::argument> {
template<typename S, typename QT>
static void
proc(S &s, QT &x) {
_proc(s, x.type);
_proc(s, x.size);
_proc(s, x.target_size);
_proc(s, x.target_align);
_proc(s, x.ext_type);
_proc(s, x.semantic);
}
};
/// (De)serialize a binary::symbol.
template<>
struct _serializer<binary::symbol> {
template<typename S, typename QT>
static void
proc(S &s, QT &x) {
_proc(s, x.name);
_proc(s, x.attributes);
_proc(s, x.reqd_work_group_size);
_proc(s, x.section);
_proc(s, x.offset);
_proc(s, x.args);
}
};
/// (De)serialize a binary.
template<>
struct _serializer<binary> {
template<typename S, typename QT>
static void
proc(S &s, QT &x) {
_proc(s, x.syms);
_proc(s, x.secs);
_proc(s, x.printf_infos);
_proc(s, x.printf_strings_in_buffer);
}
};
};
namespace clover {
void
binary::serialize(std::ostream &os) const {
_proc(os, *this);
}
binary
binary::deserialize(std::istream &is) {
return _proc<binary>(is);
}
binary::size_t
binary::size() const {
size_t sz = 0;
_proc(sz, *this);
return sz;
}
}

169
src/gallium/frontends/clover/core/binary.hpp
View file

@ -1,169 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_BINARY_HPP
#define CLOVER_CORE_BINARY_HPP
#include <vector>
#include <string>
#include "CL/cl.h"
namespace clover {
struct binary {
typedef uint32_t resource_id;
typedef uint32_t size_t;
struct section {
enum type {
text_intermediate,
text_library,
text_executable,
data_constant,
data_global,
data_local,
data_private
};
section(resource_id id, enum type type, size_t size,
const std::vector<char> &data) :
id(id), type(type), size(size), data(data) { }
section() : id(0), type(text_intermediate), size(0), data() { }
resource_id id;
type type;
size_t size;
std::vector<char> data;
};
struct printf_info {
std::vector<uint32_t> arg_sizes;
std::vector<char> strings;
};
struct arg_info {
arg_info(const std::string &arg_name, const std::string &type_name,
const cl_kernel_arg_type_qualifier type_qualifier,
const cl_kernel_arg_address_qualifier address_qualifier,
const cl_kernel_arg_access_qualifier access_qualifier) :
arg_name(arg_name), type_name(type_name),
type_qualifier(type_qualifier),
address_qualifier(address_qualifier),
access_qualifier(access_qualifier) { };
arg_info() : arg_name(""), type_name(""), type_qualifier(0),
address_qualifier(0), access_qualifier(0) { };
std::string arg_name;
std::string type_name;
cl_kernel_arg_type_qualifier type_qualifier;
cl_kernel_arg_address_qualifier address_qualifier;
cl_kernel_arg_access_qualifier access_qualifier;
};
struct argument {
enum type {
scalar,
constant,
global,
local,
image_rd,
image_wr,
sampler
};
enum ext_type {
zero_ext,
sign_ext
};
enum semantic {
general,
grid_dimension,
grid_offset,
image_size,
image_format,
constant_buffer,
printf_buffer
};
argument(enum type type, size_t size,
size_t target_size, size_t target_align,
enum ext_type ext_type,
enum semantic semantic = general) :
type(type), size(size),
target_size(target_size), target_align(target_align),
ext_type(ext_type), semantic(semantic) { }
argument(enum type type, size_t size) :
type(type), size(size),
target_size(size), target_align(1),
ext_type(zero_ext), semantic(general) { }
argument() : type(scalar), size(0),
target_size(0), target_align(1),
ext_type(zero_ext), semantic(general) { }
type type;
size_t size;
size_t target_size;
size_t target_align; // For arguments of type local, this represents
// the alignment requirement for the pointed
// type and for the pointer itself.
ext_type ext_type;
semantic semantic;
arg_info info;
};
struct symbol {
symbol(const std::string &name, const std::string &attributes,
const std::vector<::size_t> &reqd_work_group_size,
resource_id section, size_t offset,
const std::vector<argument> &args) :
name(name), attributes(attributes),
reqd_work_group_size(reqd_work_group_size),
section(section),
offset(offset), args(args) { }
symbol() : name(), attributes(), reqd_work_group_size({0, 0, 0}),
section(0), offset(0), args() { }
std::string name;
std::string attributes;
std::vector<::size_t> reqd_work_group_size;
resource_id section;
size_t offset;
std::vector<argument> args;
};
binary() : printf_strings_in_buffer(0) { }
void serialize(std::ostream &os) const;
static binary deserialize(std::istream &is);
size_t size() const;
std::vector<symbol> syms;
std::vector<section> secs;
std::vector<printf_info> printf_infos;
// printfs strings stored in output buffer
uint32_t printf_strings_in_buffer;
};
}
#endif

62
src/gallium/frontends/clover/core/compiler.hpp
View file

@ -1,62 +0,0 @@
//
// Copyright 2019 Red Hat, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_COMPILER_HPP
#define CLOVER_CORE_COMPILER_HPP
#include "core/device.hpp"
#include "core/binary.hpp"
#include "llvm/invocation.hpp"
namespace clover {
namespace compiler {
static inline binary
compile_program(const program &prog, const header_map &headers,
const device &dev, const std::string &opts,
std::string &log) {
switch (dev.ir_format()) {
case PIPE_SHADER_IR_NATIVE:
if (prog.il_type() == program::il_type::source)
return llvm::compile_program(prog.source(), headers, dev, opts, log);
else
throw error(CL_INVALID_VALUE);
default:
unreachable("device with unsupported IR");
throw error(CL_INVALID_VALUE);
}
}
static inline binary
link_program(const std::vector<binary> &bs, const device &dev,
const std::string &opts, std::string &log) {
switch (dev.ir_format()) {
case PIPE_SHADER_IR_NATIVE:
return llvm::link_program(bs, dev, opts, log);
default:
unreachable("device with unsupported IR");
throw error(CL_INVALID_VALUE);
}
}
}
}
#endif

92
src/gallium/frontends/clover/core/context.cpp
View file

@ -1,92 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/context.hpp"
using namespace clover;
context::context(const property_list &props,
const ref_vector<device> &devs,
const notify_action &notify) :
notify(notify), props(props), devs(devs) {
}
context::~context() {
while (_destroy_notify.size()) {
_destroy_notify.top()();
_destroy_notify.pop();
}
}
bool
context::operator==(const context &ctx) const {
return this == &ctx;
}
bool
context::operator!=(const context &ctx) const {
return this != &ctx;
}
void
context::destroy_notify(std::function<void ()> f) {
_destroy_notify.push(f);
}
const context::property_list &
context::properties() const {
return props;
}
context::device_range
context::devices() const {
return map(evals(), devs);
}
void
context::add_svm_allocation(const void *ptr, size_t size) {
svm_ptrs.emplace(ptr, size);
}
void
context::remove_svm_allocation(const void *ptr) {
svm_ptrs.erase(ptr);
}
context::svm_pointer_map::value_type
context::find_svm_allocation(const void *ptr) const {
// std::prev on an iterator of an empty container causes SIGSEGVs
if (svm_ptrs.empty())
return { nullptr, 0 };
auto it = std::prev(svm_ptrs.upper_bound(ptr));
if (it == svm_ptrs.end())
return { nullptr, 0 };
uintptr_t base = reinterpret_cast<uintptr_t>((*it).first);
uintptr_t end = (*it).second + base;
uintptr_t ptrv = reinterpret_cast<uintptr_t>(ptr);
if (ptrv >= base && ptrv < end)
return *it;
return { nullptr, 0 };
}

86
src/gallium/frontends/clover/core/context.hpp
View file

@ -1,86 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_CONTEXT_HPP
#define CLOVER_CORE_CONTEXT_HPP
#include <map>
#include <stack>
#include "core/object.hpp"
#include "core/device.hpp"
#include "core/property.hpp"
namespace clover {
class context : public ref_counter, public _cl_context {
private:
typedef adaptor_range<
evals, const std::vector<intrusive_ref<device>> &
> device_range;
typedef clover::property_list<cl_context_properties> property_list;
public:
~context();
typedef std::function<void (const char *)> notify_action;
typedef std::map<const void *, size_t> svm_pointer_map;
context(const property_list &props, const ref_vector<device> &devs,
const notify_action &notify);
context(const context &ctx) = delete;
context &
operator=(const context &ctx) = delete;
bool
operator==(const context &ctx) const;
bool
operator!=(const context &ctx) const;
void destroy_notify(std::function<void ()> f);
const property_list &
properties() const;
device_range
devices() const;
void
add_svm_allocation(const void *ptr, size_t size);
void
remove_svm_allocation(const void *ptr);
svm_pointer_map::value_type
find_svm_allocation(const void *ptr) const;
const notify_action notify;
private:
property_list props;
const std::vector<intrusive_ref<device>> devs;
std::stack<std::function<void ()>> _destroy_notify;
svm_pointer_map svm_ptrs;
};
}
#endif

559
src/gallium/frontends/clover/core/device.cpp
View file

@ -1,559 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include <algorithm>
#include "core/device.hpp"
#include "core/platform.hpp"
#include "pipe/p_screen.h"
#include "pipe/p_state.h"
#include "util/bitscan.h"
#include "util/disk_cache.h"
#include "util/u_debug.h"
#include "nir.h"
#include <fstream>
using namespace clover;
namespace {
cl_version
get_highest_supported_version(const device &dev) {
// All the checks below assume that the device supports FULL_PROFILE
// (which is the only profile support by clover) and that a device is
// not CUSTOM.
assert(dev.type() != CL_DEVICE_TYPE_CUSTOM);
cl_version version = CL_MAKE_VERSION(0, 0, 0);
const auto has_extension =
[extensions = dev.supported_extensions()](const char *extension_name){
return std::find_if(extensions.begin(), extensions.end(),
[extension_name](const cl_name_version &extension){
return strcmp(extension.name, extension_name) == 0;
}) != extensions.end();
};
const bool supports_images = dev.image_support();
// Check requirements for OpenCL 1.0
if (dev.max_compute_units() < 1 ||
dev.max_block_size().size() < 3 ||
// TODO: Check CL_DEVICE_MAX_WORK_ITEM_SIZES
dev.max_threads_per_block() < 1 ||
(dev.address_bits() != 32 && dev.address_bits() != 64) ||
dev.max_mem_alloc_size() < std::max(dev.max_mem_global() / 4,
(cl_ulong)128 * 1024 * 1024) ||
dev.max_mem_input() < 256 ||
dev.max_const_buffer_size() < 64 * 1024 ||
dev.max_const_buffers() < 8 ||
dev.max_mem_local() < 16 * 1024 ||
dev.clc_version < CL_MAKE_VERSION(1, 0, 0)) {
return version;
}
version = CL_MAKE_VERSION(1, 0, 0);
// Check requirements for OpenCL 1.1
if (!has_extension("cl_khr_byte_addressable_store") ||
!has_extension("cl_khr_global_int32_base_atomics") ||
!has_extension("cl_khr_global_int32_extended_atomics") ||
!has_extension("cl_khr_local_int32_base_atomics") ||
!has_extension("cl_khr_local_int32_extended_atomics") ||
// OpenCL 1.1 increased the minimum value for
// CL_DEVICE_MAX_PARAMETER_SIZE to 1024 bytes.
dev.max_mem_input() < 1024 ||
dev.mem_base_addr_align() < sizeof(cl_long16) ||
// OpenCL 1.1 increased the minimum value for
// CL_DEVICE_LOCAL_MEM_SIZE to 32 KB.
dev.max_mem_local() < 32 * 1024 ||
dev.clc_version < CL_MAKE_VERSION(1, 1, 0)) {
return version;
}
version = CL_MAKE_VERSION(1, 1, 0);
// Check requirements for OpenCL 1.2
if ((dev.has_doubles() && !has_extension("cl_khr_fp64")) ||
dev.clc_version < CL_MAKE_VERSION(1, 2, 0) ||
dev.max_printf_buffer_size() < 1 * 1024 * 1024 ||
(supports_images &&
(dev.max_image_buffer_size() < 65536 ||
dev.max_image_array_number() < 2048))) {
return version;
}
version = CL_MAKE_VERSION(1, 2, 0);
// Check requirements for OpenCL 3.0
if (dev.max_mem_alloc_size() < std::max(std::min((cl_ulong)1024 * 1024 * 1024,
dev.max_mem_global() / 4),
(cl_ulong)128 * 1024 * 1024) ||
// TODO: If pipes are supported, check:
// * CL_DEVICE_MAX_PIPE_ARGS
// * CL_DEVICE_PIPE_MAX_ACTIVE_RESERVATIONS
// * CL_DEVICE_PIPE_MAX_PACKET_SIZE
// TODO: If on-device queues are supported, check:
// * CL_DEVICE_QUEUE_ON_DEVICE_PROPERTIES
// * CL_DEVICE_QUEUE_ON_DEVICE_PREFERRED_SIZE
// * CL_DEVICE_QUEUE_ON_DEVICE_MAX_SIZE
// * CL_DEVICE_MAX_ON_DEVICE_QUEUES
// * CL_DEVICE_MAX_ON_DEVICE_EVENTS
dev.clc_version < CL_MAKE_VERSION(3, 0, 0) ||
(supports_images &&
(dev.max_images_write() < 64 ||
dev.max_image_size() < 16384))) {
return version;
}
version = CL_MAKE_VERSION(3, 0, 0);
return version;
}
static cl_device_type
parse_env_device_type() {
const char* val = getenv("CLOVER_DEVICE_TYPE");
if (!val) {
return 0;
}
if (strcmp(val, "cpu") == 0) {
return CL_DEVICE_TYPE_CPU;
}
if (strcmp(val, "gpu") == 0) {
return CL_DEVICE_TYPE_GPU;
}
if (strcmp(val, "accelerator") == 0) {
return CL_DEVICE_TYPE_ACCELERATOR;
}
/* CL_DEVICE_TYPE_CUSTOM isn't implemented
because CL_DEVICE_TYPE_CUSTOM is OpenCL 1.2
and Clover is OpenCL 1.1. */
return 0;
}
}
device::device(clover::platform &platform, pipe_loader_device *ldev) :
platform(platform), clc_cache(NULL), ldev(ldev) {
pipe = pipe_loader_create_screen(ldev, false);
if (pipe && pipe->caps.compute) {
const bool has_supported_ir = supports_ir(PIPE_SHADER_IR_NATIVE);
if (has_supported_ir) {
unsigned major = 1, minor = 1;
debug_get_version_option("CLOVER_DEVICE_CLC_VERSION_OVERRIDE",
&major, &minor);
clc_version = CL_MAKE_VERSION(major, minor, 0);
version = get_highest_supported_version(*this);
major = CL_VERSION_MAJOR(version);
minor = CL_VERSION_MINOR(version);
debug_get_version_option("CLOVER_DEVICE_VERSION_OVERRIDE", &major,
&minor);
version = CL_MAKE_VERSION(major, minor, 0);
}
if (supports_ir(PIPE_SHADER_IR_NATIVE))
return;
}
if (pipe)
pipe->destroy(pipe);
throw error(CL_INVALID_DEVICE);
}
device::~device() {
if (clc_cache)
disk_cache_destroy(clc_cache);
if (pipe)
pipe->destroy(pipe);
if (ldev)
pipe_loader_release(&ldev, 1);
}
bool
device::operator==(const device &dev) const {
return this == &dev;
}
cl_device_type
device::type() const {
cl_device_type type = parse_env_device_type();
if (type != 0) {
return type;
}
switch (ldev->type) {
case PIPE_LOADER_DEVICE_SOFTWARE:
return CL_DEVICE_TYPE_CPU;
case PIPE_LOADER_DEVICE_PCI:
case PIPE_LOADER_DEVICE_PLATFORM:
return CL_DEVICE_TYPE_GPU;
default:
unreachable("Unknown device type.");
}
}
cl_uint
device::vendor_id() const {
switch (ldev->type) {
case PIPE_LOADER_DEVICE_SOFTWARE:
case PIPE_LOADER_DEVICE_PLATFORM:
return 0;
case PIPE_LOADER_DEVICE_PCI:
return ldev->u.pci.vendor_id;
default:
unreachable("Unknown device type.");
}
}
size_t
device::max_images_read() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].max_sampler_views;
}
size_t
device::max_images_write() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].max_shader_images;
}
size_t
device::max_image_buffer_size() const {
return pipe->caps.max_texel_buffer_elements;
}
cl_uint
device::max_image_size() const {
return pipe->caps.max_texture_2d_size;
}
cl_uint
device::max_image_size_3d() const {
return 1 << (pipe->caps.max_texture_3d_levels - 1);
}
size_t
device::max_image_array_number() const {
return pipe->caps.max_texture_array_layers;
}
cl_uint
device::max_samplers() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].max_texture_samplers;
}
cl_ulong
device::max_mem_global() const {
return pipe->compute_caps.max_global_size;
}
cl_ulong
device::max_mem_local() const {
return pipe->compute_caps.max_local_size;
}
cl_ulong
device::max_mem_input() const {
return pipe->compute_caps.max_input_size;
}
cl_ulong
device::max_const_buffer_size() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].max_const_buffer0_size;
}
cl_uint
device::max_const_buffers() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].max_const_buffers;
}
size_t
device::max_threads_per_block() const {
return pipe->compute_caps.max_threads_per_block_clover;
}
cl_ulong
device::max_mem_alloc_size() const {
return pipe->compute_caps.max_mem_alloc_size;
}
cl_uint
device::max_clock_frequency() const {
return pipe->compute_caps.max_clock_frequency;
}
cl_uint
device::max_compute_units() const {
return pipe->compute_caps.max_compute_units;
}
cl_uint
device::max_printf_buffer_size() const {
return 1024 * 1024;
}
bool
device::image_support() const {
bool supports_images = pipe->compute_caps.images_supported;
if (!supports_images)
return false;
/* If the gallium driver supports images, but does not support the
* minimum requirements for opencl 1.0 images, then don't claim to
* support images.
*/
if (max_images_read() < 128 ||
max_images_write() < 8 ||
max_image_size() < 8192 ||
max_image_size_3d() < 2048 ||
max_samplers() < 16)
return false;
return true;
}
bool
device::has_doubles() const {
nir_shader_compiler_options *options =
(nir_shader_compiler_options *)pipe->get_compiler_options(pipe,
PIPE_SHADER_IR_NIR,
PIPE_SHADER_COMPUTE);
return pipe->caps.doubles &&
!(options->lower_doubles_options & nir_lower_fp64_full_software);
}
bool
device::has_halves() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].fp16;
}
bool
device::has_int64_atomics() const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].int64_atomics;
}
bool
device::has_unified_memory() const {
return pipe->caps.uma;
}
size_t
device::mem_base_addr_align() const {
uint64_t page_size = 0;
os_get_page_size(&page_size);
return std::max((size_t)page_size, sizeof(cl_long) * 16);
}
cl_device_svm_capabilities
device::svm_support() const {
// Without CAP_RESOURCE_FROM_USER_MEMORY SVM and CL_MEM_USE_HOST_PTR
// interactions won't work according to spec as clover manages a GPU side
// copy of the host data.
//
// The biggest problem are memory buffers created with CL_MEM_USE_HOST_PTR,
// but the application and/or the kernel updates the memory via SVM and not
// the cl_mem buffer.
// We can't even do proper tracking on what memory might have been accessed
// as the host ptr to the buffer could be within a SVM region, where through
// the CL API there is no reliable way of knowing if a certain cl_mem buffer
// was accessed by a kernel or not and the runtime can't reliably know from
// which side the GPU buffer content needs to be updated.
//
// Another unsolvable scenario is a cl_mem object passed by cl_mem reference
// and SVM pointer into the same kernel at the same time.
if (allows_user_pointers() && pipe->caps.system_svm)
// we can emulate all lower levels if we support fine grain system
return CL_DEVICE_SVM_FINE_GRAIN_SYSTEM |
CL_DEVICE_SVM_COARSE_GRAIN_BUFFER |
CL_DEVICE_SVM_FINE_GRAIN_BUFFER;
return 0;
}
bool
device::allows_user_pointers() const {
return pipe->caps.resource_from_user_memory ||
pipe->caps.resource_from_user_memory_compute_only;
}
std::vector<size_t>
device::max_block_size() const {
auto v = pipe->compute_caps.max_block_size_clover;
return {v[0], v[1], v[2]};
}
cl_uint
device::subgroup_size() const {
cl_uint subgroup_sizes = pipe->compute_caps.subgroup_sizes;
if (!subgroup_sizes)
return 0;
return 1 << (util_last_bit(subgroup_sizes) - 1);
}
cl_uint
device::address_bits() const {
return pipe->compute_caps.address_bits;
}
std::string
device::device_name() const {
return pipe->get_name(pipe);
}
std::string
device::vendor_name() const {
return pipe->get_device_vendor(pipe);
}
enum pipe_shader_ir
device::ir_format() const {
assert(supports_ir(PIPE_SHADER_IR_NATIVE));
return PIPE_SHADER_IR_NATIVE;
}
std::string
device::ir_target() const {
return pipe->compute_caps.ir_target;
}
enum pipe_endian
device::endianness() const {
return pipe->caps.endianness;
}
std::string
device::device_version_as_string() const {
static const std::string version_string =
std::to_string(CL_VERSION_MAJOR(version)) + "." +
std::to_string(CL_VERSION_MINOR(version));
return version_string;
}
std::string
device::device_clc_version_as_string() const {
int major = CL_VERSION_MAJOR(clc_version);
int minor = CL_VERSION_MINOR(clc_version);
/* for CL 3.0 we need this to be 1.2 until we support 2.0. */
if (major == 3) {
major = 1;
minor = 2;
}
static const std::string version_string =
std::to_string(major) + "." +
std::to_string(minor);
return version_string;
}
bool
device::supports_ir(enum pipe_shader_ir ir) const {
return pipe->shader_caps[PIPE_SHADER_COMPUTE].supported_irs & (1 << ir);
}
std::vector<cl_name_version>
device::supported_extensions() const {
std::vector<cl_name_version> vec;
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_byte_addressable_store" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_global_int32_base_atomics" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_global_int32_extended_atomics" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_local_int32_base_atomics" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_local_int32_extended_atomics" } );
if (has_int64_atomics()) {
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_int64_base_atomics" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_int64_extended_atomics" } );
}
if (has_doubles())
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_fp64" } );
if (has_halves())
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_fp16" } );
if (svm_support())
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_arm_shared_virtual_memory" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_extended_versioning" } );
return vec;
}
std::string
device::supported_extensions_as_string() const {
static std::string extensions_string;
if (!extensions_string.empty())
return extensions_string;
const auto extension_list = supported_extensions();
for (const auto &extension : extension_list) {
if (!extensions_string.empty())
extensions_string += " ";
extensions_string += extension.name;
}
return extensions_string;
}
std::vector<cl_name_version>
device::supported_il_versions() const {
return {};
}
const void *
device::get_compiler_options(enum pipe_shader_ir ir) const {
return pipe->get_compiler_options(pipe, ir, PIPE_SHADER_COMPUTE);
}
cl_version
device::device_version() const {
return version;
}
cl_version
device::device_clc_version(bool api) const {
/*
* For the API we have to limit this to 1.2,
* but internally we want 3.0 if it works.
*/
if (!api)
return clc_version;
int major = CL_VERSION_MAJOR(clc_version);
/* for CL 3.0 we need this to be 1.2 until we support 2.0. */
if (major == 3) {
return CL_MAKE_VERSION(1, 2, 0);
}
return clc_version;
}
std::vector<cl_name_version>
device::opencl_c_all_versions() const {
std::vector<cl_name_version> vec;
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "OpenCL C" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 1, 0), "OpenCL C" } );
if (CL_VERSION_MAJOR(clc_version) == 1 &&
CL_VERSION_MINOR(clc_version) == 2)
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 2, 0), "OpenCL C" } );
if (CL_VERSION_MAJOR(clc_version) == 3) {
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 2, 0), "OpenCL C" } );
vec.push_back( cl_name_version{ CL_MAKE_VERSION(3, 0, 0), "OpenCL C" } );
}
return vec;
}
std::vector<cl_name_version>
device::opencl_c_features() const {
std::vector<cl_name_version> vec;
vec.push_back( cl_name_version {CL_MAKE_VERSION(3, 0, 0), "__opencl_c_int64" });
if (has_doubles())
vec.push_back( cl_name_version {CL_MAKE_VERSION(3, 0, 0), "__opencl_c_fp64" });
return vec;
}

129
src/gallium/frontends/clover/core/device.hpp
View file

@ -1,129 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_DEVICE_HPP
#define CLOVER_CORE_DEVICE_HPP
#include <set>
#include <vector>
#include "core/object.hpp"
#include "core/format.hpp"
#include "core/binary.hpp"
#include "util/lazy.hpp"
#include "pipe-loader/pipe_loader.h"
struct nir_shader;
struct disk_cache;
namespace clover {
class platform;
class root_resource;
class hard_event;
class device : public ref_counter, public _cl_device_id {
public:
device(clover::platform &platform, pipe_loader_device *ldev);
~device();
device(const device &dev) = delete;
device &
operator=(const device &dev) = delete;
bool
operator==(const device &dev) const;
cl_device_type type() const;
cl_uint vendor_id() const;
size_t max_images_read() const;
size_t max_images_write() const;
size_t max_image_buffer_size() const;
// Use for 1D and 2D images.
cl_uint max_image_size() const;
// Use for 3D images.
cl_uint max_image_size_3d() const;
size_t max_image_array_number() const;
cl_uint max_samplers() const;
cl_ulong max_mem_global() const;
cl_ulong max_mem_local() const;
cl_ulong max_mem_input() const;
cl_ulong max_const_buffer_size() const;
cl_uint max_const_buffers() const;
size_t max_threads_per_block() const;
cl_ulong max_mem_alloc_size() const;
cl_uint max_clock_frequency() const;
cl_uint max_compute_units() const;
cl_uint max_printf_buffer_size() const;
bool image_support() const;
bool has_doubles() const;
bool has_halves() const;
bool has_int64_atomics() const;
bool has_unified_memory() const;
size_t mem_base_addr_align() const;
cl_device_svm_capabilities svm_support() const;
bool allows_user_pointers() const;
std::vector<size_t> max_block_size() const;
cl_uint subgroup_size() const;
cl_uint address_bits() const;
std::string device_name() const;
std::string vendor_name() const;
std::string device_version_as_string() const;
std::string device_clc_version_as_string() const;
enum pipe_shader_ir ir_format() const;
std::string ir_target() const;
enum pipe_endian endianness() const;
bool supports_ir(enum pipe_shader_ir ir) const;
std::string supported_extensions_as_string() const;
cl_version device_version() const;
cl_version device_clc_version(bool api = false) const;
std::vector<cl_name_version> opencl_c_all_versions() const;
std::vector<cl_name_version> supported_extensions() const;
std::vector<cl_name_version> supported_il_versions() const;
std::vector<cl_name_version> opencl_c_features() const;
friend class command_queue;
friend class root_resource;
friend class hard_event;
friend std::set<cl_image_format>
supported_formats(const context &, cl_mem_object_type, cl_mem_flags flags);
const void *get_compiler_options(enum pipe_shader_ir ir) const;
clover::platform &platform;
inline bool
has_system_svm() const {
return svm_support() & CL_DEVICE_SVM_FINE_GRAIN_SYSTEM;
}
lazy<std::shared_ptr<nir_shader>> clc_nir;
disk_cache *clc_cache;
cl_version version;
cl_version clc_version;
private:
pipe_screen *pipe;
pipe_loader_device *ldev;
};
}
#endif

207
src/gallium/frontends/clover/core/error.hpp
View file

@ -1,207 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_ERROR_HPP
#define CLOVER_CORE_ERROR_HPP
#include "CL/cl.h"
#if defined(__ALTIVEC__) && !defined(__APPLE_ALTIVEC__)
#undef vector
#undef pixel
#undef bool
#endif
#include <stdexcept>
#include <string>
namespace clover {
class command_queue;
class context;
class device;
class event;
class hard_event;
class soft_event;
class kernel;
class memory_obj;
class buffer;
class root_buffer;
class sub_buffer;
class image;
class image2d;
class image3d;
class platform;
class program;
class sampler;
///
/// Class that represents an error that can be converted to an
/// OpenCL status code.
///
class error : public std::runtime_error {
public:
error(cl_int code, std::string what = "") :
std::runtime_error(what), code(code) {
}
cl_int get() const {
return code;
}
protected:
cl_int code;
};
class invalid_build_options_error : public error {
public:
invalid_build_options_error(const std::string &what = "") :
error(CL_INVALID_BUILD_OPTIONS, what) {}
};
class build_error : public error {
public:
build_error(const std::string &what = "") :
error(CL_BUILD_PROGRAM_FAILURE, what) {}
};
template<typename O>
class invalid_object_error;
template<>
class invalid_object_error<command_queue> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_COMMAND_QUEUE, what) {}
};
template<>
class invalid_object_error<context> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_CONTEXT, what) {}
};
template<>
class invalid_object_error<device> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_DEVICE, what) {}
};
template<>
class invalid_object_error<event> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_EVENT, what) {}
};
template<>
class invalid_object_error<soft_event> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_EVENT, what) {}
};
template<>
class invalid_object_error<kernel> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_KERNEL, what) {}
};
template<>
class invalid_object_error<memory_obj> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<buffer> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<root_buffer> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<sub_buffer> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<image> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<image2d> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<image3d> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_MEM_OBJECT, what) {}
};
template<>
class invalid_object_error<platform> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_PLATFORM, what) {}
};
template<>
class invalid_object_error<program> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_PROGRAM, what) {}
};
template<>
class invalid_object_error<sampler> : public error {
public:
invalid_object_error(std::string what = "") :
error(CL_INVALID_SAMPLER, what) {}
};
class invalid_wait_list_error : public error {
public:
invalid_wait_list_error(std::string what = "") :
error(CL_INVALID_EVENT_WAIT_LIST, what) {}
};
}
#endif

272
src/gallium/frontends/clover/core/event.cpp
View file

@ -1,272 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/event.hpp"
#include "pipe/p_screen.h"
using namespace clover;
event::event(clover::context &ctx, const ref_vector<event> &deps,
action action_ok, action action_fail) :
context(ctx), _wait_count(1), _status(0),
action_ok(action_ok), action_fail(action_fail) {
for (auto &ev : deps)
ev.chain(*this);
}
event::~event() {
}
std::vector<intrusive_ref<event>>
event::trigger_self() {
std::lock_guard<std::mutex> lock(mutex);
std::vector<intrusive_ref<event>> evs;
if (_wait_count && !--_wait_count)
std::swap(_chain, evs);
cv.notify_all();
return evs;
}
void
event::trigger() try {
if (wait_count() == 1)
action_ok(*this);
for (event &ev : trigger_self())
ev.trigger();
} catch (error &e) {
abort(e.get());
}
std::vector<intrusive_ref<event>>
event::abort_self(cl_int status) {
std::lock_guard<std::mutex> lock(mutex);
std::vector<intrusive_ref<event>> evs;
_status = status;
_wait_count = 0;
std::swap(_chain, evs);
cv.notify_all();
return evs;
}
void
event::abort(cl_int status) {
action_fail(*this);
for (event &ev : abort_self(status))
ev.abort(status);
}
unsigned
event::wait_count() const {
std::lock_guard<std::mutex> lock(mutex);
return _wait_count;
}
bool
event::signalled() const {
return !wait_count();
}
cl_int
event::status() const {
std::lock_guard<std::mutex> lock(mutex);
return _status;
}
void
event::chain(event &ev) {
std::unique_lock<std::mutex> lock(mutex, std::defer_lock);
std::unique_lock<std::mutex> lock_ev(ev.mutex, std::defer_lock);
std::lock(lock, lock_ev);
if (_wait_count) {
ev._wait_count++;
_chain.push_back(ev);
}
ev.deps.push_back(*this);
}
void
event::wait_signalled() const {
std::unique_lock<std::mutex> lock(mutex);
cv.wait(lock, [=]{ return !_wait_count; });
}
void
event::wait() const {
std::vector<intrusive_ref<event>> evs;
std::swap(deps, evs);
for (event &ev : evs)
ev.wait();
wait_signalled();
}
hard_event::hard_event(command_queue &q, cl_command_type command,
const ref_vector<event> &deps, action action) :
event(q.context(), deps, profile(q, action), [](event &ev){}),
_queue(q), _command(command), _fence(NULL) {
if (q.profiling_enabled())
_time_queued = timestamp::current(q);
q.sequence(*this);
trigger();
}
hard_event::~hard_event() {
pipe_screen *screen = queue()->device().pipe;
screen->fence_reference(screen, &_fence, NULL);
}
cl_int
hard_event::status() const {
pipe_screen *screen = queue()->device().pipe;
if (event::status() < 0)
return event::status();
else if (!_fence)
return CL_QUEUED;
else if (!screen->fence_finish(screen, NULL, _fence, 0))
return CL_SUBMITTED;
else
return CL_COMPLETE;
}
command_queue *
hard_event::queue() const {
return &_queue();
}
cl_command_type
hard_event::command() const {
return _command;
}
void
hard_event::wait() const {
pipe_screen *screen = queue()->device().pipe;
event::wait();
if (status() == CL_QUEUED)
queue()->flush();
if (!_fence ||
!screen->fence_finish(screen, NULL, _fence, OS_TIMEOUT_INFINITE))
throw error(CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST);
}
const lazy<cl_ulong> &
hard_event::time_queued() const {
return _time_queued;
}
const lazy<cl_ulong> &
hard_event::time_submit() const {
return _time_submit;
}
const lazy<cl_ulong> &
hard_event::time_start() const {
return _time_start;
}
const lazy<cl_ulong> &
hard_event::time_end() const {
return _time_end;
}
void
hard_event::fence(pipe_fence_handle *fence) {
assert(fence);
pipe_screen *screen = queue()->device().pipe;
screen->fence_reference(screen, &_fence, fence);
deps.clear();
}
event::action
hard_event::profile(command_queue &q, const action &action) const {
if (q.profiling_enabled()) {
return [&q, action] (event &ev) {
auto &hev = static_cast<hard_event &>(ev);
hev._time_submit = timestamp::current(q);
hev._time_start = timestamp::query(q);
action(ev);
hev._time_end = timestamp::query(q);
};
} else {
return action;
}
}
soft_event::soft_event(clover::context &ctx, const ref_vector<event> &deps,
bool _trigger, action action) :
event(ctx, deps, action, action) {
if (_trigger)
trigger();
}
cl_int
soft_event::status() const {
if (event::status() < 0)
return event::status();
else if (!signalled() ||
any_of([](const event &ev) {
return ev.status() != CL_COMPLETE;
}, deps))
return CL_SUBMITTED;
else
return CL_COMPLETE;
}
command_queue *
soft_event::queue() const {
return NULL;
}
cl_command_type
soft_event::command() const {
return CL_COMMAND_USER;
}
void
soft_event::wait() const {
event::wait();
if (status() != CL_COMPLETE)
throw error(CL_EXEC_STATUS_ERROR_FOR_EVENTS_IN_WAIT_LIST);
}

164
src/gallium/frontends/clover/core/event.hpp
View file

@ -1,164 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_EVENT_HPP
#define CLOVER_CORE_EVENT_HPP
#include <condition_variable>
#include <functional>
#include "core/object.hpp"
#include "core/queue.hpp"
#include "core/timestamp.hpp"
#include "util/lazy.hpp"
namespace clover {
///
/// Class that represents a task that might be executed
/// asynchronously at some point in the future.
///
/// An event consists of a list of dependencies, a boolean
/// signalled() flag, and an associated task. An event is
/// considered signalled as soon as all its dependencies (if any)
/// are signalled as well, and the trigger() method is called; at
/// that point the associated task will be started through the
/// specified \a action_ok. If the abort() method is called
/// instead, the specified \a action_fail is executed and the
/// associated task will never be started. Dependent events will
/// be aborted recursively.
///
/// The execution status of the associated task can be queried
/// using the status() method, and it can be waited for completion
/// using the wait() method.
///
class event : public ref_counter, public _cl_event {
public:
typedef std::function<void (event &)> action;
event(clover::context &ctx, const ref_vector<event> &deps,
action action_ok, action action_fail);
virtual ~event();
event(const event &ev) = delete;
event &
operator=(const event &ev) = delete;
void trigger();
void abort(cl_int status);
bool signalled() const;
virtual cl_int status() const;
virtual command_queue *queue() const = 0;
virtual cl_command_type command() const = 0;
void wait_signalled() const;
virtual void wait() const;
virtual struct pipe_fence_handle *fence() const {
return NULL;
}
const intrusive_ref<clover::context> context;
protected:
void chain(event &ev);
mutable std::vector<intrusive_ref<event>> deps;
private:
std::vector<intrusive_ref<event>> trigger_self();
std::vector<intrusive_ref<event>> abort_self(cl_int status);
unsigned wait_count() const;
unsigned _wait_count;
cl_int _status;
action action_ok;
action action_fail;
std::vector<intrusive_ref<event>> _chain;
mutable std::condition_variable cv;
mutable std::mutex mutex;
};
///
/// Class that represents a task executed by a command queue.
///
/// Similar to a normal clover::event. In addition it's associated
/// with a given command queue \a q and a given OpenCL \a command.
/// hard_event instances created for the same queue are implicitly
/// ordered with respect to each other, and they are implicitly
/// triggered on construction.
///
/// A hard_event is considered complete when the associated
/// hardware task finishes execution.
///
class hard_event : public event {
public:
hard_event(command_queue &q, cl_command_type command,
const ref_vector<event> &deps,
action action = [](event &){});
~hard_event();
virtual cl_int status() const;
virtual command_queue *queue() const;
virtual cl_command_type command() const;
virtual void wait() const;
const lazy<cl_ulong> &time_queued() const;
const lazy<cl_ulong> &time_submit() const;
const lazy<cl_ulong> &time_start() const;
const lazy<cl_ulong> &time_end() const;
friend class command_queue;
virtual struct pipe_fence_handle *fence() const {
return _fence;
}
private:
virtual void fence(pipe_fence_handle *fence);
action profile(command_queue &q, const action &action) const;
const intrusive_ref<command_queue> _queue;
cl_command_type _command;
pipe_fence_handle *_fence;
lazy<cl_ulong> _time_queued, _time_submit, _time_start, _time_end;
};
///
/// Class that represents a software event.
///
/// A soft_event is not associated with any specific hardware task
/// or command queue. It's considered complete as soon as all its
/// dependencies finish execution.
///
class soft_event : public event {
public:
soft_event(clover::context &ctx, const ref_vector<event> &deps,
bool trigger, action action = [](event &){});
virtual cl_int status() const;
virtual command_queue *queue() const;
virtual cl_command_type command() const;
virtual void wait() const;
};
}
#endif

149
src/gallium/frontends/clover/core/format.cpp
View file

@ -1,149 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/format.hpp"
#include "core/memory.hpp"
#include "pipe/p_screen.h"
#include "pipe/p_context.h"
namespace clover {
// see table 16 and 17 in the 3.0 CL spec under "5.3.1.1. Image Format Descriptor"
// TODO optional channel orders:
// * CL_Rx
// * CL_RGx
// * CL_RGBx
// * CL_sRGBx
#define _FF(c, b, g) \
{ { CL_R, c }, PIPE_FORMAT_R##b##_##g }, \
{ { CL_A, c }, PIPE_FORMAT_A##b##_##g }, \
{ { CL_RG, c }, PIPE_FORMAT_R##b##G##b##_##g }, \
{ { CL_RA, c }, PIPE_FORMAT_R##b##A##b##_##g }, \
{ { CL_RGB, c }, PIPE_FORMAT_R##b##G##b##B##b##_##g }, \
{ { CL_RGBA, c }, PIPE_FORMAT_R##b##G##b##B##b##A##b##_##g }
// broken but also optional
//{ { CL_LUMINANCE, c }, PIPE_FORMAT_L##b##_##g },
//{ { CL_INTENSITY, c }, PIPE_FORMAT_I##b##_##g },
#define _FI(c, b, g) \
_FF(c##b, b, g)
static const std::map<cl_image_format, pipe_format> formats {
//required in CL 2.0 but broken
//_FI(CL_SNORM_INT, 8, SNORM),
//_FI(CL_SNORM_INT, 16, SNORM),
_FI(CL_UNORM_INT, 8, UNORM),
_FI(CL_UNORM_INT, 16, UNORM),
_FI(CL_SIGNED_INT, 8, SINT),
_FI(CL_SIGNED_INT, 16, SINT),
_FI(CL_SIGNED_INT, 32, SINT),
_FI(CL_UNSIGNED_INT, 8, UINT),
_FI(CL_UNSIGNED_INT, 16, UINT),
_FI(CL_UNSIGNED_INT, 32, UINT),
_FF(CL_HALF_FLOAT, 16, FLOAT),
_FF(CL_FLOAT, 32, FLOAT),
// TODO: next three can be CL_RGBx as well
{ { CL_RGB, CL_UNORM_SHORT_565 }, PIPE_FORMAT_B5G6R5_UNORM },
{ { CL_RGB, CL_UNORM_SHORT_555 }, PIPE_FORMAT_B5G5R5A1_UNORM },
{ { CL_RGB, CL_UNORM_INT_101010 }, PIPE_FORMAT_B10G10R10X2_UNORM },
{ { CL_RGBA, CL_UNORM_INT_101010_2 }, PIPE_FORMAT_B10G10R10A2_UNORM },
{ { CL_ARGB, CL_UNORM_INT8 }, PIPE_FORMAT_A8R8G8B8_UNORM },
{ { CL_ARGB, CL_UNSIGNED_INT8 }, PIPE_FORMAT_A8R8G8B8_UINT },
{ { CL_BGRA, CL_SNORM_INT8 }, PIPE_FORMAT_B8G8R8A8_SNORM },
{ { CL_BGRA, CL_UNORM_INT8 }, PIPE_FORMAT_B8G8R8A8_UNORM },
{ { CL_BGRA, CL_SIGNED_INT8 }, PIPE_FORMAT_B8G8R8A8_SINT },
{ { CL_BGRA, CL_UNSIGNED_INT8 }, PIPE_FORMAT_B8G8R8A8_UINT },
{ { CL_ABGR, CL_SNORM_INT8 }, PIPE_FORMAT_A8B8G8R8_SNORM },
{ { CL_ABGR, CL_UNORM_INT8 }, PIPE_FORMAT_A8B8G8R8_UNORM },
{ { CL_ABGR, CL_SIGNED_INT8 }, PIPE_FORMAT_A8B8G8R8_SINT },
{ { CL_ABGR, CL_UNSIGNED_INT8 }, PIPE_FORMAT_A8B8G8R8_UINT },
// disable for now as it needs CL C 2.0 support
//{ { CL_DEPTH, CL_UNORM_INT16 }, PIPE_FORMAT_Z16_UNORM },
//{ { CL_DEPTH, CL_FLOAT }, PIPE_FORMAT_Z32_FLOAT },
// required in CL 2.0 but broken
//{ { CL_sRGBA, CL_UNORM_INT8 }, PIPE_FORMAT_R8G8B8A8_SRGB },
// optional but broken
//{ { CL_sRGB, CL_UNORM_INT8 }, PIPE_FORMAT_R8G8B8_SRGB },
//{ { CL_sBGRA, CL_UNORM_INT8 }, PIPE_FORMAT_B8G8R8A8_SRGB },
};
#undef _FF
#undef _FI
pipe_texture_target
translate_target(cl_mem_object_type type) {
switch (type) {
case CL_MEM_OBJECT_BUFFER:
case CL_MEM_OBJECT_IMAGE1D_BUFFER:
return PIPE_BUFFER;
case CL_MEM_OBJECT_IMAGE1D:
return PIPE_TEXTURE_1D;
case CL_MEM_OBJECT_IMAGE2D:
return PIPE_TEXTURE_2D;
case CL_MEM_OBJECT_IMAGE3D:
return PIPE_TEXTURE_3D;
case CL_MEM_OBJECT_IMAGE1D_ARRAY:
return PIPE_TEXTURE_1D_ARRAY;
case CL_MEM_OBJECT_IMAGE2D_ARRAY:
return PIPE_TEXTURE_2D_ARRAY;
default:
throw error(CL_INVALID_VALUE);
}
}
pipe_format
translate_format(const cl_image_format &format) {
auto it = formats.find(format);
if (it == formats.end())
throw error(CL_IMAGE_FORMAT_NOT_SUPPORTED);
return it->second;
}
std::set<cl_image_format>
supported_formats(const context &ctx, cl_mem_object_type type, cl_mem_flags flags) {
std::set<cl_image_format> s;
pipe_texture_target target = translate_target(type);
unsigned bindings = 0;
if (flags & (CL_MEM_READ_ONLY | CL_MEM_READ_WRITE | CL_MEM_KERNEL_READ_AND_WRITE))
bindings |= PIPE_BIND_SAMPLER_VIEW;
if (flags & (CL_MEM_WRITE_ONLY | CL_MEM_READ_WRITE | CL_MEM_KERNEL_READ_AND_WRITE))
bindings |= PIPE_BIND_SHADER_IMAGE;
for (auto f : formats) {
if (all_of([=](const device &dev) {
return dev.pipe->is_format_supported(
dev.pipe, f.second, target, 1, 1, bindings);
}, ctx.devices()))
s.insert(f.first);
}
return s;
}
}

63
src/gallium/frontends/clover/core/format.hpp
View file

@ -1,63 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_FORMAT_HPP
#define CLOVER_CORE_FORMAT_HPP
#include <set>
#include "core/object.hpp"
#include "pipe/p_defines.h"
#include "util/format/u_formats.h"
namespace clover {
pipe_texture_target translate_target(cl_mem_object_type type);
pipe_format translate_format(const cl_image_format &format);
///
/// Return all the image formats supported by a given context for
/// the given memory object type.
///
std::set<cl_image_format> supported_formats(const context &ctx,
cl_mem_object_type type,
cl_mem_flags flags);
}
static inline bool
operator<(const cl_image_format &a, const cl_image_format &b) {
return (a.image_channel_order != b.image_channel_order ?
a.image_channel_order < b.image_channel_order :
a.image_channel_data_type < b.image_channel_data_type);
}
static inline bool
operator==(const cl_image_format &a, const cl_image_format &b) {
return (a.image_channel_order == b.image_channel_order &&
a.image_channel_data_type == b.image_channel_data_type);
}
static inline bool
operator!=(const cl_image_format &a, const cl_image_format &b) {
return !(a == b);
}
#endif

675
src/gallium/frontends/clover/core/kernel.cpp
View file

@ -1,675 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/kernel.hpp"
#include "core/resource.hpp"
#include "util/factor.hpp"
#include "util/u_math.h"
#include "pipe/p_context.h"
using namespace clover;
kernel::kernel(clover::program &prog, const std::string &name,
const std::vector<binary::argument> &bargs) :
program(prog), _name(name), exec(*this),
program_ref(prog._kernel_ref_counter) {
for (auto &barg : bargs) {
if (barg.semantic == binary::argument::general)
_args.emplace_back(argument::create(barg));
}
for (auto &dev : prog.devices()) {
auto &b = prog.build(dev).bin;
auto bsym = find(name_equals(name), b.syms);
const auto f = id_type_equals(bsym.section, binary::section::data_constant);
if (!any_of(f, b.secs))
continue;
auto mconst = find(f, b.secs);
auto rb = std::make_unique<root_buffer>(prog.context(), std::vector<cl_mem_properties>(),
CL_MEM_COPY_HOST_PTR | CL_MEM_READ_ONLY,
mconst.size, mconst.data.data());
_constant_buffers.emplace(&dev, std::move(rb));
}
}
template<typename V>
static inline std::vector<uint>
pad_vector(command_queue &q, const V &v, uint x) {
std::vector<uint> w { v.begin(), v.end() };
w.resize(q.device().max_block_size().size(), x);
return w;
}
void
kernel::launch(command_queue &q,
const std::vector<size_t> &grid_offset,
const std::vector<size_t> &grid_size,
const std::vector<size_t> &block_size) {
const auto b = program().build(q.device()).bin;
const auto reduced_grid_size =
map(divides(), grid_size, block_size);
if (any_of(is_zero(), grid_size))
return;
void *st = exec.bind(&q, grid_offset);
struct pipe_grid_info info = {};
// The handles are created during exec_context::bind(), so we need make
// sure to call exec_context::bind() before retrieving them.
std::vector<uint32_t *> g_handles = map([&](size_t h) {
return (uint32_t *)&exec.input[h];
}, exec.g_handles);
q.pipe->bind_compute_state(q.pipe, st);
q.pipe->bind_sampler_states(q.pipe, PIPE_SHADER_COMPUTE,
0, exec.samplers.size(),
exec.samplers.data());
q.pipe->set_sampler_views(q.pipe, PIPE_SHADER_COMPUTE, 0,
exec.sviews.size(), 0, exec.sviews.data());
q.pipe->set_shader_images(q.pipe, PIPE_SHADER_COMPUTE, 0,
exec.iviews.size(), 0, exec.iviews.data());
q.pipe->set_compute_resources(q.pipe, 0, exec.resources.size(),
exec.resources.data());
q.pipe->set_global_binding(q.pipe, 0, exec.g_buffers.size(),
exec.g_buffers.data(), g_handles.data());
// Fill information for the launch_grid() call.
info.work_dim = grid_size.size();
copy(pad_vector(q, block_size, 1), info.block);
copy(pad_vector(q, reduced_grid_size, 1), info.grid);
info.pc = find(name_equals(_name), b.syms).offset;
info.input = exec.input.data();
info.variable_shared_mem = exec.mem_local;
q.pipe->launch_grid(q.pipe, &info);
q.pipe->set_global_binding(q.pipe, 0, exec.g_buffers.size(), NULL, NULL);
q.pipe->set_compute_resources(q.pipe, 0, exec.resources.size(), NULL);
q.pipe->set_shader_images(q.pipe, PIPE_SHADER_COMPUTE, 0,
0, exec.iviews.size(), NULL);
q.pipe->set_sampler_views(q.pipe, PIPE_SHADER_COMPUTE, 0,
0, exec.sviews.size(), NULL);
q.pipe->bind_sampler_states(q.pipe, PIPE_SHADER_COMPUTE, 0,
exec.samplers.size(), NULL);
q.pipe->memory_barrier(q.pipe, PIPE_BARRIER_GLOBAL_BUFFER);
exec.unbind();
}
size_t
kernel::mem_local() const {
size_t sz = 0;
for (auto &arg : args()) {
if (dynamic_cast<local_argument *>(&arg))
sz += arg.storage();
}
return sz;
}
size_t
kernel::mem_private() const {
return 0;
}
const std::string &
kernel::name() const {
return _name;
}
std::vector<size_t>
kernel::optimal_block_size(const command_queue &q,
const std::vector<size_t> &grid_size) const {
if (any_of(is_zero(), grid_size))
return grid_size;
return factor::find_grid_optimal_factor<size_t>(
q.device().max_threads_per_block(), q.device().max_block_size(),
grid_size);
}
std::vector<size_t>
kernel::required_block_size() const {
return find(name_equals(_name), program().symbols()).reqd_work_group_size;
}
kernel::argument_range
kernel::args() {
return map(derefs(), _args);
}
kernel::const_argument_range
kernel::args() const {
return map(derefs(), _args);
}
std::vector<clover::binary::arg_info>
kernel::args_infos() {
std::vector<clover::binary::arg_info> infos;
for (auto &barg: find(name_equals(_name), program().symbols()).args)
if (barg.semantic == clover::binary::argument::general)
infos.emplace_back(barg.info);
return infos;
}
const binary &
kernel::binary(const command_queue &q) const {
return program().build(q.device()).bin;
}
kernel::exec_context::exec_context(kernel &kern) :
kern(kern), q(NULL), print_handler(), mem_local(0), st(NULL), cs() {
}
kernel::exec_context::~exec_context() {
if (st)
q->pipe->delete_compute_state(q->pipe, st);
}
void *
kernel::exec_context::bind(intrusive_ptr<command_queue> _q,
const std::vector<size_t> &grid_offset) {
std::swap(q, _q);
// Bind kernel arguments.
auto &b = kern.program().build(q->device()).bin;
auto bsym = find(name_equals(kern.name()), b.syms);
auto bargs = bsym.args;
auto msec = find(id_type_equals(bsym.section, binary::section::text_executable), b.secs);
auto explicit_arg = kern._args.begin();
for (auto &barg : bargs) {
switch (barg.semantic) {
case binary::argument::general:
(*(explicit_arg++))->bind(*this, barg);
break;
case binary::argument::grid_dimension: {
const cl_uint dimension = grid_offset.size();
auto arg = argument::create(barg);
arg->set(sizeof(dimension), &dimension);
arg->bind(*this, barg);
break;
}
case binary::argument::grid_offset: {
for (cl_uint x : pad_vector(*q, grid_offset, 0)) {
auto arg = argument::create(barg);
arg->set(sizeof(x), &x);
arg->bind(*this, barg);
}
break;
}
case binary::argument::image_size: {
auto img = dynamic_cast<image_argument &>(**(explicit_arg - 1)).get();
std::vector<cl_uint> image_size{
static_cast<cl_uint>(img->width()),
static_cast<cl_uint>(img->height()),
static_cast<cl_uint>(img->depth())};
for (auto x : image_size) {
auto arg = argument::create(barg);
arg->set(sizeof(x), &x);
arg->bind(*this, barg);
}
break;
}
case binary::argument::image_format: {
auto img = dynamic_cast<image_argument &>(**(explicit_arg - 1)).get();
cl_image_format fmt = img->format();
std::vector<cl_uint> image_format{
static_cast<cl_uint>(fmt.image_channel_data_type),
static_cast<cl_uint>(fmt.image_channel_order)};
for (auto x : image_format) {
auto arg = argument::create(barg);
arg->set(sizeof(x), &x);
arg->bind(*this, barg);
}
break;
}
case binary::argument::constant_buffer: {
auto arg = argument::create(barg);
cl_mem buf = kern._constant_buffers.at(&q->device()).get();
arg->set(sizeof(buf), &buf);
arg->bind(*this, barg);
break;
}
case binary::argument::printf_buffer: {
print_handler = printf_handler::create(q, b.printf_infos,
b.printf_strings_in_buffer,
q->device().max_printf_buffer_size());
cl_mem print_mem = print_handler->get_mem();
auto arg = argument::create(barg);
arg->set(sizeof(cl_mem), &print_mem);
arg->bind(*this, barg);
break;
}
}
}
// Create a new compute state if anything changed.
if (!st || q != _q ||
cs.req_input_mem != input.size()) {
if (st)
_q->pipe->delete_compute_state(_q->pipe, st);
cs.ir_type = q->device().ir_format();
cs.prog = &(msec.data[0]);
// we only pass in NIRs or LLVMs and both IRs decode the size
cs.static_shared_mem = 0;
cs.req_input_mem = input.size();
st = q->pipe->create_compute_state(q->pipe, &cs);
if (!st) {
unbind(); // Cleanup
throw error(CL_OUT_OF_RESOURCES);
}
}
return st;
}
void
kernel::exec_context::unbind() {
if (print_handler)
print_handler->print();
for (auto &arg : kern.args())
arg.unbind(*this);
input.clear();
samplers.clear();
sviews.clear();
iviews.clear();
resources.clear();
g_buffers.clear();
g_handles.clear();
mem_local = 0;
}
namespace {
template<typename T>
std::vector<uint8_t>
bytes(const T& x) {
return { (uint8_t *)&x, (uint8_t *)&x + sizeof(x) };
}
///
/// Transform buffer \a v from the native byte order into the byte
/// order specified by \a e.
///
template<typename T>
void
byteswap(T &v, pipe_endian e) {
if (PIPE_ENDIAN_NATIVE != e)
std::reverse(v.begin(), v.end());
}
///
/// Pad buffer \a v to the next multiple of \a n.
///
template<typename T>
void
align_vector(T &v, size_t n) {
v.resize(util_align_npot(v.size(), n));
}
bool
msb(const std::vector<uint8_t> &s) {
if (PIPE_ENDIAN_NATIVE == PIPE_ENDIAN_LITTLE)
return s.back() & 0x80;
else
return s.front() & 0x80;
}
///
/// Resize buffer \a v to size \a n using sign or zero extension
/// according to \a ext.
///
template<typename T>
void
extend(T &v, enum binary::argument::ext_type ext, size_t n) {
const size_t m = std::min(v.size(), n);
const bool sign_ext = (ext == binary::argument::sign_ext);
const uint8_t fill = (sign_ext && msb(v) ? ~0 : 0);
T w(n, fill);
if (PIPE_ENDIAN_NATIVE == PIPE_ENDIAN_LITTLE)
std::copy_n(v.begin(), m, w.begin());
else
std::copy_n(v.end() - m, m, w.end() - m);
std::swap(v, w);
}
///
/// Append buffer \a w to \a v.
///
template<typename T>
void
insert(T &v, const T &w) {
v.insert(v.end(), w.begin(), w.end());
}
///
/// Append \a n elements to the end of buffer \a v.
///
template<typename T>
size_t
allocate(T &v, size_t n) {
size_t pos = v.size();
v.resize(pos + n);
return pos;
}
}
std::unique_ptr<kernel::argument>
kernel::argument::create(const binary::argument &barg) {
switch (barg.type) {
case binary::argument::scalar:
return std::unique_ptr<kernel::argument>(new scalar_argument(barg.size));
case binary::argument::global:
return std::unique_ptr<kernel::argument>(new global_argument);
case binary::argument::local:
return std::unique_ptr<kernel::argument>(new local_argument);
case binary::argument::constant:
return std::unique_ptr<kernel::argument>(new constant_argument);
case binary::argument::image_rd:
return std::unique_ptr<kernel::argument>(new image_rd_argument);
case binary::argument::image_wr:
return std::unique_ptr<kernel::argument>(new image_wr_argument);
case binary::argument::sampler:
return std::unique_ptr<kernel::argument>(new sampler_argument);
}
throw error(CL_INVALID_KERNEL_DEFINITION);
}
kernel::argument::argument() : _set(false) {
}
bool
kernel::argument::set() const {
return _set;
}
size_t
kernel::argument::storage() const {
return 0;
}
kernel::scalar_argument::scalar_argument(size_t size) : size(size) {
}
void
kernel::scalar_argument::set(size_t size, const void *value) {
if (!value)
throw error(CL_INVALID_ARG_VALUE);
if (size != this->size)
throw error(CL_INVALID_ARG_SIZE);
v = { (uint8_t *)value, (uint8_t *)value + size };
_set = true;
}
void
kernel::scalar_argument::bind(exec_context &ctx,
const binary::argument &barg) {
auto w = v;
extend(w, barg.ext_type, barg.target_size);
byteswap(w, ctx.q->device().endianness());
align_vector(ctx.input, barg.target_align);
insert(ctx.input, w);
}
void
kernel::scalar_argument::unbind(exec_context &ctx) {
}
kernel::global_argument::global_argument() : buf(nullptr), svm(nullptr) {
}
void
kernel::global_argument::set(size_t size, const void *value) {
if (size != sizeof(cl_mem))
throw error(CL_INVALID_ARG_SIZE);
buf = pobj<buffer>(value ? *(cl_mem *)value : NULL);
svm = nullptr;
_set = true;
}
void
kernel::global_argument::set_svm(const void *value) {
svm = value;
buf = nullptr;
_set = true;
}
void
kernel::global_argument::bind(exec_context &ctx,
const binary::argument &barg) {
align_vector(ctx.input, barg.target_align);
if (buf) {
const resource &r = buf->resource_in(*ctx.q);
ctx.g_handles.push_back(ctx.input.size());
ctx.g_buffers.push_back(r.pipe);
// How to handle multi-demensional offsets?
// We don't need to. Buffer offsets are always
// one-dimensional.
auto v = bytes(r.offset[0]);
extend(v, barg.ext_type, barg.target_size);
byteswap(v, ctx.q->device().endianness());
insert(ctx.input, v);
} else if (svm) {
auto v = bytes(svm);
extend(v, barg.ext_type, barg.target_size);
byteswap(v, ctx.q->device().endianness());
insert(ctx.input, v);
} else {
// Null pointer.
allocate(ctx.input, barg.target_size);
}
}
void
kernel::global_argument::unbind(exec_context &ctx) {
}
size_t
kernel::local_argument::storage() const {
return _storage;
}
void
kernel::local_argument::set(size_t size, const void *value) {
if (value)
throw error(CL_INVALID_ARG_VALUE);
if (!size)
throw error(CL_INVALID_ARG_SIZE);
_storage = size;
_set = true;
}
void
kernel::local_argument::bind(exec_context &ctx,
const binary::argument &barg) {
ctx.mem_local = ::align(ctx.mem_local, barg.target_align);
auto v = bytes(ctx.mem_local);
extend(v, binary::argument::zero_ext, barg.target_size);
byteswap(v, ctx.q->device().endianness());
align_vector(ctx.input, ctx.q->device().address_bits() / 8);
insert(ctx.input, v);
ctx.mem_local += _storage;
}
void
kernel::local_argument::unbind(exec_context &ctx) {
}
kernel::constant_argument::constant_argument() : buf(nullptr), st(nullptr) {
}
void
kernel::constant_argument::set(size_t size, const void *value) {
if (size != sizeof(cl_mem))
throw error(CL_INVALID_ARG_SIZE);
buf = pobj<buffer>(value ? *(cl_mem *)value : NULL);
_set = true;
}
void
kernel::constant_argument::bind(exec_context &ctx,
const binary::argument &barg) {
align_vector(ctx.input, barg.target_align);
if (buf) {
resource &r = buf->resource_in(*ctx.q);
auto v = bytes(ctx.resources.size() << 24 | r.offset[0]);
extend(v, binary::argument::zero_ext, barg.target_size);
byteswap(v, ctx.q->device().endianness());
insert(ctx.input, v);
st = r.bind_surface(*ctx.q, false);
ctx.resources.push_back(st);
} else {
// Null pointer.
allocate(ctx.input, barg.target_size);
}
}
void
kernel::constant_argument::unbind(exec_context &ctx) {
if (buf)
buf->resource_in(*ctx.q).unbind_surface(*ctx.q, st);
}
kernel::image_rd_argument::image_rd_argument() : st(nullptr) {
}
void
kernel::image_rd_argument::set(size_t size, const void *value) {
if (!value)
throw error(CL_INVALID_ARG_VALUE);
if (size != sizeof(cl_mem))
throw error(CL_INVALID_ARG_SIZE);
img = &obj<image>(*(cl_mem *)value);
_set = true;
}
void
kernel::image_rd_argument::bind(exec_context &ctx,
const binary::argument &barg) {
auto v = bytes(ctx.sviews.size());
extend(v, binary::argument::zero_ext, barg.target_size);
byteswap(v, ctx.q->device().endianness());
align_vector(ctx.input, barg.target_align);
insert(ctx.input, v);
st = img->resource_in(*ctx.q).bind_sampler_view(*ctx.q);
ctx.sviews.push_back(st);
}
void
kernel::image_rd_argument::unbind(exec_context &ctx) {
img->resource_in(*ctx.q).unbind_sampler_view(*ctx.q, st);
}
void
kernel::image_wr_argument::set(size_t size, const void *value) {
if (!value)
throw error(CL_INVALID_ARG_VALUE);
if (size != sizeof(cl_mem))
throw error(CL_INVALID_ARG_SIZE);
img = &obj<image>(*(cl_mem *)value);
_set = true;
}
void
kernel::image_wr_argument::bind(exec_context &ctx,
const binary::argument &barg) {
auto v = bytes(ctx.iviews.size());
extend(v, binary::argument::zero_ext, barg.target_size);
byteswap(v, ctx.q->device().endianness());
align_vector(ctx.input, barg.target_align);
insert(ctx.input, v);
ctx.iviews.push_back(img->resource_in(*ctx.q).create_image_view(*ctx.q));
}
void
kernel::image_wr_argument::unbind(exec_context &ctx) {
}
kernel::sampler_argument::sampler_argument() : s(nullptr), st(nullptr) {
}
void
kernel::sampler_argument::set(size_t size, const void *value) {
if (!value)
throw error(CL_INVALID_SAMPLER);
if (size != sizeof(cl_sampler))
throw error(CL_INVALID_ARG_SIZE);
s = &obj(*(cl_sampler *)value);
_set = true;
}
void
kernel::sampler_argument::bind(exec_context &ctx,
const binary::argument &barg) {
st = s->bind(*ctx.q);
ctx.samplers.push_back(st);
}
void
kernel::sampler_argument::unbind(exec_context &ctx) {
s->unbind(*ctx.q, st);
}

262
src/gallium/frontends/clover/core/kernel.hpp
View file

@ -1,262 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_KERNEL_HPP
#define CLOVER_CORE_KERNEL_HPP
#include <map>
#include <memory>
#include "core/object.hpp"
#include "core/printf.hpp"
#include "core/program.hpp"
#include "core/memory.hpp"
#include "core/sampler.hpp"
#include "pipe/p_state.h"
namespace clover {
class kernel : public ref_counter, public _cl_kernel {
private:
///
/// Class containing all the state required to execute a compute
/// kernel.
///
struct exec_context {
exec_context(kernel &kern);
~exec_context();
exec_context(const exec_context &) = delete;
exec_context &
operator=(const exec_context &) = delete;
void *bind(intrusive_ptr<command_queue> _q,
const std::vector<size_t> &grid_offset);
void unbind();
kernel &kern;
intrusive_ptr<command_queue> q;
std::unique_ptr<printf_handler> print_handler;
std::vector<uint8_t> input;
std::vector<void *> samplers;
std::vector<pipe_sampler_view *> sviews;
std::vector<pipe_image_view> iviews;
std::vector<pipe_surface *> resources;
std::vector<pipe_resource *> g_buffers;
std::vector<size_t> g_handles;
size_t mem_local;
private:
void *st;
pipe_compute_state cs;
};
public:
class argument {
public:
static std::unique_ptr<argument>
create(const binary::argument &barg);
argument(const argument &arg) = delete;
argument &
operator=(const argument &arg) = delete;
/// \a true if the argument has been set.
bool set() const;
/// Storage space required for the referenced object.
virtual size_t storage() const;
/// Set this argument to some object.
virtual void set(size_t size, const void *value) = 0;
/// Set this argument to an SVM pointer.
virtual void set_svm(const void *value) {
throw error(CL_INVALID_ARG_INDEX);
};
/// Allocate the necessary resources to bind the specified
/// object to this argument, and update \a ctx accordingly.
virtual void bind(exec_context &ctx,
const binary::argument &barg) = 0;
/// Free any resources that were allocated in bind().
virtual void unbind(exec_context &ctx) = 0;
virtual ~argument() {};
protected:
argument();
bool _set;
};
private:
typedef adaptor_range<
derefs, std::vector<std::unique_ptr<argument>> &
> argument_range;
typedef adaptor_range<
derefs, const std::vector<std::unique_ptr<argument>> &
> const_argument_range;
public:
kernel(clover::program &prog, const std::string &name,
const std::vector<clover::binary::argument> &bargs);
kernel(const kernel &kern) = delete;
kernel &
operator=(const kernel &kern) = delete;
void launch(command_queue &q,
const std::vector<size_t> &grid_offset,
const std::vector<size_t> &grid_size,
const std::vector<size_t> &block_size);
size_t mem_local() const;
size_t mem_private() const;
const std::string &name() const;
std::vector<size_t>
optimal_block_size(const command_queue &q,
const std::vector<size_t> &grid_size) const;
std::vector<size_t>
required_block_size() const;
argument_range args();
const_argument_range args() const;
std::vector<clover::binary::arg_info> args_infos();
const intrusive_ref<clover::program> program;
private:
const clover::binary &binary(const command_queue &q) const;
class scalar_argument : public argument {
public:
scalar_argument(size_t size);
virtual void set(size_t size, const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
private:
size_t size;
std::vector<uint8_t> v;
};
class global_argument : public argument {
public:
global_argument();
virtual void set(size_t size, const void *value);
virtual void set_svm(const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
private:
buffer *buf;
const void *svm;
};
class local_argument : public argument {
public:
virtual size_t storage() const;
virtual void set(size_t size, const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
private:
size_t _storage = 0;
};
class constant_argument : public argument {
public:
constant_argument();
virtual void set(size_t size, const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
private:
buffer *buf;
pipe_surface *st;
};
class image_argument : public argument {
public:
const image *get() const {
return img;
}
protected:
image *img;
};
class image_rd_argument : public image_argument {
public:
image_rd_argument();
virtual void set(size_t size, const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
private:
pipe_sampler_view *st;
};
class image_wr_argument : public image_argument {
public:
virtual void set(size_t size, const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
};
class sampler_argument : public argument {
public:
sampler_argument();
virtual void set(size_t size, const void *value);
virtual void bind(exec_context &ctx,
const binary::argument &barg);
virtual void unbind(exec_context &ctx);
private:
sampler *s;
void *st;
};
std::vector<std::unique_ptr<argument>> _args;
std::map<device *, std::unique_ptr<root_buffer> > _constant_buffers;
std::string _name;
exec_context exec;
const ref_holder program_ref;
};
}
#endif

325
src/gallium/frontends/clover/core/memory.cpp
View file

@ -1,325 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/memory.hpp"
#include "core/resource.hpp"
#include "util/format/u_format.h"
using namespace clover;
memory_obj::memory_obj(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
size_t size, void *host_ptr) :
context(ctx), _properties(properties), _flags(flags),
_size(size), _host_ptr(host_ptr) {
if (flags & CL_MEM_COPY_HOST_PTR)
data.append((char *)host_ptr, size);
}
memory_obj::~memory_obj() {
while (_destroy_notify.size()) {
_destroy_notify.top()();
_destroy_notify.pop();
}
}
bool
memory_obj::operator==(const memory_obj &obj) const {
return this == &obj;
}
void
memory_obj::destroy_notify(std::function<void ()> f) {
_destroy_notify.push(f);
}
std::vector<cl_mem_properties>
memory_obj::properties() const {
return _properties;
}
cl_mem_flags
memory_obj::flags() const {
return _flags;
}
size_t
memory_obj::size() const {
return _size;
}
void *
memory_obj::host_ptr() const {
return _host_ptr;
}
buffer::buffer(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
size_t size, void *host_ptr) :
memory_obj(ctx, properties, flags, size, host_ptr) {
}
cl_mem_object_type
buffer::type() const {
return CL_MEM_OBJECT_BUFFER;
}
root_buffer::root_buffer(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
size_t size, void *host_ptr) :
buffer(ctx, properties, flags, size, host_ptr) {
}
resource &
root_buffer::resource_in(command_queue &q) {
const void *data_ptr = NULL;
if (flags() & (CL_MEM_USE_HOST_PTR | CL_MEM_COPY_HOST_PTR))
data_ptr = !data.empty() ? data.data() : host_ptr();
return resource(q, data_ptr);
}
resource &
root_buffer::resource_undef(command_queue &q) {
return resource(q, NULL);
}
resource &
root_buffer::resource(command_queue &q, const void *data_ptr) {
std::lock_guard<std::mutex> lock(resources_mtx);
// Create a new resource if there's none for this device yet.
if (!resources.count(&q.device())) {
auto r = (!resources.empty() ?
new root_resource(q.device(), *this,
*resources.begin()->second) :
new root_resource(q.device(), *this, q, data_ptr));
resources.insert(std::make_pair(&q.device(),
std::unique_ptr<root_resource>(r)));
data.clear();
}
return *resources.find(&q.device())->second;
}
void
root_buffer::resource_out(command_queue &q) {
std::lock_guard<std::mutex> lock(resources_mtx);
resources.erase(&q.device());
}
sub_buffer::sub_buffer(root_buffer &parent, cl_mem_flags flags,
size_t offset, size_t size) :
buffer(parent.context(), std::vector<cl_mem_properties>(), flags, size,
(char *)parent.host_ptr() + offset),
parent(parent), _offset(offset) {
}
resource &
sub_buffer::resource_in(command_queue &q) {
std::lock_guard<std::mutex> lock(resources_mtx);
// Create a new resource if there's none for this device yet.
if (!resources.count(&q.device())) {
auto r = new sub_resource(parent().resource_in(q), {{ offset() }});
resources.insert(std::make_pair(&q.device(),
std::unique_ptr<sub_resource>(r)));
}
return *resources.find(&q.device())->second;
}
resource &
sub_buffer::resource_undef(command_queue &q) {
return resource_in(q);
}
void
sub_buffer::resource_out(command_queue &q) {
std::lock_guard<std::mutex> lock(resources_mtx);
resources.erase(&q.device());
}
size_t
sub_buffer::offset() const {
return _offset;
}
image::image(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t height, size_t depth, size_t array_size,
size_t row_pitch, size_t slice_pitch, size_t size,
void *host_ptr, cl_mem buffer) :
memory_obj(ctx, properties, flags, size, host_ptr),
_format(*format), _width(width), _height(height), _depth(depth),
_row_pitch(row_pitch), _slice_pitch(slice_pitch), _array_size(array_size),
_buffer(buffer) {
}
resource &
image::resource_in(command_queue &q) {
const void *data_ptr = !data.empty() ? data.data() : NULL;
return resource(q, data_ptr);
}
resource &
image::resource_undef(command_queue &q) {
return resource(q, NULL);
}
resource &
image::resource(command_queue &q, const void *data_ptr) {
std::lock_guard<std::mutex> lock(resources_mtx);
// Create a new resource if there's none for this device yet.
if (!resources.count(&q.device())) {
auto r = (!resources.empty() ?
new root_resource(q.device(), *this,
*resources.begin()->second) :
new root_resource(q.device(), *this, q, data_ptr));
resources.insert(std::make_pair(&q.device(),
std::unique_ptr<root_resource>(r)));
data.clear();
}
return *resources.find(&q.device())->second;
}
void
image::resource_out(command_queue &q) {
std::lock_guard<std::mutex> lock(resources_mtx);
resources.erase(&q.device());
}
cl_image_format
image::format() const {
return _format;
}
size_t
image::width() const {
return _width;
}
size_t
image::height() const {
return _height;
}
size_t
image::depth() const {
return _depth;
}
size_t
image::pixel_size() const {
return util_format_get_blocksize(translate_format(_format));
}
size_t
image::row_pitch() const {
return _row_pitch;
}
size_t
image::slice_pitch() const {
return _slice_pitch;
}
size_t
image::array_size() const {
return _array_size;
}
cl_mem
image::buffer() const {
return _buffer;
}
image1d::image1d(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t row_pitch,
void *host_ptr) :
basic_image(ctx, properties, flags, format, width, 1, 1, 0,
row_pitch, 0, row_pitch, host_ptr, nullptr) {
}
image1d_buffer::image1d_buffer(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t row_pitch,
void *host_ptr, cl_mem buffer) :
basic_image(ctx, properties, flags, format, width, 1, 1, 0,
row_pitch, 0, row_pitch, host_ptr, buffer) {
}
image1d_array::image1d_array(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width,
size_t array_size, size_t slice_pitch,
void *host_ptr) :
basic_image(ctx, properties, flags, format, width, 1, 1, array_size,
0, slice_pitch, slice_pitch * array_size, host_ptr, nullptr) {
}
image2d::image2d(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format, size_t width,
size_t height, size_t row_pitch,
void *host_ptr) :
basic_image(ctx, properties, flags, format, width, height, 1, 0,
row_pitch, 0, height * row_pitch, host_ptr, nullptr) {
}
image2d_array::image2d_array(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t height, size_t array_size,
size_t row_pitch, size_t slice_pitch,
void *host_ptr) :
basic_image(ctx, properties, flags, format, width, height, 1, array_size,
row_pitch, slice_pitch, slice_pitch * array_size, host_ptr, nullptr) {
}
image3d::image3d(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t height, size_t depth,
size_t row_pitch, size_t slice_pitch,
void *host_ptr) :
basic_image(ctx, properties, flags, format, width, height, depth, 0,
row_pitch, slice_pitch, depth * slice_pitch,
host_ptr, nullptr) {
}

256
src/gallium/frontends/clover/core/memory.hpp
View file

@ -1,256 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_MEMORY_HPP
#define CLOVER_CORE_MEMORY_HPP
#include <functional>
#include <map>
#include <memory>
#include <stack>
#include "core/object.hpp"
#include "core/queue.hpp"
#include "core/resource.hpp"
namespace clover {
class memory_obj : public ref_counter, public _cl_mem {
protected:
memory_obj(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
size_t size, void *host_ptr);
memory_obj(const memory_obj &obj) = delete;
memory_obj &
operator=(const memory_obj &obj) = delete;
public:
virtual ~memory_obj();
bool
operator==(const memory_obj &obj) const;
virtual cl_mem_object_type type() const = 0;
virtual clover::resource &
resource_in(command_queue &q) = 0;
virtual clover::resource &
resource_undef(command_queue &q) = 0;
virtual void resource_out(command_queue &q) = 0;
void destroy_notify(std::function<void ()> f);
std::vector<cl_mem_properties> properties() const;
cl_mem_flags flags() const;
size_t size() const;
void *host_ptr() const;
const intrusive_ref<clover::context> context;
private:
std::vector<cl_mem_properties> _properties;
cl_mem_flags _flags;
size_t _size;
void *_host_ptr;
std::stack<std::function<void ()>> _destroy_notify;
protected:
std::string data;
};
class buffer : public memory_obj {
protected:
buffer(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
size_t size, void *host_ptr);
public:
virtual cl_mem_object_type type() const;
};
class root_buffer : public buffer {
public:
root_buffer(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
size_t size, void *host_ptr);
virtual clover::resource &
resource_in(command_queue &q);
virtual clover::resource &
resource_undef(command_queue &q);
virtual void
resource_out(command_queue &q);
private:
clover::resource &
resource(command_queue &q, const void *data_ptr);
std::map<device *,
std::unique_ptr<root_resource>> resources;
std::mutex resources_mtx;
};
class sub_buffer : public buffer {
public:
sub_buffer(root_buffer &parent, cl_mem_flags flags,
size_t offset, size_t size);
virtual clover::resource &
resource_in(command_queue &q);
virtual clover::resource &
resource_undef(command_queue &q);
virtual void
resource_out(command_queue &q);
size_t offset() const;
const intrusive_ref<root_buffer> parent;
private:
size_t _offset;
std::map<device *,
std::unique_ptr<sub_resource>> resources;
std::mutex resources_mtx;
};
class image : public memory_obj {
protected:
image(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t height, size_t depth, size_t array_size,
size_t row_pitch, size_t slice_pitch, size_t size,
void *host_ptr, cl_mem buffer);
public:
cl_image_format format() const;
virtual cl_uint dimensions() const = 0;
size_t width() const;
size_t height() const;
size_t depth() const;
size_t pixel_size() const;
size_t row_pitch() const;
size_t slice_pitch() const;
size_t array_size() const;
cl_mem buffer() const;
virtual clover::resource &
resource_in(command_queue &q);
virtual clover::resource &
resource_undef(command_queue &q);
virtual void
resource_out(command_queue &q);
private:
clover::resource &
resource(command_queue &q, const void *data_ptr);
cl_image_format _format;
size_t _width;
size_t _height;
size_t _depth;
size_t _row_pitch;
size_t _slice_pitch;
size_t _array_size;
cl_mem _buffer;
std::map<device *,
std::unique_ptr<root_resource>> resources;
std::mutex resources_mtx;
};
template<cl_mem_object_type Type, cl_uint Dim>
class basic_image : public image {
public:
using image::image;
virtual cl_mem_object_type type() const {
return Type;
}
virtual cl_uint dimensions() const {
return Dim;
}
};
class image1d : public basic_image<CL_MEM_OBJECT_IMAGE1D, 1> {
public:
image1d(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t row_pitch,
void *host_ptr);
};
class image1d_buffer : public basic_image<CL_MEM_OBJECT_IMAGE1D_BUFFER, 1> {
public:
image1d_buffer(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t row_pitch,
void *host_ptr, cl_mem buffer);
};
class image1d_array : public basic_image<CL_MEM_OBJECT_IMAGE1D_ARRAY, 1> {
public:
image1d_array(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width,
size_t array_size, size_t slice_pitch,
void *host_ptr);
};
class image2d : public basic_image<CL_MEM_OBJECT_IMAGE2D, 2> {
public:
image2d(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format, size_t width,
size_t height, size_t row_pitch,
void *host_ptr);
};
class image2d_array : public basic_image<CL_MEM_OBJECT_IMAGE2D_ARRAY, 2> {
public:
image2d_array(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t height, size_t array_size,
size_t row_pitch, size_t slice_pitch,
void *host_ptr);
};
class image3d : public basic_image<CL_MEM_OBJECT_IMAGE3D, 3>{
public:
image3d(clover::context &ctx,
std::vector<cl_mem_properties> properties,
cl_mem_flags flags,
const cl_image_format *format,
size_t width, size_t height, size_t depth,
size_t row_pitch, size_t slice_pitch,
void *host_ptr);
};
}
#endif

239
src/gallium/frontends/clover/core/object.hpp
View file

@ -1,239 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_OBJECT_HPP
#define CLOVER_CORE_OBJECT_HPP
#include <cassert>
#include <functional>
#include <vector>
#include "CL/cl.h"
#include "core/error.hpp"
#include "core/property.hpp"
#include "api/dispatch.hpp"
#include "util/macros.h"
///
/// Main namespace of the CL gallium frontend.
///
namespace clover {
///
/// Class that represents a CL API object.
///
template<typename T, typename S>
struct descriptor {
typedef T object_type;
typedef S descriptor_type;
descriptor() : dispatch(&_dispatch) {
static_assert(std::is_standard_layout<descriptor_type>::value,
"ICD requires CL API objects to be standard layout.");
}
const cl_icd_dispatch *dispatch;
};
struct default_tag;
struct allow_empty_tag;
struct wait_list_tag;
struct property_list_tag;
namespace detail {
template<typename T, typename D>
struct descriptor_traits {
typedef T object_type;
static void
validate(D *d) {
auto o = static_cast<typename D::object_type *>(d);
if (!o || o->dispatch != &_dispatch ||
!dynamic_cast<object_type *>(o))
throw invalid_object_error<T>();
}
static void
validate_list(D * const *ds, size_t n) {
if (!ds || !n)
throw error(CL_INVALID_VALUE);
}
};
template<typename D>
struct descriptor_traits<default_tag, D> {
typedef typename D::object_type object_type;
static void
validate(D *d) {
if (!d || d->dispatch != &_dispatch)
throw invalid_object_error<object_type>();
}
static void
validate_list(D *const *ds, size_t n) {
if (!ds || !n)
throw error(CL_INVALID_VALUE);
}
};
template<typename D>
struct descriptor_traits<allow_empty_tag, D> {
typedef typename D::object_type object_type;
static void
validate(D *d) {
if (!d || d->dispatch != &_dispatch)
throw invalid_object_error<object_type>();
}
static void
validate_list(D *const *ds, size_t n) {
if (bool(ds) != bool(n))
throw error(CL_INVALID_VALUE);
}
};
template<typename D>
struct descriptor_traits<wait_list_tag, D> {
typedef typename D::object_type object_type;
static void
validate(D *d) {
if (!d || d->dispatch != &_dispatch)
throw invalid_wait_list_error();
}
static void
validate_list(D *const *ds, size_t n) {
if (bool(ds) != bool(n))
throw invalid_wait_list_error();
}
};
}
///
/// Get a Clover object from an API object performing object
/// validation.
///
/// \a T can either be the Clover object type to return or a \a tag
/// object to select some special validation behavior by means of a
/// specialization of the detail::descriptor_traits template. The
/// default behavior is to infer the most general Clover object
/// type for the given API object.
///
template<typename T = default_tag, typename D>
typename detail::descriptor_traits<T, D>::object_type &
obj(D *d) {
detail::descriptor_traits<T, D>::validate(d);
return static_cast<
typename detail::descriptor_traits<T, D>::object_type &>(*d);
}
///
/// Get a pointer to a Clover object from an API object performing
/// object validation. Returns \c NULL if its argument is \c NULL.
///
/// \sa obj
///
template<typename T = default_tag, typename D>
typename detail::descriptor_traits<T, D>::object_type *
pobj(D *d) {
if (d)
detail::descriptor_traits<T, D>::validate(d);
return static_cast<
typename detail::descriptor_traits<T, D>::object_type *>(d);
}
///
/// Get an API object from a Clover object.
///
template<typename O>
typename O::descriptor_type *
desc(O &o) {
return static_cast<typename O::descriptor_type *>(&o);
}
///
/// Get an API object from a pointer to a Clover object.
///
template<typename O>
typename O::descriptor_type *
desc(O *o) {
return static_cast<typename O::descriptor_type *>(o);
}
///
/// Get a range of Clover objects from a range of API objects
/// performing object validation.
///
/// \sa obj
///
template<typename T = default_tag, typename D>
ref_vector<typename detail::descriptor_traits<T, D>::object_type>
objs(D *const *ds, size_t n) {
detail::descriptor_traits<T, D>::validate_list(ds, n);
return map(obj<T, D>, range(ds, n));
}
///
/// Get a range of API objects from a range of Clover objects.
///
template<typename Os>
std::vector<typename Os::value_type::descriptor_type *>
descs(const Os &os) {
return map([](typename Os::value_type &o) {
return desc(o);
}, os);
}
}
struct _cl_context :
public clover::descriptor<clover::context, _cl_context> {};
struct _cl_device_id :
public clover::descriptor<clover::device, _cl_device_id> {};
struct _cl_event :
public clover::descriptor<clover::event, _cl_event> {};
struct _cl_kernel :
public clover::descriptor<clover::kernel, _cl_kernel> {};
struct _cl_mem :
public clover::descriptor<clover::memory_obj, _cl_mem> {};
struct _cl_platform_id :
public clover::descriptor<clover::platform, _cl_platform_id> {};
struct _cl_program :
public clover::descriptor<clover::program, _cl_program> {};
struct _cl_command_queue :
public clover::descriptor<clover::command_queue, _cl_command_queue> {};
struct _cl_sampler :
public clover::descriptor<clover::sampler, _cl_sampler> {};
#endif

83
src/gallium/frontends/clover/core/platform.cpp
View file

@ -1,83 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/platform.hpp"
#include "util/u_debug.h"
using namespace clover;
platform::platform() : adaptor_range(evals(), devs) {
int n = pipe_loader_probe(NULL, 0, false);
std::vector<pipe_loader_device *> ldevs(n);
unsigned major = 1, minor = 1;
debug_get_version_option("CLOVER_PLATFORM_VERSION_OVERRIDE", &major, &minor);
version = CL_MAKE_VERSION(major, minor, 0);
pipe_loader_probe(&ldevs.front(), n, false);
for (pipe_loader_device *ldev : ldevs) {
try {
if (ldev)
devs.push_back(create<device>(*this, ldev));
} catch (error &) {
pipe_loader_release(&ldev, 1);
}
}
}
std::vector<cl_name_version>
platform::supported_extensions() const {
std::vector<cl_name_version> vec;
vec.push_back( cl_name_version{ CL_MAKE_VERSION(1, 0, 0), "cl_khr_icd" } );
return vec;
}
std::string
platform::supported_extensions_as_string() const {
static std::string extensions_string;
if (!extensions_string.empty())
return extensions_string;
const auto extension_list = supported_extensions();
for (const auto &extension : extension_list) {
if (!extensions_string.empty())
extensions_string += " ";
extensions_string += extension.name;
}
return extensions_string;
}
std::string
platform::platform_version_as_string() const {
static const std::string version_string =
std::to_string(CL_VERSION_MAJOR(version)) + "." +
std::to_string(CL_VERSION_MINOR(version));
return version_string;
}
cl_version
platform::platform_version() const {
return version;
}

57
src/gallium/frontends/clover/core/platform.hpp
View file

@ -1,57 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_PLATFORM_HPP
#define CLOVER_CORE_PLATFORM_HPP
#include <vector>
#include "core/object.hpp"
#include "core/device.hpp"
#include "util/range.hpp"
namespace clover {
class platform : public _cl_platform_id,
public adaptor_range<
evals, std::vector<intrusive_ref<device>> &> {
public:
platform();
platform(const platform &platform) = delete;
platform &
operator=(const platform &platform) = delete;
std::string supported_extensions_as_string() const;
std::vector<cl_name_version> supported_extensions() const;
std::string platform_version_as_string() const;
cl_version platform_version() const;
protected:
cl_version version;
std::vector<intrusive_ref<device>> devs;
};
platform &find_platform(cl_platform_id d_platform);
}
#endif

129
src/gallium/frontends/clover/core/printf.cpp
View file

@ -1,129 +0,0 @@
//
// Copyright 2020 Serge Martin
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include <cstring>
#include <cstdio>
#include <string>
#include <iostream>
#include "util/u_math.h"
#include "core/printf.hpp"
#include "util/u_printf.h"
using namespace clover;
namespace {
const cl_uint hdr_dwords = 2;
const cl_uint initial_buffer_offset = hdr_dwords * sizeof(cl_uint);
/* all valid chars that can appear in CL C printf string. */
const std::string clc_printf_whitelist = "%0123456789-+ #.AacdeEfFgGhilopsuvxX";
void
print_formatted(std::vector<binary::printf_info> &formatters,
bool _strings_in_buffer,
const std::vector<char> &buffer) {
static std::atomic<unsigned> warn_count;
if (buffer.empty() && !warn_count++)
std::cerr << "Printf used but no printf occurred - may cause performance issue." << std::endl;
std::vector<u_printf_info> infos;
for (auto &f : formatters) {
u_printf_info info;
info.num_args = f.arg_sizes.size();
info.arg_sizes = f.arg_sizes.data();
info.string_size = f.strings.size();
info.strings = f.strings.data();
infos.push_back(info);
}
u_printf(stdout, buffer.data(), buffer.size(), infos.data(), infos.size());
}
}
std::unique_ptr<printf_handler>
printf_handler::create(const intrusive_ptr<command_queue> &q,
const std::vector<binary::printf_info> &infos,
bool strings_in_buffer,
cl_uint size) {
return std::unique_ptr<printf_handler>(
new printf_handler(q, infos, strings_in_buffer, size));
}
printf_handler::printf_handler(const intrusive_ptr<command_queue> &q,
const std::vector<binary::printf_info> &infos,
bool strings_in_buffer,
cl_uint size) :
_q(q), _formatters(infos), _strings_in_buffer(strings_in_buffer), _size(size), _buffer() {
if (_size) {
std::string data;
data.reserve(_size);
cl_uint header[2] = { 0 };
header[0] = initial_buffer_offset;
header[1] = _size;
data.append((char *)header, (char *)(header+hdr_dwords));
_buffer = std::unique_ptr<root_buffer>(new root_buffer(_q->context,
std::vector<cl_mem_properties>(),
CL_MEM_COPY_HOST_PTR,
_size, (char*)data.data()));
}
}
cl_mem
printf_handler::get_mem() {
return (cl_mem)(_buffer.get());
}
void
printf_handler::print() {
if (!_buffer)
return;
mapping src = { *_q, _buffer->resource_in(*_q), CL_MAP_READ, true,
{{ 0 }}, {{ _size, 1, 1 }} };
cl_uint header[2] = { 0 };
std::memcpy(header,
static_cast<const char *>(src),
initial_buffer_offset);
cl_uint buffer_size = header[0];
buffer_size -= initial_buffer_offset;
std::vector<char> buf;
buf.resize(buffer_size);
std::memcpy(buf.data(),
static_cast<const char *>(src) + initial_buffer_offset,
buffer_size);
// mixed endian isn't going to work, sort it out if anyone cares later.
assert(_q->device().endianness() == PIPE_ENDIAN_NATIVE);
print_formatted(_formatters, _strings_in_buffer, buf);
}

60
src/gallium/frontends/clover/core/printf.hpp
View file

@ -1,60 +0,0 @@
//
// Copyright 2020 Serge Martin
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_PRINTF_HANDLER_HPP
#define CLOVER_CORE_PRINTF_HANDLER_HPP
#include <memory>
#include "core/memory.hpp"
namespace clover {
class printf_handler {
public:
static std::unique_ptr<printf_handler>
create(const intrusive_ptr<command_queue> &q,
const std::vector<binary::printf_info> &info,
bool strings_in_buffer, cl_uint size);
printf_handler(const printf_handler &arg) = delete;
printf_handler &
operator=(const printf_handler &arg) = delete;
~printf_handler() {};
cl_mem get_mem();
void print();
private:
printf_handler(const intrusive_ptr<command_queue> &q,
const std::vector<binary::printf_info> &infos,
bool strings_in_buffer, cl_uint size);
intrusive_ptr<command_queue> _q;
std::vector<binary::printf_info> _formatters;
bool _strings_in_buffer;
cl_uint _size;
std::unique_ptr<root_buffer> _buffer;
};
}
#endif

141
src/gallium/frontends/clover/core/program.cpp
View file

@ -1,141 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/compiler.hpp"
#include "core/program.hpp"
using namespace clover;
program::program(clover::context &ctx, std::string &&source,
enum il_type il_type) :
context(ctx), _devices(ctx.devices()), _source(std::move(source)),
_kernel_ref_counter(0), _il_type(il_type) {
}
program::program(clover::context &ctx,
const ref_vector<device> &devs,
const std::vector<binary> &binaries) :
context(ctx), _devices(devs), _kernel_ref_counter(0),
_il_type(il_type::none) {
for_each([&](device &dev, const binary &bin) {
_builds[&dev] = { bin };
},
devs, binaries);
}
void
program::compile(const ref_vector<device> &devs, const std::string &opts,
const header_map &headers) {
if (_il_type != il_type::none) {
_devices = devs;
for (auto &dev : devs) {
std::string log;
try {
const binary b =
compiler::compile_program(*this, headers, dev, opts, log);
_builds[&dev] = { b, opts, log };
} catch (...) {
_builds[&dev] = { binary(), opts, log };
throw;
}
}
}
}
void
program::link(const ref_vector<device> &devs, const std::string &opts,
const ref_vector<program> &progs) {
_devices = devs;
for (auto &dev : devs) {
const std::vector<binary> bs = map([&](const program &prog) {
return prog.build(dev).bin;
}, progs);
std::string log = _builds[&dev].log;
try {
const binary b = compiler::link_program(bs, dev, opts, log);
_builds[&dev] = { b, opts, log };
} catch (...) {
_builds[&dev] = { binary(), opts, log };
throw;
}
}
}
enum program::il_type
program::il_type() const {
return _il_type;
}
const std::string &
program::source() const {
return _source;
}
program::device_range
program::devices() const {
return map(evals(), _devices);
}
cl_build_status
program::build::status() const {
if (!bin.secs.empty())
return CL_BUILD_SUCCESS;
else if (log.size())
return CL_BUILD_ERROR;
else
return CL_BUILD_NONE;
}
cl_program_binary_type
program::build::binary_type() const {
if (any_of(type_equals(binary::section::text_intermediate), bin.secs))
return CL_PROGRAM_BINARY_TYPE_COMPILED_OBJECT;
else if (any_of(type_equals(binary::section::text_library), bin.secs))
return CL_PROGRAM_BINARY_TYPE_LIBRARY;
else if (any_of(type_equals(binary::section::text_executable), bin.secs))
return CL_PROGRAM_BINARY_TYPE_EXECUTABLE;
else
return CL_PROGRAM_BINARY_TYPE_NONE;
}
const struct program::build &
program::build(const device &dev) const {
static const struct build null;
return _builds.count(&dev) ? _builds.find(&dev)->second : null;
}
const std::vector<binary::symbol> &
program::symbols() const {
if (_builds.empty())
throw error(CL_INVALID_PROGRAM_EXECUTABLE);
return _builds.begin()->second.bin.syms;
}
unsigned
program::kernel_ref_count() const {
return _kernel_ref_counter.ref_count();
}

95
src/gallium/frontends/clover/core/program.hpp
View file

@ -1,95 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_PROGRAM_HPP
#define CLOVER_CORE_PROGRAM_HPP
#include <map>
#include "core/object.hpp"
#include "core/context.hpp"
#include "core/binary.hpp"
namespace clover {
typedef std::vector<std::pair<std::string, std::string>> header_map;
class program : public ref_counter, public _cl_program {
private:
typedef adaptor_range<
evals, const std::vector<intrusive_ref<device>> &> device_range;
public:
enum class il_type { none, source, spirv };
program(clover::context &ctx,
std::string &&il,
enum il_type il_type);
program(clover::context &ctx,
const ref_vector<device> &devs = {},
const std::vector<binary> &binaries = {});
program(const program &prog) = delete;
program &
operator=(const program &prog) = delete;
void compile(const ref_vector<device> &devs, const std::string &opts,
const header_map &headers = {});
void link(const ref_vector<device> &devs, const std::string &opts,
const ref_vector<program> &progs);
const std::string &source() const;
enum il_type il_type() const;
device_range devices() const;
struct build {
build(const binary &b = {}, const std::string &opts = {},
const std::string &log = {}) : bin(b), opts(opts), log(log) {}
cl_build_status status() const;
cl_program_binary_type binary_type() const;
binary bin;
std::string opts;
std::string log;
};
const build &build(const device &dev) const;
const std::vector<binary::symbol> &symbols() const;
unsigned kernel_ref_count() const;
const intrusive_ref<clover::context> context;
friend class kernel;
private:
std::vector<intrusive_ref<device>> _devices;
std::map<const device *, struct build> _builds;
std::string _source;
ref_counter _kernel_ref_counter;
enum il_type _il_type;
};
}
#endif

267
src/gallium/frontends/clover/core/property.hpp
View file

@ -1,267 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_PROPERTY_HPP
#define CLOVER_CORE_PROPERTY_HPP
#include <map>
#include "util/range.hpp"
#include "util/algorithm.hpp"
namespace clover {
class property_buffer;
namespace detail {
template<typename T>
class property_scalar {
public:
property_scalar(property_buffer &buf) : buf(buf) {
}
inline property_scalar &
operator=(const T &x);
private:
property_buffer &buf;
};
template<typename T>
class property_vector {
public:
property_vector(property_buffer &buf) : buf(buf) {
}
template<typename S>
inline property_vector &
operator=(const S &v);
private:
property_buffer &buf;
};
template<typename T>
class property_matrix {
public:
property_matrix(property_buffer &buf) : buf(buf) {
}
template<typename S>
inline property_matrix &
operator=(const S &v);
private:
property_buffer &buf;
};
class property_string {
public:
property_string(property_buffer &buf) : buf(buf) {
}
inline property_string &
operator=(const std::string &v);
private:
property_buffer &buf;
};
};
///
/// Return value buffer used by the CL property query functions.
///
class property_buffer {
public:
property_buffer(void *r_buf, size_t size, size_t *r_size) :
r_buf(r_buf), size(size), r_size(r_size) {
}
template<typename T>
detail::property_scalar<T>
as_scalar() {
return { *this };
}
template<typename T>
detail::property_vector<T>
as_vector() {
return { *this };
}
template<typename T>
detail::property_matrix<T>
as_matrix() {
return { *this };
}
detail::property_string
as_string() {
return { *this };
}
template<typename T>
iterator_range<T *>
allocate(size_t n) {
if (r_buf && size < n * sizeof(T))
throw error(CL_INVALID_VALUE);
if (r_size)
*r_size = n * sizeof(T);
if (r_buf)
return range((T *)r_buf, n);
else
return { };
}
private:
void *const r_buf;
const size_t size;
size_t *const r_size;
};
namespace detail {
template<typename T>
inline property_scalar<T> &
property_scalar<T>::operator=(const T &x) {
auto r = buf.allocate<T>(1);
if (!r.empty())
r.front() = x;
return *this;
}
template<typename T>
template<typename S>
inline property_vector<T> &
property_vector<T>::operator=(const S &v) {
auto r = buf.allocate<T>(v.size());
if (!r.empty())
copy(v, r.begin());
return *this;
}
template<typename T>
template<typename S>
inline property_matrix<T> &
property_matrix<T>::operator=(const S &v) {
auto r = buf.allocate<T *>(v.size());
if (!r.empty())
for_each([](typename S::value_type src, T *dst) {
if (dst)
copy(src, dst);
}, v, r);
return *this;
}
inline property_string &
property_string::operator=(const std::string &v) {
auto r = buf.allocate<char>(v.size() + 1);
if (!r.empty())
copy(range(v.begin(), r.size()), r.begin());
return *this;
}
};
template<typename T>
class property_element {
public:
property_element() : x() {
}
property_element(T x) : x(x) {
}
template<typename S>
typename std::enable_if<!std::is_convertible<T, S>::value, S>::type
as() const {
static_assert(sizeof(S) <= sizeof(T), "Ensure type fits in property list");
return reinterpret_cast<S>(x);
}
template<typename S>
typename std::enable_if<std::is_convertible<T, S>::value, S>::type
as() const {
return static_cast<S>(x);
}
private:
T x;
};
template<typename D>
using property_list = std::map<D, property_element<D>>;
struct property_list_tag;
///
/// Create a clover::property_list object from a zero-terminated
/// CL property list.
///
template<typename T, typename D,
typename = typename std::enable_if<
std::is_same<T, property_list_tag>::value>::type>
property_list<D>
obj(const D *d_props) {
property_list<D> props;
while (d_props && *d_props) {
auto key = *d_props++;
auto value = *d_props++;
if (props.count(key))
throw error(CL_INVALID_PROPERTY);
props.insert({ key, value });
}
return props;
}
///
/// Create a zero-terminated CL property list from a
/// clover::property_list object.
///
template<typename D>
std::vector<D>
desc(const property_list<D> &props) {
std::vector<D> d_props;
for (auto &prop : props) {
d_props.push_back(prop.first);
d_props.push_back(prop.second.template as<D>());
}
d_props.push_back(0);
return d_props;
}
}
#endif

158
src/gallium/frontends/clover/core/queue.cpp
View file

@ -1,158 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/queue.hpp"
#include "core/event.hpp"
#include "pipe/p_screen.h"
#include "pipe/p_context.h"
#include "pipe/p_state.h"
#include "util/u_debug.h"
using namespace clover;
namespace {
void
debug_notify_callback(void *data,
unsigned *id,
enum util_debug_type type,
const char *fmt,
va_list args) {
const command_queue *queue = (const command_queue *)data;
char buffer[1024];
vsnprintf(buffer, sizeof(buffer), fmt, args);
queue->context().notify(buffer);
}
}
command_queue::command_queue(clover::context &ctx, clover::device &dev,
cl_command_queue_properties props) :
context(ctx), device(dev), _props(props) {
pipe = dev.pipe->context_create(dev.pipe, NULL, PIPE_CONTEXT_COMPUTE_ONLY);
if (!pipe)
throw error(CL_INVALID_DEVICE);
if (ctx.notify) {
struct util_debug_callback cb;
memset(&cb, 0, sizeof(cb));
cb.debug_message = &debug_notify_callback;
cb.data = this;
if (pipe->set_debug_callback)
pipe->set_debug_callback(pipe, &cb);
}
}
command_queue::command_queue(clover::context &ctx, clover::device &dev,
std::vector<cl_queue_properties> properties) :
context(ctx), device(dev), _properties(properties), _props(0) {
for(std::vector<cl_queue_properties>::size_type i = 0; i != properties.size(); i += 2) {
if (properties[i] == 0)
break;
if (properties[i] == CL_QUEUE_PROPERTIES)
_props |= properties[i + 1];
else if (properties[i] != CL_QUEUE_SIZE)
throw error(CL_INVALID_VALUE);
}
pipe = dev.pipe->context_create(dev.pipe, NULL, PIPE_CONTEXT_COMPUTE_ONLY);
if (!pipe)
throw error(CL_INVALID_DEVICE);
if (ctx.notify) {
struct util_debug_callback cb;
memset(&cb, 0, sizeof(cb));
cb.debug_message = &debug_notify_callback;
cb.data = this;
if (pipe->set_debug_callback)
pipe->set_debug_callback(pipe, &cb);
}
}
command_queue::~command_queue() {
pipe->destroy(pipe);
}
void
command_queue::flush() {
std::lock_guard<std::mutex> lock(queued_events_mutex);
flush_unlocked();
}
void
command_queue::flush_unlocked() {
pipe_screen *screen = device().pipe;
pipe_fence_handle *fence = NULL;
if (!queued_events.empty()) {
pipe->flush(pipe, &fence, 0);
while (!queued_events.empty() &&
queued_events.front()().signalled()) {
queued_events.front()().fence(fence);
queued_events.pop_front();
}
screen->fence_reference(screen, &fence, NULL);
}
}
void
command_queue::svm_migrate(const std::vector<void const*> &svm_pointers,
const std::vector<size_t> &sizes,
cl_mem_migration_flags flags) {
if (!pipe->svm_migrate)
return;
bool to_device = !(flags & CL_MIGRATE_MEM_OBJECT_HOST);
bool mem_undefined = flags & CL_MIGRATE_MEM_OBJECT_CONTENT_UNDEFINED;
pipe->svm_migrate(pipe, svm_pointers.size(), svm_pointers.data(),
sizes.data(), to_device, mem_undefined);
}
cl_command_queue_properties
command_queue::props() const {
return _props;
}
std::vector<cl_queue_properties>
command_queue::properties() const {
return _properties;
}
bool
command_queue::profiling_enabled() const {
return _props & CL_QUEUE_PROFILING_ENABLE;
}
void
command_queue::sequence(hard_event &ev) {
std::lock_guard<std::mutex> lock(queued_events_mutex);
if (!queued_events.empty())
queued_events.back()().chain(ev);
queued_events.push_back(ev);
// Arbitrary threshold.
// The CTS tends to run a lot of subtests without flushing with the image
// tests, so flush regularly to prevent stack overflows.
if (queued_events.size() > 1000)
flush_unlocked();
}

87
src/gallium/frontends/clover/core/queue.hpp
View file

@ -1,87 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_QUEUE_HPP
#define CLOVER_CORE_QUEUE_HPP
#include <deque>
#include <mutex>
#include "core/object.hpp"
#include "core/context.hpp"
#include "core/timestamp.hpp"
#include "pipe/p_context.h"
namespace clover {
class resource;
class mapping;
class hard_event;
class command_queue : public ref_counter, public _cl_command_queue {
public:
command_queue(clover::context &ctx, clover::device &dev,
std::vector<cl_queue_properties> properties);
command_queue(clover::context &ctx, clover::device &dev,
cl_command_queue_properties props);
~command_queue();
command_queue(const command_queue &q) = delete;
command_queue &
operator=(const command_queue &q) = delete;
void flush();
void svm_migrate(const std::vector<void const *> &svm_pointers,
const std::vector<size_t> &sizes, cl_mem_migration_flags flags);
cl_command_queue_properties props() const;
std::vector<cl_queue_properties> properties() const;
bool profiling_enabled() const;
const intrusive_ref<clover::context> context;
const intrusive_ref<clover::device> device;
friend class resource;
friend class root_resource;
friend class mapping;
friend class hard_event;
friend class sampler;
friend class kernel;
friend class clover::timestamp::query;
friend class clover::timestamp::current;
private:
/// Serialize a hardware event with respect to the previous ones,
/// and push it to the pending list.
void sequence(hard_event &ev);
// Use this instead of flush() if `queued_events_mutex` is acquired.
void flush_unlocked();
std::vector<cl_queue_properties> _properties;
cl_command_queue_properties _props;
pipe_context *pipe;
std::mutex queued_events_mutex;
std::deque<intrusive_ref<hard_event>> queued_events;
};
}
#endif

281
src/gallium/frontends/clover/core/resource.cpp
View file

@ -1,281 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/resource.hpp"
#include "core/memory.hpp"
#include "pipe/p_screen.h"
#include "util/u_sampler.h"
#include "util/format/u_format.h"
#include "util/u_inlines.h"
#include "util/u_resource.h"
#include "util/u_surface.h"
using namespace clover;
namespace {
class box {
public:
box(const resource::vector &origin, const resource::vector &size) {
u_box_3d(origin[0], origin[1], origin[2], size[0], size[1], size[2], &pipe);
}
operator const pipe_box *() {
return &pipe;
}
protected:
pipe_box pipe;
};
}
resource::resource(clover::device &dev, memory_obj &obj) :
device(dev), obj(obj), pipe(NULL), offset() {
}
resource::~resource() {
}
void
resource::copy(command_queue &q, const vector &origin, const vector &region,
resource &src_res, const vector &src_origin) {
auto p = offset + origin;
q.pipe->resource_copy_region(q.pipe, pipe, 0, p[0], p[1], p[2],
src_res.pipe, 0,
box(src_res.offset + src_origin, region));
}
void
resource::clear(command_queue &q, const vector &origin, const vector &region,
const std::string &data) {
auto from = offset + origin;
if (pipe->target == PIPE_BUFFER) {
q.pipe->clear_buffer(q.pipe, pipe, from[0], region[0], data.data(), data.size());
} else {
std::string texture_data;
texture_data.reserve(util_format_get_blocksize(pipe->format));
util_format_pack_rgba(pipe->format, &texture_data[0], data.data(), 1);
if (q.pipe->clear_texture) {
q.pipe->clear_texture(q.pipe, pipe, 0, box(from, region), texture_data.data());
} else {
u_default_clear_texture(q.pipe, pipe, 0, box(from, region), texture_data.data());
}
}
}
mapping *
resource::add_map(command_queue &q, cl_map_flags flags, bool blocking,
const vector &origin, const vector &region) {
maps.emplace_back(q, *this, flags, blocking, origin, region);
return &maps.back();
}
void
resource::del_map(void *p) {
erase_if([&](const mapping &b) {
return static_cast<void *>(b) == p;
}, maps);
}
unsigned
resource::map_count() const {
return maps.size();
}
pipe_sampler_view *
resource::bind_sampler_view(command_queue &q) {
pipe_sampler_view info;
u_sampler_view_default_template(&info, pipe, pipe->format);
return q.pipe->create_sampler_view(q.pipe, pipe, &info);
}
void
resource::unbind_sampler_view(command_queue &q,
pipe_sampler_view *st) {
q.pipe->sampler_view_release(q.pipe, st);
}
pipe_image_view
resource::create_image_view(command_queue &q) {
pipe_image_view view;
view.resource = pipe;
view.format = pipe->format;
view.access = 0;
view.shader_access = PIPE_IMAGE_ACCESS_WRITE;
if (pipe->target == PIPE_BUFFER) {
view.u.buf.offset = 0;
view.u.buf.size = obj.size();
} else {
view.u.tex.first_layer = 0;
if (util_texture_is_array(pipe->target))
view.u.tex.last_layer = pipe->array_size - 1;
else
view.u.tex.last_layer = 0;
view.u.tex.level = 0;
}
return view;
}
pipe_surface *
resource::bind_surface(command_queue &q, bool rw) {
pipe_surface info {};
info.format = pipe->format;
info.writable = rw;
if (pipe->target == PIPE_BUFFER)
info.u.buf.last_element = pipe->width0 - 1;
return q.pipe->create_surface(q.pipe, pipe, &info);
}
void
resource::unbind_surface(command_queue &q, pipe_surface *st) {
q.pipe->surface_destroy(q.pipe, st);
}
root_resource::root_resource(clover::device &dev, memory_obj &obj,
command_queue &q, const void *data_ptr) :
resource(dev, obj) {
pipe_resource info {};
if (image *img = dynamic_cast<image *>(&obj)) {
info.format = translate_format(img->format());
info.width0 = img->width();
info.height0 = img->height();
info.depth0 = img->depth();
info.array_size = MAX2(1, img->array_size());
} else {
info.width0 = obj.size();
info.height0 = 1;
info.depth0 = 1;
info.array_size = 1;
}
info.target = translate_target(obj.type());
info.bind = (PIPE_BIND_SAMPLER_VIEW |
PIPE_BIND_COMPUTE_RESOURCE |
PIPE_BIND_GLOBAL);
if (obj.flags() & CL_MEM_USE_HOST_PTR && dev.allows_user_pointers()) {
// Page alignment is normally required for this, just try, hope for the
// best and fall back if it fails.
pipe = dev.pipe->resource_from_user_memory(dev.pipe, &info, obj.host_ptr());
if (pipe)
return;
}
if (obj.flags() & (CL_MEM_ALLOC_HOST_PTR | CL_MEM_USE_HOST_PTR)) {
info.usage = PIPE_USAGE_STAGING;
}
pipe = dev.pipe->resource_create(dev.pipe, &info);
if (!pipe)
throw error(CL_OUT_OF_RESOURCES);
if (data_ptr) {
box rect { {{ 0, 0, 0 }}, {{ info.width0, info.height0, info.depth0 }} };
unsigned cpp = util_format_get_blocksize(info.format);
if (pipe->target == PIPE_BUFFER)
q.pipe->buffer_subdata(q.pipe, pipe, PIPE_MAP_WRITE,
0, info.width0, data_ptr);
else
q.pipe->texture_subdata(q.pipe, pipe, 0, PIPE_MAP_WRITE,
rect, data_ptr, cpp * info.width0,
cpp * info.width0 * info.height0);
}
}
root_resource::root_resource(clover::device &dev, memory_obj &obj,
root_resource &r) :
resource(dev, obj) {
assert(0); // XXX -- resource shared among dev and r.dev
}
root_resource::~root_resource() {
pipe_resource_reference(&this->pipe, NULL);
}
sub_resource::sub_resource(resource &r, const vector &offset) :
resource(r.device(), r.obj) {
this->pipe = r.pipe;
this->offset = r.offset + offset;
}
mapping::mapping(command_queue &q, resource &r,
cl_map_flags flags, bool blocking,
const resource::vector &origin,
const resource::vector &region) :
pctx(q.pipe), pres(NULL) {
unsigned usage = ((flags & CL_MAP_WRITE ? PIPE_MAP_WRITE : 0 ) |
(flags & CL_MAP_READ ? PIPE_MAP_READ : 0 ) |
(flags & CL_MAP_WRITE_INVALIDATE_REGION ?
PIPE_MAP_DISCARD_RANGE : 0) |
(!blocking ? PIPE_MAP_UNSYNCHRONIZED : 0));
p = pctx->buffer_map(pctx, r.pipe, 0, usage,
box(origin + r.offset, region), &pxfer);
if (!p) {
pxfer = NULL;
throw error(CL_OUT_OF_RESOURCES);
}
pipe_resource_reference(&pres, r.pipe);
}
mapping::mapping(mapping &&m) :
pctx(m.pctx), pxfer(m.pxfer), pres(m.pres), p(m.p) {
m.pctx = NULL;
m.pxfer = NULL;
m.pres = NULL;
m.p = NULL;
}
mapping::~mapping() {
if (pxfer) {
pctx->buffer_unmap(pctx, pxfer);
}
pipe_resource_reference(&pres, NULL);
}
mapping &
mapping::operator=(mapping m) {
std::swap(pctx, m.pctx);
std::swap(pxfer, m.pxfer);
std::swap(pres, m.pres);
std::swap(p, m.p);
return *this;
}
resource::vector
mapping::pitch() const
{
return {
util_format_get_blocksize(pres->format),
pxfer->stride,
pxfer->layer_stride,
};
}

140
src/gallium/frontends/clover/core/resource.hpp
View file

@ -1,140 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_RESOURCE_HPP
#define CLOVER_CORE_RESOURCE_HPP
#include <list>
#include "core/queue.hpp"
#include "util/algebra.hpp"
#include "pipe/p_state.h"
namespace clover {
class memory_obj;
class mapping;
///
/// Class that represents a device-specific instance of some memory
/// object.
///
class resource {
public:
typedef std::array<size_t, 3> vector;
virtual ~resource();
resource(const resource &r) = delete;
resource &
operator=(const resource &r) = delete;
void copy(command_queue &q, const vector &origin, const vector &region,
resource &src_resource, const vector &src_origin);
void clear(command_queue &q, const vector &origin, const vector &region,
const std::string &data);
mapping *add_map(command_queue &q, cl_map_flags flags, bool blocking,
const vector &origin, const vector &region);
void del_map(void *p);
unsigned map_count() const;
const intrusive_ref<clover::device> device;
memory_obj &obj;
friend class sub_resource;
friend class mapping;
friend class kernel;
protected:
resource(clover::device &dev, memory_obj &obj);
pipe_sampler_view *bind_sampler_view(command_queue &q);
void unbind_sampler_view(command_queue &q,
pipe_sampler_view *st);
pipe_surface *bind_surface(command_queue &q, bool rw);
void unbind_surface(command_queue &q, pipe_surface *st);
pipe_image_view create_image_view(command_queue &q);
pipe_resource *pipe;
vector offset;
private:
std::list<mapping> maps;
};
///
/// Resource associated with its own top-level data storage
/// allocated in some device.
///
class root_resource : public resource {
public:
root_resource(clover::device &dev, memory_obj &obj,
command_queue &q, const void *data_ptr);
root_resource(clover::device &dev, memory_obj &obj, root_resource &r);
virtual ~root_resource();
};
///
/// Resource that reuses a portion of some other resource as data
/// storage.
///
class sub_resource : public resource {
public:
sub_resource(resource &r, const vector &offset);
};
///
/// Class that represents a mapping of some resource into the CPU
/// memory space.
///
class mapping {
public:
mapping(command_queue &q, resource &r, cl_map_flags flags,
bool blocking, const resource::vector &origin,
const resource::vector &region);
mapping(mapping &&m);
~mapping();
mapping &
operator=(mapping m);
mapping(const mapping &m) = delete;
template<typename T>
operator T *() const {
return (T *)p;
}
resource::vector pitch() const;
private:
pipe_context *pctx;
pipe_transfer *pxfer;
pipe_resource *pres;
void *p;
};
}
#endif

73
src/gallium/frontends/clover/core/sampler.cpp
View file

@ -1,73 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/sampler.hpp"
#include "pipe/p_state.h"
using namespace clover;
sampler::sampler(clover::context &ctx, bool norm_mode,
cl_addressing_mode addr_mode,
cl_filter_mode filter_mode) :
context(ctx), _norm_mode(norm_mode),
_addr_mode(addr_mode), _filter_mode(filter_mode) {
}
bool
sampler::norm_mode() {
return _norm_mode;
}
cl_addressing_mode
sampler::addr_mode() {
return _addr_mode;
}
cl_filter_mode
sampler::filter_mode() {
return _filter_mode;
}
void *
sampler::bind(command_queue &q) {
struct pipe_sampler_state info {};
info.unnormalized_coords = !norm_mode();
info.wrap_s = info.wrap_t = info.wrap_r =
(addr_mode() == CL_ADDRESS_CLAMP_TO_EDGE ? PIPE_TEX_WRAP_CLAMP_TO_EDGE :
addr_mode() == CL_ADDRESS_CLAMP ? PIPE_TEX_WRAP_CLAMP_TO_BORDER :
addr_mode() == CL_ADDRESS_REPEAT ? PIPE_TEX_WRAP_REPEAT :
addr_mode() == CL_ADDRESS_MIRRORED_REPEAT ? PIPE_TEX_WRAP_MIRROR_REPEAT :
PIPE_TEX_WRAP_CLAMP_TO_EDGE);
info.min_img_filter = info.mag_img_filter =
(filter_mode() == CL_FILTER_LINEAR ? PIPE_TEX_FILTER_LINEAR :
PIPE_TEX_FILTER_NEAREST);
return q.pipe->create_sampler_state(q.pipe, &info);
}
void
sampler::unbind(command_queue &q, void *st) {
q.pipe->delete_sampler_state(q.pipe, st);
}

58
src/gallium/frontends/clover/core/sampler.hpp
View file

@ -1,58 +0,0 @@
//
// Copyright 2012 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_SAMPLER_HPP
#define CLOVER_CORE_SAMPLER_HPP
#include "core/object.hpp"
#include "core/queue.hpp"
namespace clover {
class sampler : public ref_counter, public _cl_sampler {
public:
sampler(clover::context &ctx, bool norm_mode,
cl_addressing_mode addr_mode,
cl_filter_mode filter_mode);
sampler(const sampler &s) = delete;
sampler &
operator=(const sampler &s) = delete;
bool norm_mode();
cl_addressing_mode addr_mode();
cl_filter_mode filter_mode();
const intrusive_ref<clover::context> context;
friend class kernel;
private:
void *bind(command_queue &q);
void unbind(command_queue &q, void *st);
bool _norm_mode;
cl_addressing_mode _addr_mode;
cl_filter_mode _filter_mode;
};
}
#endif

64
src/gallium/frontends/clover/core/timestamp.cpp
View file

@ -1,64 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#include "core/timestamp.hpp"
#include "core/queue.hpp"
#include "pipe/p_screen.h"
#include "pipe/p_context.h"
using namespace clover;
timestamp::query::query(command_queue &q) :
q(q),
_query(q.pipe->create_query(q.pipe, PIPE_QUERY_TIMESTAMP, 0)) {
q.pipe->end_query(q.pipe, _query);
}
timestamp::query::query(query &&other) :
q(other.q),
_query(other._query) {
other._query = NULL;
}
timestamp::query::~query() {
if (_query)
q().pipe->destroy_query(q().pipe, _query);
}
cl_ulong
timestamp::query::operator()() const {
pipe_query_result result;
if (!q().pipe->get_query_result(q().pipe, _query, false, &result))
throw error(CL_PROFILING_INFO_NOT_AVAILABLE);
return result.u64;
}
timestamp::current::current(command_queue &q) :
result(q.pipe->screen->get_timestamp(q.pipe->screen)) {
}
cl_ulong
timestamp::current::operator()() const {
return result;
}

74
src/gallium/frontends/clover/core/timestamp.hpp
View file

@ -1,74 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_CORE_TIMESTAMP_HPP
#define CLOVER_CORE_TIMESTAMP_HPP
#include "core/object.hpp"
struct pipe_query;
namespace clover {
class command_queue;
namespace timestamp {
///
/// Emit a timestamp query that is executed asynchronously by
/// the command queue \a q.
///
class query {
public:
query(command_queue &q);
query(query &&other);
~query();
query &operator=(const query &) = delete;
///
/// Retrieve the query results.
///
cl_ulong operator()() const;
private:
const intrusive_ref<command_queue> q;
pipe_query *_query;
};
///
/// Get the current timestamp value.
///
class current {
public:
current(command_queue &q);
///
/// Retrieve the query results.
///
cl_ulong operator()() const;
private:
cl_ulong result;
};
}
}
#endif

68
src/gallium/frontends/clover/llvm/codegen.hpp
View file

@ -1,68 +0,0 @@
//
// Copyright 2016 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
///
/// \file
/// Tools to generate various forms of binary code from existing LLVM IR in
/// the given llvm::Module object and output the result as a clover::binary.
///
#ifndef CLOVER_LLVM_CODEGEN_HPP
#define CLOVER_LLVM_CODEGEN_HPP
#include "llvm/util.hpp"
#include "core/binary.hpp"
#include <llvm/IR/Module.h>
#include <clang/Frontend/CompilerInstance.h>
namespace clover {
namespace llvm {
std::string
print_module_bitcode(const ::llvm::Module &mod);
binary
build_module_library(const ::llvm::Module &mod,
enum binary::section::type section_type);
std::unique_ptr< ::llvm::Module>
parse_module_library(const binary &b, ::llvm::LLVMContext &ctx,
std::string &r_log);
binary
build_module_native(::llvm::Module &mod, const target &target,
const clang::CompilerInstance &c,
std::string &r_log);
std::string
print_module_native(const ::llvm::Module &mod, const target &target);
binary
build_module_common(const ::llvm::Module &mod,
const std::vector<char> &code,
const std::map<std::string, unsigned> &offsets,
const clang::CompilerInstance &c);
}
}
#endif

91
src/gallium/frontends/clover/llvm/codegen/bitcode.cpp
View file

@ -1,91 +0,0 @@
//
// Copyright 2012-2016 Francisco Jerez
// Copyright 2012-2016 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
///
/// \file
/// Trivial codegen back-end that simply passes through the existing LLVM IR
/// and either formats it so it can be consumed by pipe drivers (if
/// build_module_bitcode() is used) or serializes so it can be deserialized at
/// a later point and passed to the actual codegen back-end (if
/// build_module_library() / parse_module_library() is used), potentially
/// after linking against other bitcode object files.
///
#include <llvm/Support/Allocator.h>
#include "llvm/codegen.hpp"
#include "llvm/compat.hpp"
#include "llvm/metadata.hpp"
#include "core/error.hpp"
#include "util/algorithm.hpp"
#include <map>
#include <llvm/Config/llvm-config.h>
#include <llvm/Bitcode/BitcodeReader.h>
#include <llvm/Bitcode/BitcodeWriter.h>
#include <llvm/Support/raw_ostream.h>
using clover::binary;
using namespace clover::llvm;
namespace {
std::vector<char>
emit_code(const ::llvm::Module &mod) {
::llvm::SmallVector<char, 1024> data;
::llvm::raw_svector_ostream os { data };
::llvm::WriteBitcodeToFile(mod, os);
return { os.str().begin(), os.str().end() };
}
}
std::string
clover::llvm::print_module_bitcode(const ::llvm::Module &mod) {
std::string s;
::llvm::raw_string_ostream os { s };
mod.print(os, NULL);
return os.str();
}
binary
clover::llvm::build_module_library(const ::llvm::Module &mod,
enum binary::section::type section_type) {
binary b;
const auto code = emit_code(mod);
b.secs.emplace_back(0, section_type, code.size(), code);
return b;
}
std::unique_ptr< ::llvm::Module>
clover::llvm::parse_module_library(const binary &b, ::llvm::LLVMContext &ctx,
std::string &r_log) {
auto mod = ::llvm::parseBitcodeFile(::llvm::MemoryBufferRef(
as_string(b.secs[0].data), " "), ctx);
if (::llvm::Error err = mod.takeError()) {
::llvm::handleAllErrors(std::move(err), [&](::llvm::ErrorInfoBase &eib) {
fail(r_log, error(CL_INVALID_PROGRAM), eib.message());
});
}
return std::unique_ptr< ::llvm::Module>(std::move(*mod));
}

358
src/gallium/frontends/clover/llvm/codegen/common.cpp
View file

@ -1,358 +0,0 @@
//
// Copyright 2012-2016 Francisco Jerez
// Copyright 2012-2016 Advanced Micro Devices, Inc.
// Copyright 2015 Zoltan Gilian
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
///
/// \file
/// Codegen back-end-independent part of the construction of an executable
/// clover::binary, including kernel argument metadata extraction and
/// formatting of the pre-generated binary code in a form that can be
/// understood by pipe drivers.
///
#include <llvm/IR/Type.h>
#include <llvm/Support/Allocator.h>
#include "llvm/codegen.hpp"
#include "llvm/compat.hpp"
#include "llvm/metadata.hpp"
#include "CL/cl.h"
#include "pipe/p_state.h"
#include "util/u_math.h"
#include <clang/Basic/TargetInfo.h>
using clover::binary;
using clover::detokenize;
using namespace clover::llvm;
using ::llvm::Module;
using ::llvm::Function;
using ::llvm::Type;
using ::llvm::isa;
using ::llvm::cast;
using ::llvm::dyn_cast;
namespace {
enum binary::argument::type
get_image_type(const std::string &type,
const std::string &qual) {
if (type == "image1d_t" || type == "image2d_t" || type == "image3d_t") {
if (qual == "read_only")
return binary::argument::image_rd;
else if (qual == "write_only")
return binary::argument::image_wr;
}
unreachable("Unsupported image type");
}
binary::arg_info create_arg_info(const std::string &arg_name,
const std::string &type_name,
const std::string &type_qualifier,
const uint64_t address_qualifier,
const std::string &access_qualifier) {
cl_kernel_arg_type_qualifier cl_type_qualifier =
CL_KERNEL_ARG_TYPE_NONE;
if (type_qualifier.find("const") != std::string::npos)
cl_type_qualifier |= CL_KERNEL_ARG_TYPE_CONST;
if (type_qualifier.find("restrict") != std::string::npos)
cl_type_qualifier |= CL_KERNEL_ARG_TYPE_RESTRICT;
if (type_qualifier.find("volatile") != std::string::npos)
cl_type_qualifier |= CL_KERNEL_ARG_TYPE_VOLATILE;
cl_kernel_arg_address_qualifier cl_address_qualifier =
CL_KERNEL_ARG_ADDRESS_PRIVATE;
if (address_qualifier == 1)
cl_address_qualifier = CL_KERNEL_ARG_ADDRESS_GLOBAL;
else if (address_qualifier == 2)
cl_address_qualifier = CL_KERNEL_ARG_ADDRESS_CONSTANT;
else if (address_qualifier == 3)
cl_address_qualifier = CL_KERNEL_ARG_ADDRESS_LOCAL;
cl_kernel_arg_access_qualifier cl_access_qualifier =
CL_KERNEL_ARG_ACCESS_NONE;
if (access_qualifier == "read_only")
cl_access_qualifier = CL_KERNEL_ARG_ACCESS_READ_ONLY;
else if (access_qualifier == "write_only")
cl_access_qualifier = CL_KERNEL_ARG_ACCESS_WRITE_ONLY;
else if (access_qualifier == "read_write")
cl_access_qualifier = CL_KERNEL_ARG_ACCESS_READ_WRITE;
return binary::arg_info(arg_name, type_name, cl_type_qualifier,
cl_address_qualifier, cl_access_qualifier);
}
std::vector<size_t>
get_reqd_work_group_size(const Module &mod,
const std::string &kernel_name) {
const Function &f = *mod.getFunction(kernel_name);
auto vector_metadata = get_uint_vector_kernel_metadata(f, "reqd_work_group_size");
return vector_metadata.empty() ? std::vector<size_t>({0, 0, 0}) : vector_metadata;
}
std::string
kernel_attributes(const Module &mod, const std::string &kernel_name) {
std::vector<std::string> attributes;
const Function &f = *mod.getFunction(kernel_name);
auto vec_type_hint = get_type_kernel_metadata(f, "vec_type_hint");
if (!vec_type_hint.empty())
attributes.emplace_back("vec_type_hint(" + vec_type_hint + ")");
auto work_group_size_hint = get_uint_vector_kernel_metadata(f, "work_group_size_hint");
if (!work_group_size_hint.empty()) {
std::string s = "work_group_size_hint(";
s += detokenize(work_group_size_hint, ",");
s += ")";
attributes.emplace_back(s);
}
auto reqd_work_group_size = get_uint_vector_kernel_metadata(f, "reqd_work_group_size");
if (!reqd_work_group_size.empty()) {
std::string s = "reqd_work_group_size(";
s += detokenize(reqd_work_group_size, ",");
s += ")";
attributes.emplace_back(s);
}
auto nosvm = get_str_kernel_metadata(f, "nosvm");
if (!nosvm.empty())
attributes.emplace_back("nosvm");
return detokenize(attributes, " ");
}
// Parse the type which are pointers to CL vector types with no prefix.
// so e.g. char/uchar, short/ushort, int/uint, long/ulong
// half/float/double, followed by the vector length, followed by *.
// uint8 is 8x32-bit integer, short4 is 4x16-bit integer etc.
// Since this is a pointer only path, assert the * is on the end.
::llvm::Type *
ptr_arg_to_llvm_type(const Module &mod, std::string type_name) {
int len = type_name.length();
assert (type_name[len-1] == '*');
::llvm::Type *base_type = NULL;
if (type_name.find("void") != std::string::npos)
base_type = ::llvm::Type::getVoidTy(mod.getContext());
else if (type_name.find("char") != std::string::npos)
base_type = ::llvm::Type::getInt8Ty(mod.getContext());
else if (type_name.find("short") != std::string::npos)
base_type = ::llvm::Type::getInt16Ty(mod.getContext());
else if (type_name.find("int") != std::string::npos)
base_type = ::llvm::Type::getInt32Ty(mod.getContext());
else if (type_name.find("long") != std::string::npos)
base_type = ::llvm::Type::getInt64Ty(mod.getContext());
else if (type_name.find("half") != std::string::npos)
base_type = ::llvm::Type::getHalfTy(mod.getContext());
else if (type_name.find("float") != std::string::npos)
base_type = ::llvm::Type::getFloatTy(mod.getContext());
else if (type_name.find("double") != std::string::npos)
base_type = ::llvm::Type::getDoubleTy(mod.getContext());
assert(base_type);
if (type_name.find("2") != std::string::npos)
base_type = ::llvm::FixedVectorType::get(base_type, 2);
else if (type_name.find("3") != std::string::npos)
base_type = ::llvm::FixedVectorType::get(base_type, 3);
else if (type_name.find("4") != std::string::npos)
base_type = ::llvm::FixedVectorType::get(base_type, 4);
else if (type_name.find("8") != std::string::npos)
base_type = ::llvm::FixedVectorType::get(base_type, 8);
else if (type_name.find("16") != std::string::npos)
base_type = ::llvm::FixedVectorType::get(base_type, 16);
return base_type;
}
std::vector<binary::argument>
make_kernel_args(const Module &mod, const std::string &kernel_name,
const clang::CompilerInstance &c) {
std::vector<binary::argument> args;
const Function &f = *mod.getFunction(kernel_name);
#if LLVM_VERSION_MAJOR >= 20
const ::llvm::DataLayout &dl = mod.getDataLayout();
#else
::llvm::DataLayout dl(&mod);
#endif
const auto size_type =
dl.getSmallestLegalIntType(mod.getContext(), sizeof(cl_uint) * 8);
const unsigned size_align = compat::get_abi_type_alignment(dl, size_type);
for (const auto &arg : f.args()) {
const auto arg_type = arg.getType();
// OpenCL 1.2 specification, Ch. 6.1.5: "A built-in data
// type that is not a power of two bytes in size must be
// aligned to the next larger power of two.
// This rule applies to built-in types only, not structs or unions."
const unsigned arg_api_size = dl.getTypeAllocSize(arg_type);
const unsigned target_size = dl.getTypeStoreSize(arg_type);
const unsigned target_align = compat::get_abi_type_alignment(dl, arg_type);
const auto type_name = get_str_argument_metadata(f, arg,
"kernel_arg_type");
if (type_name == "image2d_t" || type_name == "image3d_t") {
// Image.
const auto access_qual = get_str_argument_metadata(
f, arg, "kernel_arg_access_qual");
args.emplace_back(get_image_type(type_name, access_qual),
target_size, target_size,
target_align, binary::argument::zero_ext);
} else if (type_name == "sampler_t") {
args.emplace_back(binary::argument::sampler, arg_api_size,
target_size, target_align,
binary::argument::zero_ext);
} else if (type_name == "__llvm_image_size") {
// Image size implicit argument.
args.emplace_back(binary::argument::scalar, sizeof(cl_uint),
dl.getTypeStoreSize(size_type),
size_align,
binary::argument::zero_ext,
binary::argument::image_size);
} else if (type_name == "__llvm_image_format") {
// Image format implicit argument.
args.emplace_back(binary::argument::scalar, sizeof(cl_uint),
dl.getTypeStoreSize(size_type),
size_align,
binary::argument::zero_ext,
binary::argument::image_format);
} else {
// Other types.
const auto actual_type =
isa< ::llvm::PointerType>(arg_type) && arg.hasByValAttr() ?
ptr_arg_to_llvm_type(mod, type_name) : arg_type;
if (actual_type->isPointerTy()) {
const unsigned address_space =
cast< ::llvm::PointerType>(actual_type)->getAddressSpace();
const auto &map = c.getTarget().getAddressSpaceMap();
const auto offset =
static_cast<unsigned>(clang::LangAS::opencl_local);
if (address_space == map[offset]) {
const auto pointee_type = ptr_arg_to_llvm_type(mod, type_name);
args.emplace_back(binary::argument::local, arg_api_size,
target_size,
(pointee_type->isVoidTy()) ? 8 :
compat::get_abi_type_alignment(dl, pointee_type),
binary::argument::zero_ext);
} else {
// XXX: Correctly handle constant address space. There is no
// way for r600g to pass a handle for constant buffers back
// to clover like it can for global buffers, so
// creating constant arguments will break r600g. For now,
// continue treating constant buffers as global buffers
// until we can come up with a way to create handles for
// constant buffers.
args.emplace_back(binary::argument::global, arg_api_size,
target_size, target_align,
binary::argument::zero_ext);
}
} else {
const bool needs_sign_ext = f.getAttributes().hasParamAttr(
arg.getArgNo(), ::llvm::Attribute::SExt);
args.emplace_back(binary::argument::scalar, arg_api_size,
target_size, target_align,
(needs_sign_ext ? binary::argument::sign_ext :
binary::argument::zero_ext));
}
// Add kernel argument infos if built with -cl-kernel-arg-info.
if (c.getCodeGenOpts().EmitOpenCLArgMetadata) {
args.back().info = create_arg_info(
get_str_argument_metadata(f, arg, "kernel_arg_name"),
type_name,
get_str_argument_metadata(f, arg, "kernel_arg_type_qual"),
get_uint_argument_metadata(f, arg, "kernel_arg_addr_space"),
get_str_argument_metadata(f, arg, "kernel_arg_access_qual"));
}
}
}
// Append implicit arguments. XXX - The types, ordering and
// vector size of the implicit arguments should depend on the
// target according to the selected calling convention.
args.emplace_back(binary::argument::scalar, sizeof(cl_uint),
dl.getTypeStoreSize(size_type),
size_align,
binary::argument::zero_ext,
binary::argument::grid_dimension);
args.emplace_back(binary::argument::scalar, sizeof(cl_uint),
dl.getTypeStoreSize(size_type),
size_align,
binary::argument::zero_ext,
binary::argument::grid_offset);
return args;
}
binary::section
make_text_section(const std::vector<char> &code) {
const pipe_binary_program_header header { uint32_t(code.size()) };
binary::section text { 0, binary::section::text_executable,
header.num_bytes, {} };
text.data.insert(text.data.end(), reinterpret_cast<const char *>(&header),
reinterpret_cast<const char *>(&header) + sizeof(header));
text.data.insert(text.data.end(), code.begin(), code.end());
return text;
}
}
binary
clover::llvm::build_module_common(const Module &mod,
const std::vector<char> &code,
const std::map<std::string,
unsigned> &offsets,
const clang::CompilerInstance &c) {
binary b;
for (const auto &llvm_name : map(std::mem_fn(&Function::getName),
get_kernels(mod))) {
const ::std::string name(llvm_name);
if (offsets.count(name))
b.syms.emplace_back(name, kernel_attributes(mod, name),
get_reqd_work_group_size(mod, name),
0, offsets.at(name),
make_kernel_args(mod, name, c));
}
b.secs.push_back(make_text_section(code));
return b;
}

190
src/gallium/frontends/clover/llvm/codegen/native.cpp
View file

@ -1,190 +0,0 @@
//
// Copyright 2012-2016 Francisco Jerez
// Copyright 2012-2016 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
///
/// \file
/// Generate code using an arbitrary LLVM back-end capable of emitting
/// executable code as an ELF object file.
///
#include <llvm/Target/TargetMachine.h>
#include <llvm/Transforms/Utils/Cloning.h>
#include "llvm/codegen.hpp"
#include "llvm/compat.hpp"
#include "llvm/util.hpp"
#include "core/error.hpp"
using clover::binary;
using clover::build_error;
using namespace clover::llvm;
using ::llvm::TargetMachine;
#if defined(USE_LIBELF)
#include <libelf.h>
#include <gelf.h>
namespace {
namespace elf {
std::unique_ptr<Elf, int (*)(Elf *)>
get(const std::vector<char> &code) {
// One of the libelf implementations
// (http://www.mr511.de/software/english.htm) requires calling
// elf_version() before elf_memory().
elf_version(EV_CURRENT);
return { elf_memory(const_cast<char *>(code.data()), code.size()),
elf_end };
}
Elf_Scn *
get_symbol_table(Elf *elf) {
size_t section_str_index;
elf_getshdrstrndx(elf, &section_str_index);
for (Elf_Scn *s = elf_nextscn(elf, NULL); s; s = elf_nextscn(elf, s)) {
GElf_Shdr header;
if (gelf_getshdr(s, &header) != &header)
return nullptr;
if (!std::strcmp(elf_strptr(elf, section_str_index, header.sh_name),
".symtab"))
return s;
}
return nullptr;
}
std::map<std::string, unsigned>
get_symbol_offsets(Elf *elf, Elf_Scn *symtab) {
Elf_Data *const symtab_data = elf_getdata(symtab, NULL);
GElf_Shdr header;
if (gelf_getshdr(symtab, &header) != &header)
return {};
std::map<std::string, unsigned> symbol_offsets;
GElf_Sym symbol;
unsigned i = 0;
while (GElf_Sym *s = gelf_getsym(symtab_data, i++, &symbol)) {
const char *name = elf_strptr(elf, header.sh_link, s->st_name);
symbol_offsets[name] = s->st_value;
}
return symbol_offsets;
}
}
std::map<std::string, unsigned>
get_symbol_offsets(const std::vector<char> &code, std::string &r_log) {
const auto elf = elf::get(code);
const auto symtab = elf::get_symbol_table(elf.get());
if (!symtab)
fail(r_log, build_error(), "Unable to find symbol table.");
return elf::get_symbol_offsets(elf.get(), symtab);
}
std::vector<char>
emit_code(::llvm::Module &mod, const target &target,
compat::CodeGenFileType ft,
std::string &r_log) {
std::string err;
auto t = ::llvm::TargetRegistry::lookupTarget(target.triple, err);
if (!t)
fail(r_log, build_error(), err);
std::unique_ptr<TargetMachine> tm {
t->createTargetMachine(target.triple, target.cpu, "", {},
#if LLVM_VERSION_MAJOR >= 16
std::nullopt, std::nullopt,
#else
::llvm::None, ::llvm::None,
#endif
#if LLVM_VERSION_MAJOR >= 18
::llvm::CodeGenOptLevel::Default) };
#else
::llvm::CodeGenOpt::Default) };
#endif
if (!tm)
fail(r_log, build_error(),
"Could not create TargetMachine: " + target.triple);
::llvm::SmallVector<char, 1024> data;
{
::llvm::legacy::PassManager pm;
::llvm::raw_svector_ostream os { data };
mod.setDataLayout(tm->createDataLayout());
tm->Options.MCOptions.AsmVerbose =
(ft == compat::CGFT_AssemblyFile);
if (tm->addPassesToEmitFile(pm, os, nullptr, ft))
fail(r_log, build_error(), "TargetMachine can't emit this file");
pm.run(mod);
}
return { data.begin(), data.end() };
}
}
binary
clover::llvm::build_module_native(::llvm::Module &mod, const target &target,
const clang::CompilerInstance &c,
std::string &r_log) {
const auto code = emit_code(mod, target,
compat::CGFT_ObjectFile, r_log);
return build_module_common(mod, code, get_symbol_offsets(code, r_log), c);
}
std::string
clover::llvm::print_module_native(const ::llvm::Module &mod,
const target &target) {
std::string log;
try {
std::unique_ptr< ::llvm::Module> cmod { ::llvm::CloneModule(mod) };
return as_string(emit_code(*cmod, target,
compat::CGFT_AssemblyFile, log));
} catch (...) {
return "Couldn't output native disassembly: " + log;
}
}
#else
binary
clover::llvm::build_module_native(::llvm::Module &mod, const target &target,
const clang::CompilerInstance &c,
std::string &r_log) {
unreachable("Native codegen support disabled at build time");
}
std::string
clover::llvm::print_module_native(const ::llvm::Module &mod,
const target &target) {
unreachable("Native codegen support disabled at build time");
}
#endif

140
src/gallium/frontends/clover/llvm/compat.hpp
View file

@ -1,140 +0,0 @@
//
// Copyright 2016 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
///
/// \file
/// Some thin wrappers around the Clang/LLVM API used to preserve
/// compatibility with older API versions while keeping the ifdef clutter low
/// in the rest of the clover::llvm subtree. In case of an API break please
/// consider whether it's possible to preserve backwards compatibility by
/// introducing a new one-liner inline function or typedef here under the
/// compat namespace in order to keep the running code free from preprocessor
/// conditionals.
///
#ifndef CLOVER_LLVM_COMPAT_HPP
#define CLOVER_LLVM_COMPAT_HPP
#include "util/algorithm.hpp"
#include <llvm/Config/llvm-config.h>
#include <llvm/Analysis/TargetLibraryInfo.h>
#include <llvm/IR/LegacyPassManager.h>
#include <llvm/IR/LLVMContext.h>
#include <llvm/IR/Type.h>
#include <llvm/Linker/Linker.h>
#include <llvm/Support/CodeGen.h>
#include <llvm/Target/TargetMachine.h>
#include <llvm/Transforms/IPO.h>
#include <llvm/Transforms/Utils/Cloning.h>
#include <clang/Basic/TargetInfo.h>
#include <clang/Frontend/CompilerInstance.h>
#include <clang/Lex/PreprocessorOptions.h>
#if LLVM_VERSION_MAJOR >= 14
#include <llvm/MC/TargetRegistry.h>
#else
#include <llvm/Support/TargetRegistry.h>
#endif
#if LLVM_VERSION_MAJOR >= 17
#include <llvm/TargetParser/Triple.h>
#else
#include <llvm/ADT/Triple.h>
#endif
namespace clover {
namespace llvm {
namespace compat {
#if LLVM_VERSION_MAJOR >= 18
const auto CGFT_ObjectFile = ::llvm::CodeGenFileType::ObjectFile;
const auto CGFT_AssemblyFile = ::llvm::CodeGenFileType::AssemblyFile;
#else
const auto CGFT_ObjectFile = ::llvm::CGFT_ObjectFile;
const auto CGFT_AssemblyFile = ::llvm::CGFT_AssemblyFile;
#endif
typedef ::llvm::CodeGenFileType CodeGenFileType;
const clang::InputKind ik_opencl = clang::Language::OpenCL;
template<typename T> inline bool
create_compiler_invocation_from_args(clang::CompilerInvocation &cinv,
T copts,
clang::DiagnosticsEngine &diag)
{
return clang::CompilerInvocation::CreateFromArgs(
cinv, copts, diag);
}
static inline void
compiler_set_lang_defaults(std::unique_ptr<clang::CompilerInstance> &c,
clang::InputKind ik, const ::llvm::Triple& triple,
clang::LangStandard::Kind d)
{
#if LLVM_VERSION_MAJOR >= 15
c->getLangOpts().setLangDefaults(c->getLangOpts(), ik.getLanguage(), triple,
#else
c->getInvocation().setLangDefaults(c->getLangOpts(), ik, triple,
#endif
#if LLVM_VERSION_MAJOR >= 12
c->getPreprocessorOpts().Includes,
#else
c->getPreprocessorOpts(),
#endif
d);
}
static inline unsigned
get_abi_type_alignment(::llvm::DataLayout dl, ::llvm::Type *type)
{
#if LLVM_VERSION_MAJOR >= 16
return dl.getABITypeAlign(type).value();
#else
return dl.getABITypeAlignment(type);
#endif
}
static inline bool
is_scalable_vector(const ::llvm::Type *type)
{
return ::llvm::isa<::llvm::ScalableVectorType>(type);
}
static inline bool
is_fixed_vector(const ::llvm::Type *type)
{
return ::llvm::isa<::llvm::FixedVectorType>(type);
}
static inline unsigned
get_fixed_vector_elements(const ::llvm::Type *type)
{
return ::llvm::cast<::llvm::FixedVectorType>(type)->getNumElements();
}
}
}
}
#endif

580
src/gallium/frontends/clover/llvm/invocation.cpp
View file

@ -1,580 +0,0 @@
//
// Copyright 2012-2016 Francisco Jerez
// Copyright 2012-2016 Advanced Micro Devices, Inc.
// Copyright 2014-2016 Jan Vesely
// Copyright 2014-2015 Serge Martin
// Copyright 2015 Zoltan Gilian
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifdef HAVE_DLFCN_H
#include <dlfcn.h>
#endif
#include <llvm/IR/DiagnosticPrinter.h>
#include <llvm/IR/DiagnosticInfo.h>
#include <llvm/IR/LLVMContext.h>
#include <llvm/IR/Module.h>
#include <llvm/Support/raw_ostream.h>
#include <llvm/Transforms/IPO/Internalize.h>
#include <llvm-c/Target.h>
#include <llvm-c/TargetMachine.h>
#include <llvm-c/Transforms/PassBuilder.h>
#include <llvm/Support/CBindingWrapping.h>
#include <clang/CodeGen/CodeGenAction.h>
#include <clang/Lex/PreprocessorOptions.h>
#include <clang/Frontend/TextDiagnosticBuffer.h>
#include <clang/Frontend/TextDiagnosticPrinter.h>
#include <clang/Basic/TargetInfo.h>
#include <clang/Config/config.h>
#include <clang/Driver/Driver.h>
#if LLVM_VERSION_MAJOR >= 20
#include <llvm/Support/VirtualFileSystem.h>
#endif
// We need to include internal headers last, because the internal headers
// include CL headers which have #define's like:
//
//#define cl_khr_gl_sharing 1
//#define cl_khr_icd 1
//
// Which will break the compilation of clang/Basic/OpenCLOptions.h
#include "core/error.hpp"
#include "llvm/codegen.hpp"
#include "llvm/compat.hpp"
#include "llvm/invocation.hpp"
#include "llvm/metadata.hpp"
#include "llvm/util.hpp"
#include "util/algorithm.hpp"
using clover::binary;
using clover::device;
using clover::build_error;
using clover::invalid_build_options_error;
using clover::map;
using clover::header_map;
using namespace clover::llvm;
using ::llvm::Function;
using ::llvm::LLVMContext;
using ::llvm::Module;
using ::llvm::raw_string_ostream;
namespace {
static const cl_version ANY_VERSION = CL_MAKE_VERSION(9, 9, 9);
const cl_version cl_versions[] = {
CL_MAKE_VERSION(1, 1, 0),
CL_MAKE_VERSION(1, 2, 0),
CL_MAKE_VERSION(2, 0, 0),
CL_MAKE_VERSION(2, 1, 0),
CL_MAKE_VERSION(2, 2, 0),
CL_MAKE_VERSION(3, 0, 0),
};
struct clc_version_lang_std {
cl_version version_number; // CLC Version
clang::LangStandard::Kind clc_lang_standard;
};
const clc_version_lang_std cl_version_lang_stds[] = {
{ CL_MAKE_VERSION(1, 0, 0), clang::LangStandard::lang_opencl10},
{ CL_MAKE_VERSION(1, 1, 0), clang::LangStandard::lang_opencl11},
{ CL_MAKE_VERSION(1, 2, 0), clang::LangStandard::lang_opencl12},
{ CL_MAKE_VERSION(2, 0, 0), clang::LangStandard::lang_opencl20},
#if LLVM_VERSION_MAJOR >= 12
{ CL_MAKE_VERSION(3, 0, 0), clang::LangStandard::lang_opencl30},
#endif
};
bool
are_equal(cl_version_khr version1, cl_version_khr version2,
bool ignore_patch_version = false) {
if (ignore_patch_version) {
version1 &= ~CL_VERSION_PATCH_MASK_KHR;
version2 &= ~CL_VERSION_PATCH_MASK_KHR;
}
return version1 == version2;
}
void
init_targets() {
static bool targets_initialized = false;
if (!targets_initialized) {
LLVMInitializeAllTargets();
LLVMInitializeAllTargetInfos();
LLVMInitializeAllTargetMCs();
LLVMInitializeAllAsmParsers();
LLVMInitializeAllAsmPrinters();
targets_initialized = true;
}
}
void
#if LLVM_VERSION_MAJOR >= 19
diagnostic_handler(const ::llvm::DiagnosticInfo *di, void *data) {
if (di->getSeverity() == ::llvm::DS_Error) {
#else
diagnostic_handler(const ::llvm::DiagnosticInfo &di, void *data) {
if (di.getSeverity() == ::llvm::DS_Error) {
#endif
raw_string_ostream os { *reinterpret_cast<std::string *>(data) };
::llvm::DiagnosticPrinterRawOStream printer { os };
#if LLVM_VERSION_MAJOR >= 19
di->print(printer);
#else
di.print(printer);
#endif
throw build_error();
}
}
std::unique_ptr<LLVMContext>
create_context(std::string &r_log) {
init_targets();
std::unique_ptr<LLVMContext> ctx { new LLVMContext };
ctx->setDiagnosticHandlerCallBack(diagnostic_handler, &r_log);
return ctx;
}
const struct clc_version_lang_std&
get_cl_lang_standard(unsigned requested, unsigned max = ANY_VERSION) {
for (const struct clc_version_lang_std &version : cl_version_lang_stds) {
if (version.version_number == max ||
version.version_number == requested) {
return version;
}
}
throw build_error("Unknown/Unsupported language version");
}
const cl_version
get_cl_version(cl_version requested,
cl_version max = ANY_VERSION) {
for (const auto &version : cl_versions) {
if (are_equal(version, max, true) ||
are_equal(version, requested, true)) {
return version;
}
}
throw build_error("Unknown/Unsupported language version");
}
clang::LangStandard::Kind
get_lang_standard_from_version(const cl_version input_version,
bool is_build_opt = false) {
//Per CL 2.0 spec, section 5.8.4.5:
// If it's an option, use the value directly.
// If it's a device version, clamp to max 1.x version, a.k.a. 1.2
const cl_version version =
get_cl_version(input_version, is_build_opt ? ANY_VERSION : 120);
const struct clc_version_lang_std standard =
get_cl_lang_standard(version);
return standard.clc_lang_standard;
}
clang::LangStandard::Kind
get_language_version(const std::vector<std::string> &opts,
const cl_version device_version) {
const std::string search = "-cl-std=CL";
for (auto &opt: opts) {
auto pos = opt.find(search);
if (pos == 0){
std::stringstream ver_str(opt.substr(pos + search.size()));
unsigned int ver_major = 0;
char separator = '\0';
unsigned int ver_minor = 0;
ver_str >> ver_major >> separator >> ver_minor;
if (ver_str.fail() || ver_str.bad() || !ver_str.eof() ||
separator != '.') {
throw build_error();
}
const auto ver = CL_MAKE_VERSION_KHR(ver_major, ver_minor, 0);
const auto device_ver = get_cl_version(device_version);
const auto requested = get_cl_version(ver);
if (requested > device_ver) {
throw build_error();
}
return get_lang_standard_from_version(ver, true);
}
}
return get_lang_standard_from_version(device_version);
}
std::unique_ptr<clang::CompilerInstance>
create_compiler_instance(const device &dev, const std::string& ir_target,
const std::vector<std::string> &opts,
std::string &r_log) {
std::unique_ptr<clang::CompilerInstance> c { new clang::CompilerInstance };
clang::TextDiagnosticBuffer *diag_buffer = new clang::TextDiagnosticBuffer;
clang::DiagnosticsEngine diag { new clang::DiagnosticIDs,
new clang::DiagnosticOptions, diag_buffer };
// Parse the compiler options. A file name should be present at the end
// and must have the .cl extension in order for the CompilerInvocation
// class to recognize it as an OpenCL source file.
#if LLVM_VERSION_MAJOR >= 12
std::vector<const char *> copts;
#if LLVM_VERSION_MAJOR == 15 || LLVM_VERSION_MAJOR == 16
// Before LLVM commit 702d5de4 opaque pointers were supported but not enabled
// by default when building LLVM. They were made default in commit 702d5de4.
// LLVM commit d69e9f9d introduced -opaque-pointers/-no-opaque-pointers cc1
// options to enable or disable them whatever the LLVM default is.
// Those two commits follow llvmorg-15-init and precede llvmorg-15.0.0-rc1 tags.
// Since LLVM commit d785a8ea, the CLANG_ENABLE_OPAQUE_POINTERS build option of
// LLVM is removed, meaning there is no way to build LLVM with opaque pointers
// enabled by default.
// It was said at the time it was still possible to explicitly disable opaque
// pointers via cc1 -no-opaque-pointers option, but it is known a later commit
// broke backward compatibility provided by -no-opaque-pointers as verified with
// arbitrary commit d7d586e5, so there is no way to use opaque pointers starting
// with LLVM 16.
// Those two commits follow llvmorg-16-init and precede llvmorg-16.0.0-rc1 tags.
// Since Mesa commit 977dbfc9 opaque pointers are properly implemented in Clover
// and used.
// If we don't pass -opaque-pointers to Clang on LLVM versions supporting opaque
// pointers but disabling them by default, there will be an API mismatch between
// Mesa and LLVM and Clover will not work.
copts.push_back("-opaque-pointers");
#endif
for (auto &opt : opts) {
if (opt == "-cl-denorms-are-zero")
copts.push_back("-fdenormal-fp-math=positive-zero");
else
copts.push_back(opt.c_str());
}
#else
const std::vector<const char *> copts =
map(std::mem_fn(&std::string::c_str), opts);
#endif
const target &target = ir_target;
const cl_version device_clc_version = dev.device_clc_version();
if (!compat::create_compiler_invocation_from_args(
c->getInvocation(), copts, diag))
throw invalid_build_options_error();
diag_buffer->FlushDiagnostics(diag);
if (diag.hasErrorOccurred())
throw invalid_build_options_error();
c->getTargetOpts().CPU = target.cpu;
c->getTargetOpts().Triple = target.triple;
c->getLangOpts().NoBuiltin = true;
#if LLVM_VERSION_MAJOR >= 13
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_generic_address_space");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_pipes");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_device_enqueue");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_program_scope_global_variables");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_subgroups");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_work_group_collective_functions");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_atomic_scope_device");
c->getTargetOpts().OpenCLExtensionsAsWritten.push_back("-__opencl_c_atomic_order_seq_cst");
#endif
// This is a workaround for a Clang bug which causes the number
// of warnings and errors to be printed to stderr.
// http://www.llvm.org/bugs/show_bug.cgi?id=19735
c->getDiagnosticOpts().ShowCarets = false;
compat::compiler_set_lang_defaults(c, compat::ik_opencl,
::llvm::Triple(target.triple),
get_language_version(opts, device_clc_version));
c->createDiagnostics(
#if LLVM_VERSION_MAJOR >= 20
*llvm::vfs::getRealFileSystem(),
#endif
new clang::TextDiagnosticPrinter(
*new raw_string_ostream(r_log),
&c->getDiagnosticOpts(), true));
c->setTarget(clang::TargetInfo::CreateTargetInfo(
c->getDiagnostics(), c->getInvocation().TargetOpts));
return c;
}
std::string getResourceDirectory() {
#ifdef HAVE_DLFCN_H
Dl_info info;
if (dladdr((void *)clang::CompilerInvocation::CreateFromArgs, &info) == 0) {
return FALLBACK_CLANG_RESOURCE_DIR;
}
char *libclang_path = realpath(info.dli_fname, NULL);
if (libclang_path == nullptr) {
return FALLBACK_CLANG_RESOURCE_DIR;
}
// GetResourcePath is a way to retrieve the actual libclang resource dir based on a given
// binary or library.
std::string clang_resource_dir =
#if LLVM_VERSION_MAJOR >= 20
clang::driver::Driver::GetResourcesPath(std::string(libclang_path));
#else
clang::driver::Driver::GetResourcesPath(std::string(libclang_path), CLANG_RESOURCE_DIR);
#endif
free(libclang_path);
return clang_resource_dir;
#else
return FALLBACK_CLANG_RESOURCE_DIR;
#endif
}
std::unique_ptr<Module>
compile(LLVMContext &ctx, clang::CompilerInstance &c,
const std::string &name, const std::string &source,
const header_map &headers, const device &dev,
const std::string &opts, bool use_libclc, std::string &r_log) {
c.getFrontendOpts().ProgramAction = clang::frontend::EmitLLVMOnly;
c.getHeaderSearchOpts().UseBuiltinIncludes = true;
c.getHeaderSearchOpts().UseStandardSystemIncludes = true;
std::string clang_resource_dir = getResourceDirectory();
c.getHeaderSearchOpts().ResourceDir = clang_resource_dir;
// Add opencl-c generic search path
std::string clang_include_path = clang_resource_dir + "/include";
c.getHeaderSearchOpts().AddPath(clang_include_path,
clang::frontend::Angled,
false, false);
// Add opencl include
c.getPreprocessorOpts().Includes.push_back("opencl-c.h");
// Add definition for the OpenCL version
const auto dev_version = dev.device_version();
c.getPreprocessorOpts().addMacroDef("__OPENCL_VERSION__=" +
std::to_string(CL_VERSION_MAJOR_KHR(dev_version)) +
std::to_string(CL_VERSION_MINOR_KHR(dev_version)) + "0");
if (CL_VERSION_MAJOR(dev.version) >= 3) {
const auto features = dev.opencl_c_features();
for (const auto &feature : features)
c.getPreprocessorOpts().addMacroDef(feature.name);
}
// clc.h requires that this macro be defined:
c.getPreprocessorOpts().addMacroDef("cl_clang_storage_class_specifiers");
c.getPreprocessorOpts().addRemappedFile(
name, ::llvm::MemoryBuffer::getMemBuffer(source).release());
if (headers.size()) {
const std::string tmp_header_path = "/tmp/clover/";
c.getHeaderSearchOpts().AddPath(tmp_header_path,
clang::frontend::Angled,
false, false);
for (const auto &header : headers)
c.getPreprocessorOpts().addRemappedFile(
tmp_header_path + header.first,
::llvm::MemoryBuffer::getMemBuffer(header.second).release());
}
// Tell clang to link this file before performing any
// optimizations. This is required so that we can replace calls
// to the OpenCL C barrier() builtin with calls to target
// intrinsics that have the noduplicate attribute. This
// attribute will prevent Clang from creating illegal uses of
// barrier() (e.g. Moving barrier() inside a conditional that is
// no executed by all threads) during its optimizaton passes.
if (use_libclc) {
clang::CodeGenOptions::BitcodeFileToLink F;
F.Filename = LIBCLC_LIBEXECDIR + dev.ir_target() + ".bc";
F.PropagateAttrs = true;
F.LinkFlags = ::llvm::Linker::Flags::None;
c.getCodeGenOpts().LinkBitcodeFiles.emplace_back(F);
}
// undefine __IMAGE_SUPPORT__ for device without image support
if (!dev.image_support())
c.getPreprocessorOpts().addMacroUndef("__IMAGE_SUPPORT__");
// Compile the code
clang::EmitLLVMOnlyAction act(&ctx);
if (!c.ExecuteAction(act))
throw build_error();
return act.takeModule();
}
}
binary
clover::llvm::compile_program(const std::string &source,
const header_map &headers,
const device &dev,
const std::string &opts,
std::string &r_log) {
if (has_flag(debug::clc))
debug::log(".cl", "// Options: " + opts + '\n' + source);
auto ctx = create_context(r_log);
auto c = create_compiler_instance(dev, dev.ir_target(),
tokenize(opts + " input.cl"), r_log);
auto mod = compile(*ctx, *c, "input.cl", source, headers, dev, opts, true,
r_log);
if (has_flag(debug::llvm))
debug::log(".ll", print_module_bitcode(*mod));
return build_module_library(*mod, binary::section::text_intermediate);
}
namespace {
void
optimize(Module &mod,
const std::string& ir_target,
unsigned optimization_level,
bool internalize_symbols) {
// By default, the function internalizer pass will look for a function
// called "main" and then mark all other functions as internal. Marking
// functions as internal enables the optimizer to perform optimizations
// like function inlining and global dead-code elimination.
//
// When there is no "main" function in a binary, the internalize pass will
// treat the binary like a library, and it won't internalize any functions.
// Since there is no "main" function in our kernels, we need to tell
// the internalizer pass that this binary is not a library by passing a
// list of kernel functions to the internalizer. The internalizer will
// treat the functions in the list as "main" functions and internalize
// all of the other functions.
if (internalize_symbols) {
std::vector<std::string> names =
map(std::mem_fn(&Function::getName), get_kernels(mod));
internalizeModule(mod,
[=](const ::llvm::GlobalValue &gv) {
return std::find(names.begin(), names.end(),
gv.getName()) != names.end();
});
}
const char *opt_str = NULL;
LLVMCodeGenOptLevel level;
switch (optimization_level) {
case 0:
default:
opt_str = "default<O0>";
level = LLVMCodeGenLevelNone;
break;
case 1:
opt_str = "default<O1>";
level = LLVMCodeGenLevelLess;
break;
case 2:
opt_str = "default<O2>";
level = LLVMCodeGenLevelDefault;
break;
case 3:
opt_str = "default<O3>";
level = LLVMCodeGenLevelAggressive;
break;
}
const target &target = ir_target;
LLVMTargetRef targ;
char *err_message;
if (LLVMGetTargetFromTriple(target.triple.c_str(), &targ, &err_message))
return;
LLVMTargetMachineRef tm =
LLVMCreateTargetMachine(targ, target.triple.c_str(),
target.cpu.c_str(), "", level,
LLVMRelocDefault, LLVMCodeModelDefault);
if (!tm)
return;
LLVMPassBuilderOptionsRef opts = LLVMCreatePassBuilderOptions();
LLVMRunPasses(wrap(&mod), opt_str, tm, opts);
LLVMDisposeTargetMachine(tm);
LLVMDisposePassBuilderOptions(opts);
}
std::unique_ptr<Module>
link(LLVMContext &ctx, const clang::CompilerInstance &c,
const std::vector<binary> &binaries, std::string &r_log) {
std::unique_ptr<Module> mod { new Module("link", ctx) };
std::unique_ptr< ::llvm::Linker> linker { new ::llvm::Linker(*mod) };
for (auto &b : binaries) {
if (linker->linkInModule(parse_module_library(b, ctx, r_log)))
throw build_error();
}
return mod;
}
}
binary
clover::llvm::link_program(const std::vector<binary> &binaries,
const device &dev, const std::string &opts,
std::string &r_log) {
std::vector<std::string> options = tokenize(opts + " input.cl");
const bool create_library = count("-create-library", options);
erase_if(equals("-create-library"), options);
auto ctx = create_context(r_log);
auto c = create_compiler_instance(dev, dev.ir_target(), options, r_log);
auto mod = link(*ctx, *c, binaries, r_log);
optimize(*mod, dev.ir_target(), c->getCodeGenOpts().OptimizationLevel, !create_library);
static std::atomic_uint seq(0);
const std::string id = "." + mod->getModuleIdentifier() + "-" +
std::to_string(seq++);
if (has_flag(debug::llvm))
debug::log(id + ".ll", print_module_bitcode(*mod));
if (create_library) {
return build_module_library(*mod, binary::section::text_library);
} else if (dev.ir_format() == PIPE_SHADER_IR_NATIVE) {
if (has_flag(debug::native))
debug::log(id + ".asm", print_module_native(*mod, dev.ir_target()));
return build_module_native(*mod, dev.ir_target(), *c, r_log);
} else {
unreachable("Unsupported IR.");
}
}

46
src/gallium/frontends/clover/llvm/invocation.hpp
View file

@ -1,46 +0,0 @@
//
// Copyright 2016 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_LLVM_INVOCATION_HPP
#define CLOVER_LLVM_INVOCATION_HPP
#include "core/error.hpp"
#include "core/binary.hpp"
#include "core/program.hpp"
#include "pipe/p_defines.h"
namespace clover {
namespace llvm {
binary compile_program(const std::string &source,
const header_map &headers,
const device &device,
const std::string &opts,
std::string &r_log);
binary link_program(const std::vector<binary> &binaries,
const device &device,
const std::string &opts,
std::string &r_log);
}
}
#endif

203
src/gallium/frontends/clover/llvm/metadata.hpp
View file

@ -1,203 +0,0 @@
//
// Copyright 2016 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
///
/// \file
/// Utility functions for LLVM IR metadata introspection.
///
#ifndef CLOVER_LLVM_METADATA_HPP
#define CLOVER_LLVM_METADATA_HPP
#include "llvm/compat.hpp"
#include "util/algorithm.hpp"
#include <vector>
#include <llvm/Config/llvm-config.h>
#include <llvm/IR/Constants.h>
#include <llvm/IR/Module.h>
#include <llvm/IR/Metadata.h>
namespace clover {
namespace llvm {
namespace detail {
inline bool
is_kernel(const ::llvm::Function &f) {
return f.getMetadata("kernel_arg_type");
}
inline iterator_range< ::llvm::MDNode::op_iterator>
get_kernel_metadata_operands(const ::llvm::Function &f,
const std::string &name) {
const auto data_node = f.getMetadata(name);
if (data_node)
return range(data_node->op_begin(), data_node->op_end());
else
return iterator_range< ::llvm::MDNode::op_iterator>();
}
}
///
/// Extract the string metadata node \p name.
///
inline std::string
get_str_kernel_metadata(const ::llvm::Function &f,
const std::string &name) {
auto operands = detail::get_kernel_metadata_operands(f, name);
if (operands.size()) {
return ::llvm::cast< ::llvm::MDString>(
detail::get_kernel_metadata_operands(f, name)[0])
->getString().str();
} else {
return "";
}
}
///
/// Extract the string metadata node \p name.
///
inline std::vector<size_t>
get_uint_vector_kernel_metadata(const ::llvm::Function &f,
const std::string &name) {
auto operands = detail::get_kernel_metadata_operands(f, name);
if (operands.size()) {
return map([=](const ::llvm::MDOperand& o) {
auto value = ::llvm::cast< ::llvm::ConstantAsMetadata>(o)
->getValue();
return ::llvm::cast< ::llvm::ConstantInt>(value)
->getLimitedValue(UINT_MAX);
}, operands);
} else {
return {};
}
}
///
/// Extract the string metadata node \p name.
///
inline std::string
get_type_kernel_metadata(const ::llvm::Function &f,
const std::string &name) {
auto operands = detail::get_kernel_metadata_operands(f, name);
if (operands.size()) {
auto value = ::llvm::cast< ::llvm::ConstantAsMetadata>(operands[0])
->getValue();
auto type = ::llvm::cast< ::llvm::UndefValue>(value)
->getType();
value = ::llvm::cast< ::llvm::ConstantAsMetadata>(operands[1])
->getValue();
bool is_signed = ::llvm::cast< ::llvm::ConstantInt>(value)
->getLimitedValue(UINT_MAX);
std::string data;
if (type->isIntOrIntVectorTy()) {
if (!is_signed)
data = "unsigned ";
const auto size = type->getScalarSizeInBits();
switch(size) {
case 8:
data += "char";
break;
case 16:
data += "short";
break;
case 32:
data += "int";
break;
case 64:
data += "long";
break;
}
if (compat::is_scalable_vector(type))
throw build_error("hit unexpected scalable vector");
if (compat::is_fixed_vector(type))
data += std::to_string(compat::get_fixed_vector_elements(type));
} else {
::llvm::raw_string_ostream os { data };
type->print(os);
os.flush();
}
return data;
} else {
return "";
}
}
///
/// Extract the string metadata node \p name corresponding to the kernel
/// argument given by \p arg.
///
inline std::string
get_str_argument_metadata(const ::llvm::Function &f,
const ::llvm::Argument &arg,
const std::string &name) {
auto operands = detail::get_kernel_metadata_operands(f, name);
if (operands.size() > arg.getArgNo()) {
return ::llvm::cast< ::llvm::MDString>(operands[arg.getArgNo()])
->getString().str();
} else {
return "";
}
}
///
/// Extract the int metadata node \p name corresponding to the kernel
/// argument given by \p arg.
///
inline uint64_t
get_uint_argument_metadata(const ::llvm::Function &f,
const ::llvm::Argument &arg,
const std::string &name) {
auto operands = detail::get_kernel_metadata_operands(f, name);
if (operands.size() >= arg.getArgNo()) {
auto meta_arg_value = ::llvm::cast< ::llvm::ConstantAsMetadata>(
operands[arg.getArgNo()])->getValue();
return ::llvm::cast< ::llvm::ConstantInt>(meta_arg_value)
->getLimitedValue(UINT_MAX);
} else {
return 0;
}
}
///
/// Return a vector with all CL kernel functions found in the LLVM
/// module \p mod.
///
inline std::vector<const ::llvm::Function *>
get_kernels(const ::llvm::Module &mod) {
std::vector<const ::llvm::Function *> fs;
for (auto &f : mod.getFunctionList()) {
if (detail::is_kernel(f))
fs.push_back(&f);
}
return fs;
}
}
}
#endif

93
src/gallium/frontends/clover/llvm/util.hpp
View file

@ -1,93 +0,0 @@
//
// Copyright 2012-2016 Francisco Jerez
// Copyright 2012-2016 Advanced Micro Devices, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_LLVM_UTIL_HPP
#define CLOVER_LLVM_UTIL_HPP
#include "core/error.hpp"
#include "util/u_debug.h"
#include <vector>
#include <fstream>
#include <iostream>
namespace clover {
namespace llvm {
template<typename E> void
fail(std::string &r_log, E &&e, const std::string &s) {
r_log += s;
throw std::forward<E>(e);
}
inline std::string
as_string(const std::vector<char> &v) {
return { v.begin(), v.end() };
}
struct target {
target(const std::string &s) :
cpu(s.begin(), s.begin() + s.find_first_of("-")),
triple(s.begin() + s.find_first_of("-") + 1, s.end()) {}
std::string cpu;
std::string triple;
};
namespace debug {
enum flag {
clc = 1 << 0,
llvm = 1 << 1,
native = 1 << 2,
spirv = 1 << 3,
};
inline bool
has_flag(flag f) {
static const struct debug_named_value debug_options[] = {
{ "clc", clc, "Dump the OpenCL C code for all kernels." },
{ "llvm", llvm, "Dump the generated LLVM IR for all kernels." },
{ "native", native, "Dump kernel assembly code for targets "
"specifying PIPE_SHADER_IR_NATIVE" },
{ "spirv", spirv, "Dump the generated SPIR-V for all kernels." },
DEBUG_NAMED_VALUE_END
};
static const unsigned flags =
debug_get_flags_option("CLOVER_DEBUG", debug_options, 0);
return flags & f;
}
inline void
log(const std::string &suffix, const std::string &s) {
const std::string path = debug_get_option("CLOVER_DEBUG_FILE",
"stderr");
if (path == "stderr")
std::cerr << s;
else
std::ofstream(path + suffix, std::ios::app) << s;
}
}
}
}
#endif

125
src/gallium/frontends/clover/meson.build
View file

@ -1,125 +0,0 @@
# Copyright © 2017-2018 Intel Corporation
# SPDX-License-Identifier: MIT
clover_cpp_args = []
clover_opencl_cpp_args = [
'-DCL_TARGET_OPENCL_VERSION=300',
'-DCL_USE_DEPRECATED_OPENCL_1_0_APIS',
'-DCL_USE_DEPRECATED_OPENCL_1_1_APIS',
'-DCL_USE_DEPRECATED_OPENCL_1_2_APIS',
'-DCL_USE_DEPRECATED_OPENCL_2_0_APIS',
'-DCL_USE_DEPRECATED_OPENCL_2_1_APIS',
'-DCL_USE_DEPRECATED_OPENCL_2_2_APIS',
'-DLIBCLC_LIBEXECDIR="@0@/"'.format(dep_clc.get_variable(pkgconfig : 'libexecdir'))
]
clover_incs = [inc_include, inc_src, inc_gallium, inc_gallium_aux]
# the CL header files declare attributes on the CL types. Compilers warn if
# we use them as template arguments. Disable the warning as there isn't
# anything we can do about it
if cpp.has_argument('-Wno-ignored-attributes')
clover_cpp_args += '-Wno-ignored-attributes'
endif
if with_opencl_icd
clover_cpp_args += '-DHAVE_CLOVER_ICD'
endif
libclllvm = static_library(
'clllvm',
files(
'llvm/codegen/bitcode.cpp',
'llvm/codegen/common.cpp',
'llvm/codegen/native.cpp',
'llvm/codegen.hpp',
'llvm/compat.hpp',
'llvm/invocation.cpp',
'llvm/invocation.hpp',
'llvm/metadata.hpp',
'llvm/util.hpp',
),
include_directories : clover_incs,
cpp_args : [
clover_cpp_args,
clover_opencl_cpp_args,
'-DFALLBACK_CLANG_RESOURCE_DIR="@0@"'.format(join_paths(
dep_llvm.get_variable(cmake : 'LLVM_LIBRARY_DIR', configtool: 'libdir'), 'clang',
dep_llvm.version()
)),
],
gnu_symbol_visibility : 'hidden',
dependencies : [dep_llvm, dep_elf, dep_llvmspirvlib, idep_mesautil],
)
clover_files = files(
'api/context.cpp',
'api/device.cpp',
'api/dispatch.cpp',
'api/dispatch.hpp',
'api/event.cpp',
'api/interop.cpp',
'api/invalid.cpp',
'api/kernel.cpp',
'api/memory.cpp',
'api/platform.cpp',
'api/program.cpp',
'api/queue.cpp',
'api/sampler.cpp',
'api/transfer.cpp',
'api/util.hpp',
'core/binary.cpp',
'core/binary.hpp',
'core/compiler.hpp',
'core/context.cpp',
'core/context.hpp',
'core/device.cpp',
'core/device.hpp',
'core/error.hpp',
'core/event.cpp',
'core/event.hpp',
'core/format.cpp',
'core/format.hpp',
'core/kernel.cpp',
'core/kernel.hpp',
'core/memory.cpp',
'core/memory.hpp',
'core/object.hpp',
'core/platform.cpp',
'core/platform.hpp',
'core/printf.cpp',
'core/printf.hpp',
'core/program.cpp',
'core/program.hpp',
'core/property.hpp',
'core/queue.cpp',
'core/queue.hpp',
'core/resource.cpp',
'core/resource.hpp',
'core/sampler.cpp',
'core/sampler.hpp',
'core/timestamp.cpp',
'core/timestamp.hpp',
'util/adaptor.hpp',
'util/algebra.hpp',
'util/algorithm.hpp',
'util/compat.hpp',
'util/factor.hpp',
'util/functional.hpp',
'util/lazy.hpp',
'util/pointer.hpp',
'util/range.hpp',
'util/tuple.hpp',
)
libclover = static_library(
'clover',
[clover_files, sha1_h],
include_directories : clover_incs,
cpp_args : [
clover_opencl_cpp_args,
clover_cpp_args,
],
gnu_symbol_visibility : 'hidden',
link_with : [libclllvm],
dependencies : [idep_mesautil, idep_nir],
)

184
src/gallium/frontends/clover/util/adaptor.hpp
View file

@ -1,184 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_ADAPTOR_HPP
#define CLOVER_UTIL_ADAPTOR_HPP
#include <iterator>
#include "util/compat.hpp"
#include "util/tuple.hpp"
#include "util/pointer.hpp"
#include "util/functional.hpp"
namespace clover {
namespace detail {
///
/// Implementation of the iterator concept that transforms the
/// value of the source iterators \a Is on dereference by use of
/// a functor \a F.
///
/// The exact category of the resulting iterator should be the
/// least common denominator of the source iterator categories.
///
template<typename F, typename... Is>
class iterator_adaptor {
public:
typedef std::forward_iterator_tag iterator_category;
typedef typename invoke_result<
F, typename std::iterator_traits<Is>::reference...
>::type reference;
typedef typename std::remove_reference<reference>::type value_type;
typedef pseudo_ptr<value_type> pointer;
typedef std::ptrdiff_t difference_type;
iterator_adaptor() {
}
iterator_adaptor(F f, std::tuple<Is...> &&its) :
f(f), its(std::move(its)) {
}
reference
operator*() const {
return tuple::apply(f, tuple::map(derefs(), its));
}
iterator_adaptor &
operator++() {
tuple::map(preincs(), its);
return *this;
}
iterator_adaptor
operator++(int) {
auto jt = *this;
++*this;
return jt;
}
bool
operator==(const iterator_adaptor &jt) const {
return its == jt.its;
}
bool
operator!=(const iterator_adaptor &jt) const {
return its != jt.its;
}
pointer
operator->() const {
return { **this };
}
iterator_adaptor &
operator--() {
tuple::map(predecs(), its);
return *this;
}
iterator_adaptor
operator--(int) {
auto jt = *this;
--*this;
return jt;
}
iterator_adaptor &
operator+=(difference_type n) {
tuple::map(advances_by(n), its);
return *this;
}
iterator_adaptor &
operator-=(difference_type n) {
tuple::map(advances_by(-n), its);
return *this;
}
iterator_adaptor
operator+(difference_type n) const {
auto jt = *this;
jt += n;
return jt;
}
iterator_adaptor
operator-(difference_type n) const {
auto jt = *this;
jt -= n;
return jt;
}
difference_type
operator-(const iterator_adaptor &jt) const {
return std::get<0>(its) - std::get<0>(jt.its);
}
reference
operator[](difference_type n) const {
return *(*this + n);
}
bool
operator<(iterator_adaptor &jt) const {
return *this - jt < 0;
}
bool
operator>(iterator_adaptor &jt) const {
return *this - jt > 0;
}
bool
operator>=(iterator_adaptor &jt) const {
return !(*this < jt);
}
bool
operator<=(iterator_adaptor &jt) const {
return !(*this > jt);
}
protected:
F f;
std::tuple<Is...> its;
};
template<typename F, typename... Is>
iterator_adaptor<F, Is...>
operator+(typename iterator_adaptor<F, Is...>::difference_type n,
const iterator_adaptor<F, Is...> &jt) {
return (jt + n);
}
template<typename F, typename... Is>
iterator_adaptor<F, Is...>
operator-(typename iterator_adaptor<F, Is...>::difference_type n,
const iterator_adaptor<F, Is...> &jt) {
return (jt - n);
}
}
}
#endif

160
src/gallium/frontends/clover/util/algebra.hpp
View file

@ -1,160 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_ALGEBRA_HPP
#define CLOVER_UTIL_ALGEBRA_HPP
#include <type_traits>
#include "util/range.hpp"
#include "util/functional.hpp"
namespace clover {
///
/// Class that identifies vectors (in the linear-algebraic sense).
///
/// There should be a definition of this class for each type that
/// makes sense as vector arithmetic operand.
///
template<typename V, typename = void>
struct vector_traits;
///
/// References of vectors are vectors.
///
template<typename T>
struct vector_traits<T &, typename vector_traits<T>::enable> {
typedef void enable;
};
///
/// Constant vectors are vectors.
///
template<typename T>
struct vector_traits<const T, typename vector_traits<T>::enable> {
typedef void enable;
};
///
/// Arrays of arithmetic types are vectors.
///
template<typename T, std::size_t N>
struct vector_traits<std::array<T, N>,
typename std::enable_if<
std::is_arithmetic<T>::value>::type> {
typedef void enable;
};
namespace detail {
template<typename... Ts>
struct are_defined {
typedef void enable;
};
}
///
/// The result of mapping a vector is a vector.
///
template<typename F, typename... Vs>
struct vector_traits<adaptor_range<F, Vs...>,
typename detail::are_defined<
typename vector_traits<Vs>::enable...>::enable> {
typedef void enable;
};
///
/// Vector sum.
///
template<typename U, typename V,
typename = typename vector_traits<U>::enable,
typename = typename vector_traits<V>::enable>
adaptor_range<plus, U, V>
operator+(U &&u, V &&v) {
return map(plus(), std::forward<U>(u), std::forward<V>(v));
}
///
/// Vector difference.
///
template<typename U, typename V,
typename = typename vector_traits<U>::enable,
typename = typename vector_traits<V>::enable>
adaptor_range<minus, U, V>
operator-(U &&u, V &&v) {
return map(minus(), std::forward<U>(u), std::forward<V>(v));
}
///
/// Scalar multiplication.
///
template<typename U, typename T,
typename = typename vector_traits<U>::enable>
adaptor_range<multiplies_by_t<T>, U>
operator*(U &&u, T &&a) {
return map(multiplies_by<T>(std::forward<T>(a)), std::forward<U>(u));
}
///
/// Scalar multiplication.
///
template<typename U, typename T,
typename = typename vector_traits<U>::enable>
adaptor_range<multiplies_by_t<T>, U>
operator*(T &&a, U &&u) {
return map(multiplies_by<T>(std::forward<T>(a)), std::forward<U>(u));
}
///
/// Additive inverse.
///
template<typename U,
typename = typename vector_traits<U>::enable>
adaptor_range<negate, U>
operator-(U &&u) {
return map(negate(), std::forward<U>(u));
}
namespace detail {
template<typename U, typename V>
using dot_type = typename std::common_type<
typename std::remove_reference<U>::type::value_type,
typename std::remove_reference<V>::type::value_type
>::type;
}
///
/// Dot product of two vectors.
///
/// It can also do matrix multiplication if \a u or \a v is a
/// vector of vectors.
///
template<typename U, typename V,
typename = typename vector_traits<U>::enable,
typename = typename vector_traits<V>::enable>
detail::dot_type<U, V>
dot(U &&u, V &&v) {
return fold(plus(), detail::dot_type<U, V>(),
map(multiplies(), u, v));
}
}
#endif

294
src/gallium/frontends/clover/util/algorithm.hpp
View file

@ -1,294 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_ALGORITHM_HPP
#define CLOVER_UTIL_ALGORITHM_HPP
#include <algorithm>
#include <sstream>
#include <stdexcept>
#include "util/range.hpp"
#include "util/functional.hpp"
namespace clover {
namespace detail {
template<typename R>
using preferred_reference_type = decltype(*std::declval<R>().begin());
}
///
/// Return the first element in a range.
///
template<typename R>
detail::preferred_reference_type<R>
head(R &&r) {
assert(!r.empty());
return r.front();
}
///
/// Return all elements in a range but the first.
///
template<typename R>
slice_range<R>
tail(R &&r) {
assert(!r.empty());
return { std::forward<R>(r), 1, r.size() };
}
///
/// Return the only element in a range.
///
template<typename R>
detail::preferred_reference_type<R>
unique(R &&r) {
if (r.size() != 1)
throw std::out_of_range("");
return r.front();
}
///
/// Combine a variable number of ranges element-wise in a single
/// range of tuples.
///
template<typename... Rs>
adaptor_range<zips, Rs...>
zip(Rs &&... rs) {
return map(zips(), std::forward<Rs>(rs)...);
}
///
/// Evaluate the elements of a range.
///
/// Useful because most of the range algorithms evaluate their
/// result lazily.
///
template<typename R>
void
eval(R &&r) {
for (auto i = r.begin(), e = r.end(); i != e; ++i)
*i;
}
///
/// Apply functor \a f element-wise on a variable number of ranges
/// \a rs.
///
/// The functor \a f should take as many arguments as ranges are
/// provided.
///
template<typename F, typename... Rs>
void
for_each(F &&f, Rs &&... rs) {
eval(map(std::forward<F>(f), std::forward<Rs>(rs)...));
}
///
/// Copy all elements from range \a r into an output container
/// starting from iterator \a i.
///
template<typename R, typename I>
void
copy(R &&r, I i) {
for (detail::preferred_reference_type<R> x : r)
*(i++) = x;
}
///
/// Reduce the elements of range \a r by applying functor \a f
/// element by element.
///
/// \a f should take an accumulator value (which is initialized to
/// \a a) and an element value as arguments, and return an updated
/// accumulator value.
///
/// \returns The final value of the accumulator.
///
template<typename F, typename A, typename R>
A
fold(F &&f, A a, R &&r) {
for (detail::preferred_reference_type<R> x : r)
a = f(a, x);
return a;
}
///
/// Return how many elements of range \a r are equal to \a x.
///
template<typename T, typename R>
typename std::remove_reference<R>::type::size_type
count(T &&x, R &&r) {
typename std::remove_reference<R>::type::size_type n = 0;
for (detail::preferred_reference_type<R> y : r) {
if (x == y)
n++;
}
return n;
}
///
/// Return the first element in range \a r for which predicate \a f
/// evaluates to true.
///
template<typename F, typename R>
detail::preferred_reference_type<R>
find(F &&f, R &&r) {
for (detail::preferred_reference_type<R> x : r) {
if (f(x))
return x;
}
throw std::out_of_range("");
}
///
/// Return true if the element-wise application of predicate \a f
/// on \a rs evaluates to true for all elements.
///
template<typename F, typename... Rs>
bool
all_of(F &&f, Rs &&... rs) {
for (auto b : map(f, rs...)) {
if (!b)
return false;
}
return true;
}
///
/// Return true if the element-wise application of predicate \a f
/// on \a rs evaluates to true for any element.
///
template<typename F, typename... Rs>
bool
any_of(F &&f, Rs &&... rs) {
for (auto b : map(f, rs...)) {
if (b)
return true;
}
return false;
}
///
/// Erase elements for which predicate \a f evaluates to true from
/// container \a r.
///
template<typename F, typename R>
void
erase_if(F &&f, R &&r) {
auto i = r.begin(), e = r.end();
for (auto j = r.begin(); j != e; ++j) {
if (!f(*j)) {
if (j != i)
*i = std::move(*j);
++i;
}
}
r.erase(i, e);
}
///
/// Build a vector of string from a space separated string
/// quoted parts content is preserved and unquoted
///
inline std::vector<std::string>
tokenize(const std::string &s) {
std::vector<std::string> ss;
std::ostringstream oss;
// OpenCL programs can pass a quoted argument, most frequently the
// include path. This is useful so that path containing spaces is
// treated as a single argument instead of being split by the spaces.
// Additionally, the argument should also be unquoted before being
// passed to the compiler. We avoid using std::string::replace here to
// remove quotes, as the single and double quote characters can be a
// part of the file name.
bool escape_next = false;
bool in_quote_double = false;
bool in_quote_single = false;
for (auto c : s) {
if (escape_next) {
oss.put(c);
escape_next = false;
} else if (c == '\\') {
escape_next = true;
} else if (c == '"' && !in_quote_single) {
in_quote_double = !in_quote_double;
} else if (c == '\'' && !in_quote_double) {
in_quote_single = !in_quote_single;
} else if (c != ' ' || in_quote_single || in_quote_double) {
oss.put(c);
} else if (oss.tellp() > 0) {
ss.emplace_back(oss.str());
oss.str("");
}
}
if (oss.tellp() > 0)
ss.emplace_back(oss.str());
if (in_quote_double || in_quote_single)
throw invalid_build_options_error();
return ss;
}
///
/// Build a \a sep separated string from a vector of T
///
template<typename T>
std::string
detokenize(const std::vector<T> &ss, const std::string &sep) {
std::string r;
for (const auto &s : ss)
r += (r.empty() ? "" : sep) + std::to_string(s);
return r;
}
///
/// Build a \a sep separated string from a vector of string
///
template <>
inline std::string
detokenize(const std::vector<std::string> &ss, const std::string &sep) {
std::string r;
for (const auto &s : ss)
r += (r.empty() || s.empty() ? "" : sep) + s;
return r;
}
}
#endif

43
src/gallium/frontends/clover/util/compat.hpp
View file

@ -1,43 +0,0 @@
//
// Copyright © Microsoft Corporation
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_COMPAT_HPP
#define CLOVER_UTIL_COMPAT_HPP
#include <type_traits>
namespace clover {
template<typename F, typename... Args>
struct invoke_result {
#if __cplusplus >= 201703L
typedef typename std::invoke_result<
F, Args...
>::type type;
#else
typedef typename std::result_of<
F(Args...)
>::type type;
#endif
};
}
#endif

131
src/gallium/frontends/clover/util/factor.hpp
View file

@ -1,131 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_FACTOR_HPP
#define CLOVER_UTIL_FACTOR_HPP
#include "util/range.hpp"
namespace clover {
namespace factor {
///
/// Calculate all prime integer factors of \p x.
///
/// If \p limit is non-zero, terminate early as soon as enough
/// factors have been collected to reach the product \p limit.
///
template<typename T>
std::vector<T>
find_integer_prime_factors(T x, T limit = 0)
{
const T max_d = (limit > 0 && limit < x ? limit : x);
const T min_x = x / max_d;
std::vector<T> factors;
for (T d = 2; d <= max_d && x > min_x; d++) {
if (x % d == 0) {
for (; x % d == 0; x /= d);
factors.push_back(d);
}
}
return factors;
}
namespace detail {
///
/// Walk the power set of prime factors of the n-dimensional
/// integer array \p grid subject to the constraints given by
/// \p limits.
///
template<typename T>
std::pair<T, std::vector<T>>
next_grid_factor(const std::pair<T, std::vector<T>> &limits,
const std::vector<T> &grid,
const std::vector<std::vector<T>> &factors,
std::pair<T, std::vector<T>> block,
unsigned d = 0, unsigned i = 0) {
if (d >= factors.size()) {
// We're done.
return {};
} else if (i >= factors[d].size()) {
// We're done with this grid dimension, try the next.
return next_grid_factor(limits, grid, factors,
std::move(block), d + 1, 0);
} else {
T f = factors[d][i];
// Try the next power of this factor.
block.first *= f;
block.second[d] *= f;
if (block.first <= limits.first &&
block.second[d] <= limits.second[d] &&
grid[d] % block.second[d] == 0) {
// We've found a valid grid divisor.
return block;
} else {
// Overflow, back off to the zeroth power,
while (block.second[d] % f == 0) {
block.second[d] /= f;
block.first /= f;
}
// ...and carry to the next factor.
return next_grid_factor(limits, grid, factors,
std::move(block), d, i + 1);
}
}
}
}
///
/// Find the divisor of the integer array \p grid that gives the
/// highest possible product not greater than \p product_limit
/// subject to the constraints given by \p coord_limit.
///
template<typename T>
std::vector<T>
find_grid_optimal_factor(T product_limit,
const std::vector<T> &coord_limit,
const std::vector<T> &grid) {
const std::vector<std::vector<T>> factors =
map(find_integer_prime_factors<T>, grid, coord_limit);
const auto limits = std::make_pair(product_limit, coord_limit);
auto best = std::make_pair(T(1), std::vector<T>(grid.size(), T(1)));
for (auto block = best;
block.first != 0 && best.first != product_limit;
block = detail::next_grid_factor(limits, grid, factors, block)) {
if (block.first > best.first)
best = block;
}
return best.second;
}
}
}
#endif

428
src/gallium/frontends/clover/util/functional.hpp
View file

@ -1,428 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_FUNCTIONAL_HPP
#define CLOVER_UTIL_FUNCTIONAL_HPP
#include <type_traits>
namespace clover {
struct identity {
template<typename T>
typename std::remove_reference<T>::type
operator()(T &&x) const {
return x;
}
};
struct plus {
template<typename T, typename S>
typename std::common_type<T, S>::type
operator()(T x, S y) const {
return x + y;
}
};
struct minus {
template<typename T, typename S>
typename std::common_type<T, S>::type
operator()(T x, S y) const {
return x - y;
}
};
struct negate {
template<typename T>
T
operator()(T x) const {
return -x;
}
};
struct multiplies {
template<typename T, typename S>
typename std::common_type<T, S>::type
operator()(T x, S y) const {
return x * y;
}
};
struct divides {
template<typename T, typename S>
typename std::common_type<T, S>::type
operator()(T x, S y) const {
return x / y;
}
};
struct modulus {
template<typename T, typename S>
typename std::common_type<T, S>::type
operator()(T x, S y) const {
return x % y;
}
};
struct minimum {
template<typename T>
T
operator()(T x) const {
return x;
}
template<typename T, typename... Ts>
T
operator()(T x, Ts... xs) const {
T y = minimum()(xs...);
return x < y ? x : y;
}
};
struct maximum {
template<typename T>
T
operator()(T x) const {
return x;
}
template<typename T, typename... Ts>
T
operator()(T x, Ts... xs) const {
T y = maximum()(xs...);
return x < y ? y : x;
}
};
struct preincs {
template<typename T>
T &
operator()(T &x) const {
return ++x;
}
};
struct predecs {
template<typename T>
T &
operator()(T &x) const {
return --x;
}
};
template<typename T>
class multiplies_by_t {
public:
multiplies_by_t(T x) : x(x) {
}
template<typename S>
typename std::common_type<T, S>::type
operator()(S y) const {
return x * y;
}
private:
T x;
};
template<typename T>
multiplies_by_t<T>
multiplies_by(T x) {
return { x };
}
template<typename T>
class preincs_by_t {
public:
preincs_by_t(T n) : n(n) {
}
template<typename S>
S &
operator()(S &x) const {
return x += n;
}
private:
T n;
};
template<typename T>
preincs_by_t<T>
preincs_by(T n) {
return { n };
}
template<typename T>
class predecs_by_t {
public:
predecs_by_t(T n) : n(n) {
}
template<typename S>
S &
operator()(S &x) const {
return x -= n;
}
private:
T n;
};
template<typename T>
predecs_by_t<T>
predecs_by(T n) {
return { n };
}
struct greater {
template<typename T, typename S>
bool
operator()(T x, S y) const {
return x > y;
}
};
struct evals {
template<typename T>
auto
operator()(T &&x) const -> decltype(x()) {
return x();
}
};
struct derefs {
template<typename T>
auto
operator()(T &&x) const -> decltype(*x) {
return *x;
}
};
struct addresses {
template<typename T>
T *
operator()(T &x) const {
return &x;
}
template<typename T>
T *
operator()(std::reference_wrapper<T> x) const {
return &x.get();
}
};
struct begins {
template<typename T>
auto
operator()(T &x) const -> decltype(x.begin()) {
return x.begin();
}
};
struct ends {
template<typename T>
auto
operator()(T &x) const -> decltype(x.end()) {
return x.end();
}
};
struct sizes {
template<typename T>
auto
operator()(T &x) const -> decltype(x.size()) {
return x.size();
}
};
template<typename T>
class advances_by_t {
public:
advances_by_t(T n) : n(n) {
}
template<typename S>
S
operator()(S &&it) const {
std::advance(it, n);
return std::forward<S>(it);
}
private:
T n;
};
template<typename T>
advances_by_t<T>
advances_by(T n) {
return { n };
}
struct zips {
template<typename... Ts>
std::tuple<Ts...>
operator()(Ts &&... xs) const {
return std::tuple<Ts...>(std::forward<Ts>(xs)...);
}
};
struct is_zero {
template<typename T>
bool
operator()(const T &x) const {
return x == 0;
}
};
struct keys {
template<typename P>
auto
operator()(P &&p) const -> decltype(std::get<0>(std::forward<P>(p))) {
return std::get<0>(std::forward<P>(p));
}
};
struct values {
template<typename P>
auto
operator()(P &&p) const -> decltype(std::get<1>(std::forward<P>(p))) {
return std::get<1>(std::forward<P>(p));
}
};
template<typename T>
class equals_t {
public:
equals_t(T &&x) : x(x) {}
template<typename S>
bool
operator()(S &&y) const {
return x == y;
}
private:
T x;
};
template<typename T>
equals_t<T>
equals(T &&x) {
return { std::forward<T>(x) };
}
class name_equals {
public:
name_equals(const std::string &name) : name(name) {
}
template<typename T>
bool
operator()(const T &x) const {
return std::string(x.name.begin(), x.name.end()) == name;
}
private:
const std::string &name;
};
template<typename T>
class key_equals_t {
public:
key_equals_t(T &&x) : x(x) {
}
template<typename P>
bool
operator()(const P &p) const {
return p.first == x;
}
private:
T x;
};
template<typename T>
key_equals_t<T>
key_equals(T &&x) {
return { std::forward<T>(x) };
}
template<typename T>
class type_equals_t {
public:
type_equals_t(T type) : type(type) {
}
template<typename S>
bool
operator()(const S &x) const {
return x.type == type;
}
private:
T type;
};
template<typename T>
type_equals_t<T>
type_equals(T x) {
return { x };
}
template<typename T>
class id_type_equals_t {
public:
id_type_equals_t(const uint32_t id, T t) :
id(id), type(t) {
}
template<typename X>
bool
operator()(const X &x) const {
return id == x.id && type(x);
}
private:
const uint32_t id;
type_equals_t<T> type;
};
template<typename T>
id_type_equals_t<T>
id_type_equals(const uint32_t id, T x) {
return { id, x };
}
struct interval_overlaps {
template<typename T>
bool
operator()(T x0, T x1, T y0, T y1) {
return ((x0 <= y0 && y0 < x1) ||
(y0 <= x0 && x0 < y1));
}
};
}
#endif

161
src/gallium/frontends/clover/util/lazy.hpp
View file

@ -1,161 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_LAZY_HPP
#define CLOVER_UTIL_LAZY_HPP
#include <type_traits>
#include <stdexcept>
#include <memory>
namespace clover {
namespace detail {
template<typename T>
class basic_lazy {
public:
virtual
~basic_lazy() {
}
virtual basic_lazy *
clone() const = 0;
virtual
operator T() const = 0;
};
template<typename T, typename F>
class deferred_lazy : public basic_lazy<T> {
public:
template<typename G>
deferred_lazy(G &&f) : f(new F(std::forward<G>(f))) {
}
virtual basic_lazy<T> *
clone() const {
return new deferred_lazy(*this);
}
operator T() const {
if (f) {
x = (*f)();
f = {};
}
return x;
}
private:
mutable std::shared_ptr<F> f;
mutable T x;
};
template<typename T>
class strict_lazy : public basic_lazy<T> {
public:
template<typename S>
strict_lazy(S &&x) : x(std::forward<S>(x)) {
}
virtual basic_lazy<T> *
clone() const {
return new strict_lazy(*this);
}
operator T() const {
return x;
}
private:
T x;
};
}
///
/// Object that represents a value of type \a T that is calculated
/// lazily as soon as it is required.
///
template<typename T>
class lazy {
public:
class undefined_error : std::logic_error {
public:
undefined_error() : std::logic_error("") {
}
};
///
/// Initialize to some fixed value \a x which isn't calculated
/// lazily.
///
lazy(T x) : obj(new detail::strict_lazy<T>(x)) {
}
///
/// Initialize by providing a functor \a f that will calculate
/// the value on-demand.
///
template<typename F>
lazy(F &&f) : obj(new detail::deferred_lazy<
T, typename std::remove_reference<F>::type
>(std::forward<F>(f))) {
}
///
/// Initialize to undefined.
///
lazy() : lazy([]() {
throw undefined_error();
return T();
}) {
}
lazy(const lazy &other) : obj(obj->clone()) {
}
lazy(lazy &&other) : obj(NULL) {
std::swap(obj, other.obj);
}
~lazy() {
delete obj;
}
lazy &
operator=(lazy other) {
std::swap(obj, other.obj);
return *this;
}
///
/// Evaluate the value.
///
operator T() const {
return *obj;
}
private:
detail::basic_lazy<T> *obj;
};
}
#endif

284
src/gallium/frontends/clover/util/pointer.hpp
View file

@ -1,284 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_POINTER_HPP
#define CLOVER_UTIL_POINTER_HPP
#include <atomic>
namespace clover {
///
/// Some helper functions for raw pointer operations
///
template <class T>
static bool
ptr_is_aligned(const T *ptr, uintptr_t a) noexcept {
assert(a == (a & -a));
uintptr_t ptr_value = reinterpret_cast<uintptr_t>(ptr);
return (ptr_value & (a - 1)) == 0;
}
///
/// Base class for objects that support reference counting.
///
class ref_counter {
public:
ref_counter(unsigned value = 1) : _ref_count(value) {}
unsigned
ref_count() const {
return _ref_count;
}
void
retain() {
_ref_count++;
}
bool
release() {
return (--_ref_count) == 0;
}
private:
std::atomic<unsigned> _ref_count;
};
///
/// Simple reference to a clover::ref_counter object. Unlike
/// clover::intrusive_ptr and clover::intrusive_ref, it does nothing
/// special when the reference count drops to zero.
///
class ref_holder {
public:
ref_holder(ref_counter &o) : p(&o) {
p->retain();
}
ref_holder(const ref_holder &ref) :
ref_holder(*ref.p) {
}
ref_holder(ref_holder &&ref) :
p(ref.p) {
ref.p = NULL;
}
~ref_holder() {
if (p)
p->release();
}
ref_holder &
operator=(ref_holder ref) {
std::swap(ref.p, p);
return *this;
}
bool
operator==(const ref_holder &ref) const {
return p == ref.p;
}
bool
operator!=(const ref_holder &ref) const {
return p != ref.p;
}
private:
ref_counter *p;
};
///
/// Intrusive smart pointer for objects that implement the
/// clover::ref_counter interface.
///
template<typename T>
class intrusive_ptr {
public:
intrusive_ptr(T *q = NULL) : p(q) {
if (p)
p->retain();
}
intrusive_ptr(const intrusive_ptr &ptr) :
intrusive_ptr(ptr.p) {
}
intrusive_ptr(intrusive_ptr &&ptr) :
p(ptr.p) {
ptr.p = NULL;
}
~intrusive_ptr() {
if (p && p->release())
delete p;
}
intrusive_ptr &
operator=(intrusive_ptr ptr) {
std::swap(ptr.p, p);
return *this;
}
bool
operator==(const intrusive_ptr &ref) const {
return p == ref.p;
}
bool
operator!=(const intrusive_ptr &ref) const {
return p != ref.p;
}
T &
operator*() const {
return *p;
}
T *
operator->() const {
return p;
}
T *
operator()() const {
return p;
}
explicit operator bool() const {
return p;
}
explicit operator T *() const {
return p;
}
private:
T *p;
};
///
/// Intrusive smart reference for objects that implement the
/// clover::ref_counter interface.
///
template<typename T>
class intrusive_ref {
public:
intrusive_ref(T &o) : p(&o) {
p->retain();
}
intrusive_ref(const intrusive_ref &ref) :
intrusive_ref(*ref.p) {
}
intrusive_ref(intrusive_ref &&ref) :
p(ref.p) {
ref.p = NULL;
}
~intrusive_ref() {
if (p && p->release())
delete p;
}
intrusive_ref &
operator=(intrusive_ref ref) {
std::swap(ref.p, p);
return *this;
}
bool
operator==(const intrusive_ref &ref) const {
return p == ref.p;
}
bool
operator!=(const intrusive_ref &ref) const {
return p != ref.p;
}
T &
operator()() const {
return *p;
}
operator T &() const {
return *p;
}
private:
T *p;
};
///
/// Initialize a clover::intrusive_ref from a newly created object
/// using the specified constructor arguments.
///
template<typename T, typename... As>
intrusive_ref<T>
create(As &&... as) {
intrusive_ref<T> ref { *new T(std::forward<As>(as)...) };
ref().release();
return ref;
}
///
/// Class that implements the usual pointer interface but in fact
/// contains the object it seems to be pointing to.
///
template<typename T>
class pseudo_ptr {
public:
pseudo_ptr(T x) : x(x) {
}
pseudo_ptr(const pseudo_ptr &p) : x(p.x) {
}
pseudo_ptr &
operator=(const pseudo_ptr &p) {
x = p.x;
return *this;
}
T &
operator*() {
return x;
}
T *
operator->() {
return &x;
}
explicit operator bool() const {
return true;
}
private:
T x;
};
}
#endif

419
src/gallium/frontends/clover/util/range.hpp
View file

@ -1,419 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_RANGE_HPP
#define CLOVER_UTIL_RANGE_HPP
#include <array>
#include <vector>
#include "util/adaptor.hpp"
namespace clover {
///
/// Class that identifies container types where the elements of a
/// range can be stored by the type conversion operator.
///
/// \a T identifies the range element type.
///
template<typename T, typename V>
struct range_store_traits;
template<typename T, typename S>
struct range_store_traits<T, std::vector<S>> {
typedef void enable;
template<typename R>
static std::vector<S>
create(const R &r) {
return { r.begin(), r.end() };
}
};
template<typename T, typename S, std::size_t N>
struct range_store_traits<T, std::array<S, N>> {
typedef void enable;
template<typename R>
static std::array<S, N>
create(const R &r) {
std::array<S, N> v;
assert(r.size() == v.size());
copy(r, v.begin());
return v;
}
};
namespace detail {
///
/// Common functionality that is shared by other implementations
/// of the container concept.
///
template<typename R, typename I, typename CI>
class basic_range {
public:
typedef I iterator;
typedef CI const_iterator;
typedef typename std::iterator_traits<iterator>::value_type value_type;
typedef typename std::iterator_traits<iterator>::reference
reference;
typedef typename std::iterator_traits<const_iterator>::reference
const_reference;
typedef typename std::iterator_traits<iterator>::difference_type
difference_type;
typedef std::size_t size_type;
bool
operator==(const basic_range &r) const {
return *static_cast<const R *>(this) == r;
}
bool
operator!=(const basic_range &r) const {
return !(*this == r);
}
iterator
begin() {
return static_cast<R *>(this)->begin();
}
iterator
end() {
return static_cast<R *>(this)->end();
}
const_iterator
begin() const {
return static_cast<const R *>(this)->begin();
}
const_iterator
end() const {
return static_cast<const R *>(this)->end();
}
std::reverse_iterator<iterator>
rbegin() {
return { begin() };
}
std::reverse_iterator<iterator>
rend() {
return { end() };
}
reference
front() {
return *begin();
}
reference
back() {
return *(end() - 1);
}
bool
empty() const {
return begin() == end();
}
reference
at(size_type i) {
if (i >= static_cast<const R *>(this)->size())
throw std::out_of_range("");
return begin()[i];
}
const_reference
at(size_type i) const {
if (i >= static_cast<const R *>(this)->size())
throw std::out_of_range("");
return begin()[i];
}
reference
operator[](size_type i) {
return begin()[i];
}
const_reference
operator[](size_type i) const {
return begin()[i];
}
template<typename V>
using store_traits = range_store_traits<
typename std::remove_cv<value_type>::type, V
>;
template<typename V,
typename = typename store_traits<V>::enable>
operator V() const {
return store_traits<V>::create(*static_cast<const R *>(this));
}
};
}
///
/// Range that contains all elements delimited by an iterator pair
/// (\a i, \a j). Use range() as convenience constructor.
///
template<typename I>
class iterator_range : public detail::basic_range<iterator_range<I>, I, I> {
public:
typedef detail::basic_range<iterator_range<I>, I, I> super;
iterator_range() : i(), j() {
}
iterator_range(I i, I j) : i(i), j(j) {
}
bool
operator==(const iterator_range &r) const {
return i == r.i && j == r.j;
}
I
begin() const {
return i;
}
I
end() const {
return j;
}
typename super::size_type
size() const {
return end() - begin();
}
private:
I i, j;
};
namespace detail {
template<typename T>
using preferred_iterator_type = decltype(std::declval<T>().begin());
}
///
/// Range that transforms the contents of a number of source ranges
/// \a os element-wise by using the provided functor \a f. Use
/// map() as convenience constructor.
///
template<typename F, typename... Os>
class adaptor_range :
public detail::basic_range<adaptor_range<F, Os...>,
detail::iterator_adaptor<
F, detail::preferred_iterator_type<Os>...>,
detail::iterator_adaptor<
F, detail::preferred_iterator_type<const Os>...>
> {
public:
typedef detail::basic_range<adaptor_range<F, Os...>,
detail::iterator_adaptor<
F, detail::preferred_iterator_type<Os>...>,
detail::iterator_adaptor<
F, detail::preferred_iterator_type<const Os>...>
> super;
template<typename G, typename... Rs>
adaptor_range(G &&f, Rs &&... os) :
f(std::forward<G>(f)), os(std::forward<Rs>(os)...) {
}
bool
operator==(const adaptor_range &r) const {
return f == r.f && os == r.os;
}
typename super::iterator
begin() {
return { f, tuple::map(begins(), os) };
}
typename super::iterator
end() {
return { f, tuple::map(advances_by(size()),
tuple::map(begins(), os)) };
}
typename super::const_iterator
begin() const {
return { f, tuple::map(begins(), os) };
}
typename super::const_iterator
end() const {
return { f, tuple::map(advances_by(size()),
tuple::map(begins(), os)) };
}
typename super::size_type
size() const {
return tuple::apply(minimum(), tuple::map(sizes(), os));
}
private:
F f;
std::tuple<Os...> os;
};
///
/// Range that contains all elements delimited by the index pair
/// (\a i, \a j) in the source range \a r. Use slice() as
/// convenience constructor.
///
template<typename O>
class slice_range :
public detail::basic_range<slice_range<O>,
detail::preferred_iterator_type<O>,
detail::preferred_iterator_type<const O>> {
public:
typedef detail::basic_range<slice_range<O>,
detail::preferred_iterator_type<O>,
detail::preferred_iterator_type<const O>
> super;
template<typename R>
slice_range(R &&r, typename super::size_type i,
typename super::size_type j) :
o(std::forward<R>(r)), i(i), j(j) {
}
bool
operator==(const slice_range &r) const {
return o == r.o && i == r.i && j == r.j;
}
typename super::iterator
begin() {
return std::next(o.begin(), i);
}
typename super::iterator
end() {
return std::next(o.begin(), j);
}
typename super::const_iterator
begin() const {
return std::next(o.begin(), i);
}
typename super::const_iterator
end() const {
return std::next(o.begin(), j);
}
typename super::size_type
size() const {
return j - i;
}
private:
O o;
typename super::size_type i, j;
};
///
/// Create a range from an iterator pair (\a i, \a j).
///
/// \sa iterator_range.
///
template<typename T>
iterator_range<T>
range(T i, T j) {
return { i, j };
}
///
/// Create a range of \a n elements starting from iterator \a i.
///
/// \sa iterator_range.
///
template<typename T>
iterator_range<T>
range(T i, typename std::iterator_traits<T>::difference_type n) {
return { i, i + n };
}
///
/// Create a range by transforming the contents of a number of
/// source ranges \a rs element-wise using a provided functor \a f.
///
/// \sa adaptor_range.
///
template<typename F, typename... Rs>
adaptor_range<F, Rs...>
map(F &&f, Rs &&... rs) {
return { std::forward<F>(f), std::forward<Rs>(rs)... };
}
///
/// Create a range identical to another range \a r.
///
template<typename R>
adaptor_range<identity, R>
range(R &&r) {
return { identity(), std::forward<R>(r) };
}
///
/// Create a range by taking the elements delimited by the index
/// pair (\a i, \a j) in a source range \a r.
///
/// \sa slice_range.
///
template<typename R>
slice_range<R>
slice(R &&r, typename slice_range<R>::size_type i,
typename slice_range<R>::size_type j) {
return { std::forward<R>(r), i, j };
}
///
/// Range that behaves as a vector of references of type \a T.
///
/// Useful because STL containers cannot contain references to
/// objects as elements.
///
template<typename T>
class ref_vector : public adaptor_range<derefs, std::vector<T *>> {
public:
ref_vector(std::initializer_list<std::reference_wrapper<T>> il) :
adaptor_range<derefs, std::vector<T *>>(derefs(), map(addresses(), il)) {
}
template<typename R>
ref_vector(R &&r) : adaptor_range<derefs, std::vector<T *>>(
derefs(), map(addresses(), std::forward<R>(r))) {
}
};
}
#endif

117
src/gallium/frontends/clover/util/tuple.hpp
View file

@ -1,117 +0,0 @@
//
// Copyright 2013 Francisco Jerez
//
// Permission is hereby granted, free of charge, to any person obtaining a
// copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
// THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
// OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
//
#ifndef CLOVER_UTIL_TUPLE_HPP
#define CLOVER_UTIL_TUPLE_HPP
#include <tuple>
namespace clover {
namespace tuple {
///
/// Static sequence of integers.
///
template<int... Is>
struct integral_sequence;
///
/// Static sequence containing all integers from 0 to N-1.
///
template<int N, int... Is>
struct enumerate {
typedef typename enumerate<N-1, N-1, Is...>::type
type;
};
template<int... Is>
struct enumerate<0, Is...> {
typedef integral_sequence<Is...> type;
};
namespace detail {
template<typename F, typename T,
typename E = typename enumerate<std::tuple_size<
typename std::remove_reference<T>::type>::value
>::type>
struct _apply;
template<typename F, typename T, int... Is>
struct _apply<F, T, integral_sequence<Is...>> {
typedef typename std::remove_reference<F>::type func_type;
typedef decltype(
std::declval<func_type>()(std::get<Is>(std::declval<T &&>())...)
) value_type;
static value_type
eval(F &&f, T &&t) {
return f(std::get<Is>(std::forward<T>(t))...);
}
};
}
///
/// Evaluate function \a f with the elements of tuple \a t
/// expanded as arguments.
///
template<typename F, typename T>
typename detail::_apply<F, T>::value_type
apply(F &&f, T &&t) {
return detail::_apply<F, T>::eval(std::forward<F>(f),
std::forward<T>(t));
}
namespace detail {
template<typename F, typename T,
typename E = typename enumerate<std::tuple_size<
typename std::remove_reference<T>::type>::value
>::type>
struct _map;
template<typename F, typename T, int... Is>
struct _map<F, T, integral_sequence<Is...>> {
typedef typename std::remove_reference<F>::type func_type;
typedef std::tuple<
decltype(std::declval<func_type>()(
std::get<Is>(std::declval<T &&>())))...
> value_type;
static value_type
eval(F &&f, T &&t) {
return value_type(f(std::get<Is>(std::forward<T>(t)))...);
}
};
}
///
/// Evaluate function \a f on each element of the tuple \a t and
/// return the resulting values as a new tuple.
///
template<typename F, typename T>
typename detail::_map<F, T>::value_type
map(F &&f, T &&t) {
return detail::_map<F, T>::eval(std::forward<F>(f),
std::forward<T>(t));
}
}
}
#endif

9
src/gallium/meson.build
View file

@ -195,15 +195,6 @@ if with_gallium_d3d12
else
driver_d3d12 = declare_dependency()
endif
if with_gallium_clover or with_tests
# At the moment, clover and gallium/tests are the only two consumers
# for pipe-loader
subdir('targets/pipe-loader')
endif
if with_gallium_clover
subdir('frontends/clover')
subdir('targets/opencl')
endif
if with_gallium_rusticl
subdir('frontends/rusticl')
subdir('targets/rusticl')

1
src/gallium/targets/opencl/mesa.icd.in
View file

@ -1 +0,0 @@
lib@OPENCL_LIBNAME@.so.@OPENCL_VERSION@

69
src/gallium/targets/opencl/meson.build
View file

@ -1,69 +0,0 @@
# Copyright © 2017 Intel Corporation
# SPDX-License-Identifier: MIT
opencl_link_args = []
opencl_link_deps = []
opencl_version = '1'
if with_ld_version_script
opencl_link_args += [
'-Wl,--version-script', join_paths(meson.current_source_dir(), 'opencl.sym')
]
opencl_link_deps += files('opencl.sym')
endif
llvm_libdir = dep_llvm.get_variable(cmake : 'LLVM_LIBRARY_DIR', configtool: 'libdir')
opencl_libname = with_opencl_icd ? 'MesaOpenCL' : 'OpenCL'
polly_dep = null_dep
polly_isl_dep = null_dep
if dep_llvm.version().version_compare('>=10.0.0')
polly_dep = cpp.find_library('Polly', dirs : llvm_libdir, required : false)
polly_isl_dep = cpp.find_library('PollyISL', dirs : llvm_libdir, required : false)
endif
ocldef_in = files(opencl_libname + '.def.in')[0]
ocldef = custom_target(
'ocldef.def',
input: ocldef_in,
output : 'ocldef.def',
command : gen_vs_module_defs_normal_command,
)
libopencl = shared_library(
opencl_libname,
[],
vs_module_defs : ocldef,
link_args : [ld_args_gc_sections, opencl_link_args],
link_depends : opencl_link_deps,
link_whole : libclover,
link_with : [libpipe_loader_dynamic, libgallium],
dependencies : [
idep_mesautil,
dep_clock, dep_dl, dep_unwind, dep_elf, dep_clang, polly_dep, polly_isl_dep, dep_version
],
name_prefix : host_machine.system() == 'windows' ? '' : [], # otherwise mingw will create libOpenCL-1.dll or libMesaOpenCL-1.dll
version : '@0@.0.0'.format(opencl_version),
soversion : host_machine.system() == 'windows' ? '' : opencl_version,
install : true,
)
if with_opencl_icd
_config = configuration_data()
_config.set('OPENCL_LIBNAME', 'MesaOpenCL')
_config.set('OPENCL_VERSION', opencl_version)
configure_file(
configuration : _config,
input : 'mesa.icd.in',
output : 'mesa.icd',
install : true,
install_tag : 'runtime',
install_dir : join_paths(get_option('sysconfdir'), 'OpenCL', 'vendors'),
)
# .so is hardcoded in the icd as well
devenv.prepend(
'OCL_ICD_FILENAMES',
meson.current_build_dir() / 'libMesaOpenCL.so.@0@'.format(opencl_version)
)
endif

7
src/gallium/targets/opencl/opencl.sym
View file

@ -1,7 +0,0 @@
{
global:
cl*;
opencl_dri_*;
local:
*;
};