fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2025-12-21 22:20:14 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
mesa/src/intel/compiler/brw_vec4.cpp

2912 lines
90 KiB
C++
Raw Normal View History

i965/vs: Copy the live intervals calculation over from the FS. This is a rather pessimistic calculation, since it doesn't distinguish individual channels of a vec4, or elements of an array, but should be a minimum start for register allocation.
2011-08-16 15:09:48 -07:00
/*
* Copyright © 2011 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#include "brw_vec4.h"
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
#include "brw_fs.h"
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
#include "brw_cfg.h"
i965: Create NIR during LinkShader() and ProgramStringNotify(). Previously, we translated into NIR and did all the optimizations and lowering as part of running fs_visitor. This meant that we did all of that work twice for fragment shaders - once for SIMD8, and again for SIMD16. We also had to redo it every time we hit a state based recompile. We now generate NIR once at link time. ARB programs don't have linking, so we instead generate it at ProgramStringNotify time. Mesa's fixed function vertex program handling doesn't bother to inform the driver about new programs at all (which is rather mean), so we generate NIR at the last minute, if it hasn't happened already. shader-db runs ~9.4% faster on my i7-5600U, with a release build. v2: Check NirOptions != NULL in ProgramStringNotify(). Don't bother using _mesa_program_enum_to_shader_stage as we already know it. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-07 15:15:09 -07:00
#include "brw_nir.h"
i965: Lower min/max after optimization on Gen4/5. Gen4/5's SEL instruction cannot use conditional modifiers, so min/max are implemented as CMP + SEL. Handling that after optimization lets us CSE more. On Ironlake: total instructions in shared programs: 6426035 -> 6422753 (-0.05%) instructions in affected programs: 326604 -> 323322 (-1.00%) helped: 1411 total cycles in shared programs: 129184700 -> 129101586 (-0.06%) cycles in affected programs: 18950290 -> 18867176 (-0.44%) helped: 2419 HURT: 328 Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-02-11 12:27:02 -08:00
#include "brw_vec4_builder.h"
i965/vec4: Define helpers to calculate the common live interval of a range of variables. These will be especially useful when we start keeping track of liveness information for each subregister. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 20:49:43 +02:00
#include "brw_vec4_live_variables.h"
i965: Add a header for brw_vec4_vs_visitor brw_vs.h is not a compiler file but brw_vec4_visitor is definitely a compiler thing. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-02-28 19:00:45 -08:00
#include "brw_vec4_vs.h"
i965: Add a pass to remove dead control flow. Removes IF/ENDIF and IF/ELSE/ENDIF with no intervening instructions. total instructions in shared programs: 1360393 -> 1360387 (-0.00%) instructions in affected programs: 157 -> 151 (-3.82%) (no change in vertex shaders) Reviewed-by: Paul Berry <stereotype441@gmail.com> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-10-30 10:32:12 -07:00
#include "brw_dead_control_flow.h"
i965: Move intel_debug.h to intel/common/gen_debug.h This is shared between the Vulkan and GL drivers as it's a requirement of the back-end compiler. However, it doesn't really belong in the compiler. We rename the file to match the prefix of the other stuff in common and because libdrm defines an intel_debug.h and this avoids a pile of possible name conflicts. Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2017-03-01 11:20:25 -08:00
#include "common/gen_debug.h"
i965/vs: Copy the live intervals calculation over from the FS. This is a rather pessimistic calculation, since it doesn't distinguish individual channels of a vec4, or elements of an array, but should be a minimum start for register allocation.
2011-08-16 15:09:48 -07:00
#include "program/prog_parameter.h"
#define MAX_INSTRUCTION (1 << 30)
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
using namespace brw;
i965/vs: Copy the live intervals calculation over from the FS. This is a rather pessimistic calculation, since it doesn't distinguish individual channels of a vec4, or elements of an array, but should be a minimum start for register allocation.
2011-08-16 15:09:48 -07:00
namespace brw {
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
void
src_reg::init()
{
memset(this, 0, sizeof(*this));
this->file = BAD_FILE;
}
i965: Combine register file field. The first four values (2-bits) are hardware values, and VGRF, ATTR, and UNIFORM remain values used in the IR. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 18:41:27 -07:00
src_reg::src_reg(enum brw_reg_file file, int nr, const glsl_type *type)
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
{
init();
this->file = file;
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
this->nr = nr;
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
if (type && (type->is_scalar() || type->is_vector() || type->is_matrix()))
i965/vec4: Remove swizzle_for_size() in favour of brw_swizzle_for_size(). It could be objected that swizzle_for_size() is "faster" than brw_swizzle_for_size(). It's not measurably better in any reasonable CPU-bound benchmark on VLV according to the Finnish benchmarking system (including the SynMark2 DrvShComp shader compilation benchmark). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:18:08 +02:00
this->swizzle = brw_swizzle_for_size(type->vector_elements);
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
else
i965/vec4: Fix confusion between SWIZZLE and BRW_SWIZZLE macros. Most of the VEC4 back-end agrees on src_reg::swizzle being one of the BRW_SWIZZLE macros defined in brw_reg.h, except in two places where we use Mesa's SWIZZLE macros. There is even a doxygen comment saying that Mesa's macros are the right ones. They are incompatible swizzle representations (3 bits vs. 2 bits per component), and the code using Mesa's works by pure luck. Fix it. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-11-27 22:21:45 -08:00
this->swizzle = BRW_SWIZZLE_XYZW;
i965/vec4: fill src_reg type using the constructor type parameter The src_reg constructor that received the glsl_type was using it only to build the swizzle, but not to fill this->type as dst_reg is doing. This caused some type mismatch between movs and alu operations on the NIR path, so copy propagation optimization was not applied to remove unneeded movs if negate modifier was involved. This was first detected on minus (negate+add) operations. Shader DB results (taking into account only vec4): total instructions in shared programs: 20019 -> 19934 (-0.42%) instructions in affected programs: 2918 -> 2833 (-2.91%) helped: 79 HURT: 0 GAINED: 0 LOST: 0 Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-01 17:02:20 +02:00
if (type)
this->type = brw_type_for_base_type(type);
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
}
/** Generic unset register constructor. */
src_reg::src_reg()
{
init();
}
i965: Use scope operator to ensure brw_reg is interpreted as a type. In the next patch, I make backend_reg's inheritance from brw_reg private, which confuses clang when it sees the type "struct brw_reg" in the derived class constructors, thinking it is referring to the privately inherited brw_reg: brw_fs.cpp:366:23: error: 'brw_reg' is a private member of 'brw_reg' fs_reg::fs_reg(struct brw_reg reg) : ^ brw_shader.h:39:22: note: constrained by private inheritance here struct backend_reg : private brw_reg ^~~~~~~~~~~~~~~ brw_reg.h:232:8: note: member is declared here struct brw_reg { ^ Avoid this by marking brw_reg with the scope resolution operator.
2015-11-23 16:17:28 -08:00
src_reg::src_reg(struct ::brw_reg reg) :
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
backend_reg(reg)
i965/vec4: Add constructor of src_reg from a fixed hardware reg. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-11-22 20:22:03 -08:00
{
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
this->offset = 0;
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
this->reladdr = NULL;
i965/vec4: Add constructor of src_reg from a fixed hardware reg. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-11-22 20:22:03 -08:00
}
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
src_reg::src_reg(const dst_reg &reg) :
i965: Use implicit backend_reg copy-constructor. In order to do this, we have to change the signature of the backend_reg(brw_reg) constructor to take a reference to a brw_reg in order to avoid unresolvable ambiguity about which constructor is actually being called in the other modifications in this patch. As far as I understand it, the rule in C++ is that if multiple constructors are available for parent classes, the one closest to you in the class heirarchy is closen, but if one of them didn't take a reference, that screws things up. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-20 21:18:26 -08:00
backend_reg(reg)
i965/vs: Move the other two src_reg/dst_reg constructors to brw_vec4.cpp. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:31:46 -07:00
{
this->reladdr = reg.reladdr;
i965/vec4: Improve src_reg/dst_reg conversion constructors. This simplifies the src_reg/dst_reg conversion constructors using the swizzle utils introduced in a previous patch. It also makes them more useful by changing their semantics slightly: dst_reg(src_reg) used to set the writemask to XYZW if the src_reg swizzle was anything other than XXXX, which was almost certainly not what the caller intended if the swizzle was non-trivial. After this patch the same components that are present in the swizzle will be enabled in the resulting writemask. src_reg(dst_reg) used to set the first components of the swizzle to the enabled components of the writemask and then replicate the last enabled component to fill the swizzle, which, in cases where the writemask didn't have exactly the first n components set, would in general not be compatible with the original dst_reg. E.g.: | ADD(tmp, src_reg(tmp), src_reg(1)); would *not* do what one would expect (add one to each of the enabled components of tmp) if tmp didn't have a writemask of the described form (e.g. YZ, YW, XZW would all fail). This pattern actually occurs in many different places in the VEC4 back-end, it's a wonder that it hasn't caused piglit failures until now. After this patch src_reg(dst_reg) will construct a swizzle with each enabled component at its natural position (e.g. Y at the second position, Z at the third, and so on). The resulting swizzle will behave like the identity when used in any instruction with the original writemask. I've manually verified that *none* of the callers of both conversion constructors were relying on the previous broken semantics. There are no piglit regressions on any generation. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 15:27:58 +02:00
this->swizzle = brw_swizzle_for_mask(reg.writemask);
i965/vs: Move the other two src_reg/dst_reg constructors to brw_vec4.cpp. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:31:46 -07:00
}
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
void
dst_reg::init()
{
memset(this, 0, sizeof(*this));
this->file = BAD_FILE;
this->writemask = WRITEMASK_XYZW;
}
dst_reg::dst_reg()
{
init();
}
i965: Combine register file field. The first four values (2-bits) are hardware values, and VGRF, ATTR, and UNIFORM remain values used in the IR. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 18:41:27 -07:00
dst_reg::dst_reg(enum brw_reg_file file, int nr)
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
{
init();
this->file = file;
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
this->nr = nr;
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
}
i965: Combine register file field. The first four values (2-bits) are hardware values, and VGRF, ATTR, and UNIFORM remain values used in the IR. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 18:41:27 -07:00
dst_reg::dst_reg(enum brw_reg_file file, int nr, const glsl_type *type,
i965/vec4: Fix signedness of dst_reg::writemask. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 15:24:06 +02:00
unsigned writemask)
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
{
init();
i965: Combine register file field. The first four values (2-bits) are hardware values, and VGRF, ATTR, and UNIFORM remain values used in the IR. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 18:41:27 -07:00
this->file = file;
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
this->nr = nr;
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
this->type = brw_type_for_base_type(type);
this->writemask = writemask;
}
i965: Combine register file field. The first four values (2-bits) are hardware values, and VGRF, ATTR, and UNIFORM remain values used in the IR. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 18:41:27 -07:00
dst_reg::dst_reg(enum brw_reg_file file, int nr, brw_reg_type type,
i965/vec4: Add a new dst_reg constructor accepting a brw_reg_type This is useful for the upcoming texture support in NIR->vec4 pass, as we found several cases where the brw_type is available, but not the glsl_type. Without this new constructor, the alternative would be: dst_reg reg(MRF, <reg>) reg.type = <brw_type> reg.writemask = <mask> Adding a new constructor makes code easier to read. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-05-23 23:42:58 +02:00
unsigned writemask)
{
init();
this->file = file;
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
this->nr = nr;
i965/vec4: Add a new dst_reg constructor accepting a brw_reg_type This is useful for the upcoming texture support in NIR->vec4 pass, as we found several cases where the brw_type is available, but not the glsl_type. Without this new constructor, the alternative would be: dst_reg reg(MRF, <reg>) reg.type = <brw_type> reg.writemask = <mask> Adding a new constructor makes code easier to read. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-05-23 23:42:58 +02:00
this->type = type;
this->writemask = writemask;
}
i965: Use scope operator to ensure brw_reg is interpreted as a type. In the next patch, I make backend_reg's inheritance from brw_reg private, which confuses clang when it sees the type "struct brw_reg" in the derived class constructors, thinking it is referring to the privately inherited brw_reg: brw_fs.cpp:366:23: error: 'brw_reg' is a private member of 'brw_reg' fs_reg::fs_reg(struct brw_reg reg) : ^ brw_shader.h:39:22: note: constrained by private inheritance here struct backend_reg : private brw_reg ^~~~~~~~~~~~~~~ brw_reg.h:232:8: note: member is declared here struct brw_reg { ^ Avoid this by marking brw_reg with the scope resolution operator.
2015-11-23 16:17:28 -08:00
dst_reg::dst_reg(struct ::brw_reg reg) :
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
backend_reg(reg)
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
{
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
this->offset = 0;
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
this->reladdr = NULL;
i965/vs: Move class functions to brw_vec4.cpp. This has less impact than for the FS (4k savings), because it was partially done already, but makes things more consistent. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:25:27 -07:00
}
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
dst_reg::dst_reg(const src_reg &reg) :
i965: Use implicit backend_reg copy-constructor. In order to do this, we have to change the signature of the backend_reg(brw_reg) constructor to take a reference to a brw_reg in order to avoid unresolvable ambiguity about which constructor is actually being called in the other modifications in this patch. As far as I understand it, the rule in C++ is that if multiple constructors are available for parent classes, the one closest to you in the class heirarchy is closen, but if one of them didn't take a reference, that screws things up. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-20 21:18:26 -08:00
backend_reg(reg)
i965/vs: Move the other two src_reg/dst_reg constructors to brw_vec4.cpp. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:31:46 -07:00
{
i965/vec4: Improve src_reg/dst_reg conversion constructors. This simplifies the src_reg/dst_reg conversion constructors using the swizzle utils introduced in a previous patch. It also makes them more useful by changing their semantics slightly: dst_reg(src_reg) used to set the writemask to XYZW if the src_reg swizzle was anything other than XXXX, which was almost certainly not what the caller intended if the swizzle was non-trivial. After this patch the same components that are present in the swizzle will be enabled in the resulting writemask. src_reg(dst_reg) used to set the first components of the swizzle to the enabled components of the writemask and then replicate the last enabled component to fill the swizzle, which, in cases where the writemask didn't have exactly the first n components set, would in general not be compatible with the original dst_reg. E.g.: | ADD(tmp, src_reg(tmp), src_reg(1)); would *not* do what one would expect (add one to each of the enabled components of tmp) if tmp didn't have a writemask of the described form (e.g. YZ, YW, XZW would all fail). This pattern actually occurs in many different places in the VEC4 back-end, it's a wonder that it hasn't caused piglit failures until now. After this patch src_reg(dst_reg) will construct a swizzle with each enabled component at its natural position (e.g. Y at the second position, Z at the third, and so on). The resulting swizzle will behave like the identity when used in any instruction with the original writemask. I've manually verified that *none* of the callers of both conversion constructors were relying on the previous broken semantics. There are no piglit regressions on any generation. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 15:27:58 +02:00
this->writemask = brw_mask_for_swizzle(reg.swizzle);
i965/vs: Move the other two src_reg/dst_reg constructors to brw_vec4.cpp. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-07-04 13:31:46 -07:00
this->reladdr = reg.reladdr;
}
i965/vec4: Implement equals() method for dst_reg too. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:40:07 +02:00
bool
dst_reg::equals(const dst_reg &r) const
{
i965: Add and use backend_reg::equals(). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-22 13:25:05 -08:00
return (this->backend_reg::equals(r) &&
i965/vec4: Implement equals() method for dst_reg too. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:40:07 +02:00
(reladdr == r.reladdr ||
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
(reladdr && r.reladdr && reladdr->equals(*r.reladdr))));
i965/vec4: Implement equals() method for dst_reg too. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:40:07 +02:00
}
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
bool
vec4_instruction::is_send_from_grf()
{
i965/vs: Use GRFs for pull constant offsets on gen7. This allows the computation of the offset to get written directly into the message source. shader-db results: total instructions in shared programs: 3308390 -> 3283025 (-0.77%) instructions in affected programs: 442998 -> 417633 (-5.73%) No difference in GLB2.7 low res (n=9). Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-04-04 14:10:18 -07:00
switch (opcode) {
case SHADER_OPCODE_SHADER_TIME_ADD:
case VS_OPCODE_PULL_CONSTANT_LOAD_GEN7:
i965/vec4: Add support for untyped surface message sends from GRF. This doesn't actually enable untyped surface message sends from GRF yet, the upcoming atomic counter and image intrinsic lowering code will. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 17:42:47 +02:00
case SHADER_OPCODE_UNTYPED_ATOMIC:
case SHADER_OPCODE_UNTYPED_SURFACE_READ:
i965: Add untyped surface write opcode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:24:14 +03:00
case SHADER_OPCODE_UNTYPED_SURFACE_WRITE:
i965: Add typed surface access opcodes. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:28:25 +03:00
case SHADER_OPCODE_TYPED_ATOMIC:
case SHADER_OPCODE_TYPED_SURFACE_READ:
case SHADER_OPCODE_TYPED_SURFACE_WRITE:
i965: Add tessellation control shaders. The TCS is the first tessellation shader stage, and the most complicated. It has access to each of the control points in the input patch, and computes a new output patch. There is one logical invocation per output control point; all invocations run in parallel, and can communicate by reading and writing output variables. One of the main responsibilities of the TCS is to write the special gl_TessLevelOuter[] and gl_TessLevelInner[] output variables which control how much new geometry the hardware tessellation engine will produce. Otherwise, it simply writes outputs that are passed along to the TES. We run in SIMD4x2 mode, handling two logical invocations per EU thread. The hardware doesn't properly manage the dispatch mask for us; it always initializes it to 0xFF. We wrap the whole program in an IF..ENDIF block to handle an odd number of invocations, essentially falling back to SIMD4x1 on the last thread. v2: Update comments (requested by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:07:39 -08:00
case VEC4_OPCODE_URB_READ:
case TCS_OPCODE_URB_WRITE:
i965: Relase input URB Handles on Gen7/7.5 when TCS threads finish. Pre-Broadwell hardware requires us to manually release the ICP Handles by issuing URB read messages with the "Complete" bit set. We can do this in pairs to use fewer URB read messages. Based heavily on work from Chris Forbes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Edward O'Callaghan <eocallaghan@alterapraxis.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-25 17:54:22 -08:00
case TCS_OPCODE_RELEASE_INPUT:
i965: Add tessellation control shaders. The TCS is the first tessellation shader stage, and the most complicated. It has access to each of the control points in the input patch, and computes a new output patch. There is one logical invocation per output control point; all invocations run in parallel, and can communicate by reading and writing output variables. One of the main responsibilities of the TCS is to write the special gl_TessLevelOuter[] and gl_TessLevelInner[] output variables which control how much new geometry the hardware tessellation engine will produce. Otherwise, it simply writes outputs that are passed along to the TES. We run in SIMD4x2 mode, handling two logical invocations per EU thread. The hardware doesn't properly manage the dispatch mask for us; it always initializes it to 0xFF. We wrap the whole program in an IF..ENDIF block to handle an odd number of invocations, essentially falling back to SIMD4x1 on the last thread. v2: Update comments (requested by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:07:39 -08:00
case SHADER_OPCODE_BARRIER:
i965/vs: Use GRFs for pull constant offsets on gen7. This allows the computation of the offset to get written directly into the message source. shader-db results: total instructions in shared programs: 3308390 -> 3283025 (-0.77%) instructions in affected programs: 442998 -> 417633 (-5.73%) No difference in GLB2.7 low res (n=9). Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-04-04 14:10:18 -07:00
return true;
default:
return false;
}
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
}
i965: Add src/dst interference for certain instructions with hazards. When working on tessellation shaders, I created some vec4 virtual opcodes for creating message headers through a sequence like: mov(8) g7<1>UD 0x00000000UD { align1 WE_all 1Q compacted }; mov(1) g7.5<1>UD 0x00000100UD { align1 WE_all }; mov(1) g7<1>UD g0<0,1,0>UD { align1 WE_all compacted }; mov(1) g7.3<1>UD g8<0,1,0>UD { align1 WE_all }; This is done in the generator since the vec4 backend can't handle align1 regioning. From the visitor's point of view, this is a single opcode: hs_set_output_urb_offsets vgrf7.0:UD, 1U, vgrf8.xxxx:UD Normally, there's no hazard between sources and destinations - an instruction (naturally) reads its sources, then writes the result to the destination. However, when the virtual instruction generates multiple hardware instructions, we can get into trouble. In the above example, if the register allocator assigned vgrf7 and vgrf8 to the same hardware register, then we'd clobber the source with 0 in the first instruction, and read back the wrong value in the last one. It occured to me that this is exactly the same problem we have with SIMD16 instructions that use W/UW or B/UB types with 0 stride. The hardware implicitly decodes them as two SIMD8 instructions, and with the overlapping regions, the first would clobber the second. Previously, we handled that by incrementing the live range end IP by 1, which works, but is excessive: the next instruction doesn't actually care about that. It might also be the end of control flow. This might keep values alive too long. What we really want is to say "my source and destinations interfere". This patch creates new infrastructure for doing just that, and teaches the register allocator to add interference when there's a hazard. For my vec4 case, we can determine this by switching on opcodes. For the SIMD16 case, we just move the existing code there. I audited our existing virtual opcodes that generate multiple instructions; I believe FS_OPCODE_PACK_HALF_2x16_SPLIT needs this treatment as well, but no others. v2: Rebased by mattst88. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-19 16:00:18 -08:00
/**
* Returns true if this instruction's sources and destinations cannot
* safely be the same register.
*
* In most cases, a register can be written over safely by the same
* instruction that is its last use. For a single instruction, the
* sources are dereferenced before writing of the destination starts
* (naturally).
*
* However, there are a few cases where this can be problematic:
*
* - Virtual opcodes that translate to multiple instructions in the
* code generator: if src == dst and one instruction writes the
* destination before a later instruction reads the source, then
* src will have been clobbered.
*
* The register allocator uses this information to set up conflicts between
* GRF sources and the destination.
*/
bool
vec4_instruction::has_source_and_destination_hazard() const
{
switch (opcode) {
i965: Add tessellation control shaders. The TCS is the first tessellation shader stage, and the most complicated. It has access to each of the control points in the input patch, and computes a new output patch. There is one logical invocation per output control point; all invocations run in parallel, and can communicate by reading and writing output variables. One of the main responsibilities of the TCS is to write the special gl_TessLevelOuter[] and gl_TessLevelInner[] output variables which control how much new geometry the hardware tessellation engine will produce. Otherwise, it simply writes outputs that are passed along to the TES. We run in SIMD4x2 mode, handling two logical invocations per EU thread. The hardware doesn't properly manage the dispatch mask for us; it always initializes it to 0xFF. We wrap the whole program in an IF..ENDIF block to handle an odd number of invocations, essentially falling back to SIMD4x1 on the last thread. v2: Update comments (requested by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:07:39 -08:00
case TCS_OPCODE_SET_INPUT_URB_OFFSETS:
case TCS_OPCODE_SET_OUTPUT_URB_OFFSETS:
i965: Port tessellation evaluation shaders to vec4 mode. This can be used on Broadwell by setting INTEL_SCALAR_TES=0. More importantly, it will be used for Ivybridge and Haswell. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Edward O'Callaghan <eocallaghan@alterapraxis.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:30:35 -08:00
case TES_OPCODE_ADD_INDIRECT_URB_OFFSET:
i965: Add tessellation control shaders. The TCS is the first tessellation shader stage, and the most complicated. It has access to each of the control points in the input patch, and computes a new output patch. There is one logical invocation per output control point; all invocations run in parallel, and can communicate by reading and writing output variables. One of the main responsibilities of the TCS is to write the special gl_TessLevelOuter[] and gl_TessLevelInner[] output variables which control how much new geometry the hardware tessellation engine will produce. Otherwise, it simply writes outputs that are passed along to the TES. We run in SIMD4x2 mode, handling two logical invocations per EU thread. The hardware doesn't properly manage the dispatch mask for us; it always initializes it to 0xFF. We wrap the whole program in an IF..ENDIF block to handle an odd number of invocations, essentially falling back to SIMD4x1 on the last thread. v2: Update comments (requested by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:07:39 -08:00
return true;
i965: Add src/dst interference for certain instructions with hazards. When working on tessellation shaders, I created some vec4 virtual opcodes for creating message headers through a sequence like: mov(8) g7<1>UD 0x00000000UD { align1 WE_all 1Q compacted }; mov(1) g7.5<1>UD 0x00000100UD { align1 WE_all }; mov(1) g7<1>UD g0<0,1,0>UD { align1 WE_all compacted }; mov(1) g7.3<1>UD g8<0,1,0>UD { align1 WE_all }; This is done in the generator since the vec4 backend can't handle align1 regioning. From the visitor's point of view, this is a single opcode: hs_set_output_urb_offsets vgrf7.0:UD, 1U, vgrf8.xxxx:UD Normally, there's no hazard between sources and destinations - an instruction (naturally) reads its sources, then writes the result to the destination. However, when the virtual instruction generates multiple hardware instructions, we can get into trouble. In the above example, if the register allocator assigned vgrf7 and vgrf8 to the same hardware register, then we'd clobber the source with 0 in the first instruction, and read back the wrong value in the last one. It occured to me that this is exactly the same problem we have with SIMD16 instructions that use W/UW or B/UB types with 0 stride. The hardware implicitly decodes them as two SIMD8 instructions, and with the overlapping regions, the first would clobber the second. Previously, we handled that by incrementing the live range end IP by 1, which works, but is excessive: the next instruction doesn't actually care about that. It might also be the end of control flow. This might keep values alive too long. What we really want is to say "my source and destinations interfere". This patch creates new infrastructure for doing just that, and teaches the register allocator to add interference when there's a hazard. For my vec4 case, we can determine this by switching on opcodes. For the SIMD16 case, we just move the existing code there. I audited our existing virtual opcodes that generate multiple instructions; I believe FS_OPCODE_PACK_HALF_2x16_SPLIT needs this treatment as well, but no others. v2: Rebased by mattst88. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-19 16:00:18 -08:00
default:
i965/vec4: prevent src/dst hazards during 64-bit register allocation 8-wide compressed DF operations are executed as two separate 4-wide DF operations. In that scenario, we have to be careful when we allocate register space for their operands to prevent the case where the first half of the instruction overwrites the source of the second half. To do this we mark compressed instructions as having hazards to make sure that ther register allocators assigns a register regions for the destination that does not overlap with the region assigned for any of its source operands. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-09-01 14:23:26 +02:00
/* 8-wide compressed DF operations are executed as two 4-wide operations,
* so we have a src/dst hazard if the first half of the instruction
* overwrites the source of the second half. Prevent this by marking
* compressed instructions as having src/dst hazards, so the register
* allocator assigns safe register regions for dst and srcs.
*/
return size_written > REG_SIZE;
i965: Add src/dst interference for certain instructions with hazards. When working on tessellation shaders, I created some vec4 virtual opcodes for creating message headers through a sequence like: mov(8) g7<1>UD 0x00000000UD { align1 WE_all 1Q compacted }; mov(1) g7.5<1>UD 0x00000100UD { align1 WE_all }; mov(1) g7<1>UD g0<0,1,0>UD { align1 WE_all compacted }; mov(1) g7.3<1>UD g8<0,1,0>UD { align1 WE_all }; This is done in the generator since the vec4 backend can't handle align1 regioning. From the visitor's point of view, this is a single opcode: hs_set_output_urb_offsets vgrf7.0:UD, 1U, vgrf8.xxxx:UD Normally, there's no hazard between sources and destinations - an instruction (naturally) reads its sources, then writes the result to the destination. However, when the virtual instruction generates multiple hardware instructions, we can get into trouble. In the above example, if the register allocator assigned vgrf7 and vgrf8 to the same hardware register, then we'd clobber the source with 0 in the first instruction, and read back the wrong value in the last one. It occured to me that this is exactly the same problem we have with SIMD16 instructions that use W/UW or B/UB types with 0 stride. The hardware implicitly decodes them as two SIMD8 instructions, and with the overlapping regions, the first would clobber the second. Previously, we handled that by incrementing the live range end IP by 1, which works, but is excessive: the next instruction doesn't actually care about that. It might also be the end of control flow. This might keep values alive too long. What we really want is to say "my source and destinations interfere". This patch creates new infrastructure for doing just that, and teaches the register allocator to add interference when there's a hazard. For my vec4 case, we can determine this by switching on opcodes. For the SIMD16 case, we just move the existing code there. I audited our existing virtual opcodes that generate multiple instructions; I believe FS_OPCODE_PACK_HALF_2x16_SPLIT needs this treatment as well, but no others. v2: Rebased by mattst88. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-19 16:00:18 -08:00
}
}
i965/vec4: Fix the scheduler to take into account reads and writes of multiple registers. v2: Avoid nested ternary operators in vec4_instruction::regs_read(). (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:39:33 +02:00
unsigned
i965/vec4: Replace vec4_instruction::regs_read with ::size_read using byte units. The previous regs_read value can be recovered by rewriting each reference of regs_read() like 'x = i.regs_read(j)' to 'x = DIV_ROUND_UP(i.size_read(j), reg_unit)'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 17:00:30 -07:00
vec4_instruction::size_read(unsigned arg) const
i965/vec4: Fix the scheduler to take into account reads and writes of multiple registers. v2: Avoid nested ternary operators in vec4_instruction::regs_read(). (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:39:33 +02:00
{
switch (opcode) {
case SHADER_OPCODE_SHADER_TIME_ADD:
i965/vec4: Add support for untyped surface message sends from GRF. This doesn't actually enable untyped surface message sends from GRF yet, the upcoming atomic counter and image intrinsic lowering code will. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 17:42:47 +02:00
case SHADER_OPCODE_UNTYPED_ATOMIC:
case SHADER_OPCODE_UNTYPED_SURFACE_READ:
i965: Add untyped surface write opcode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:24:14 +03:00
case SHADER_OPCODE_UNTYPED_SURFACE_WRITE:
i965: Add typed surface access opcodes. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:28:25 +03:00
case SHADER_OPCODE_TYPED_ATOMIC:
case SHADER_OPCODE_TYPED_SURFACE_READ:
case SHADER_OPCODE_TYPED_SURFACE_WRITE:
i965: Add tessellation control shaders. The TCS is the first tessellation shader stage, and the most complicated. It has access to each of the control points in the input patch, and computes a new output patch. There is one logical invocation per output control point; all invocations run in parallel, and can communicate by reading and writing output variables. One of the main responsibilities of the TCS is to write the special gl_TessLevelOuter[] and gl_TessLevelInner[] output variables which control how much new geometry the hardware tessellation engine will produce. Otherwise, it simply writes outputs that are passed along to the TES. We run in SIMD4x2 mode, handling two logical invocations per EU thread. The hardware doesn't properly manage the dispatch mask for us; it always initializes it to 0xFF. We wrap the whole program in an IF..ENDIF block to handle an odd number of invocations, essentially falling back to SIMD4x1 on the last thread. v2: Update comments (requested by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:07:39 -08:00
case TCS_OPCODE_URB_WRITE:
i965/vec4: Replace vec4_instruction::regs_read with ::size_read using byte units. The previous regs_read value can be recovered by rewriting each reference of regs_read() like 'x = i.regs_read(j)' to 'x = DIV_ROUND_UP(i.size_read(j), reg_unit)'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 17:00:30 -07:00
if (arg == 0)
return mlen * REG_SIZE;
break;
i965/vec4: Fix the scheduler to take into account reads and writes of multiple registers. v2: Avoid nested ternary operators in vec4_instruction::regs_read(). (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:39:33 +02:00
case VS_OPCODE_PULL_CONSTANT_LOAD_GEN7:
i965/vec4: Replace vec4_instruction::regs_read with ::size_read using byte units. The previous regs_read value can be recovered by rewriting each reference of regs_read() like 'x = i.regs_read(j)' to 'x = DIV_ROUND_UP(i.size_read(j), reg_unit)'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 17:00:30 -07:00
if (arg == 1)
return mlen * REG_SIZE;
break;
default:
break;
}
i965/vec4: Fix the scheduler to take into account reads and writes of multiple registers. v2: Avoid nested ternary operators in vec4_instruction::regs_read(). (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:39:33 +02:00
i965/vec4: Replace vec4_instruction::regs_read with ::size_read using byte units. The previous regs_read value can be recovered by rewriting each reference of regs_read() like 'x = i.regs_read(j)' to 'x = DIV_ROUND_UP(i.size_read(j), reg_unit)'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 17:00:30 -07:00
switch (src[arg].file) {
case BAD_FILE:
return 0;
case IMM:
case UNIFORM:
return 4 * type_sz(src[arg].type);
i965/vec4: Fix the scheduler to take into account reads and writes of multiple registers. v2: Avoid nested ternary operators in vec4_instruction::regs_read(). (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:39:33 +02:00
default:
i965/vec4: fix regs_read() for doubles Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-30 13:20:31 +02:00
/* XXX - Represent actual vertical stride. */
return exec_size * type_sz(src[arg].type);
i965/vec4: Fix the scheduler to take into account reads and writes of multiple registers. v2: Avoid nested ternary operators in vec4_instruction::regs_read(). (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:39:33 +02:00
}
}
i965/vs: Teach copy propagation about sends from GRFs. This incidentally also teaches it a bit about gen6 math -- we now allow unswizzled, unmodified GRF temps as the sources for math. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:03:02 -08:00
bool
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
vec4_instruction::can_do_source_mods(const struct gen_device_info *devinfo)
i965/vs: Teach copy propagation about sends from GRFs. This incidentally also teaches it a bit about gen6 math -- we now allow unswizzled, unmodified GRF temps as the sources for math. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:03:02 -08:00
{
i965: Add a devinfo field to backend_visitor and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 18:00:05 -07:00
if (devinfo->gen == 6 && is_math())
i965/vs: Teach copy propagation about sends from GRFs. This incidentally also teaches it a bit about gen6 math -- we now allow unswizzled, unmodified GRF temps as the sources for math. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:03:02 -08:00
return false;
i965: Make can_do_source_mods() a member of the instruction classes. Pretty nonsensical to have it as a method of the visitor just for access to brw. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-23 21:57:31 -07:00
if (is_send_from_grf())
i965/vs: Teach copy propagation about sends from GRFs. This incidentally also teaches it a bit about gen6 math -- we now allow unswizzled, unmodified GRF temps as the sources for math. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:03:02 -08:00
return false;
i965: Make can_do_source_mods() a member of the instruction classes. Pretty nonsensical to have it as a method of the visitor just for access to brw. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-23 21:57:31 -07:00
if (!backend_instruction::can_do_source_mods())
i965: Don't copy prop source mods into instructions that can't take them.
2013-09-19 19:48:22 -07:00
return false;
i965/vs: Teach copy propagation about sends from GRFs. This incidentally also teaches it a bit about gen6 math -- we now allow unswizzled, unmodified GRF temps as the sources for math. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:03:02 -08:00
return true;
}
i965/vec4: Move can_do_writemask to vec4_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:06:10 -07:00
bool
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
vec4_instruction::can_do_writemask(const struct gen_device_info *devinfo)
i965/vec4: Move can_do_writemask to vec4_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:06:10 -07:00
{
switch (opcode) {
case SHADER_OPCODE_GEN4_SCRATCH_READ:
i965/vec4: split VEC4_OPCODE_FROM_DOUBLE into one opcode per destination's type This way we can set the destination type as double to all these new opcodes, avoiding any optimizer's confusion that was happening before. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> [ Francisco Jerez: Drop no_spill workaround originally needed due to the bogus destination type of VEC4_OPCODE_FROM_DOUBLE. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-03-24 08:46:13 +01:00
case VEC4_OPCODE_DOUBLE_TO_F32:
case VEC4_OPCODE_DOUBLE_TO_D32:
case VEC4_OPCODE_DOUBLE_TO_U32:
i965/vec4: Rename DF to/from F generator opcodes The opcodes are not specific for conversions to/from float since we need the same for conversions to/from other 32-bit types. Rename the opcodes accordingly and change the asserts to check the size of the types involved instead. v2: - Rename to VEC4_OPCODE_TO_DOUBLE and VEC4_OPCODE_FROM_DOUBLE (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-02-17 10:32:05 +01:00
case VEC4_OPCODE_TO_DOUBLE:
i965/vec4: add VEC4_OPCODE_PICK_{LOW,HIGH}_32BIT opcodes These opcodes will pick the low/high 32-bit in each 64-bit data element using Align1 mode. We will use this, for example, to do things like unpackDouble2x32. We use Align1 mode because in order to implement this in Align16 mode we would need to use 32-bit logical swizzles (XZ for low, YW for high), but the IR works in terms of 64-bit logical swizzles for DF operands all the way up to codegen. v2: - use suboffset() instead of get_element_ud() - no need to set the width on the dst Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-31 10:17:37 +02:00
case VEC4_OPCODE_PICK_LOW_32BIT:
case VEC4_OPCODE_PICK_HIGH_32BIT:
i965/vec4: add VEC4_OPCODE_SET_{LOW,HIGH}_32BIT opcodes These opcodes will set the low/high 32-bit in each 64-bit data element using Align1 mode. We will use this to implement packDouble2x32. We use Align1 mode because in order to implement this in Align16 mode we would need to use 32-bit logical swizzles (XZ for low, YW for high), but the IR works in terms of 64-bit logical swizzles for DF operands all the way up to codegen. v2: - use suboffset() instead of get_element_ud() - no need to set the width on the dst Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-17 12:19:35 +02:00
case VEC4_OPCODE_SET_LOW_32BIT:
case VEC4_OPCODE_SET_HIGH_32BIT:
i965/vec4: Move can_do_writemask to vec4_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:06:10 -07:00
case VS_OPCODE_PULL_CONSTANT_LOAD:
case VS_OPCODE_PULL_CONSTANT_LOAD_GEN7:
case VS_OPCODE_SET_SIMD4X2_HEADER_GEN9:
case TCS_OPCODE_SET_INPUT_URB_OFFSETS:
case TCS_OPCODE_SET_OUTPUT_URB_OFFSETS:
case TES_OPCODE_CREATE_INPUT_READ_HEADER:
case TES_OPCODE_ADD_INDIRECT_URB_OFFSET:
case VEC4_OPCODE_URB_READ:
i965/vec4: Add support for SHADER_OPCODE_MOV_INDIRECT Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-23 21:39:15 -08:00
case SHADER_OPCODE_MOV_INDIRECT:
i965/vec4: Move can_do_writemask to vec4_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:06:10 -07:00
return false;
default:
/* The MATH instruction on Gen6 only executes in align1 mode, which does
* not support writemasking.
*/
if (devinfo->gen == 6 && is_math())
return false;
if (is_tex())
return false;
return true;
}
}
i965: Extract can_change_source_types() functions. Make them members of fs_inst/vec4_instruction for use elsewhere. Also fix the fs version to check that dst.type == src[1].type and for !saturate. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-10-14 02:12:09 -07:00
bool
vec4_instruction::can_change_types() const
{
return dst.type == src[0].type &&
!src[0].abs && !src[0].negate && !saturate &&
(opcode == BRW_OPCODE_MOV ||
(opcode == BRW_OPCODE_SEL &&
dst.type == src[1].type &&
predicate != BRW_PREDICATE_NONE &&
!src[1].abs && !src[1].negate));
}
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
/**
* Returns how many MRFs an opcode will write over.
*
* Note that this is not the 0 or 1 implied writes in an actual gen
* instruction -- the generate_* functions generate additional MOVs
* for setup.
*/
int
vec4_visitor::implied_mrf_writes(vec4_instruction *inst)
{
i965/vec4: Make vec4_visitor::implied_mrf_writes() return zero for sends from GRF. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-05 22:58:03 +02:00
if (inst->mlen == 0 || inst->is_send_from_grf())
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
return 0;
switch (inst->opcode) {
case SHADER_OPCODE_RCP:
case SHADER_OPCODE_RSQ:
case SHADER_OPCODE_SQRT:
case SHADER_OPCODE_EXP2:
case SHADER_OPCODE_LOG2:
case SHADER_OPCODE_SIN:
case SHADER_OPCODE_COS:
return 1;
i965/vs: Fix implied_mrf_writes() for integer division pre-gen6. Previously it would assertion fail in debug builds (though the correct value was returned in a non-debug build). Marking it as a candidate for stable even though it has no current consumers in the stable branches, in case one shows up in a later backport. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=64727 NOTE: This is a candidate for stable branches. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-05-28 14:35:19 -07:00
case SHADER_OPCODE_INT_QUOTIENT:
case SHADER_OPCODE_INT_REMAINDER:
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
case SHADER_OPCODE_POW:
i965: Explicitly write the "TR DS Cache Disable" bit at TCS EOT. Bit 0 of the Patch Header is "TR DS Cache Disable". Setting that bit disables the DS Cache for tessellator-output topologies resulting in stitch-transition regions (but leaves it enabled for other cases). We probably shouldn't leave this to chance - the URB could contain garbage - which could result in the cache randomly being turned on or off. This patch makes the final EOT write 0 to the first DWord (which only contains this one bit). This ensures the cache is always on. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-05 01:53:57 -08:00
case TCS_OPCODE_THREAD_END:
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
return 2;
case VS_OPCODE_URB_WRITE:
return 1;
case VS_OPCODE_PULL_CONSTANT_LOAD:
return 2;
i965: Merge together opcodes for SHADER_OPCODE_GEN4_SCRATCH_READ/WRITE I'm going to be introducing gen7 variants, and the previous naming was going to get confusing. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:45:06 -07:00
case SHADER_OPCODE_GEN4_SCRATCH_READ:
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
return 2;
i965: Merge together opcodes for SHADER_OPCODE_GEN4_SCRATCH_READ/WRITE I'm going to be introducing gen7 variants, and the previous naming was going to get confusing. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:45:06 -07:00
case SHADER_OPCODE_GEN4_SCRATCH_WRITE:
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
return 3;
i965/gs: Add GS_OPCODE_URB_WRITE. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-03-21 09:11:12 -07:00
case GS_OPCODE_URB_WRITE:
i965/gen6/gs: Implement GS_OPCODE_URB_WRITE_ALLOCATE. Gen6 geometry shaders need to allocate URB handles for each new vertex they emit after the first (the URB handle for the first vertex is obtained via the FF_SYNC message). This opcode adds the URB allocation mechanism to regular URB writes. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-07-09 16:28:30 +02:00
case GS_OPCODE_URB_WRITE_ALLOCATE:
i965/gs: Add GS_OPCODE_THREAD_END. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-03-23 07:42:32 -07:00
case GS_OPCODE_THREAD_END:
i965/gs: Add GS_OPCODE_URB_WRITE. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-03-21 09:11:12 -07:00
return 0;
i965/gen6/gs: Implement GS_OPCODE_FF_SYNC. This implements the FF_SYNC message required in gen6 geometry shaders to get the initial URB handle. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-07-09 08:46:17 +02:00
case GS_OPCODE_FF_SYNC:
return 1;
i965: Add tessellation control shaders. The TCS is the first tessellation shader stage, and the most complicated. It has access to each of the control points in the input patch, and computes a new output patch. There is one logical invocation per output control point; all invocations run in parallel, and can communicate by reading and writing output variables. One of the main responsibilities of the TCS is to write the special gl_TessLevelOuter[] and gl_TessLevelInner[] output variables which control how much new geometry the hardware tessellation engine will produce. Otherwise, it simply writes outputs that are passed along to the TES. We run in SIMD4x2 mode, handling two logical invocations per EU thread. The hardware doesn't properly manage the dispatch mask for us; it always initializes it to 0xFF. We wrap the whole program in an IF..ENDIF block to handle an odd number of invocations, essentially falling back to SIMD4x1 on the last thread. v2: Update comments (requested by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-11-17 01:07:39 -08:00
case TCS_OPCODE_URB_WRITE:
return 0;
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
case SHADER_OPCODE_SHADER_TIME_ADD:
return 0;
i965/vs: Add instruction scheduling. While this is ignorant of dependency control, it's still good for a 0.39% +/- 0.08% performance improvement on GLBenchmark 2.7 (n=548) v2: Rewrite as a subclass of the base class for the FS instruction scheduler, inheriting the same latency information. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2012-11-30 16:13:34 -08:00
case SHADER_OPCODE_TEX:
case SHADER_OPCODE_TXL:
case SHADER_OPCODE_TXD:
case SHADER_OPCODE_TXF:
i965: rename tex_ms to tex_cms Prepares for the introduction of non-compressed multi-sampled lookup used in the blorp programs. v2: now also taking into account gen8 Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-10 16:36:31 +02:00
case SHADER_OPCODE_TXF_CMS:
i965/vec4/skl+: Use ld2dms_w instead of ld2dms In order to support 16x MSAA, skl+ has a wider version of ld2dms that takes two parameters for the MCS data. The MCS data in the response still fits in a single register so we just need to ensure we copy both values rather than just the lower one. Acked-by: Ben Widawsky <ben@bwidawsk.net>
2015-09-09 15:59:36 +01:00
case SHADER_OPCODE_TXF_CMS_W:
i965: Add shader opcode for sampling MCS surface Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Paul Berry <stereotype441@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-30 10:32:16 +13:00
case SHADER_OPCODE_TXF_MCS:
i965/vs: Add instruction scheduling. While this is ignorant of dependency control, it's still good for a 0.39% +/- 0.08% performance improvement on GLBenchmark 2.7 (n=548) v2: Rewrite as a subclass of the base class for the FS instruction scheduler, inheriting the same latency information. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2012-11-30 16:13:34 -08:00
case SHADER_OPCODE_TXS:
i965: add SHADER_OPCODE_TG4 Adds the Gen7 message IDs, a new SHADER_OPCODE_TG4 pseudo-op, and low-level support for emitting it via generate_tex(). V3: Updated for changes in master. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-31 21:31:12 +13:00
case SHADER_OPCODE_TG4:
i965: Add SHADER_OPCODE_TG4_OFFSET for gather with nonconstant offsets. The generator code ends up clearer this way than if we had to sniff via the message length. Implemented via the gather4_po message in hardware, which is present in Gen7 and later. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2013-10-08 21:42:10 +13:00
case SHADER_OPCODE_TG4_OFFSET:
i965: add support for textureSamples function Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu> [v2: kayden-supplied code in fs_nir replacing need for logical opcode] Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-08-11 20:37:32 -04:00
case SHADER_OPCODE_SAMPLEINFO:
i965/vec4: Implement VS_OPCODE_GET_BUFFER_SIZE Notice that Skylake needs to include a header in the sampler message so it will need some tweaks to work there. Signed-off-by: Samuel Iglesias Gonsalvez <siglesias@igalia.com> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-08-28 09:39:49 +02:00
case VS_OPCODE_GET_BUFFER_SIZE:
i965: Change header_present to header_size in backend_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-24 10:17:32 -07:00
return inst->header_size;
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
default:
i965: Use unreachable() instead of unconditional assert(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-29 14:54:01 -07:00
unreachable("not reached");
i965/vs: Add a function for how many MRFs get written as part of a SEND. This will be used for compute-to-mrf, which needs to know when MRFs get overwritten.
2011-09-06 12:26:42 -07:00
}
}
i965/vs: Add support for copy propagation of the UNIFORM and ATTR files. Removes 2.0% of the instructions from 35.7% of vertex shaders in shader-db.
2011-09-01 16:21:14 -07:00
i965/vs: Add constant propagation to a few opcodes. This differs from the FS in that we track constants in each destination channel, and we we have to look at all the swizzled source channels. Also, the instruction stream walk is done in an O(n) manner instead of O(n^2). Across shader-db, this reduces 8.0% of the instructions from 60.0% of the vertex shaders, leaving us now behind the old backend by 11.1% overall.
2011-08-30 13:25:15 -07:00
bool
i965: Make src_reg::equals() take a constant reference, not a pointer. This is more typical C++ style. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2013-03-06 08:51:44 -08:00
src_reg::equals(const src_reg &r) const
i965/vs: Add constant propagation to a few opcodes. This differs from the FS in that we track constants in each destination channel, and we we have to look at all the swizzled source channels. Also, the instruction stream walk is done in an O(n) manner instead of O(n^2). Across shader-db, this reduces 8.0% of the instructions from 60.0% of the vertex shaders, leaving us now behind the old backend by 11.1% overall.
2011-08-30 13:25:15 -07:00
{
i965: Add and use backend_reg::equals(). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-22 13:25:05 -08:00
return (this->backend_reg::equals(r) &&
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
!reladdr && !r.reladdr);
i965/vs: Add constant propagation to a few opcodes. This differs from the FS in that we track constants in each destination channel, and we we have to look at all the swizzled source channels. Also, the instruction stream walk is done in an O(n) manner instead of O(n^2). Across shader-db, this reduces 8.0% of the instructions from 60.0% of the vertex shaders, leaving us now behind the old backend by 11.1% overall.
2011-08-30 13:25:15 -07:00
}
i965/vec4: Add pass to gather constants into a vector-float MOV. Currently only handles consecutive instructions with the same destination that collectively write all channels. total instructions in shared programs: 5879798 -> 5869011 (-0.18%) instructions in affected programs: 465236 -> 454449 (-2.32%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-12-20 11:50:31 -08:00
bool
vec4_visitor::opt_vector_float()
{
bool progress = false;
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
foreach_block(block, cfg) {
i965/vec4: Change opt_vector_float to keep track of the last offset seen in bytes. This simplifies things slightly and makes the pass more correct in presence of sub-GRF offsets. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 21:51:07 -07:00
int last_reg = -1, last_offset = -1;
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
enum brw_reg_file last_reg_file = BAD_FILE;
uint8_t imm[4] = { 0 };
int inst_count = 0;
vec4_instruction *imm_inst[4];
unsigned writemask = 0;
enum brw_reg_type dest_type = BRW_REGISTER_TYPE_F;
foreach_inst_in_block_safe(vec4_instruction, inst, block) {
int vf = -1;
enum brw_reg_type need_type;
/* Look for unconditional MOVs from an immediate with a partial
* writemask. Skip type-conversion MOVs other than integer 0,
* where the type doesn't matter. See if the immediate can be
* represented as a VF.
*/
if (inst->opcode == BRW_OPCODE_MOV &&
inst->src[0].file == IMM &&
inst->predicate == BRW_PREDICATE_NONE &&
inst->dst.writemask != WRITEMASK_XYZW &&
i965/vec4: make opt_vector_float ignore doubles The pass does not support doubles in its current form. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-02-12 13:10:06 +01:00
type_sz(inst->src[0].type) < 8 &&
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
(inst->src[0].type == inst->dst.type || inst->src[0].d == 0)) {
vf = brw_float_to_vf(inst->src[0].d);
need_type = BRW_REGISTER_TYPE_D;
if (vf == -1) {
vf = brw_float_to_vf(inst->src[0].f);
need_type = BRW_REGISTER_TYPE_F;
}
} else {
last_reg = -1;
i965: Properly handle integer types in opt_vector_float(). Previously, opt_vector_float() always interpreted MOV sources as floating point, and always created a MOV with a F-type destination. This meant that we could mess up sequences of integer loads, such as: mov vgrf6.0.x:D, 0D mov vgrf6.0.y:D, 1D mov vgrf6.0.z:D, 2D mov vgrf6.0.w:D, 3D Here, integer 0/1/2/3 become approximately 0.0f, so we generated: mov vgrf6.0:F, [0F, 0F, 0F, 0F] which is clearly wrong. We can properly handle this by converting integer values to float (rather than bitcasting), and emitting a type converting MOV: mov vgrf6.0:D, [0F, 1F, 2F, 3F] To do this, see first see if the integer values (converted to float) are representable. If so, we use a D-type MOV. If not, we then try the floating point values and an F-type MOV. We make zero not impose type restrictions. This is important because 0D would imply a D-type MOV, but is often used in sequences such as MOV 0D, MOV 0x3f800000D, where we want to use an F-type MOV. Fixes about 54 dEQP-GLES2 failures with the vec4 VS backend. This recently became visible due to changes in opt_vector_float() which made it optimize more cases, but it was a pre-existing bug. Apparently it also manages to turn more integer loads into VFs, producing the following shader-db statistics on Haswell: total instructions in shared programs: 7084195 -> 7082191 (-0.03%) instructions in affected programs: 246027 -> 244023 (-0.81%) helped: 1937 total cycles in shared programs: 65669642 -> 65651968 (-0.03%) cycles in affected programs: 531064 -> 513390 (-3.33%) helped: 1177 v2: Handle the type of zero better. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-13 16:58:10 -07:00
}
i965: Rework opt_vector_float() control flow. This reworks opt_vector_float() so that there's only one place that flushes out any accumulated state and emits a VF. v2: Don't break the sequence for non-representable numbers - just skip recording their values. Only break it for non-MOVs or register changes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-13 15:56:07 -07:00
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
/* If this wasn't a MOV, or the destination register doesn't match,
* or we have to switch destination types, then this breaks our
* sequence. Combine anything we've accumulated so far.
*/
if (last_reg != inst->dst.nr ||
i965/vec4: Change opt_vector_float to keep track of the last offset seen in bytes. This simplifies things slightly and makes the pass more correct in presence of sub-GRF offsets. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 21:51:07 -07:00
last_offset != inst->dst.offset ||
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
last_reg_file != inst->dst.file ||
(vf > 0 && dest_type != need_type)) {
if (inst_count > 1) {
unsigned vf;
memcpy(&vf, imm, sizeof(vf));
vec4_instruction *mov = MOV(imm_inst[0]->dst, brw_imm_vf(vf));
mov->dst.type = dest_type;
mov->dst.writemask = writemask;
inst->insert_before(block, mov);
for (int i = 0; i < inst_count; i++) {
imm_inst[i]->remove(block);
}
progress = true;
i965: Fold vectorize_mov() back into the one caller. After the previous patch, this helper is only called in one place. So, just fold it back in - there are a lot of parameters here and not much code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-13 16:04:04 -07:00
}
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
inst_count = 0;
last_reg = -1;
writemask = 0;
dest_type = BRW_REGISTER_TYPE_F;
i965: Fold vectorize_mov() back into the one caller. After the previous patch, this helper is only called in one place. So, just fold it back in - there are a lot of parameters here and not much code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-13 16:04:04 -07:00
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
for (int i = 0; i < 4; i++) {
imm[i] = 0;
}
i965/vec4: add opportunistic behaviour to opt_vector_float() opt_vector_float() transforms several scalar MOV operations to a single vectorial MOV. This is done when those MOV covers all the components of the destination register. So something like: mov vgrf3.0.xy:D, 0D mov vgrf3.0.w:D, 1065353216D mov vgrf3.0.z:D, 0D is transformed in: mov vgrf3.0:F, [0F, 0F, 0F, 1F] But there are cases where not all the components are written. For example, in: mov vgrf2.0.x:D, 1073741824D mov vgrf3.0.xy:D, 0D mov vgrf3.0.w:D, 1065353216D mov vgrf4.0.xy:D, 1065353216D mov vgrf4.0.w:D, 0D mov vgrf6.0:UD, u4.xyzw:UD Nor vgrf3 nor vgrf4 .z components are written, so the optimization is not applied. But it could be applied anyway with the components covered, using a writemask to select the ones written. So we could transform it in: mov vgrf2.0.x:D, 1073741824D mov vgrf3.0.xyw:F, [0F, 0F, 0F, 1F] mov vgrf4.0.xyw:F, [1F, 1F, 0F, 0F] mov vgrf6.0:UD, u4.xyzw:UD This commit does precisely that: opportunistically apply opt_vector_float() when possible. total instructions in shared programs: 7124660 -> 7114784 (-0.14%) instructions in affected programs: 443078 -> 433202 (-2.23%) helped: 4998 HURT: 0 total cycles in shared programs: 64757760 -> 64728016 (-0.05%) cycles in affected programs: 1401686 -> 1371942 (-2.12%) helped: 3243 HURT: 38 v2: change vectorize_mov() signature (Matt). v3: take in account predicates (Juan). v4 [mattst88]: Update shader-db numbers. Fix some whitespace issues. Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
2016-03-02 13:21:02 +01:00
}
i965/vec4: Add pass to gather constants into a vector-float MOV. Currently only handles consecutive instructions with the same destination that collectively write all channels. total instructions in shared programs: 5879798 -> 5869011 (-0.18%) instructions in affected programs: 465236 -> 454449 (-2.32%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-12-20 11:50:31 -08:00
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
/* Record this instruction's value (if it was representable). */
if (vf != -1) {
if ((inst->dst.writemask & WRITEMASK_X) != 0)
imm[0] = vf;
if ((inst->dst.writemask & WRITEMASK_Y) != 0)
imm[1] = vf;
if ((inst->dst.writemask & WRITEMASK_Z) != 0)
imm[2] = vf;
if ((inst->dst.writemask & WRITEMASK_W) != 0)
imm[3] = vf;
writemask |= inst->dst.writemask;
imm_inst[inst_count++] = inst;
last_reg = inst->dst.nr;
i965/vec4: Change opt_vector_float to keep track of the last offset seen in bytes. This simplifies things slightly and makes the pass more correct in presence of sub-GRF offsets. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 21:51:07 -07:00
last_offset = inst->dst.offset;
i965/vec4: Make opt_vector_float reset at the top of each block The pass isn't really control-flow aware and you can get into case where it tries to combine instructions from different blocks. This can actually lead to an assertion failure when removing unneeded instructions if part of the vector is set in one block and part in another. This prevents regressions in the next commit. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: "12.0" <mesa-stable@lists.freedesktop.org>
2016-07-20 12:21:41 -07:00
last_reg_file = inst->dst.file;
if (vf > 0)
dest_type = need_type;
}
i965: Rework opt_vector_float() control flow. This reworks opt_vector_float() so that there's only one place that flushes out any accumulated state and emits a VF. v2: Don't break the sequence for non-representable numbers - just skip recording their values. Only break it for non-MOVs or register changes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-13 15:56:07 -07:00
}
i965/vec4: Add pass to gather constants into a vector-float MOV. Currently only handles consecutive instructions with the same destination that collectively write all channels. total instructions in shared programs: 5879798 -> 5869011 (-0.18%) instructions in affected programs: 465236 -> 454449 (-2.32%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-12-20 11:50:31 -08:00
}
if (progress)
invalidate_live_intervals();
return progress;
}
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
/* Replaces unused channels of a swizzle with channels that are used.
*
* For instance, this pass transforms
*
* mov vgrf4.yz, vgrf5.wxzy
*
* into
*
* mov vgrf4.yz, vgrf5.xxzx
*
* This eliminates false uses of some channels, letting dead code elimination
* remove the instructions that wrote them.
*/
bool
vec4_visitor::opt_reduce_swizzle()
{
bool progress = false;
i965/vec4: Don't use instruction list after calculating the cfg. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 13:42:51 -07:00
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
if (inst->dst.file == BAD_FILE ||
inst->dst.file == ARF ||
inst->dst.file == FIXED_GRF ||
i965/vec4: Don't attempt to reduce swizzles of send from GRF instructions. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-03 20:34:17 +02:00
inst->is_send_from_grf())
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
continue;
i965/vec4: Simplify opt_reduce_swizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:08:52 +02:00
unsigned swizzle;
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
/* Determine which channels of the sources are read. */
switch (inst->opcode) {
i965/vec4: Add VEC4_OPCODE_PACK_4_BYTES. Will be used by emit_pack_{s,u}norm_4x8().
2014-03-10 13:26:30 -07:00
case VEC4_OPCODE_PACK_BYTES:
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
case BRW_OPCODE_DP4:
case BRW_OPCODE_DPH: /* FINISHME: DPH reads only three channels of src0,
* but all four of src1.
*/
i965/vec4: Simplify opt_reduce_swizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:08:52 +02:00
swizzle = brw_swizzle_for_size(4);
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
break;
case BRW_OPCODE_DP3:
i965/vec4: Simplify opt_reduce_swizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:08:52 +02:00
swizzle = brw_swizzle_for_size(3);
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
break;
case BRW_OPCODE_DP2:
i965/vec4: Simplify opt_reduce_swizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:08:52 +02:00
swizzle = brw_swizzle_for_size(2);
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
break;
i965/vec4: add double/float conversion pseudo-opcodes These need to be emitted as align1 MOV's, since they need to have a stride of 2 on the float register (whether src or dest) so that data from another thread doesn't cross the middle of a SIMD8 register. v2 (Iago): - The float-to-double needs to align 32-bit data to 64-bit before doing the conversion. This was doable in align16 when we tried to use an execsize of 4, but with an execsize of 8 we would need another align1 opcode to do that (since we need data to cross the middle of a SIMD register). Just making the opcode handle this internally seems more practical that adding another opcode just for this purpose and having the caller know about this before converting. - The double-to-float conversion produces 32-bit elements aligned to 64-bit so we make the opcode re-pack the result to 32-bit and fit in one register, as expected by SIMD4x2 operation. This still requires that callers reserve two registers for the float data destination because we need to produce 64-bit aligned data first, and repack it later on the same destination register, but it saves the need for a re-pack opcode only to achieve this making the operation complete in a single opcode. Hopefully that is worth the weirdness of the double register allocation... Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Signed-off-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-08-13 15:36:05 -07:00
i965/vec4: Rename DF to/from F generator opcodes The opcodes are not specific for conversions to/from float since we need the same for conversions to/from other 32-bit types. Rename the opcodes accordingly and change the asserts to check the size of the types involved instead. v2: - Rename to VEC4_OPCODE_TO_DOUBLE and VEC4_OPCODE_FROM_DOUBLE (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-02-17 10:32:05 +01:00
case VEC4_OPCODE_TO_DOUBLE:
i965/vec4: split VEC4_OPCODE_FROM_DOUBLE into one opcode per destination's type This way we can set the destination type as double to all these new opcodes, avoiding any optimizer's confusion that was happening before. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> [ Francisco Jerez: Drop no_spill workaround originally needed due to the bogus destination type of VEC4_OPCODE_FROM_DOUBLE. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-03-24 08:46:13 +01:00
case VEC4_OPCODE_DOUBLE_TO_F32:
case VEC4_OPCODE_DOUBLE_TO_D32:
case VEC4_OPCODE_DOUBLE_TO_U32:
i965/vec4: add VEC4_OPCODE_PICK_{LOW,HIGH}_32BIT opcodes These opcodes will pick the low/high 32-bit in each 64-bit data element using Align1 mode. We will use this, for example, to do things like unpackDouble2x32. We use Align1 mode because in order to implement this in Align16 mode we would need to use 32-bit logical swizzles (XZ for low, YW for high), but the IR works in terms of 64-bit logical swizzles for DF operands all the way up to codegen. v2: - use suboffset() instead of get_element_ud() - no need to set the width on the dst Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-31 10:17:37 +02:00
case VEC4_OPCODE_PICK_LOW_32BIT:
case VEC4_OPCODE_PICK_HIGH_32BIT:
i965/vec4: add VEC4_OPCODE_SET_{LOW,HIGH}_32BIT opcodes These opcodes will set the low/high 32-bit in each 64-bit data element using Align1 mode. We will use this to implement packDouble2x32. We use Align1 mode because in order to implement this in Align16 mode we would need to use 32-bit logical swizzles (XZ for low, YW for high), but the IR works in terms of 64-bit logical swizzles for DF operands all the way up to codegen. v2: - use suboffset() instead of get_element_ud() - no need to set the width on the dst Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-17 12:19:35 +02:00
case VEC4_OPCODE_SET_LOW_32BIT:
case VEC4_OPCODE_SET_HIGH_32BIT:
i965/vec4: add double/float conversion pseudo-opcodes These need to be emitted as align1 MOV's, since they need to have a stride of 2 on the float register (whether src or dest) so that data from another thread doesn't cross the middle of a SIMD8 register. v2 (Iago): - The float-to-double needs to align 32-bit data to 64-bit before doing the conversion. This was doable in align16 when we tried to use an execsize of 4, but with an execsize of 8 we would need another align1 opcode to do that (since we need data to cross the middle of a SIMD register). Just making the opcode handle this internally seems more practical that adding another opcode just for this purpose and having the caller know about this before converting. - The double-to-float conversion produces 32-bit elements aligned to 64-bit so we make the opcode re-pack the result to 32-bit and fit in one register, as expected by SIMD4x2 operation. This still requires that callers reserve two registers for the float data destination because we need to produce 64-bit aligned data first, and repack it later on the same destination register, but it saves the need for a re-pack opcode only to achieve this making the operation complete in a single opcode. Hopefully that is worth the weirdness of the double register allocation... Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Signed-off-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-08-13 15:36:05 -07:00
swizzle = brw_swizzle_for_size(4);
break;
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
default:
i965/vec4: Simplify opt_reduce_swizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:08:52 +02:00
swizzle = brw_swizzle_for_mask(inst->dst.writemask);
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
break;
}
/* Update sources' swizzles. */
for (int i = 0; i < 3; i++) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
if (inst->src[i].file != VGRF &&
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
inst->src[i].file != ATTR &&
inst->src[i].file != UNIFORM)
continue;
i965/vec4: Simplify opt_reduce_swizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:08:52 +02:00
const unsigned new_swizzle =
brw_compose_swizzle(swizzle, inst->src[i].swizzle);
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
if (inst->src[i].swizzle != new_swizzle) {
inst->src[i].swizzle = new_swizzle;
progress = true;
}
}
}
if (progress)
i965: Remove cfg-invalidating parameter from invalidate_live_intervals. Everything has been converted to preserve the CFG. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 10:54:00 -07:00
invalidate_live_intervals();
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
return progress;
}
i965/vs: Track uniforms as separate vectors once we've done array access. This will make it easier to figure out which elements are totally unused and not upload them. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 10:22:50 -07:00
void
vec4_visitor::split_uniform_registers()
{
/* Prior to this, uniforms have been in an array sized according to
* the number of vector uniforms present, sparsely filled (so an
* aggregate results in reg indices being skipped over). Now we're
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
* going to cut those aggregates up so each .nr index is one
i965/vs: Track uniforms as separate vectors once we've done array access. This will make it easier to figure out which elements are totally unused and not upload them. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 10:22:50 -07:00
* vector. The goal is to make elimination of unused uniform
* components easier later.
*/
i965/vec4: Don't use instruction list after calculating the cfg. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 13:42:51 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
i965/vs: Track uniforms as separate vectors once we've done array access. This will make it easier to figure out which elements are totally unused and not upload them. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 10:22:50 -07:00
for (int i = 0 ; i < 3; i++) {
if (inst->src[i].file != UNIFORM)
continue;
assert(!inst->src[i].reladdr);
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
inst->src[i].nr += inst->src[i].offset / 16;
inst->src[i].offset %= 16;
i965/vs: Track uniforms as separate vectors once we've done array access. This will make it easier to figure out which elements are totally unused and not upload them. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 10:22:50 -07:00
}
}
}
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
/* This function returns the register number where we placed the uniform */
static int
set_push_constant_loc(const int nr_uniforms, int *new_uniform_count,
const int src, const int size, const int channel_size,
int *new_loc, int *new_chan,
int *new_chans_used)
{
int dst;
/* Find the lowest place we can slot this uniform in. */
for (dst = 0; dst < nr_uniforms; dst++) {
if (ALIGN(new_chans_used[dst], channel_size) + size <= 4)
break;
}
assert(dst < nr_uniforms);
new_loc[src] = dst;
new_chan[src] = ALIGN(new_chans_used[dst], channel_size);
new_chans_used[dst] = ALIGN(new_chans_used[dst], channel_size) + size;
*new_uniform_count = MAX2(*new_uniform_count, dst + 1);
return dst;
}
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
void
vec4_visitor::pack_uniform_registers()
{
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
uint8_t chans_used[this->uniforms];
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
int new_loc[this->uniforms];
int new_chan[this->uniforms];
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
bool is_aligned_to_dvec4[this->uniforms];
int new_chans_used[this->uniforms];
int channel_sizes[this->uniforms];
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
memset(chans_used, 0, sizeof(chans_used));
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
memset(new_loc, 0, sizeof(new_loc));
memset(new_chan, 0, sizeof(new_chan));
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
memset(new_chans_used, 0, sizeof(new_chans_used));
memset(is_aligned_to_dvec4, 0, sizeof(is_aligned_to_dvec4));
memset(channel_sizes, 0, sizeof(channel_sizes));
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
/* Find which uniform vectors are actually used by the program. We
* expect unused vector elements when we've moved array access out
* to pull constants, and from some GLSL code generators like wine.
*/
i965/vec4: Don't use instruction list after calculating the cfg. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 13:42:51 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
unsigned readmask;
switch (inst->opcode) {
case VEC4_OPCODE_PACK_BYTES:
case BRW_OPCODE_DP4:
case BRW_OPCODE_DPH:
readmask = 0xf;
break;
case BRW_OPCODE_DP3:
readmask = 0x7;
break;
case BRW_OPCODE_DP2:
readmask = 0x3;
break;
default:
readmask = inst->dst.writemask;
break;
}
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
for (int i = 0 ; i < 3; i++) {
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
if (inst->src[i].file != UNIFORM)
continue;
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: fix pack_uniform_registers for doubles We need to consider the fact that dvec3/4 require two vec4 slots. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-29 10:49:47 +02:00
assert(type_sz(inst->src[i].type) % 4 == 0);
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
int channel_size = type_sz(inst->src[i].type) / 4;
i965/vec4: fix pack_uniform_registers for doubles We need to consider the fact that dvec3/4 require two vec4 slots. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-29 10:49:47 +02:00
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
int reg = inst->src[i].nr;
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
for (int c = 0; c < 4; c++) {
if (!(readmask & (1 << c)))
continue;
i965/vec4: fix pack_uniform_registers for doubles We need to consider the fact that dvec3/4 require two vec4 slots. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-29 10:49:47 +02:00
unsigned channel = BRW_GET_SWZ(inst->src[i].swizzle, c) + 1;
unsigned used = MAX2(chans_used[reg], channel * channel_size);
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
if (used <= 4) {
i965/vec4: fix pack_uniform_registers for doubles We need to consider the fact that dvec3/4 require two vec4 slots. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-29 10:49:47 +02:00
chans_used[reg] = used;
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
channel_sizes[reg] = MAX2(channel_sizes[reg], channel_size);
} else {
is_aligned_to_dvec4[reg] = true;
is_aligned_to_dvec4[reg + 1] = true;
i965/vec4: fix pack_uniform_registers for doubles We need to consider the fact that dvec3/4 require two vec4 slots. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-29 10:49:47 +02:00
chans_used[reg + 1] = used - 4;
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
channel_sizes[reg + 1] = MAX2(channel_sizes[reg + 1], channel_size);
}
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
}
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
}
i965/vec4: Handle MOV_INDIRECT in pack_uniform_registers Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 15:43:48 -07:00
if (inst->opcode == SHADER_OPCODE_MOV_INDIRECT &&
inst->src[0].file == UNIFORM) {
assert(inst->src[2].file == BRW_IMMEDIATE_VALUE);
assert(inst->src[0].subnr == 0);
unsigned bytes_read = inst->src[2].ud;
assert(bytes_read % 4 == 0);
unsigned vec4s_read = DIV_ROUND_UP(bytes_read, 16);
/* We just mark every register touched by a MOV_INDIRECT as being
* fully used. This ensures that it doesn't broken up piecewise by
* the next part of our packing algorithm.
*/
int reg = inst->src[0].nr;
for (unsigned i = 0; i < vec4s_read; i++)
chans_used[reg + i] = 4;
}
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
}
int new_uniform_count = 0;
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
/* As the uniforms are going to be reordered, take the data from a temporary
* copy of the original param[].
*/
gl_constant_value **param = ralloc_array(NULL, gl_constant_value*,
stage_prog_data->nr_params);
memcpy(param, stage_prog_data->param,
sizeof(gl_constant_value*) * stage_prog_data->nr_params);
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
/* Now, figure out a packing of the live uniform vectors into our
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
* push constants. Start with dvec{3,4} because they are aligned to
* dvec4 size (2 vec4).
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
*/
for (int src = 0; src < uniforms; src++) {
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
int size = chans_used[src];
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
if (size == 0 || !is_aligned_to_dvec4[src])
i965/vec4: Use the actual channels used in pack_uniform_registers Previously, pack_uniform_registers worked based on the size of the uniform as given to us when we initially set up the uniforms. However, we have to walk through the uniforms and figure out liveness anyway, so we migh as well record the number of channels used as we go. This may also allow us to pack things tighter in a few cases. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-09-30 10:31:03 -07:00
continue;
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
/* dvec3 are aligned to dvec4 size, apply the alignment of the size
* to 4 to avoid moving last component of a dvec3 to the available
* location at the end of a previous dvec3. These available locations
* could be filled by smaller variables in next loop.
*/
size = ALIGN(size, 4);
int dst = set_push_constant_loc(uniforms, &new_uniform_count,
src, size, channel_sizes[src],
new_loc, new_chan,
new_chans_used);
/* Move the references to the data */
for (int j = 0; j < size; j++) {
stage_prog_data->param[dst * 4 + new_chan[src] + j] =
param[src * 4 + j];
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
}
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
}
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
/* Continue with the rest of data, which is aligned to vec4. */
for (int src = 0; src < uniforms; src++) {
int size = chans_used[src];
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
if (size == 0 || is_aligned_to_dvec4[src])
continue;
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
int dst = set_push_constant_loc(uniforms, &new_uniform_count,
src, size, channel_sizes[src],
new_loc, new_chan,
new_chans_used);
/* Move the references to the data */
for (int j = 0; j < size; j++) {
stage_prog_data->param[dst * 4 + new_chan[src] + j] =
param[src * 4 + j];
}
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
}
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
ralloc_free(param);
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
this->uniforms = new_uniform_count;
/* Now, update the instructions for our repacked uniforms. */
i965/vec4: Don't use instruction list after calculating the cfg. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 13:42:51 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
for (int i = 0 ; i < 3; i++) {
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
int src = inst->src[i].nr;
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: fix indentation in pack_uniform_registers Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-30 09:08:04 +02:00
if (inst->src[i].file != UNIFORM)
continue;
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
int chan = new_chan[src] / channel_sizes[src];
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
inst->src[i].nr = new_loc[src];
i965/vec4: load dvec3/4 uniforms first in the push constant buffer Reorder the uniforms to load first the dvec4-aligned variables in the push constant buffer and then push the vec4-aligned ones. It takes into account that the relocated uniforms should be aligned to their channel size. This fixes a bug were the dvec3/4 might be loaded one part on a GRF and the rest in next GRF, so the region parameters to read that could break the HW rules. v2: - Fix broken logic. - Add a comment to explain what should be needed to optimise the usage of the push constant buffer slots, as this patch does not pack the uniforms. v3: - Implemented the push constant buffer usage optimization. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Acked-by: Francisco Jerez <currojerez@riseup.net>
2017-04-19 11:18:35 +02:00
inst->src[i].swizzle += BRW_SWIZZLE4(chan, chan, chan, chan);
i965/vs: Pack live uniform vectors together in the push constant upload. At some point we need to also move uniform accesses out to pull constants when there are just too many in use, but we lack tests for that at the moment. Fixes glsl-vs-large-uniform-array. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-23 12:13:14 -07:00
}
}
}
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
/**
* Does algebraic optimizations (0 * a = 0, 1 * a = a, a + 0 = a).
*
* While GLSL IR also performs this optimization, we end up with it in
* our instruction stream for a couple of reasons. One is that we
* sometimes generate silly instructions, for example in array access
* where we'll generate "ADD offset, index, base" even if base is 0.
* The other is that GLSL IR's constant propagation doesn't track the
* components of aggregates, so some VS patterns (initialize matrix to
* 0, accumulate in vertex blending factors) end up breaking down to
* instructions involving 0.
*/
bool
vec4_visitor::opt_algebraic()
{
bool progress = false;
i965/vec4: Preserve the CFG in a few more places. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-08-24 19:27:09 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
switch (inst->opcode) {
i965: Add support for saturating immediates. I don't feel great about assert(!"unimplemented: ...") but these cases do only seem possible under some currently impossible circumstances. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-12-21 06:56:54 -08:00
case BRW_OPCODE_MOV:
if (inst->src[0].file != IMM)
break;
if (inst->saturate) {
if (inst->dst.type != inst->src[0].type)
assert(!"unimplemented: saturate mixed types");
i965: Prevent implicit upcasts to brw_reg. Now that backend_reg inherits from brw_reg, we have to be careful to avoid the object slicing problem. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-19 21:51:37 -08:00
if (brw_saturate_immediate(inst->dst.type,
&inst->src[0].as_brw_reg())) {
i965: Add support for saturating immediates. I don't feel great about assert(!"unimplemented: ...") but these cases do only seem possible under some currently impossible circumstances. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-12-21 06:56:54 -08:00
inst->saturate = false;
progress = true;
}
}
break;
i965/vec4: Allow CSE on uniform-vec4 expansion MOVs. Three source instructions cannot directly source a packed vec4 (<0,4,1> regioning) like vec4 uniforms, so we emit a MOV that expands the vec4 to both halves of a register. If these uniform values are used by multiple three-source instructions, we'll emit multiple expansion moves, which we cannot combine in CSE (because CSE emits moves itself). So emit a virtual instruction that we can CSE. Sometimes we demote a uniform to to a pull constant after emitting an expansion move for it. In that case, recognize in opt_algebraic that if the .file of the new instruction is GRF then it's just a real move that we can copy propagate and such. total instructions in shared programs: 5822418 -> 5812335 (-0.17%) instructions in affected programs: 351841 -> 341758 (-2.87%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-23 23:22:09 -07:00
case VEC4_OPCODE_UNPACK_UNIFORM:
if (inst->src[0].file != UNIFORM) {
inst->opcode = BRW_OPCODE_MOV;
progress = true;
}
break;
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
case BRW_OPCODE_ADD:
if (inst->src[1].is_zero()) {
inst->opcode = BRW_OPCODE_MOV;
inst->src[1] = src_reg();
progress = true;
}
break;
case BRW_OPCODE_MUL:
if (inst->src[1].is_zero()) {
inst->opcode = BRW_OPCODE_MOV;
switch (inst->src[0].type) {
case BRW_REGISTER_TYPE_F:
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
inst->src[0] = brw_imm_f(0.0f);
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
break;
case BRW_REGISTER_TYPE_D:
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
inst->src[0] = brw_imm_d(0);
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
break;
case BRW_REGISTER_TYPE_UD:
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
inst->src[0] = brw_imm_ud(0u);
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
break;
default:
i965: Use unreachable() instead of unconditional assert(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-29 14:54:01 -07:00
unreachable("not reached");
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
}
inst->src[1] = src_reg();
progress = true;
} else if (inst->src[1].is_one()) {
inst->opcode = BRW_OPCODE_MOV;
inst->src[1] = src_reg();
progress = true;
i965: Optimize multiplication by -1 into a negated MOV. instructions in affected programs: 968 -> 942 (-2.69%) helped: 4 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-02-04 18:08:30 -08:00
} else if (inst->src[1].is_negative_one()) {
inst->opcode = BRW_OPCODE_MOV;
inst->src[0].negate = !inst->src[0].negate;
inst->src[1] = src_reg();
progress = true;
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
}
break;
i965: Convert CMP.GE -(abs)reg 0 -> CMP.Z reg 0. total instructions in shared programs: 5952059 -> 5951603 (-0.01%) instructions in affected programs: 138812 -> 138356 (-0.33%) GAINED: 1 LOST: 0 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-01-05 13:51:03 -08:00
case BRW_OPCODE_CMP:
if (inst->conditional_mod == BRW_CONDITIONAL_GE &&
inst->src[0].abs &&
inst->src[0].negate &&
inst->src[1].is_zero()) {
inst->src[0].abs = false;
inst->src[0].negate = false;
inst->conditional_mod = BRW_CONDITIONAL_Z;
progress = true;
break;
}
break;
i965: Perform basic optimizations on the BROADCAST opcode. v2: Style fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-19 14:52:24 +02:00
case SHADER_OPCODE_BROADCAST:
if (is_uniform(inst->src[0]) ||
inst->src[1].is_zero()) {
inst->opcode = BRW_OPCODE_MOV;
inst->src[1] = src_reg();
inst->force_writemask_all = true;
progress = true;
}
break;
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
default:
break;
}
}
if (progress)
i965: Remove cfg-invalidating parameter from invalidate_live_intervals. Everything has been converted to preserve the CFG. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 10:54:00 -07:00
invalidate_live_intervals();
i965/vs: Add support for simple algebraic optimizations. We generate silly code for array access, and it's easier to generally support the cleanup than to specifically avoid the bad code in each place we might generate it. Removes 4.6% of instructions from 41.6% of shaders in shader-db, particularly savage2/hon and unigine. v2: Fixes by Ken: Make is_zero/one member functions, and fix a progress flag. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-02 15:18:29 -07:00
return progress;
}
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
/**
* Only a limited number of hardware registers may be used for push
* constants, so this turns access to the overflowed constants into
* pull constants.
*/
void
vec4_visitor::move_push_constants_to_pull_constants()
{
int pull_constant_loc[this->uniforms];
/* Only allow 32 registers (256 uniform components) as push constants,
* which is the limit on gen6.
i965: Update a ton of comments about constant buffers. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-19 08:51:12 -07:00
*
* If changing this value, note the limitation about total_regs in
* brw_curbe.c.
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
*/
int max_uniform_components = 32 * 8;
if (this->uniforms * 4 <= max_uniform_components)
return;
/* Make some sort of choice as to which uniforms get sent to pull
* constants. We could potentially do something clever here like
* look for the most infrequently used uniform vec4s, but leave
* that for later.
*/
for (int i = 0; i < this->uniforms * 4; i += 4) {
pull_constant_loc[i / 4] = -1;
if (i >= max_uniform_components) {
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
const gl_constant_value **values = &stage_prog_data->param[i];
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
/* Try to find an existing copy of this uniform in the pull
* constants if it was part of an array access already.
*/
for (unsigned int j = 0; j < stage_prog_data->nr_pull_params; j += 4) {
int matches;
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
for (matches = 0; matches < 4; matches++) {
if (stage_prog_data->pull_param[j + matches] != values[matches])
break;
}
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
if (matches == 4) {
pull_constant_loc[i / 4] = j / 4;
break;
}
}
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
if (pull_constant_loc[i / 4] == -1) {
assert(stage_prog_data->nr_pull_params % 4 == 0);
pull_constant_loc[i / 4] = stage_prog_data->nr_pull_params / 4;
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
for (int j = 0; j < 4; j++) {
stage_prog_data->pull_param[stage_prog_data->nr_pull_params++] =
i965: Move up duplicated fields from stage-specific prog_data to brw_stage_prog_data. There doesn't seem to be any reason for nr_params, nr_pull_params, param, and pull_param to be duplicated in the stage-specific subclasses of brw_stage_prog_data. Moving their definition to the common base class will allow some code sharing in a future commit, the removal of brw_vec4_prog_data_compare and brw_*_prog_data_free, and the simplification of the stage-specific brw_*_prog_data_compare. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2014-02-19 15:14:02 +01:00
values[j];
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
}
}
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
}
}
/* Now actually rewrite usage of the things we've moved to pull
* constants.
*/
i965/vec4: Preserve CFG in spill_reg(). Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-08-24 19:38:21 -07:00
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
for (int i = 0 ; i < 3; i++) {
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
if (inst->src[i].file != UNIFORM ||
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
pull_constant_loc[inst->src[i].nr] == -1)
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
continue;
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
int uniform = inst->src[i].nr;
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: make emit_pull_constant_load support 64-bit loads This way callers don't need to know about 64-bit particularities and we reuse some code. v2: - use byte_offset() instead of offset() - only mark the surface as used once Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-15 13:02:27 +02:00
const glsl_type *temp_type = type_sz(inst->src[i].type) == 8 ?
glsl_type::dvec4_type : glsl_type::vec4_type;
dst_reg temp = dst_reg(this, temp_type);
emit_pull_constant_load(block, inst, temp, inst->src[i],
pull_constant_loc[uniform], src_reg());
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
inst->src[i].file = temp.file;
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
inst->src[i].nr = temp.nr;
i965/vec4: fix indentation in move_push_constants_to_pull_constants() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-13 10:21:18 +02:00
inst->src[i].offset %= 16;
inst->src[i].reladdr = NULL;
i965/vs: Add support for overflowing the number of available push constants. Fixes glsl-vs-uniform-array-4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=33742 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2011-09-06 22:32:33 -07:00
}
}
/* Repack push constants to remove the now-unused ones. */
pack_uniform_registers();
}
i965/vec4: Extract depctrl hazards Move this to a separate function so that we can begin to add other little caveats without making too big a mess. NOTE: There is some desire to improve this function eventually, but we need to fix a bug first. v2: Use const for the inst for the hazard check (Matt) Invert safe logic to get rid of the double negative (Matt) Add PRM reference for predicates (Matt) Add note about empirical evidence for math (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:37 -08:00
/* Conditions for which we want to avoid setting the dependency control bits */
i965/vec4/gen8: Handle the MUL dest hazard exception Fix one of the few cases where we can't reliable touch the destination hazard bits. I am explicitly doing this patch individually so it is easy to backport. I was tempted to do this patch before the previous patch which reorganized the code, but I believe even doing that first, this is still easy to backport. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=84212 Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:41 -08:00
bool
vec4_visitor::is_dep_ctrl_unsafe(const vec4_instruction *inst)
i965/vec4: Extract depctrl hazards Move this to a separate function so that we can begin to add other little caveats without making too big a mess. NOTE: There is some desire to improve this function eventually, but we need to fix a bug first. v2: Use const for the inst for the hazard check (Matt) Invert safe logic to get rid of the double negative (Matt) Add PRM reference for predicates (Matt) Add note about empirical evidence for math (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:37 -08:00
{
i965/vec4/gen8: Handle the MUL dest hazard exception Fix one of the few cases where we can't reliable touch the destination hazard bits. I am explicitly doing this patch individually so it is easy to backport. I was tempted to do this patch before the previous patch which reorganized the code, but I believe even doing that first, this is still easy to backport. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=84212 Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:41 -08:00
#define IS_DWORD(reg) \
(reg.type == BRW_REGISTER_TYPE_UD || \
reg.type == BRW_REGISTER_TYPE_D)
i965/vec4: Do not use DepCtrl with 64-bit instructions The BDW PRM says that it is not supported, but it seems that gen7 is also affected, since doing DepCtrl on double-float instructions leads to GPU hangs in some cases, which is probably not surprising knowing that this is not supported in new hardware iterations. The SKL PRMs do not mention this restriction, so it is probably fine. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-23 10:40:47 +02:00
#define IS_64BIT(reg) (reg.file != BAD_FILE && type_sz(reg.type) == 8)
i965/vec4: extend the DWORD multiply DepCtrl restriction to all gen8 platforms v2: - Add Broxton as Intel's internal PRMs says that it is needed (Matt). Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-23 10:35:50 +02:00
/* From the Cherryview and Broadwell PRMs:
*
* "When source or destination datatype is 64b or operation is integer DWord
i965/vec4: Correct MUL destination hazard As it turns out, we were over-thinking the cause of the hang on Cherryview. It's simply errata for Cherryview. commit 88fea85f09e2252035bec66ab26c375b45b000f5 Author: Ben Widawsky <benjamin.widawsky@intel.com> Date: Fri Nov 21 10:47:41 2014 -0800 i965/vec4/gen8: Handle the MUL dest hazard exception This is an explanation to why we never saw the hang on BDW. NOTE: The problem the original patch was trying to fix does still exist. It will have to be fixed at some point. v2: Modify commit message, s/CHV/BDW Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=84212 Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-02 17:30:49 -08:00
* multiply, DepCtrl must not be used."
i965/vec4: extend the DWORD multiply DepCtrl restriction to all gen8 platforms v2: - Add Broxton as Intel's internal PRMs says that it is needed (Matt). Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-23 10:35:50 +02:00
*
i965/vec4: Do not use DepCtrl with 64-bit instructions The BDW PRM says that it is not supported, but it seems that gen7 is also affected, since doing DepCtrl on double-float instructions leads to GPU hangs in some cases, which is probably not surprising knowing that this is not supported in new hardware iterations. The SKL PRMs do not mention this restriction, so it is probably fine. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-23 10:40:47 +02:00
* SKL PRMs don't include this restriction, however, gen7 seems to be
* affected, at least by the 64b restriction, since DepCtrl with double
* precision instructions seems to produce GPU hangs in some cases.
i965/vec4/gen8: Handle the MUL dest hazard exception Fix one of the few cases where we can't reliable touch the destination hazard bits. I am explicitly doing this patch individually so it is easy to backport. I was tempted to do this patch before the previous patch which reorganized the code, but I believe even doing that first, this is still easy to backport. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=84212 Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:41 -08:00
*/
intel: compiler/i965: fix is_broxton checks In 5f2fe9302c is_geminilake was introduced for the differenciate broxton from geminilake. Unfortunately I failed as verifying that is_broxton is throughout the code base to mean Gen9lp. Fixes: 5f2fe9302c ("intel: common: add flag to identify platforms by name") Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-20 11:06:24 +01:00
if (devinfo->gen == 8 || gen_device_info_is_9lp(devinfo)) {
i965/vec4/gen8: Handle the MUL dest hazard exception Fix one of the few cases where we can't reliable touch the destination hazard bits. I am explicitly doing this patch individually so it is easy to backport. I was tempted to do this patch before the previous patch which reorganized the code, but I believe even doing that first, this is still easy to backport. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=84212 Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:41 -08:00
if (inst->opcode == BRW_OPCODE_MUL &&
IS_DWORD(inst->src[0]) &&
IS_DWORD(inst->src[1]))
return true;
}
i965/vec4: Do not use DepCtrl with 64-bit instructions The BDW PRM says that it is not supported, but it seems that gen7 is also affected, since doing DepCtrl on double-float instructions leads to GPU hangs in some cases, which is probably not surprising knowing that this is not supported in new hardware iterations. The SKL PRMs do not mention this restriction, so it is probably fine. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-23 10:40:47 +02:00
if (devinfo->gen >= 7 && devinfo->gen <= 8) {
if (IS_64BIT(inst->dst) || IS_64BIT(inst->src[0]) ||
IS_64BIT(inst->src[1]) || IS_64BIT(inst->src[2]))
return true;
}
#undef IS_64BIT
i965/vec4/gen8: Handle the MUL dest hazard exception Fix one of the few cases where we can't reliable touch the destination hazard bits. I am explicitly doing this patch individually so it is easy to backport. I was tempted to do this patch before the previous patch which reorganized the code, but I believe even doing that first, this is still easy to backport. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=84212 Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:41 -08:00
#undef IS_DWORD
i965: Add a devinfo field to backend_visitor and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 18:00:05 -07:00
if (devinfo->gen >= 8) {
i965/vec4: Don't set any dependency control bits for F32TO16 on Gen8. It's expanded to several instructions. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:37:46 +02:00
if (inst->opcode == BRW_OPCODE_F32TO16)
return true;
}
i965/vec4: Extract depctrl hazards Move this to a separate function so that we can begin to add other little caveats without making too big a mess. NOTE: There is some desire to improve this function eventually, but we need to fix a bug first. v2: Use const for the inst for the hazard check (Matt) Invert safe logic to get rid of the double negative (Matt) Add PRM reference for predicates (Matt) Add note about empirical evidence for math (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:37 -08:00
/*
* mlen:
* In the presence of send messages, totally interrupt dependency
* control. They're long enough that the chance of dependency
* control around them just doesn't matter.
*
* predicate:
* From the Ivy Bridge PRM, volume 4 part 3.7, page 80:
* When a sequence of NoDDChk and NoDDClr are used, the last instruction that
* completes the scoreboard clear must have a non-zero execution mask. This
* means, if any kind of predication can change the execution mask or channel
* enable of the last instruction, the optimization must be avoided. This is
* to avoid instructions being shot down the pipeline when no writes are
* required.
*
* math:
* Dependency control does not work well over math instructions.
* NB: Discovered empirically
*/
return (inst->mlen || inst->predicate || inst->is_math());
}
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
/**
* Sets the dependency control fields on instructions after register
* allocation and before the generator is run.
*
* When you have a sequence of instructions like:
*
* DP4 temp.x vertex uniform[0]
* DP4 temp.y vertex uniform[0]
* DP4 temp.z vertex uniform[0]
* DP4 temp.w vertex uniform[0]
*
* The hardware doesn't know that it can actually run the later instructions
* while the previous ones are in flight, producing stalls. However, we have
* manual fields we can set in the instructions that let it do so.
*/
void
vec4_visitor::opt_set_dependency_control()
{
vec4_instruction *last_grf_write[BRW_MAX_GRF];
uint8_t grf_channels_written[BRW_MAX_GRF];
vec4_instruction *last_mrf_write[BRW_MAX_GRF];
uint8_t mrf_channels_written[BRW_MAX_GRF];
i965/vs: move VS-specific data members to vs_vec4_visitor. This patch moves the following data structures from vec4_visitor to vec4_vs_visitor, since they contain VS-specific data: - struct brw_vs_compile *c (renamed to vs_compile) - struct brw_vs_prog_data *prog_data (renamed to vs_prog_data) - src_reg *vp_temp_regs - src_reg vp_addr_reg Since brw_vs_compile and brw_vs_prog_data also contain vec4-generic data, the following pointers are added to the base class, to allow it to access the vec4-generic portions of these data structures: - struct brw_vec4_compile *c - struct brw_vec4_prog_key *key - struct brw_vec4_prog_data *prog_data Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net> v2: Use shorter names in the base class and longer names in the derived class. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-02-17 09:09:45 -08:00
assert(prog_data->total_grf ||
i965/vs: split brw_vs_prog_data into generic and VS-specific parts. This will allow the generic parts to be re-used for geometry shaders. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> v2: Put urb_read_length and urb_entry_size in the generic struct. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-02-17 07:48:21 -08:00
!"Must be called after register allocation");
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
i965: Add and use foreach_block macro. Use this as an opportunity to rename 'block_num' to 'num'. block->num is clear, and block->block_num has always been redundant.
2014-07-11 22:31:39 -07:00
foreach_block (block, cfg) {
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
memset(last_grf_write, 0, sizeof(last_grf_write));
memset(last_mrf_write, 0, sizeof(last_mrf_write));
i965: Add and use foreach_block macro. Use this as an opportunity to rename 'block_num' to 'num'. block->num is clear, and block->block_num has always been redundant.
2014-07-11 22:31:39 -07:00
foreach_inst_in_block (vec4_instruction, inst, block) {
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
/* If we read from a register that we were doing dependency control
* on, don't do dependency control across the read.
*/
for (int i = 0; i < 3; i++) {
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
int reg = inst->src[i].nr + inst->src[i].offset / REG_SIZE;
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
if (inst->src[i].file == VGRF) {
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
last_grf_write[reg] = NULL;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
} else if (inst->src[i].file == FIXED_GRF) {
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
memset(last_grf_write, 0, sizeof(last_grf_write));
break;
}
assert(inst->src[i].file != MRF);
}
i965/vec4: Extract depctrl hazards Move this to a separate function so that we can begin to add other little caveats without making too big a mess. NOTE: There is some desire to improve this function eventually, but we need to fix a bug first. v2: Use const for the inst for the hazard check (Matt) Invert safe logic to get rid of the double negative (Matt) Add PRM reference for predicates (Matt) Add note about empirical evidence for math (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-21 10:47:37 -08:00
if (is_dep_ctrl_unsafe(inst)) {
i965: Avoid dependency hints on math opcodes Putting NoDDClr and NoDDChk dependency control on instruction sequences that include math opcodes can cause corruption of channels. Treat math opcodes like send opcodes and suppress dependency hinting. Signed-off-by: Mike Stroyan <mike@LunarG.com> Tested-by: Tony Bertapelli <anthony.p.bertapelli@intel.com> Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-02-12 17:24:55 -07:00
memset(last_grf_write, 0, sizeof(last_grf_write));
memset(last_mrf_write, 0, sizeof(last_mrf_write));
continue;
}
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
/* Now, see if we can do dependency control for this instruction
* against a previous one writing to its destination.
*/
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
int reg = inst->dst.nr + inst->dst.offset / REG_SIZE;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
if (inst->dst.file == VGRF || inst->dst.file == FIXED_GRF) {
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
if (last_grf_write[reg] &&
i965/vec4: Check that the write offsets match when setting dependency controls. For simplicity just assume that two writes to the same GRF with different sub-GRF offsets will potentially interfere and break the dependency control chain. This is in preparation for adding sub-GRF offset support to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:02:00 -07:00
last_grf_write[reg]->dst.offset == inst->dst.offset &&
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
!(inst->dst.writemask & grf_channels_written[reg])) {
last_grf_write[reg]->no_dd_clear = true;
inst->no_dd_check = true;
} else {
grf_channels_written[reg] = 0;
}
last_grf_write[reg] = inst;
grf_channels_written[reg] |= inst->dst.writemask;
} else if (inst->dst.file == MRF) {
if (last_mrf_write[reg] &&
i965/vec4: Check that the write offsets match when setting dependency controls. For simplicity just assume that two writes to the same GRF with different sub-GRF offsets will potentially interfere and break the dependency control chain. This is in preparation for adding sub-GRF offset support to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:02:00 -07:00
last_mrf_write[reg]->dst.offset == inst->dst.offset &&
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
!(inst->dst.writemask & mrf_channels_written[reg])) {
last_mrf_write[reg]->no_dd_clear = true;
inst->no_dd_check = true;
} else {
mrf_channels_written[reg] = 0;
}
last_mrf_write[reg] = inst;
mrf_channels_written[reg] |= inst->dst.writemask;
}
}
}
}
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
bool
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
vec4_instruction::can_reswizzle(const struct gen_device_info *devinfo,
i965/vec4: Don't coalesce regs in Gen6 MATH ops if reswizzle/writemask needed Gen6 MATH instructions can not execute in align16 mode, so swizzles or writemasking are not allowed. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92033 Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-22 18:17:45 +02:00
int dst_writemask,
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
int swizzle,
int swizzle_mask)
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
{
i965/vec4: Don't coalesce regs in Gen6 MATH ops if reswizzle/writemask needed Gen6 MATH instructions can not execute in align16 mode, so swizzles or writemasking are not allowed. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92033 Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-22 18:17:45 +02:00
/* Gen6 MATH instructions can not execute in align16 mode, so swizzles
i965/vec4: Use can_do_writemask in can_reswizzle Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:10:04 -07:00
* are not allowed.
i965/vec4: Don't coalesce regs in Gen6 MATH ops if reswizzle/writemask needed Gen6 MATH instructions can not execute in align16 mode, so swizzles or writemasking are not allowed. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92033 Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-22 18:17:45 +02:00
*/
i965/vec4: Use can_do_writemask in can_reswizzle Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:10:04 -07:00
if (devinfo->gen == 6 && is_math() && swizzle != BRW_SWIZZLE_XYZW)
return false;
i965/vec4: Use reads_accumulator_implicitly(), not MACH checks. Curro pointed out that I should not just check for MACH, but use the reads_accumulator_implicitly() helper, which would also prevent the same bug with MAC and SADA2 (if we ever decide to use them). Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-22 15:02:08 -07:00
/* We can't swizzle implicit accumulator access. We'd have to
* reswizzle the producer of the accumulator value in addition
* to the consumer (i.e. both MUL and MACH). Just skip this.
i965/vec4: Avoid reswizzling MACH instructions in opt_register_coalesce(). opt_register_coalesce() was optimizing sequences such as: mul(8) acc0:D, attr18.xyyy:D, attr19.xyyy:D mach(8) vgrf5.xy:D, attr18.xyyy:D, attr19.xyyy:D mov(8) m4.zw:F, vgrf5.xxxy:F into: mul(8) acc0:D, attr18.xyyy:D, attr19.xyyy:D mach(8) m4.zw:D, attr18.xxxy:D, attr19.xxxy:D This doesn't work - if we're going to reswizzle MACH, we'd need to reswizzle the MUL as well. Here, the MUL fills the accumulator's .zw components with attr18.yy * attr19.yy. But the MACH instruction expects .z to contain attr18.x * attr19.x. Bogus results ensue. No change in shader-db on Haswell. Prevents regressions in Timothy's patches to use enhanced layouts for varying packing (which rearrange code just enough to trigger this pre-existing bug, but were fine themselves). Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-04-21 01:28:13 -07:00
*/
i965/vec4: Use reads_accumulator_implicitly(), not MACH checks. Curro pointed out that I should not just check for MACH, but use the reads_accumulator_implicitly() helper, which would also prevent the same bug with MAC and SADA2 (if we ever decide to use them). Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-22 15:02:08 -07:00
if (reads_accumulator_implicitly())
i965/vec4: Avoid reswizzling MACH instructions in opt_register_coalesce(). opt_register_coalesce() was optimizing sequences such as: mul(8) acc0:D, attr18.xyyy:D, attr19.xyyy:D mach(8) vgrf5.xy:D, attr18.xyyy:D, attr19.xyyy:D mov(8) m4.zw:F, vgrf5.xxxy:F into: mul(8) acc0:D, attr18.xyyy:D, attr19.xyyy:D mach(8) m4.zw:D, attr18.xxxy:D, attr19.xxxy:D This doesn't work - if we're going to reswizzle MACH, we'd need to reswizzle the MUL as well. Here, the MUL fills the accumulator's .zw components with attr18.yy * attr19.yy. But the MACH instruction expects .z to contain attr18.x * attr19.x. Bogus results ensue. No change in shader-db on Haswell. Prevents regressions in Timothy's patches to use enhanced layouts for varying packing (which rearrange code just enough to trigger this pre-existing bug, but were fine themselves). Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-04-21 01:28:13 -07:00
return false;
i965/vec4: Use can_do_writemask in can_reswizzle Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-05 14:10:04 -07:00
if (!can_do_writemask(devinfo) && dst_writemask != WRITEMASK_XYZW)
i965/vec4: Don't coalesce regs in Gen6 MATH ops if reswizzle/writemask needed Gen6 MATH instructions can not execute in align16 mode, so swizzles or writemasking are not allowed. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92033 Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-22 18:17:45 +02:00
return false;
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
/* If this instruction sets anything not referenced by swizzle, then we'd
* totally break it when we reswizzle.
*/
if (dst.writemask & ~swizzle_mask)
return false;
i965/fs: Brown bag fix.
2014-09-05 10:29:38 -07:00
if (mlen > 0)
i965/vec4: Add ability to reswizzle arbitrary swizzles. Before commit 04895f5c we would only reswizzle dot product instructions (since they wrote the same value into all channels, and we didn't have to think about anything else). That commit extended reswizzling to cases when the swizzle was single valued -- i.e., writing the same result into all channels. But allowing reswizzling of arbitrary things is actually really easy and is even less code. (Why didn't we do this in the first place?!) total instructions in shared programs: 4266079 -> 4261000 (-0.12%) instructions in affected programs: 351933 -> 346854 (-1.44%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:27:40 -07:00
return false;
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
i965/vec4: Don't reswizzle hardware registers Cc: "11.0 10.6" <mesa-stable@lists.freedesktop.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=91719 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-10 16:19:42 -07:00
for (int i = 0; i < 3; i++) {
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
if (src[i].is_accumulator())
i965/vec4: Don't reswizzle hardware registers Cc: "11.0 10.6" <mesa-stable@lists.freedesktop.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=91719 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-10 16:19:42 -07:00
return false;
}
i965/vec4: Add ability to reswizzle arbitrary swizzles. Before commit 04895f5c we would only reswizzle dot product instructions (since they wrote the same value into all channels, and we didn't have to think about anything else). That commit extended reswizzling to cases when the swizzle was single valued -- i.e., writing the same result into all channels. But allowing reswizzling of arbitrary things is actually really easy and is even less code. (Why didn't we do this in the first place?!) total instructions in shared programs: 4266079 -> 4261000 (-0.12%) instructions in affected programs: 351933 -> 346854 (-1.44%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:27:40 -07:00
return true;
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
}
/**
* For any channels in the swizzle's source that were populated by this
* instruction, rewrite the instruction to put the appropriate result directly
* in those channels.
*
* e.g. for swizzle=yywx, MUL a.xy b c -> MUL a.yy_x b.yy z.yy_x
*/
void
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
vec4_instruction::reswizzle(int dst_writemask, int swizzle)
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
{
i965/vec4: Simplify reswizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:09:51 +02:00
/* Destination write mask doesn't correspond to source swizzle for the dot
* product and pack_bytes instructions.
*/
i965/vec4: Add ability to reswizzle arbitrary swizzles. Before commit 04895f5c we would only reswizzle dot product instructions (since they wrote the same value into all channels, and we didn't have to think about anything else). That commit extended reswizzling to cases when the swizzle was single valued -- i.e., writing the same result into all channels. But allowing reswizzling of arbitrary things is actually really easy and is even less code. (Why didn't we do this in the first place?!) total instructions in shared programs: 4266079 -> 4261000 (-0.12%) instructions in affected programs: 351933 -> 346854 (-1.44%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:27:40 -07:00
if (opcode != BRW_OPCODE_DP4 && opcode != BRW_OPCODE_DPH &&
i965/vec4: Simplify reswizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:09:51 +02:00
opcode != BRW_OPCODE_DP3 && opcode != BRW_OPCODE_DP2 &&
opcode != VEC4_OPCODE_PACK_BYTES) {
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
for (int i = 0; i < 3; i++) {
if (src[i].file == BAD_FILE || src[i].file == IMM)
continue;
i965/vec4: Simplify reswizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:09:51 +02:00
src[i].swizzle = brw_compose_swizzle(swizzle, src[i].swizzle);
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
}
i965/vec4: Add ability to reswizzle arbitrary swizzles. Before commit 04895f5c we would only reswizzle dot product instructions (since they wrote the same value into all channels, and we didn't have to think about anything else). That commit extended reswizzling to cases when the swizzle was single valued -- i.e., writing the same result into all channels. But allowing reswizzling of arbitrary things is actually really easy and is even less code. (Why didn't we do this in the first place?!) total instructions in shared programs: 4266079 -> 4261000 (-0.12%) instructions in affected programs: 351933 -> 346854 (-1.44%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:27:40 -07:00
}
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
i965/vec4: Simplify reswizzle() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:09:51 +02:00
/* Apply the specified swizzle and writemask to the original mask of
* written components.
*/
dst.writemask = dst_writemask &
brw_apply_swizzle_to_mask(swizzle, dst.writemask);
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
}
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
/*
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
* Tries to reduce extra MOV instructions by taking temporary GRFs that get
* just written and then MOVed into another reg and making the original write
* of the GRF write directly to the final destination instead.
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
*/
bool
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
vec4_visitor::opt_register_coalesce()
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
{
bool progress = false;
int next_ip = 0;
calculate_live_intervals();
i965: Use basic-block aware insertion/removal functions. To avoid invalidating and recreating the control flow graph. Also stop invalidating the CFG in places we didn't add or remove an instruction. cfg calculations: 202951 -> 80307 (-60.43%) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-07-12 21:18:39 -07:00
foreach_block_and_inst_safe (block, vec4_instruction, inst, cfg) {
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
int ip = next_ip;
next_ip++;
if (inst->opcode != BRW_OPCODE_MOV ||
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
(inst->dst.file != VGRF && inst->dst.file != MRF) ||
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
inst->predicate ||
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
inst->src[0].file != VGRF ||
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
inst->dst.type != inst->src[0].type ||
inst->src[0].abs || inst->src[0].negate || inst->src[0].reladdr)
continue;
i965/vec4: Detect and delete useless MOVs. With NIR: instructions in affected programs: 111508 -> 109193 (-2.08%) helped: 507 Without NIR: instructions in affected programs: 28763 -> 28474 (-1.00%) helped: 186 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-21 13:58:19 -07:00
/* Remove no-op MOVs */
if (inst->dst.file == inst->src[0].file &&
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
inst->dst.nr == inst->src[0].nr &&
i965/vec4: Compare full register offsets in opt_register_coalesce nop move check. In preparation for adding support for sub-GRF offsets to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:04:02 -07:00
inst->dst.offset == inst->src[0].offset) {
i965/vec4: Detect and delete useless MOVs. With NIR: instructions in affected programs: 111508 -> 109193 (-2.08%) helped: 507 Without NIR: instructions in affected programs: 28763 -> 28474 (-1.00%) helped: 186 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-21 13:58:19 -07:00
bool is_nop_mov = true;
for (unsigned c = 0; c < 4; c++) {
if ((inst->dst.writemask & (1 << c)) == 0)
continue;
if (BRW_GET_SWZ(inst->src[0].swizzle, c) != c) {
is_nop_mov = false;
break;
}
}
if (is_nop_mov) {
inst->remove(block);
i965/vec4: Consider removal of no-op MOVs as progress during register coalesce. Bug found by the liveness analysis validation pass that will be introduced in a later commit. The no-op MOV check in opt_register_coalesce() was removing instructions which makes the cached liveness analysis calculation inconsistent with the shader IR. We were failing to set progress to true in that case though, which means that invalidate_live_intervals() wouldn't necessarily be called at the end of the function. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-03-13 19:15:45 -07:00
progress = true;
i965/vec4: Detect and delete useless MOVs. With NIR: instructions in affected programs: 111508 -> 109193 (-2.08%) helped: 507 Without NIR: instructions in affected programs: 28763 -> 28474 (-1.00%) helped: 186 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-21 13:58:19 -07:00
continue;
}
}
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
bool to_mrf = (inst->dst.file == MRF);
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* Can't coalesce this GRF if someone else was going to
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
* read it later.
*/
i965/vec4: handle 32 and 64 bit channels in liveness analysis Our current data flow analysis does not take into account that channels on 64-bit operands are 64-bit. This is a problem when the same register is accessed using both 64-bit and 32-bit channels. This is very common in operations where we need to access 64-bit data in 32-bit chunks, such as the double packing and packing operations. This patch changes the analysis by checking the bits that each source or destination datatype needs. Actually, rather than bits, we use blocks of 32bits, which is the minimum channel size. Because a vgrf can contain a dvec4 (256 bits), we reserve 8 32-bit blocks to map the channels. v2 (Curro): - Simplify code by making the var_from_reg helpers take an extra argument with the register component we want. - Fix a couple of cases where we had to update the code to the new way of representing live variables. v3: - Fix indent in multiline expressions (Matt) - Fix comment's closing tag (Matt) - Use DIV_ROUND_UP(inst->size_written, 16) instead of 2 * regs_written(inst) to avoid rounding issues. The same for regs_read(i). (Curro). - Add asserts in var_from_reg() to avoid exceeding the allocated registers (Curro). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-06-10 13:55:00 +02:00
if (var_range_end(var_from_reg(alloc, dst_reg(inst->src[0])), 8) > ip)
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
continue;
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* We need to check interference with the final destination between this
* instruction and the earliest instruction involved in writing the GRF
* we're eliminating. To do that, keep track of which of our source
* channels we've seen initialized.
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
*/
i965/vec4: Simplify opt_register_coalesce() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:11:20 +02:00
const unsigned chans_needed =
brw_apply_inv_swizzle_to_mask(inst->src[0].swizzle,
inst->dst.writemask);
unsigned chans_remaining = chans_needed;
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* Now walk up the instruction stream trying to see if we can rewrite
* everything writing to the temporary to write into the destination
* instead.
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
*/
i965/vec4: Don't iterate between blocks with inst->next/prev. The register coalescing portion of this patch hurts three shaders in Guacamelee by one instruction each, but examining the diff makes me believe that what we were generating was (perhaps harmlessly) incorrect.
2014-09-02 14:43:43 -07:00
vec4_instruction *_scan_inst = (vec4_instruction *)inst->prev;
foreach_inst_in_block_reverse_starting_from(vec4_instruction, scan_inst,
i965: Remove block arg from foreach_inst_in_block_*_starting_from Since 49374fab5d793 these macros no longer actually use the block argument. I think this is worth doing to make the macros easier to use because they already have really long names and a confusing set of arguments. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-20 11:16:00 +02:00
inst) {
i965/vec4: Don't iterate between blocks with inst->next/prev. The register coalescing portion of this patch hurts three shaders in Guacamelee by one instruction each, but examining the diff makes me believe that what we were generating was (perhaps harmlessly) incorrect.
2014-09-02 14:43:43 -07:00
_scan_inst = scan_inst;
i965/vec4: Drop backend_reg::in_range() in favor of regions_overlap(). This makes sure that overlap checks are done correctly throughout the back-end when the '*this' register starts before the register/size pair provided as argument, and is actually less annoying to use than in_range() at this point since regions_overlap() takes its size arguments in bytes. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 19:42:40 -07:00
if (regions_overlap(inst->src[0], inst->size_read(0),
scan_inst->dst, scan_inst->size_written)) {
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* Found something writing to the reg we want to coalesce away. */
if (to_mrf) {
/* SEND instructions can't have MRF as a destination. */
if (scan_inst->mlen)
break;
i965: Add a devinfo field to backend_visitor and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 18:00:05 -07:00
if (devinfo->gen == 6) {
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* gen6 math instructions must have the destination be
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
* VGRF, so no compute-to-MRF for them.
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
*/
if (scan_inst->is_math()) {
break;
}
}
}
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
i965/vec4: Fix saturation errors when coalescing registers If the register types do not match and the instruction that contains the final destination is saturated, register coalescing generated non-equivalent code. This did not happen when using IR because types usually matched, but it is visible in nir-vec4. For example, mov vgrf7:D vgrf2:D mov.sat m4:F vgrf7:F is coalesced to: mov.sat m4:D vgrf2:D The patch prevents coalescing in such scenario, unless the instruction we want to coalesce into is a MOV (without type conversion implied). In that case, the patch sets the register types to the type of the final destination. Shader-db results in HSW (only vec4 instructions shown): total instructions in shared programs: 1754415 -> 1754416 (0.00%) instructions in affected programs: 74 -> 75 (1.35%) helped: 0 HURT: 1 GAINED: 0 LOST: 0 Only one extra instruction in one of the shaders, that comes from eliminating a saturation error by preventing register coalesce. Cc: "10.6 11.0" <mesa-stable@lists.freedesktop.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-08-05 15:57:33 +02:00
/* This doesn't handle saturation on the instruction we
* want to coalesce away if the register types do not match.
* But if scan_inst is a non type-converting 'mov', we can fix
* the types later.
*/
if (inst->saturate &&
inst->dst.type != scan_inst->dst.type &&
!(scan_inst->opcode == BRW_OPCODE_MOV &&
scan_inst->dst.type == scan_inst->src[0].type))
break;
i965/vec4: teach register coalescing about 64-bit Specifically, at least for now, we don't want to deal with the fact that channel sizes for fp64 instructions are twice the size, so prevent coalescing from instructions with a different type size. Also, we should check that if we are coalescing a register from another MOV we should be writing the same amount of data in both operations, otherwise we end up wiring more or less than the original instruction. This can happen, for example, when we have split fp64 MOVs with an exec size of 4 that only write one register each and then a MOV with exec size of 8 that reads both. We want to avoid the pass to think that it can coalesce from the first split MOV alone. Ideally we would like the pass to see that it can coalesce from both split MOVs instead, but for now we keep it simple. Finally, the pass doesn't support coalescing of multiple registers but in the case of normal SIMD4x2 double-precision instructions they naturally write two registers (one per vertex) and there is no reason why we should not allow coalescing in this case. Change the restriction to bail if we see instructions that write more than 8 channels, where the channels can be 32-bit or 64-bit. v2: - Make sure that scan_inst and inst write the same amount of data. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-18 12:04:13 +02:00
/* Only allow coalescing between registers of the same type size.
* Otherwise we would need to make the pass aware of the fact that
* channel sizes are different for single and double precision.
*/
if (type_sz(inst->src[0].type) != type_sz(scan_inst->src[0].type))
break;
/* Check that scan_inst writes the same amount of data as the
* instruction, otherwise coalescing would lead to writing a
* different (larger or smaller) region of the destination
*/
if (scan_inst->size_written != inst->size_written)
break;
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
/* If we can't handle the swizzle, bail. */
i965/vec4: Don't coalesce regs in Gen6 MATH ops if reswizzle/writemask needed Gen6 MATH instructions can not execute in align16 mode, so swizzles or writemasking are not allowed. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92033 Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-22 18:17:45 +02:00
if (!scan_inst->can_reswizzle(devinfo, inst->dst.writemask,
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
inst->src[0].swizzle,
i965/vec4: Simplify opt_register_coalesce() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:11:20 +02:00
chans_needed)) {
i965/vs: Extend opt_compute_to_mrf to handle limited "reswizzling" The way our visitor works, scalar expression/swizzle results that get stored in channels other than .x will have an intermediate MOV from their result in the .x channel to the real .y (or whatever) channel, and similarly for vec2/vec3 results. By knowing how to adjust DP4-type instructions for optimizing out a swizzled MOV, we can reduce instructions in common matrix multiplication cases. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 22:29:26 -08:00
break;
}
i965/vec4: teach register coalescing about 64-bit Specifically, at least for now, we don't want to deal with the fact that channel sizes for fp64 instructions are twice the size, so prevent coalescing from instructions with a different type size. Also, we should check that if we are coalescing a register from another MOV we should be writing the same amount of data in both operations, otherwise we end up wiring more or less than the original instruction. This can happen, for example, when we have split fp64 MOVs with an exec size of 4 that only write one register each and then a MOV with exec size of 8 that reads both. We want to avoid the pass to think that it can coalesce from the first split MOV alone. Ideally we would like the pass to see that it can coalesce from both split MOVs instead, but for now we keep it simple. Finally, the pass doesn't support coalescing of multiple registers but in the case of normal SIMD4x2 double-precision instructions they naturally write two registers (one per vertex) and there is no reason why we should not allow coalescing in this case. Change the restriction to bail if we see instructions that write more than 8 channels, where the channels can be 32-bit or 64-bit. v2: - Make sure that scan_inst and inst write the same amount of data. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-18 12:04:13 +02:00
/* This only handles coalescing writes of 8 channels (1 register
* for single-precision and 2 registers for double-precision)
* starting at the source offset of the copy instruction.
i965/vec4: Don't coalesce registers with overlapping writes not matching the MOV source. In preparation for adding support for sub-GRF offsets to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:08:29 -07:00
*/
i965/vec4: teach register coalescing about 64-bit Specifically, at least for now, we don't want to deal with the fact that channel sizes for fp64 instructions are twice the size, so prevent coalescing from instructions with a different type size. Also, we should check that if we are coalescing a register from another MOV we should be writing the same amount of data in both operations, otherwise we end up wiring more or less than the original instruction. This can happen, for example, when we have split fp64 MOVs with an exec size of 4 that only write one register each and then a MOV with exec size of 8 that reads both. We want to avoid the pass to think that it can coalesce from the first split MOV alone. Ideally we would like the pass to see that it can coalesce from both split MOVs instead, but for now we keep it simple. Finally, the pass doesn't support coalescing of multiple registers but in the case of normal SIMD4x2 double-precision instructions they naturally write two registers (one per vertex) and there is no reason why we should not allow coalescing in this case. Change the restriction to bail if we see instructions that write more than 8 channels, where the channels can be 32-bit or 64-bit. v2: - Make sure that scan_inst and inst write the same amount of data. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-18 12:04:13 +02:00
if (DIV_ROUND_UP(scan_inst->size_written,
type_sz(scan_inst->dst.type)) > 8 ||
i965/vec4: Don't coalesce registers with overlapping writes not matching the MOV source. In preparation for adding support for sub-GRF offsets to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:08:29 -07:00
scan_inst->dst.offset != inst->src[0].offset)
i965/vec4: Fix handling of multiple register reads and writes in opt_register_coalesce(). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 19:43:44 +02:00
break;
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
/* Mark which channels we found unconditional writes for. */
i965/vec4: Simplify opt_register_coalesce() using the swizzle utils. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 21:11:20 +02:00
if (!scan_inst->predicate)
chans_remaining &= ~scan_inst->dst.writemask;
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
if (chans_remaining == 0)
break;
}
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* You can't read from an MRF, so if someone else reads our MRF's
* source GRF that we wanted to rewrite, that stops us. If it's a
* GRF we're trying to coalesce to, we don't actually handle
* rewriting sources so bail in that case as well.
*/
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
bool interfered = false;
for (int i = 0; i < 3; i++) {
i965/vec4: Drop backend_reg::in_range() in favor of regions_overlap(). This makes sure that overlap checks are done correctly throughout the back-end when the '*this' register starts before the register/size pair provided as argument, and is actually less annoying to use than in_range() at this point since regions_overlap() takes its size arguments in bytes. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 19:42:40 -07:00
if (regions_overlap(inst->src[0], inst->size_read(0),
scan_inst->src[i], scan_inst->size_read(i)))
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
interfered = true;
}
if (interfered)
break;
i965/vec4: check writemask when bailing out at register coalesce opt_register_coalesce stopped to check previous instructions to coalesce with if somebody else was writing on the same destination. This can be optimized to check if somebody else was writing to the same channels of the same destination using the writemask. Shader DB results (taking into account only vec4): total instructions in shared programs: 1781593 -> 1734957 (-2.62%) instructions in affected programs: 1238390 -> 1191754 (-3.77%) helped: 12782 HURT: 0 GAINED: 0 LOST: 0 v2: removed some parenthesis, fixed indentation, as suggested by Matt Turner v3: added brackets, for consistency, as suggested by Eduardo Lima Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-11 12:21:13 +02:00
/* If somebody else writes the same channels of our destination here,
* we can't coalesce before that.
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
*/
i965/vec4: Drop backend_reg::in_range() in favor of regions_overlap(). This makes sure that overlap checks are done correctly throughout the back-end when the '*this' register starts before the register/size pair provided as argument, and is actually less annoying to use than in_range() at this point since regions_overlap() takes its size arguments in bytes. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 19:42:40 -07:00
if (regions_overlap(inst->dst, inst->size_written,
scan_inst->dst, scan_inst->size_written) &&
i965/vec4: check writemask when bailing out at register coalesce opt_register_coalesce stopped to check previous instructions to coalesce with if somebody else was writing on the same destination. This can be optimized to check if somebody else was writing to the same channels of the same destination using the writemask. Shader DB results (taking into account only vec4): total instructions in shared programs: 1781593 -> 1734957 (-2.62%) instructions in affected programs: 1238390 -> 1191754 (-3.77%) helped: 12782 HURT: 0 GAINED: 0 LOST: 0 v2: removed some parenthesis, fixed indentation, as suggested by Matt Turner v3: added brackets, for consistency, as suggested by Eduardo Lima Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-11 12:21:13 +02:00
(inst->dst.writemask & scan_inst->dst.writemask) != 0) {
break;
}
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* Check for reads of the register we're trying to coalesce into. We
* can't go rewriting instructions above that to put some other value
* in the register instead.
*/
if (to_mrf && scan_inst->mlen > 0) {
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
if (inst->dst.nr >= scan_inst->base_mrf &&
inst->dst.nr < scan_inst->base_mrf + scan_inst->mlen) {
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
break;
}
} else {
for (int i = 0; i < 3; i++) {
i965/vec4: Drop backend_reg::in_range() in favor of regions_overlap(). This makes sure that overlap checks are done correctly throughout the back-end when the '*this' register starts before the register/size pair provided as argument, and is actually less annoying to use than in_range() at this point since regions_overlap() takes its size arguments in bytes. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 19:42:40 -07:00
if (regions_overlap(inst->dst, inst->size_written,
scan_inst->src[i], scan_inst->size_read(i)))
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
interfered = true;
}
if (interfered)
break;
}
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
}
if (chans_remaining == 0) {
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
/* If we've made it here, we have an MOV we want to coalesce out, and
* a scan_inst pointing to the earliest instruction involved in
* computing the value. Now go rewrite the instruction stream
* between the two.
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
*/
i965/vec4: Don't iterate between blocks with inst->next/prev. The register coalescing portion of this patch hurts three shaders in Guacamelee by one instruction each, but examining the diff makes me believe that what we were generating was (perhaps harmlessly) incorrect.
2014-09-02 14:43:43 -07:00
vec4_instruction *scan_inst = _scan_inst;
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
while (scan_inst != inst) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
if (scan_inst->dst.file == VGRF &&
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
scan_inst->dst.nr == inst->src[0].nr &&
i965/vec4: Don't coalesce registers with overlapping writes not matching the MOV source. In preparation for adding support for sub-GRF offsets to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:08:29 -07:00
scan_inst->dst.offset == inst->src[0].offset) {
i965/vec4: Reswizzle sources when necessary. Despite the comment above the function claiming otherwise, the function did not reswizzle sources, which would lead to bad code generation since commit 04895f5c, which began claiming we could do such swizzling when we could not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82932 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-31 11:10:02 -07:00
scan_inst->reswizzle(inst->dst.writemask,
inst->src[0].swizzle);
i965: Generalize VS compute-to-MRF for compute-to-another-GRF, too. No statistically significant performance difference on glbenchmark 2.7 (n=60). It reduces cycles spent in the vertex shader by 3.3% +/- 0.8% (n=5), but that's only about .3% of all cycles spent according to the fixed shader_time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-01 19:35:18 -07:00
scan_inst->dst.file = inst->dst.file;
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
scan_inst->dst.nr = inst->dst.nr;
i965/vec4: Assign correct destination offset to rewritten instruction in register coalesce. Because the pass already checks that the destination offset of each 'scan_inst' that needs to be rewritten matches 'inst->src[0].offset' exactly, the final offset of the rewritten instruction is just the original destination offset of the copy. This is in preparation for adding support for sub-GRF offsets to the VEC4 IR. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 22:12:04 -07:00
scan_inst->dst.offset = inst->dst.offset;
i965/vec4: Fix saturation errors when coalescing registers If the register types do not match and the instruction that contains the final destination is saturated, register coalescing generated non-equivalent code. This did not happen when using IR because types usually matched, but it is visible in nir-vec4. For example, mov vgrf7:D vgrf2:D mov.sat m4:F vgrf7:F is coalesced to: mov.sat m4:D vgrf2:D The patch prevents coalescing in such scenario, unless the instruction we want to coalesce into is a MOV (without type conversion implied). In that case, the patch sets the register types to the type of the final destination. Shader-db results in HSW (only vec4 instructions shown): total instructions in shared programs: 1754415 -> 1754416 (0.00%) instructions in affected programs: 74 -> 75 (1.35%) helped: 0 HURT: 1 GAINED: 0 LOST: 0 Only one extra instruction in one of the shaders, that comes from eliminating a saturation error by preventing register coalesce. Cc: "10.6 11.0" <mesa-stable@lists.freedesktop.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-08-05 15:57:33 +02:00
if (inst->saturate &&
inst->dst.type != scan_inst->dst.type) {
/* If we have reached this point, scan_inst is a non
* type-converting 'mov' and we can modify its register types
* to match the ones in inst. Otherwise, we could have an
* incorrect saturation result.
*/
scan_inst->dst.type = inst->dst.type;
scan_inst->src[0].type = inst->src[0].type;
}
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
scan_inst->saturate |= inst->saturate;
}
scan_inst = (vec4_instruction *)scan_inst->next;
}
i965: Use basic-block aware insertion/removal functions. To avoid invalidating and recreating the control flow graph. Also stop invalidating the CFG in places we didn't add or remove an instruction. cfg calculations: 202951 -> 80307 (-60.43%) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-07-12 21:18:39 -07:00
inst->remove(block);
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
progress = true;
}
}
if (progress)
i965: Remove cfg-invalidating parameter from invalidate_live_intervals. Everything has been converted to preserve the CFG. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 10:54:00 -07:00
invalidate_live_intervals();
i965/vs: Add support for compute-to-MRF. Removes 1.8% of the instructions from 97% of the vertex shaders in shader-db.
2011-09-06 13:21:24 -07:00
return progress;
}
i965: Perform basic optimizations on the FIND_LIVE_CHANNEL opcode. v2: Save some CPU cycles by doing 'return progress' rather than 'depth++' in the discard jump special case. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-20 20:25:04 +02:00
/**
* Eliminate FIND_LIVE_CHANNEL instructions occurring outside any control
* flow. We could probably do better here with some form of divergence
* analysis.
*/
bool
vec4_visitor::eliminate_find_live_channel()
{
bool progress = false;
unsigned depth = 0;
i965/ir: Skip eliminate_find_live_channel() for stages with sparse thread dispatch. The eliminate_find_live_channel optimization eliminates FIND_LIVE_CHANNEL instructions in cases where control flow is known to be uniform, and replaces them with 'MOV 0', which in turn unblocks subsequent elimination of the BROADCAST instruction frequently used on the result of FIND_LIVE_CHANNEL. This is however not correct in per-sample fragment shader dispatch because the PSD can dispatch a fully unlit sample under certain conditions. Disable the optimization in that case. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> v2: Add devinfo argument to brw_stage_has_packed_dispatch() to implement hardware generation check.
2016-09-15 17:20:23 -07:00
if (!brw_stage_has_packed_dispatch(devinfo, stage, stage_prog_data)) {
/* The optimization below assumes that channel zero is live on thread
* dispatch, which may not be the case if the fixed function dispatches
* threads sparsely.
*/
return false;
}
i965: Perform basic optimizations on the FIND_LIVE_CHANNEL opcode. v2: Save some CPU cycles by doing 'return progress' rather than 'depth++' in the discard jump special case. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-20 20:25:04 +02:00
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
switch (inst->opcode) {
case BRW_OPCODE_IF:
case BRW_OPCODE_DO:
depth++;
break;
case BRW_OPCODE_ENDIF:
case BRW_OPCODE_WHILE:
depth--;
break;
case SHADER_OPCODE_FIND_LIVE_CHANNEL:
if (depth == 0) {
inst->opcode = BRW_OPCODE_MOV;
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
inst->src[0] = brw_imm_d(0);
i965: Perform basic optimizations on the FIND_LIVE_CHANNEL opcode. v2: Save some CPU cycles by doing 'return progress' rather than 'depth++' in the discard jump special case. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-20 20:25:04 +02:00
inst->force_writemask_all = true;
progress = true;
}
break;
default:
break;
}
}
return progress;
}
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
/**
* Splits virtual GRFs requesting more than one contiguous physical register.
*
* We initially create large virtual GRFs for temporary structures, arrays,
i965/ir: Update several stale comments. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-02 13:53:13 -07:00
* and matrices, so that the visitor functions can add offsets to work their
* way down to the actual member being accessed. But when it comes to
* optimization, we'd like to treat each register as individual storage if
* possible.
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
*
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
* So far, the only thing that might prevent splitting is a send message from
* a GRF on IVB.
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
*/
void
vec4_visitor::split_virtual_grfs()
{
i965: Factor out virtual GRF allocation to a separate object. Right now virtual GRF book-keeping and allocation is performed in each visitor class separately (among other hundred different things), leading to duplicated logic in each visitor and preventing layering as it forces any code that manipulates i965 IR and needs to allocate virtual registers to depend on the specific visitor that happens to be used to translate from GLSL IR. v2: Use realloc()/free() to allocate VGRF book-keeping arrays (Connor). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-10 15:51:34 +02:00
int num_vars = this->alloc.count;
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
int new_virtual_grf[num_vars];
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
bool split_grf[num_vars];
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
memset(new_virtual_grf, 0, sizeof(new_virtual_grf));
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
/* Try to split anything > 0 sized. */
for (int i = 0; i < num_vars; i++) {
i965: Factor out virtual GRF allocation to a separate object. Right now virtual GRF book-keeping and allocation is performed in each visitor class separately (among other hundred different things), leading to duplicated logic in each visitor and preventing layering as it forces any code that manipulates i965 IR and needs to allocate virtual registers to depend on the specific visitor that happens to be used to translate from GLSL IR. v2: Use realloc()/free() to allocate VGRF book-keeping arrays (Connor). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-10 15:51:34 +02:00
split_grf[i] = this->alloc.sizes[i] != 1;
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
}
/* Check that the instructions are compatible with the registers we're trying
* to split.
*/
i965/vec4: Don't use instruction list after calculating the cfg. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 13:42:51 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
i965/vec4: Add wrapper functions for vec4_instruction::regs_read and ::regs_written. This is in preparation for dropping vec4_instruction::regs_read and ::regs_written in favor of more accurate alternatives expressed in byte units. The main reason these wrappers are useful is that a number of optimization passes implement dataflow analysis with register granularity, so these helpers will come in handy once we've switched register offsets and sizes to the byte representation. The wrapper functions will also make sure that GRF misalignment (currently neglected by most of the back-end) is taken into account correctly in the calculation of regs_read and regs_written. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 16:55:46 -07:00
if (inst->dst.file == VGRF && regs_written(inst) > 1)
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
split_grf[inst->dst.nr] = false;
i965/vec4: Fix handling of multiple register reads and writes in split_virtual_grfs(). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-03-18 19:45:40 +02:00
for (int i = 0; i < 3; i++) {
i965/vec4: Add wrapper functions for vec4_instruction::regs_read and ::regs_written. This is in preparation for dropping vec4_instruction::regs_read and ::regs_written in favor of more accurate alternatives expressed in byte units. The main reason these wrappers are useful is that a number of optimization passes implement dataflow analysis with register granularity, so these helpers will come in handy once we've switched register offsets and sizes to the byte representation. The wrapper functions will also make sure that GRF misalignment (currently neglected by most of the back-end) is taken into account correctly in the calculation of regs_read and regs_written. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 16:55:46 -07:00
if (inst->src[i].file == VGRF && regs_read(inst, i) > 1)
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
split_grf[inst->src[i].nr] = false;
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
}
}
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
/* Allocate new space for split regs. Note that the virtual
* numbers will be contiguous.
*/
for (int i = 0; i < num_vars; i++) {
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
if (!split_grf[i])
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
continue;
i965: Factor out virtual GRF allocation to a separate object. Right now virtual GRF book-keeping and allocation is performed in each visitor class separately (among other hundred different things), leading to duplicated logic in each visitor and preventing layering as it forces any code that manipulates i965 IR and needs to allocate virtual registers to depend on the specific visitor that happens to be used to translate from GLSL IR. v2: Use realloc()/free() to allocate VGRF book-keeping arrays (Connor). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-10 15:51:34 +02:00
new_virtual_grf[i] = alloc.allocate(1);
for (unsigned j = 2; j < this->alloc.sizes[i]; j++) {
unsigned reg = alloc.allocate(1);
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
assert(reg == new_virtual_grf[i] + j - 1);
(void) reg;
}
i965: Factor out virtual GRF allocation to a separate object. Right now virtual GRF book-keeping and allocation is performed in each visitor class separately (among other hundred different things), leading to duplicated logic in each visitor and preventing layering as it forces any code that manipulates i965 IR and needs to allocate virtual registers to depend on the specific visitor that happens to be used to translate from GLSL IR. v2: Use realloc()/free() to allocate VGRF book-keeping arrays (Connor). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-10 15:51:34 +02:00
this->alloc.sizes[i] = 1;
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
}
i965/vec4: Don't use instruction list after calculating the cfg. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 13:42:51 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
if (inst->dst.file == VGRF && split_grf[inst->dst.nr] &&
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
inst->dst.offset / REG_SIZE != 0) {
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
inst->dst.nr = (new_virtual_grf[inst->dst.nr] +
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
inst->dst.offset / REG_SIZE - 1);
inst->dst.offset %= REG_SIZE;
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
}
for (int i = 0; i < 3; i++) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
if (inst->src[i].file == VGRF && split_grf[inst->src[i].nr] &&
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
inst->src[i].offset / REG_SIZE != 0) {
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
inst->src[i].nr = (new_virtual_grf[inst->src[i].nr] +
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
inst->src[i].offset / REG_SIZE - 1);
inst->src[i].offset %= REG_SIZE;
i965/vs: Prepare split_virtual_grfs() for the presence of SENDs from GRFs. v2: Fix silly bool handling, and don't add new tabs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 16:48:20 -08:00
}
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
}
}
i965: Remove cfg-invalidating parameter from invalidate_live_intervals. Everything has been converted to preserve the CFG. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 10:54:00 -07:00
invalidate_live_intervals();
i965/vs: Add support for splitting virtual GRFs. This should improve our ability to register allocate without spilling. Unfortuantely, due to the live variable analysis being ignorant of loops, we still have register allocation failures on some programs. v2: Add more context to the comment explaining the function. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2012-10-03 10:04:22 -07:00
}
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
void
i965: Make dump_instructions be a virtual method of the visitor. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-04-29 14:21:14 -07:00
vec4_visitor::dump_instruction(backend_instruction *be_inst)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
{
dump_instruction(be_inst, stderr);
}
void
vec4_visitor::dump_instruction(backend_instruction *be_inst, FILE *file)
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
{
i965: Make dump_instructions be a virtual method of the visitor. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-04-29 14:21:14 -07:00
vec4_instruction *inst = (vec4_instruction *)be_inst;
i965/vec4: Print the predicate in dump_instructions(). Reviewed-by: Eric Anholt <eric@anholt.net>
2014-03-13 11:22:08 -07:00
if (inst->predicate) {
i965/vec4: print predicate control at brw_vec4 dump_instruction v2: externalize pred_ctrl_align16 from brw_disasm.c instead of adding a copy on brw_vec4.c, as suggested by Matt Turner Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-09 18:39:42 +02:00
fprintf(file, "(%cf0.%d%s) ",
i965: Move up fs_inst::flag_subreg to backend_instruction. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-06 01:28:12 +02:00
inst->predicate_inverse ? '-' : '+',
i965/vec4: print predicate control at brw_vec4 dump_instruction v2: externalize pred_ctrl_align16 from brw_disasm.c instead of adding a copy on brw_vec4.c, as suggested by Matt Turner Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-09 18:39:42 +02:00
inst->flag_subreg,
pred_ctrl_align16[inst->predicate]);
i965/vec4: Print the predicate in dump_instructions(). Reviewed-by: Eric Anholt <eric@anholt.net>
2014-03-13 11:22:08 -07:00
}
i965/vec4: dump the instruction execution size Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-30 13:36:30 +02:00
fprintf(file, "%s(%d)", brw_instruction_name(devinfo, inst->opcode),
inst->exec_size);
i965/vec4: Handle saturate in dump_instruction(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-03-05 11:16:07 -08:00
if (inst->saturate)
fprintf(file, ".sat");
i965: Print conditional mod in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-10-20 15:48:14 -07:00
if (inst->conditional_mod) {
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "%s", conditional_modifier[inst->conditional_mod]);
i965: Move up fs_inst::flag_subreg to backend_instruction. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-06 01:28:12 +02:00
if (!inst->predicate &&
i965: Add a devinfo field to backend_visitor and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 18:00:05 -07:00
(devinfo->gen < 5 || (inst->opcode != BRW_OPCODE_SEL &&
inst->opcode != BRW_OPCODE_IF &&
inst->opcode != BRW_OPCODE_WHILE))) {
i965: Move up fs_inst::flag_subreg to backend_instruction. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-06 01:28:12 +02:00
fprintf(file, ".f0.%d", inst->flag_subreg);
}
i965: Print conditional mod in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-10-20 15:48:14 -07:00
}
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, " ");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
switch (inst->dst.file) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
case VGRF:
i965/vec4: Print src/dst_reg::offset field consistently for all register files. C.f. 'i965/fs: Print fs_reg::offset field consistently for all register files.'. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-02 13:52:54 -07:00
fprintf(file, "vgrf%d", inst->dst.nr);
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
case FIXED_GRF:
fprintf(file, "g%d", inst->dst.nr);
break;
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
case MRF:
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
fprintf(file, "m%d", inst->dst.nr);
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
case ARF:
switch (inst->dst.nr) {
case BRW_ARF_NULL:
fprintf(file, "null");
break;
case BRW_ARF_ADDRESS:
fprintf(file, "a0.%d", inst->dst.subnr);
break;
case BRW_ARF_ACCUMULATOR:
fprintf(file, "acc%d", inst->dst.subnr);
break;
case BRW_ARF_FLAG:
fprintf(file, "f%d.%d", inst->dst.nr & 0xf, inst->dst.subnr);
break;
default:
fprintf(file, "arf%d.%d", inst->dst.nr & 0xf, inst->dst.subnr);
break;
i965/vec4: Add support for printing HW_REGs in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-11-25 22:17:29 -08:00
}
break;
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
case BAD_FILE:
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "(null)");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
i965: Replace default case with list of enum values. If we add a new file type, we'd like to get warnings if it's not handled. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-10-26 06:58:56 -07:00
case IMM:
case ATTR:
case UNIFORM:
unreachable("not reached");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
}
i965/vec4: Print src/dst_reg::offset field consistently for all register files. C.f. 'i965/fs: Print fs_reg::offset field consistently for all register files.'. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-02 13:52:54 -07:00
if (inst->dst.offset ||
(inst->dst.file == VGRF &&
alloc.sizes[inst->dst.nr] * REG_SIZE != inst->size_written)) {
const unsigned reg_size = (inst->dst.file == UNIFORM ? 16 : REG_SIZE);
fprintf(file, "+%d.%d", inst->dst.offset / reg_size,
inst->dst.offset % reg_size);
}
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
if (inst->dst.writemask != WRITEMASK_XYZW) {
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, ".");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
if (inst->dst.writemask & 1)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "x");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
if (inst->dst.writemask & 2)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "y");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
if (inst->dst.writemask & 4)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "z");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
if (inst->dst.writemask & 8)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "w");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
}
i965/vec4: slightly improve insn dumping with no srcs Previously, we would get a trailing ', ' which looked strange. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-09-09 19:55:28 +12:00
fprintf(file, ":%s", brw_reg_type_letters(inst->dst.type));
if (inst->src[0].file != BAD_FILE)
fprintf(file, ", ");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
i965: Don't print extra (null) arguments in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-11-25 15:15:25 -08:00
for (int i = 0; i < 3 && inst->src[i].file != BAD_FILE; i++) {
i965/vec4: Print negate and absolute value for src args. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:41:16 -08:00
if (inst->src[i].negate)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "-");
i965/vec4: Print negate and absolute value for src args. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:41:16 -08:00
if (inst->src[i].abs)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "|");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
switch (inst->src[i].file) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
case VGRF:
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
fprintf(file, "vgrf%d", inst->src[i].nr);
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
case FIXED_GRF:
i965/vec4: dump subnr for FIXED_GRF This came in handy when debugging the payload setup for Tess Eval, since it prints correct subnr for attributes that can be loaded in the second half of a register. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-01 09:28:31 +02:00
fprintf(file, "g%d.%d", inst->src[i].nr, inst->src[i].subnr);
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
break;
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
case ATTR:
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
fprintf(file, "attr%d", inst->src[i].nr);
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
case UNIFORM:
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 04:35:14 -07:00
fprintf(file, "u%d", inst->src[i].nr);
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
case IMM:
switch (inst->src[i].type) {
case BRW_REGISTER_TYPE_F:
i965: Use immediate storage in inherited brw_reg. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 14:55:57 -07:00
fprintf(file, "%fF", inst->src[i].f);
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
break;
i965/vec4: add support for printing DF immediates Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-08-13 14:18:04 -07:00
case BRW_REGISTER_TYPE_DF:
fprintf(file, "%fDF", inst->src[i].df);
break;
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
case BRW_REGISTER_TYPE_D:
i965: Use immediate storage in inherited brw_reg. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 14:55:57 -07:00
fprintf(file, "%dD", inst->src[i].d);
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
break;
case BRW_REGISTER_TYPE_UD:
i965: Use immediate storage in inherited brw_reg. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 14:55:57 -07:00
fprintf(file, "%uU", inst->src[i].ud);
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
break;
i965/vec4: Add vector float immediate infrastructure. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-03-08 17:22:22 -08:00
case BRW_REGISTER_TYPE_VF:
i965: Fix INTEL_DEBUG=optimizer with VF types. Hardcoding stderr is wrong; INTEL_DEBUG=optimizer uses other files. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-31 16:54:44 -08:00
fprintf(file, "[%-gF, %-gF, %-gF, %-gF]",
i965: Use immediate storage in inherited brw_reg. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 14:55:57 -07:00
brw_vf_to_float((inst->src[i].ud >> 0) & 0xff),
brw_vf_to_float((inst->src[i].ud >> 8) & 0xff),
brw_vf_to_float((inst->src[i].ud >> 16) & 0xff),
brw_vf_to_float((inst->src[i].ud >> 24) & 0xff));
i965/vec4: Add vector float immediate infrastructure. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-03-08 17:22:22 -08:00
break;
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
default:
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "???");
i965/vs: Add IR dumping for immediates. This makes dump_instructions more useful. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-11 11:10:34 -07:00
break;
}
break;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
case ARF:
switch (inst->src[i].nr) {
case BRW_ARF_NULL:
fprintf(file, "null");
break;
case BRW_ARF_ADDRESS:
fprintf(file, "a0.%d", inst->src[i].subnr);
break;
case BRW_ARF_ACCUMULATOR:
fprintf(file, "acc%d", inst->src[i].subnr);
break;
case BRW_ARF_FLAG:
fprintf(file, "f%d.%d", inst->src[i].nr & 0xf, inst->src[i].subnr);
break;
default:
fprintf(file, "arf%d.%d", inst->src[i].nr & 0xf, inst->src[i].subnr);
break;
i965/vec4: Add support for printing HW_REGs in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-11-25 22:17:29 -08:00
}
break;
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
case BAD_FILE:
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "(null)");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
break;
i965: Replace default case with list of enum values. If we add a new file type, we'd like to get warnings if it's not handled. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-10-26 06:58:56 -07:00
case MRF:
unreachable("not reached");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
}
i965/vec4: Print src/dst_reg::offset field consistently for all register files. C.f. 'i965/fs: Print fs_reg::offset field consistently for all register files.'. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-02 13:52:54 -07:00
if (inst->src[i].offset ||
(inst->src[i].file == VGRF &&
alloc.sizes[inst->src[i].nr] * REG_SIZE != inst->size_read(i))) {
const unsigned reg_size = (inst->src[i].file == UNIFORM ? 16 : REG_SIZE);
fprintf(file, "+%d.%d", inst->src[i].offset / reg_size,
inst->src[i].offset % reg_size);
}
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
i965/vec4: Don't print swizzles for immediate values. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:43:50 -08:00
if (inst->src[i].file != IMM) {
static const char *chans[4] = {"x", "y", "z", "w"};
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, ".");
i965/vec4: Don't print swizzles for immediate values. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:43:50 -08:00
for (int c = 0; c < 4; c++) {
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "%s", chans[BRW_GET_SWZ(inst->src[i].swizzle, c)]);
i965/vec4: Don't print swizzles for immediate values. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:43:50 -08:00
}
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
}
i965/vec4: Print negate and absolute value for src args. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:41:16 -08:00
if (inst->src[i].abs)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "|");
i965/vec4: Print negate and absolute value for src args. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 12:41:16 -08:00
i965: Print argument types in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 13:10:29 -08:00
if (inst->src[i].file != IMM) {
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, ":%s", brw_reg_type_letters(inst->src[i].type));
i965: Print argument types in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-12-02 13:10:29 -08:00
}
i965: Don't print extra (null) arguments in dump_instruction(). Reviewed-by: Eric Anholt <eric@anholt.net>
2013-11-25 15:15:25 -08:00
if (i < 2 && inst->src[i + 1].file != BAD_FILE)
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, ", ");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
}
i965: Print force_writemask_all in dump_instructions(). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-09 23:55:58 -08:00
if (inst->force_writemask_all)
fprintf(file, " NoMask");
i965/vec4: dump NibCtrl for instructions with execsize != 8 v2: do it in the same fashion as the FS backend for consistency (Curro) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-29 08:40:27 +02:00
if (inst->exec_size != 8)
fprintf(file, " group%d", inst->group);
i965: Give dump_instruction() a FILE* argument. Use function overloading rather than default arguments, since gdb doesn't know about default arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-29 11:45:15 -07:00
fprintf(file, "\n");
i965/vs: Add a little bit of IR-level debug ability. This is super basic, but it let me visualize a problem I had with opt_compute_to_mrf(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 16:11:26 -07:00
}
i965/vec4: Add the ability for attributes to be interleaved. When geometry shaders are operated in "single" or "dual instanced" mode, a single set of geometry shader inputs is interleaved into the thread payload (with each payload register containing a pair of inputs) in order to save register space. This patch modifies vec4_visitor::lower_attributes_to_hw_regs so that it can handle the interleaved format. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-10-16 11:40:41 -07:00
i965/vs: Generalize attribute setup code in preparation for GS. This patch introduces a new function, vec4_visitor::lower_attributes_to_hw_regs(), which replaces registers of type ATTR in the instruction stream with the hardware registers that store those attributes. This logic will need to be common between the vertex and geometry shaders. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-22 08:47:49 -07:00
int
vec4_vs_visitor::setup_attributes(int payload_reg)
{
i965/vec4: Use NIR remapping for VS attributes The NIR pass already handles remapping system values to attributes for us so we delete the system value code as part of the conversion. We also change nir_lower_vs_inputs to take an explicit inputs_read bitmask and pass in the inputs_read from prog_data instead from pulling it out of NIR. This is because the version in prog_data may get EDGEFLAG added to it on some old platforms. Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-03 17:40:54 -07:00
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
for (int i = 0; i < 3; i++) {
if (inst->src[i].file == ATTR) {
assert(inst->src[i].offset % REG_SIZE == 0);
int grf = payload_reg + inst->src[i].nr +
inst->src[i].offset / REG_SIZE;
struct brw_reg reg = brw_vec8_grf(grf, 0);
reg.swizzle = inst->src[i].swizzle;
reg.type = inst->src[i].type;
reg.abs = inst->src[i].abs;
reg.negate = inst->src[i].negate;
inst->src[i] = reg;
}
i965/vs: Generalize attribute setup code in preparation for GS. This patch introduces a new function, vec4_visitor::lower_attributes_to_hw_regs(), which replaces registers of type ATTR in the instruction stream with the hardware registers that store those attributes. This logic will need to be common between the vertex and geometry shaders. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-22 08:47:49 -07:00
}
}
i965/vec4: use attribute slots for first non payload GRF As part of the payload setup, setup_attributes is called with the first GRF that can be used for the attributes (first ones are used for uniforms for example) and returns the first GRF that is not part of the payload. Before this patch, it adds directly the number of attributes. But as with 64-bit attributes can consume more than one slot, that is not valid anymore. This patch change the addition to use the number of slots consumed. gen >= 8 would not be affected, as they use the scalar mode. For that case, the vs configuration is done at fs_visitor::assign_vs_urb_setup. v2: add explanation in commit log (Jordan) Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-04-21 12:23:47 +02:00
return payload_reg + vs_prog_data->nr_attribute_slots;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
}
int
vec4_visitor::setup_uniforms(int reg)
{
i965: Move dispatch_grf_start_reg and first_curbe_grf into stage_prog_data. I wanted to access this value from stage-generic code, so stop storing it under two different names. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-19 10:10:01 -07:00
prog_data->base.dispatch_grf_start_reg = reg;
i965/vec4: Allow for dispatch_grf_start_reg to vary. Both 3DSTATE_VS and 3DSTATE_GS have a dispatch_grf_start_reg control, which determines the register where the hardware delivers data sourced from the URB (push constants followed by per-vertex input data). For vertex shaders, we always set dispatch_grf_start_reg to 1, since R1 is always the first register available for push constants in vertex shaders. For geometry shaders, we'll need the flexibility to set dispatch_grf_start_reg to different values depending on the behvaiour of the geometry shader; if it accesses gl_PrimitiveIDIn, we'll need to set it to 2 to allow the primitive ID to be delivered to the thread in R1. This patch eliminates the assumption that dispatch_grf_start_reg is always 1. In vec4_visitor, we record the regnum that was passed to vec4_visitor::setup_uniforms() in prog_data for later use. In vec4_generator, we consult this value when converting an abstract UNIFORM register to a concrete hardware register. And in the code that emits 3DSTATE_VS, we set dispatch_grf_start_reg based on the value recorded in prog_data. This will allow us to set dispatch_grf_start_reg to the appropriate value when compiling geometry shaders. Vertex shaders will continue to always use a dispatch_grf_start_reg of 1. v2: Make dispatch_grf_start_reg "unsigned" rather than "GLuint". Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-07-13 07:09:54 -07:00
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
/* The pre-gen6 VS requires that some push constants get loaded no
* matter what, or the GPU would hang.
*/
i965: Add a devinfo field to backend_visitor and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 18:00:05 -07:00
if (devinfo->gen < 6 && this->uniforms == 0) {
i965: Move up duplicated fields from stage-specific prog_data to brw_stage_prog_data. There doesn't seem to be any reason for nr_params, nr_pull_params, param, and pull_param to be duplicated in the stage-specific subclasses of brw_stage_prog_data. Moving their definition to the common base class will allow some code sharing in a future commit, the removal of brw_vec4_prog_data_compare and brw_*_prog_data_free, and the simplification of the stage-specific brw_*_prog_data_compare. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2014-02-19 15:14:02 +01:00
stage_prog_data->param =
i965: Store uniform constant values in a gl_constant_value instead of float The brw_stage_prog_data struct previously contained an array of float pointers to the values of parameters. These were then copied into a batch buffer to upload the values using a regular assignment. However the float values were also being overloaded to store integer values for integer uniforms. This can break if x87 floating-point registers are used to do the assignment because the fst instruction tries to fix up invalid float values. If an integer constant happened to look like an invalid float value then it would get altered when it was copied into the batch buffer. This patch changes the pointers to be gl_constant_value instead so that the assignment should end up copying without any alteration. This also makes it more obvious that the values being stored here are overloaded for multiple types. There are some static asserts where the values are uploaded to ensure that the size of gl_constant_value is the same as a float. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=81150 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-11 12:21:44 +01:00
reralloc(NULL, stage_prog_data->param, const gl_constant_value *, 4);
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
for (unsigned int i = 0; i < 4; i++) {
unsigned int slot = this->uniforms * 4 + i;
i965: fix compiler error in union initiliazer gcc 4.6.3 chokes with the following error: brw_vec4.cpp: In member function 'int brw::vec4_visitor::setup_uniforms(int)': brw_vec4.cpp:1496:37: error: expected primary-expression before '.' token Apparently C++ does not do named initializers for unions, except maybe as a gcc extension, which is not present here. As .f is the first element of the union, just drop it. Fixes the build error. Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-14 21:17:04 +03:00
static gl_constant_value zero = { 0.0 };
i965: Move up duplicated fields from stage-specific prog_data to brw_stage_prog_data. There doesn't seem to be any reason for nr_params, nr_pull_params, param, and pull_param to be duplicated in the stage-specific subclasses of brw_stage_prog_data. Moving their definition to the common base class will allow some code sharing in a future commit, the removal of brw_vec4_prog_data_compare and brw_*_prog_data_free, and the simplification of the stage-specific brw_*_prog_data_compare. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2014-02-19 15:14:02 +01:00
stage_prog_data->param[slot] = &zero;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
}
this->uniforms++;
reg++;
} else {
reg += ALIGN(uniforms, 2) / 2;
}
i965: Move up duplicated fields from stage-specific prog_data to brw_stage_prog_data. There doesn't seem to be any reason for nr_params, nr_pull_params, param, and pull_param to be duplicated in the stage-specific subclasses of brw_stage_prog_data. Moving their definition to the common base class will allow some code sharing in a future commit, the removal of brw_vec4_prog_data_compare and brw_*_prog_data_free, and the simplification of the stage-specific brw_*_prog_data_compare. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2014-02-19 15:14:02 +01:00
stage_prog_data->nr_params = this->uniforms * 4;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
i965: Move curb_read_length/total_scratch to brw_stage_prog_data. All shader stages have these fields, so it makes sense to store them in the common base structure, rather than duplicating them in each. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-09-02 11:38:29 -07:00
prog_data->base.curb_read_length =
reg - prog_data->base.dispatch_grf_start_reg;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
return reg;
}
void
i965/vec4: Virtualize setup_payload instead of setup_attributes. When I initially generalized the vec4_visitor class in preparation for geometry shaders, I assumed that the setup_attributes() function would need to be different between vertex and geometry shaders, but its caller, setup_payload(), could be shared. So I made setup_attributes() a virtual function. It turns out this isn't true; setup_payload() needs to be different too, since the geometry shader payload sometimes includes an extra register (primitive ID) that has to come before uniforms. So setup_payload() needs to be the virtual function instead of setup_attributes(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-07-13 09:03:18 -07:00
vec4_vs_visitor::setup_payload(void)
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
{
int reg = 0;
/* The payload always contains important data in g0, which contains
* the URB handles that are passed on to the URB write at the end
* of the thread. So, we always start push constants at g1.
*/
reg++;
reg = setup_uniforms(reg);
reg = setup_attributes(reg);
this->first_non_payload_grf = reg;
}
i965: Lower min/max after optimization on Gen4/5. Gen4/5's SEL instruction cannot use conditional modifiers, so min/max are implemented as CMP + SEL. Handling that after optimization lets us CSE more. On Ironlake: total instructions in shared programs: 6426035 -> 6422753 (-0.05%) instructions in affected programs: 326604 -> 323322 (-1.00%) helped: 1411 total cycles in shared programs: 129184700 -> 129101586 (-0.06%) cycles in affected programs: 18950290 -> 18867176 (-0.44%) helped: 2419 HURT: 328 Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-02-11 12:27:02 -08:00
bool
vec4_visitor::lower_minmax()
{
assert(devinfo->gen < 6);
bool progress = false;
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
const vec4_builder ibld(this, block, inst);
if (inst->opcode == BRW_OPCODE_SEL &&
inst->predicate == BRW_PREDICATE_NONE) {
/* FIXME: Using CMP doesn't preserve the NaN propagation semantics of
* the original SEL.L/GE instruction
*/
ibld.CMP(ibld.null_reg_d(), inst->src[0], inst->src[1],
inst->conditional_mod);
inst->predicate = BRW_PREDICATE_NORMAL;
inst->conditional_mod = BRW_CONDITIONAL_NONE;
progress = true;
}
}
if (progress)
invalidate_live_intervals();
return progress;
}
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
src_reg
vec4_visitor::get_timestamp()
{
i965: Add a devinfo field to backend_visitor and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 18:00:05 -07:00
assert(devinfo->gen >= 7);
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
src_reg ts = src_reg(brw_reg(BRW_ARCHITECTURE_REGISTER_FILE,
BRW_ARF_TIMESTAMP,
0,
i965/brw_reg: struct constructor now needs explicit negate and abs values. We were assuming, when constructing a new brw_reg struct, that the negate and abs register modifiers would not be present by default in the new register. Now, we force explicitly setting these values when constructing a new register. This will avoid problems like forgetting to properly set them when we are using a previous register to generate this new register, as it was happening in the dFdx and dFdy generation functions. Fixes piglit test shaders/glsl-deriv-varyings Cc: "10.4 10.3" <mesa-stable@lists.freedesktop.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82991 Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-12 17:19:07 +01:00
0,
0,
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
BRW_REGISTER_TYPE_UD,
BRW_VERTICAL_STRIDE_0,
BRW_WIDTH_4,
BRW_HORIZONTAL_STRIDE_4,
BRW_SWIZZLE_XYZW,
WRITEMASK_XYZW));
dst_reg dst = dst_reg(this, glsl_type::uvec4_type);
vec4_instruction *mov = emit(MOV(dst, ts));
/* We want to read the 3 fields we care about (mostly field 0, but also 2)
* even if it's not enabled in the dispatch.
*/
mov->force_writemask_all = true;
return src_reg(dst);
}
void
vec4_visitor::emit_shader_time_begin()
{
current_annotation = "shader time start";
shader_start_time = get_timestamp();
}
void
vec4_visitor::emit_shader_time_end()
{
current_annotation = "shader time end";
src_reg shader_end_time = get_timestamp();
/* Check that there weren't any timestamp reset events (assuming these
* were the only two timestamp reads that happened).
*/
i965: Adjust the split between shader_time_end() and shader_time_write(). I'm about to emit other kinds of writes besides time deltas, and it turns out with the frequency of resets, we couldn't really use the old time delta write() function more than once in a shader.
2012-12-10 09:21:34 -08:00
src_reg reset_end = shader_end_time;
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
reset_end.swizzle = BRW_SWIZZLE_ZZZZ;
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
vec4_instruction *test = emit(AND(dst_null_ud(), reset_end, brw_imm_ud(1u)));
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
test->conditional_mod = BRW_CONDITIONAL_Z;
emit(IF(BRW_PREDICATE_NORMAL));
/* Take the current timestamp and get the delta. */
i965: Adjust the split between shader_time_end() and shader_time_write(). I'm about to emit other kinds of writes besides time deltas, and it turns out with the frequency of resets, we couldn't really use the old time delta write() function more than once in a shader.
2012-12-10 09:21:34 -08:00
shader_start_time.negate = true;
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
dst_reg diff = dst_reg(this, glsl_type::uint_type);
i965: Adjust the split between shader_time_end() and shader_time_write(). I'm about to emit other kinds of writes besides time deltas, and it turns out with the frequency of resets, we couldn't really use the old time delta write() function more than once in a shader.
2012-12-10 09:21:34 -08:00
emit(ADD(diff, shader_start_time, shader_end_time));
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
/* If there were no instructions between the two timestamp gets, the diff
* is 2 cycles. Remove that overhead, so I can forget about that when
* trying to determine the time taken for single instructions.
*/
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
emit(ADD(diff, src_reg(diff), brw_imm_ud(-2u)));
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
i965: Pull calls to get_shader_time_index out of the visitor Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-06-19 15:40:09 -07:00
emit_shader_time_write(0, src_reg(diff));
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
emit_shader_time_write(1, brw_imm_ud(1u));
i965: Scale shader_time to compensate for resets. Some shaders experience resets more than others, which skews the numbers reported. Attempt to correct for this by linearly scaling according to the number of resets that happen. Note that will not be accurate if invocations of shaders have varying times and longer invocations are more likely to reset. However, this should at least be better than the previous situation.
2012-12-10 09:44:19 -08:00
emit(BRW_OPCODE_ELSE);
i965/vec4: Replace src_reg(imm) constructors with brw_imm_*(). Cuts 1.5k of .text. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:12:44 -08:00
emit_shader_time_write(2, brw_imm_ud(1u));
i965: Adjust the split between shader_time_end() and shader_time_write(). I'm about to emit other kinds of writes besides time deltas, and it turns out with the frequency of resets, we couldn't really use the old time delta write() function more than once in a shader.
2012-12-10 09:21:34 -08:00
emit(BRW_OPCODE_ENDIF);
}
void
i965: Pull calls to get_shader_time_index out of the visitor Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-06-19 15:40:09 -07:00
vec4_visitor::emit_shader_time_write(int shader_time_subindex, src_reg value)
i965: Adjust the split between shader_time_end() and shader_time_write(). I'm about to emit other kinds of writes besides time deltas, and it turns out with the frequency of resets, we couldn't really use the old time delta write() function more than once in a shader.
2012-12-10 09:21:34 -08:00
{
i965/vs: Use a send from a 2-register VGRF for shader time writes. This will let us emit it later, after we're setting up MRFs for the URB write. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:11:21 -08:00
dst_reg dst =
dst_reg(this, glsl_type::get_array_instance(glsl_type::vec4_type, 2));
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
i965/vs: Use a send from a 2-register VGRF for shader time writes. This will let us emit it later, after we're setting up MRFs for the URB write. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:11:21 -08:00
dst_reg offset = dst;
dst_reg time = dst;
i965/vec4: Replace dst/src_reg::reg_offset with dst/src_reg::offset expressed in bytes. The dst/src_reg::offset field in byte units introduced in the previous patch is a more straightforward alternative to an offset representation split between ::reg_offset and ::subreg_offset fields. The split representation makes it too easy to forget about one of the offsets while dealing with the other, which has led to multiple FS back-end bugs in the past. To make the matter worse the unit reg_offset was expressed in was rather inconsistent, for uniforms it would be expressed in either 4B or 16B units depending on the back-end, and for most other things it would be expressed in 32B units. This encodes reg_offset as a new offset field expressed consistently in byte units. Each rvalue reference of reg_offset in existing code like 'x = r.reg_offset' is rewritten to 'x = r.offset / reg_unit', and each lvalue reference like 'r.reg_offset = x' is rewritten to 'r.offset = r.offset % reg_unit + x * reg_unit'. Because the change affects a lot of places and is rather non-trivial to verify due to the inconsistent value of reg_unit, I've tried to avoid making any additional changes other than applying the rewrite rule above in order to keep the patch as simple as possible, sometimes at the cost of introducing obvious stupidity (e.g. algebraic expressions that could be simplified given some knowledge of the context) -- I'll clean those up later on in a second pass. v2: Fix division by the wrong reg_unit in the UNIFORM case of convert_to_hw_regs(). (Iago) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-01 13:10:36 -07:00
time.offset += REG_SIZE;
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
i965/vs: Use a send from a 2-register VGRF for shader time writes. This will let us emit it later, after we're setting up MRFs for the URB write. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:11:21 -08:00
offset.type = BRW_REGISTER_TYPE_UD;
i965: Use a single index per shader for shader_time. Previously, each shader took 3 shader time indices which were potentially at arbirary points in the shader time buffer. Now, each shader gets a single index which refers to 3 consecutive locations in the buffer. This simplifies some of the logic at the cost of having a magic 3 a few places. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-06-19 14:46:03 -07:00
int index = shader_time_index * 3 + shader_time_subindex;
i965: Move SHADER_TIME_STRIDE to brw_compiler.h Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-02-28 18:10:53 -08:00
emit(MOV(offset, brw_imm_d(index * BRW_SHADER_TIME_STRIDE)));
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
i965/vs: Use a send from a 2-register VGRF for shader time writes. This will let us emit it later, after we're setting up MRFs for the URB write. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:11:21 -08:00
time.type = BRW_REGISTER_TYPE_UD;
i965: remove unneeded src_reg copy in emit_shader_time_write The variable is already of type src_reg. creating a new instance only to destroy it seems unnecessary. Signed-off-by: Emil Velikov <emil.velikov@collabora.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-07 12:38:12 +01:00
emit(MOV(time, value));
i965/vs: Use a send from a 2-register VGRF for shader time writes. This will let us emit it later, after we're setting up MRFs for the URB write. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-17 17:11:21 -08:00
i965/vec4: Init mlen for several send from GRF instructions. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-03 22:52:37 +02:00
vec4_instruction *inst =
emit(SHADER_OPCODE_SHADER_TIME_ADD, dst_reg(), src_reg(dst));
inst->mlen = 2;
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
}
i965/vec4: fix vertical stride to avoid breaking region parameter rule From IVB PRM, vol4, part3, "General Restrictions on Regioning Parameters": "If ExecSize = Width and HorzStride ≠ 0, VertStride must be set to Width * HorzStride." In next patch, we are going to modify the region parameter for uniforms and vgrf. For uniforms that are the source of DF align1 instructions, they will have <0, 4, 1> regioning and the execsize for those instructions will be 4, so they will break the regioning rule. This will be the same for VGRF sources where we use the vstride == 0 exploit. As we know we are not going to cross the GRF boundary with that execsize and parameters (not even with the exploit), we just fix the vstride here. v2: - Move is_align1_df() (Curro) - Refactor exec_size == width calculation (Curro) Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-26 10:12:03 +02:00
static bool
is_align1_df(vec4_instruction *inst)
{
switch (inst->opcode) {
case VEC4_OPCODE_DOUBLE_TO_F32:
case VEC4_OPCODE_DOUBLE_TO_D32:
case VEC4_OPCODE_DOUBLE_TO_U32:
case VEC4_OPCODE_TO_DOUBLE:
case VEC4_OPCODE_PICK_LOW_32BIT:
case VEC4_OPCODE_PICK_HIGH_32BIT:
case VEC4_OPCODE_SET_LOW_32BIT:
case VEC4_OPCODE_SET_HIGH_32BIT:
return true;
default:
return false;
}
}
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
void
vec4_visitor::convert_to_hw_regs()
{
foreach_block_and_inst(block, vec4_instruction, inst, cfg) {
for (int i = 0; i < 3; i++) {
struct src_reg &src = inst->src[i];
struct brw_reg reg;
switch (src.file) {
i965/vec4: set correct register regions for 32-bit and 64-bit For 32-bit instructions we want to use <4,4,1> regions for VGRF sources so we should really set a width of 4 (we were setting 8). For 64-bit instructions we want to use a width of 2 because the hardware uses 32-bit swizzles, meaning that we can only address 2 consecutive 64-bit components in a row. Also, Curro suggested that the hardware is probably fixing the width to 2 for 64-bit instructions anyway, so just go with that and use <2,2,1>. v2: - No need to explicitly set the vertical stride of 64-bit regions to 2, brw_vecn_grf with a width of 2 will do that for us. - No need to adjust the width of dst registers. v3 (Ian): - Make type_size and width const. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-18 14:00:58 +01:00
case VGRF: {
i965/vec4: fix register width for DF VGRF and UNIFORM On gen7, the swizzles used in DF align16 instructions works for element size of 32 bits, so we can address only 2 consecutive DFs. As we assumed that in the rest of the code and prepare the instructions for this (scalarize_df()), we need to set it to two again. However, for DF align1 instructions, a width of 2 is wrong as we are not reading the data we want. For example, an uniform would have a region of <0, 2, 1> so it would repeat the first 2 DFs, when we wanted to access to the first 4. This patch sets the default one to 4 and then modifies the width of align16 instruction's DF sources when we translate the logical swizzle to the physical one. v2: - Remove conditional (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-25 12:18:17 +02:00
reg = byte_offset(brw_vecn_grf(4, src.nr, 0), src.offset);
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
reg.type = src.type;
reg.abs = src.abs;
reg.negate = src.negate;
break;
i965/vec4: set correct register regions for 32-bit and 64-bit For 32-bit instructions we want to use <4,4,1> regions for VGRF sources so we should really set a width of 4 (we were setting 8). For 64-bit instructions we want to use a width of 2 because the hardware uses 32-bit swizzles, meaning that we can only address 2 consecutive 64-bit components in a row. Also, Curro suggested that the hardware is probably fixing the width to 2 for 64-bit instructions anyway, so just go with that and use <2,2,1>. v2: - No need to explicitly set the vertical stride of 64-bit regions to 2, brw_vecn_grf with a width of 2 will do that for us. - No need to adjust the width of dst registers. v3 (Ian): - Make type_size and width const. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-18 14:00:58 +01:00
}
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
i965/vec4: set correct register regions for 32-bit and 64-bit For 32-bit instructions we want to use <4,4,1> regions for VGRF sources so we should really set a width of 4 (we were setting 8). For 64-bit instructions we want to use a width of 2 because the hardware uses 32-bit swizzles, meaning that we can only address 2 consecutive 64-bit components in a row. Also, Curro suggested that the hardware is probably fixing the width to 2 for 64-bit instructions anyway, so just go with that and use <2,2,1>. v2: - No need to explicitly set the vertical stride of 64-bit regions to 2, brw_vecn_grf with a width of 2 will do that for us. - No need to adjust the width of dst registers. v3 (Ian): - Make type_size and width const. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-18 14:00:58 +01:00
case UNIFORM: {
i965/vec4: Simplify src/dst_reg to brw_reg conversion by using byte_offset(). This should also have the side effect of fixing convert_to_hw_regs() to handle sub-GRF register offsets. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-08 18:00:11 -07:00
reg = stride(byte_offset(brw_vec4_grf(
prog_data->base.dispatch_grf_start_reg +
src.nr / 2, src.nr % 2 * 4),
src.offset),
i965/vec4: fix register width for DF VGRF and UNIFORM On gen7, the swizzles used in DF align16 instructions works for element size of 32 bits, so we can address only 2 consecutive DFs. As we assumed that in the rest of the code and prepare the instructions for this (scalarize_df()), we need to set it to two again. However, for DF align1 instructions, a width of 2 is wrong as we are not reading the data we want. For example, an uniform would have a region of <0, 2, 1> so it would repeat the first 2 DFs, when we wanted to access to the first 4. This patch sets the default one to 4 and then modifies the width of align16 instruction's DF sources when we translate the logical swizzle to the physical one. v2: - Remove conditional (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-25 12:18:17 +02:00
0, 4, 1);
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
reg.type = src.type;
reg.abs = src.abs;
reg.negate = src.negate;
/* This should have been moved to pull constants. */
assert(!src.reladdr);
break;
i965/vec4: set correct register regions for 32-bit and 64-bit For 32-bit instructions we want to use <4,4,1> regions for VGRF sources so we should really set a width of 4 (we were setting 8). For 64-bit instructions we want to use a width of 2 because the hardware uses 32-bit swizzles, meaning that we can only address 2 consecutive 64-bit components in a row. Also, Curro suggested that the hardware is probably fixing the width to 2 for 64-bit instructions anyway, so just go with that and use <2,2,1>. v2: - No need to explicitly set the vertical stride of 64-bit regions to 2, brw_vecn_grf with a width of 2 will do that for us. - No need to adjust the width of dst registers. v3 (Ian): - Make type_size and width const. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-18 14:00:58 +01:00
}
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
case FIXED_GRF:
i965/vec4: translate 64-bit swizzles to 32-bit The hardware can only operate with 32-bit swizzles, which is a rather limiting restriction. However, the idea is not to expose this to the optimization passes, which would be a mess to deal with. Instead, we let the bulk of the vec4 backend ignore this fact and we fix the swizzles right at codegen time. At the moment the pass only needs to handle single value swizzles thanks to the scalarization pass that runs before it. Notice that this only works for X/Y swizzles. We will add support for Z/W swizzles in the next patch, since they need a bit more work. v2 (Sam): - Do not expand swizzle of 64-bit immediate values. v3: - Do this after translation to hardware registers instead of doing it right before so we don't need the force_vstride0 flag (Curro). - Squashed patch that included FIXED_GRF in the list of register files that need this translation (Iago). - Remove swizzle assignments for VGRF and UNIFORM files in convert_to_hw_regs(), they will be set by apply_logical_swizzle() (Iago). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 11:01:27 +02:00
if (type_sz(src.type) == 8) {
reg = src.as_brw_reg();
break;
}
/* fallthrough */
case ARF:
i965: Drop IMM fs_reg/src_reg -> brw_reg conversions. The previous two commits make this unnecessary. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 12:25:24 -08:00
case IMM:
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
continue;
case BAD_FILE:
/* Probably unused. */
reg = brw_null_reg();
i965/vec4: keep original type when dealing with null registers Keep the original type when dealing with null registers. Especially because we do no want to introduce an implicit conversion between types that could affect the conditional flags. This affects especially when the original type is DF, and we are working on Ivybridge/Baytrail. v2 (Curro) - Fix typo. - Use retype() instead of applying the type directly. - Remove unneeded retype. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-12 16:06:22 +00:00
reg = retype(reg, src.type);
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
break;
i965: Replace default case with list of enum values. If we add a new file type, we'd like to get warnings if it's not handled. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-10-26 06:58:56 -07:00
case MRF:
case ATTR:
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
unreachable("not reached");
}
i965: Move 3-src subnr swizzle handling into the vec4 backend. While most align16 instructions only support a SubRegNum of 0 or 4 (using swizzling to control the other channels), 3-src instructions actually support arbitrary SubRegNums. When the RepCtrl bit is set, we believe it ignores the swizzle and uses the equivalent of a <0,1,0> region from the subnr. In the past, we adopted a vec4-centric approach of specifying subnr of 0 or 4 and a swizzle, then having brw_eu_emit.c convert that to a proper SubRegNum. This isn't a great fit for the scalar backend, where we don't set swizzles at all, and happily set subnrs in the range [0, 7]. This patch changes brw_eu_emit.c to use subnr and swizzle directly, relying on the higher levels to set them sensibly. This should fix problems where scalar sources get copy propagated into 3-src instructions in the FS backend. I've only observed this with TES push model inputs, but I suppose it could happen in other cases. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-12-31 12:47:19 -08:00
i965/vec4: translate 64-bit swizzles to 32-bit The hardware can only operate with 32-bit swizzles, which is a rather limiting restriction. However, the idea is not to expose this to the optimization passes, which would be a mess to deal with. Instead, we let the bulk of the vec4 backend ignore this fact and we fix the swizzles right at codegen time. At the moment the pass only needs to handle single value swizzles thanks to the scalarization pass that runs before it. Notice that this only works for X/Y swizzles. We will add support for Z/W swizzles in the next patch, since they need a bit more work. v2 (Sam): - Do not expand swizzle of 64-bit immediate values. v3: - Do this after translation to hardware registers instead of doing it right before so we don't need the force_vstride0 flag (Curro). - Squashed patch that included FIXED_GRF in the list of register files that need this translation (Iago). - Remove swizzle assignments for VGRF and UNIFORM files in convert_to_hw_regs(), they will be set by apply_logical_swizzle() (Iago). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 11:01:27 +02:00
apply_logical_swizzle(&reg, inst, i);
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
src = reg;
i965/vec4: fix vertical stride to avoid breaking region parameter rule From IVB PRM, vol4, part3, "General Restrictions on Regioning Parameters": "If ExecSize = Width and HorzStride ≠ 0, VertStride must be set to Width * HorzStride." In next patch, we are going to modify the region parameter for uniforms and vgrf. For uniforms that are the source of DF align1 instructions, they will have <0, 4, 1> regioning and the execsize for those instructions will be 4, so they will break the regioning rule. This will be the same for VGRF sources where we use the vstride == 0 exploit. As we know we are not going to cross the GRF boundary with that execsize and parameters (not even with the exploit), we just fix the vstride here. v2: - Move is_align1_df() (Curro) - Refactor exec_size == width calculation (Curro) Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-26 10:12:03 +02:00
/* From IVB PRM, vol4, part3, "General Restrictions on Regioning
* Parameters":
*
* "If ExecSize = Width and HorzStride ≠ 0, VertStride must be set
* to Width * HorzStride."
*
* We can break this rule with DF sources on DF align1
* instructions, because the exec_size would be 4 and width is 4.
* As we know we are not accessing to next GRF, it is safe to
* set vstride to the formula given by the rule itself.
*/
if (is_align1_df(inst) && (cvt(inst->exec_size) - 1) == src.width)
src.vstride = src.width + src.hstride;
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
}
i965: Pass devinfo pointer to is_3src() helpers. This is not strictly required for the following changes because none of the three-source opcodes we support at the moment in the compiler back-end has been removed or redefined, but that's likely to change in the future. In any case having hardware instructions specified as a pair of hardware device and opcode number explicitly in all cases will simplify the opcode look-up interface introduced in a subsequent commit, since the opcode number alone is in general ambiguous. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-28 00:19:13 -07:00
if (inst->is_3src(devinfo)) {
i965: Move 3-src subnr swizzle handling into the vec4 backend. While most align16 instructions only support a SubRegNum of 0 or 4 (using swizzling to control the other channels), 3-src instructions actually support arbitrary SubRegNums. When the RepCtrl bit is set, we believe it ignores the swizzle and uses the equivalent of a <0,1,0> region from the subnr. In the past, we adopted a vec4-centric approach of specifying subnr of 0 or 4 and a swizzle, then having brw_eu_emit.c convert that to a proper SubRegNum. This isn't a great fit for the scalar backend, where we don't set swizzles at all, and happily set subnrs in the range [0, 7]. This patch changes brw_eu_emit.c to use subnr and swizzle directly, relying on the higher levels to set them sensibly. This should fix problems where scalar sources get copy propagated into 3-src instructions in the FS backend. I've only observed this with TES push model inputs, but I suppose it could happen in other cases. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-12-31 12:47:19 -08:00
/* 3-src instructions with scalar sources support arbitrary subnr,
* but don't actually use swizzles. Convert swizzle into subnr.
i965/vec4: Skip swizzle to subnr in 3src instructions with DF operands We make scalar sources in 3src instructions use subnr instead of swizzles because they don't really use swizzles. With doubles it is more complicated because we use vstride=0 in more scenarios in which they don't produce scalar regions. Also RepCtrl=1 is not allowed with 64-bit operands, so we should avoid this. v2: Fix typo (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-01 08:35:37 +02:00
* Skip this for double-precision instructions: RepCtrl=1 is not
* allowed for them and needs special handling.
i965: Move 3-src subnr swizzle handling into the vec4 backend. While most align16 instructions only support a SubRegNum of 0 or 4 (using swizzling to control the other channels), 3-src instructions actually support arbitrary SubRegNums. When the RepCtrl bit is set, we believe it ignores the swizzle and uses the equivalent of a <0,1,0> region from the subnr. In the past, we adopted a vec4-centric approach of specifying subnr of 0 or 4 and a swizzle, then having brw_eu_emit.c convert that to a proper SubRegNum. This isn't a great fit for the scalar backend, where we don't set swizzles at all, and happily set subnrs in the range [0, 7]. This patch changes brw_eu_emit.c to use subnr and swizzle directly, relying on the higher levels to set them sensibly. This should fix problems where scalar sources get copy propagated into 3-src instructions in the FS backend. I've only observed this with TES push model inputs, but I suppose it could happen in other cases. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-12-31 12:47:19 -08:00
*/
for (int i = 0; i < 3; i++) {
i965/vec4: Skip swizzle to subnr in 3src instructions with DF operands We make scalar sources in 3src instructions use subnr instead of swizzles because they don't really use swizzles. With doubles it is more complicated because we use vstride=0 in more scenarios in which they don't produce scalar regions. Also RepCtrl=1 is not allowed with 64-bit operands, so we should avoid this. v2: Fix typo (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-01 08:35:37 +02:00
if (inst->src[i].vstride == BRW_VERTICAL_STRIDE_0 &&
type_sz(inst->src[i].type) < 8) {
i965: Move 3-src subnr swizzle handling into the vec4 backend. While most align16 instructions only support a SubRegNum of 0 or 4 (using swizzling to control the other channels), 3-src instructions actually support arbitrary SubRegNums. When the RepCtrl bit is set, we believe it ignores the swizzle and uses the equivalent of a <0,1,0> region from the subnr. In the past, we adopted a vec4-centric approach of specifying subnr of 0 or 4 and a swizzle, then having brw_eu_emit.c convert that to a proper SubRegNum. This isn't a great fit for the scalar backend, where we don't set swizzles at all, and happily set subnrs in the range [0, 7]. This patch changes brw_eu_emit.c to use subnr and swizzle directly, relying on the higher levels to set them sensibly. This should fix problems where scalar sources get copy propagated into 3-src instructions in the FS backend. I've only observed this with TES push model inputs, but I suppose it could happen in other cases. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-12-31 12:47:19 -08:00
assert(brw_is_single_value_swizzle(inst->src[i].swizzle));
inst->src[i].subnr += 4 * BRW_GET_SWZ(inst->src[i].swizzle, 0);
}
}
}
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
dst_reg &dst = inst->dst;
struct brw_reg reg;
switch (inst->dst.file) {
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:09:25 -07:00
case VGRF:
i965/vec4: Simplify src/dst_reg to brw_reg conversion by using byte_offset(). This should also have the side effect of fixing convert_to_hw_regs() to handle sub-GRF register offsets. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-08 18:00:11 -07:00
reg = byte_offset(brw_vec8_grf(dst.nr, 0), dst.offset);
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
reg.type = dst.type;
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-22 19:41:30 -07:00
reg.writemask = dst.writemask;
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
break;
case MRF:
i965/vec4: Simplify src/dst_reg to brw_reg conversion by using byte_offset(). This should also have the side effect of fixing convert_to_hw_regs() to handle sub-GRF register offsets. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-08 18:00:11 -07:00
reg = byte_offset(brw_message_reg(dst.nr), dst.offset);
assert((reg.nr & ~BRW_MRF_COMPR4) < BRW_MAX_MRF(devinfo->gen));
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
reg.type = dst.type;
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-22 19:41:30 -07:00
reg.writemask = dst.writemask;
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
break;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 17:52:57 -07:00
case ARF:
case FIXED_GRF:
i965: Prevent implicit upcasts to brw_reg. Now that backend_reg inherits from brw_reg, we have to be careful to avoid the object slicing problem. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-19 21:51:37 -08:00
reg = dst.as_brw_reg();
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
break;
case BAD_FILE:
reg = brw_null_reg();
i965/vec4: keep original type when dealing with null registers Keep the original type when dealing with null registers. Especially because we do no want to introduce an implicit conversion between types that could affect the conditional flags. This affects especially when the original type is DF, and we are working on Ivybridge/Baytrail. v2 (Curro) - Fix typo. - Use retype() instead of applying the type directly. - Remove unneeded retype. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-12 16:06:22 +00:00
reg = retype(reg, dst.type);
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
break;
i965: Replace default case with list of enum values. If we add a new file type, we'd like to get warnings if it's not handled. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-10-26 06:58:56 -07:00
case IMM:
case ATTR:
case UNIFORM:
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
unreachable("not reached");
}
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 15:29:03 -07:00
dst = reg;
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
}
}
i965/vec4: split instructions that read 64-bit interleaved attributes Stages that use interleaved attributes generate regions with a vstride=0 that can hit the gen7 hardware decompression bug. v2: - Make static the function and fix indent (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-01 09:26:32 +02:00
static bool
stage_uses_interleaved_attributes(unsigned stage,
enum shader_dispatch_mode dispatch_mode)
{
switch (stage) {
case MESA_SHADER_TESS_EVAL:
return true;
case MESA_SHADER_GEOMETRY:
return dispatch_mode != DISPATCH_MODE_4X2_DUAL_OBJECT;
default:
return false;
}
}
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
/**
* Get the closest native SIMD width supported by the hardware for instruction
* \p inst. The instruction will be left untouched by
* vec4_visitor::lower_simd_width() if the returned value matches the
* instruction's original execution size.
*/
static unsigned
get_lowered_simd_width(const struct gen_device_info *devinfo,
i965/vec4: split instructions that read 64-bit interleaved attributes Stages that use interleaved attributes generate regions with a vstride=0 that can hit the gen7 hardware decompression bug. v2: - Make static the function and fix indent (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-01 09:26:32 +02:00
enum shader_dispatch_mode dispatch_mode,
unsigned stage, const vec4_instruction *inst)
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
{
i965/vec4: do not split scratch read/write opcodes 64-bit scratch read/writes require to shuffle data around so we need to have access to the full 64-bit data. We will do the right thing for these when we emit the messages. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-28 11:54:07 +02:00
/* Do not split some instructions that require special handling */
switch (inst->opcode) {
case SHADER_OPCODE_GEN4_SCRATCH_READ:
case SHADER_OPCODE_GEN4_SCRATCH_WRITE:
return inst->exec_size;
default:
break;
}
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
unsigned lowered_width = MIN2(16, inst->exec_size);
/* We need to split some cases of double-precision instructions that write
* 2 registers. We only need to care about this in gen7 because that is the
* only hardware that implements fp64 in Align16.
*/
if (devinfo->gen == 7 && inst->size_written > REG_SIZE) {
i965/vec4: split double-precision SEL There is a hardware bug affecting compressed double-precision SEL instructions in align16 mode by which they won't read predication mask properly. The bug does not affect other predicated instructions and it does not affect SEL in Align1 mode either. This was found empirically and verified by Curro in the simulator. Fix this by splitting double-precision SEL in Align16 mode to use an execution size of 4. v2: Check that the dst type is 64-bit, since we can have 16-wide single precision bcsel instructions that also write 2 registers. v3: Replace bcsel by SEL in all the comments as bcsel is the nir opcode but SEL is the actual assembly instruction (Matt). Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-17 08:47:29 +02:00
/* Align16 8-wide double-precision SEL does not work well. Verified
* empirically.
*/
if (inst->opcode == BRW_OPCODE_SEL && type_sz(inst->dst.type) == 8)
lowered_width = MIN2(lowered_width, 4);
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
/* HSW PRM, 3D Media GPGPU Engine, Region Alignment Rules for Direct
* Register Addressing:
*
* "When destination spans two registers, the source MUST span two
* registers."
*/
for (unsigned i = 0; i < 3; i++) {
if (inst->src[i].file == BAD_FILE)
continue;
if (inst->size_read(i) <= REG_SIZE)
lowered_width = MIN2(lowered_width, 4);
i965/vec4: split instructions that read 64-bit interleaved attributes Stages that use interleaved attributes generate regions with a vstride=0 that can hit the gen7 hardware decompression bug. v2: - Make static the function and fix indent (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-01 09:26:32 +02:00
/* Interleaved attribute setups use a vertical stride of 0, which
* makes them hit the associated instruction decompression bug in gen7.
* Split them to prevent this.
*/
if (inst->src[i].file == ATTR &&
stage_uses_interleaved_attributes(stage, dispatch_mode))
lowered_width = MIN2(lowered_width, 4);
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
}
}
i965/vec4: split DF instructions and later double its execsize in IVB/BYT We need to split DF instructions in two on IVB/BYT as it needs an execsize 8 to process 4 DF values (one GRF in total). v2: - Rename helper and make it static inline function (Matt). - Fix indention and add braces (Matt). v3: - Don't edit IR instruction when doubling exec_size (Curro) - Add comment into the code (Curro). - Manage ARF registers like the others (Curro) v4: - Add get_exec_type() function and use it to calculate the execution size. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> [ Francisco Jerez: Fix bogus 'type != BAD_FILE' check. Take destination type as execution type where there is no valid source. Assert-fail if the deduced execution type is byte. Clarify comment in get_lowered_simd_width(). Move SIMD width workaround outside of 'if (...inst->size_written > REG_SIZE)' conditional block, since the problem should be independent of whether the amount of data written by the instruction is greater or lower than a GRF. Drop redundant is_ivb_df definition. Drop bogus inst->exec_size < 8 check. Simplify channel group assertion. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-08-29 10:10:30 +02:00
/* IvyBridge can manage a maximum of 4 DFs per SIMD4x2 instruction, since
* it doesn't support compression in Align16 mode, no matter if it has
* force_writemask_all enabled or disabled (the latter is affected by the
* compressed instruction bug in gen7, which is another reason to enforce
* this limit).
*/
if (devinfo->gen == 7 && !devinfo->is_haswell &&
(get_exec_type_size(inst) == 8 || type_sz(inst->dst.type) == 8))
lowered_width = MIN2(lowered_width, 4);
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
return lowered_width;
}
static bool
dst_src_regions_overlap(vec4_instruction *inst)
{
if (inst->size_written == 0)
return false;
unsigned dst_start = inst->dst.offset;
unsigned dst_end = dst_start + inst->size_written - 1;
for (int i = 0; i < 3; i++) {
if (inst->src[i].file == BAD_FILE)
continue;
if (inst->dst.file != inst->src[i].file ||
inst->dst.nr != inst->src[i].nr)
continue;
unsigned src_start = inst->src[i].offset;
unsigned src_end = src_start + inst->size_read(i) - 1;
if ((dst_start >= src_start && dst_start <= src_end) ||
(dst_end >= src_start && dst_end <= src_end) ||
(dst_start <= src_start && dst_end >= src_end)) {
return true;
}
}
return false;
}
bool
vec4_visitor::lower_simd_width()
{
bool progress = false;
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
i965/vec4: split instructions that read 64-bit interleaved attributes Stages that use interleaved attributes generate regions with a vstride=0 that can hit the gen7 hardware decompression bug. v2: - Make static the function and fix indent (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-01 09:26:32 +02:00
const unsigned lowered_width =
get_lowered_simd_width(devinfo, prog_data->dispatch_mode, stage, inst);
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
assert(lowered_width <= inst->exec_size);
if (lowered_width == inst->exec_size)
continue;
/* We need to deal with source / destination overlaps when splitting.
* The hardware supports reading from and writing to the same register
* in the same instruction, but we need to be careful that each split
* instruction we produce does not corrupt the source of the next.
*
* The easiest way to handle this is to make the split instructions write
* to temporaries if there is an src/dst overlap and then move from the
* temporaries to the original destination. We also need to consider
* instructions that do partial writes via align1 opcodes, in which case
* we need to make sure that the we initialize the temporary with the
* value of the instruction's dst.
*/
bool needs_temp = dst_src_regions_overlap(inst);
for (unsigned n = 0; n < inst->exec_size / lowered_width; n++) {
unsigned channel_offset = lowered_width * n;
unsigned size_written = lowered_width * type_sz(inst->dst.type);
/* Create the split instruction from the original so that we copy all
* relevant instruction fields, then set the width and calculate the
* new dst/src regions.
*/
vec4_instruction *linst = new(mem_ctx) vec4_instruction(*inst);
linst->exec_size = lowered_width;
linst->group = channel_offset;
linst->size_written = size_written;
/* Compute split dst region */
dst_reg dst;
if (needs_temp) {
unsigned num_regs = DIV_ROUND_UP(size_written, REG_SIZE);
dst = retype(dst_reg(VGRF, alloc.allocate(num_regs)),
inst->dst.type);
if (inst->is_align1_partial_write()) {
vec4_instruction *copy = MOV(dst, src_reg(inst->dst));
copy->exec_size = lowered_width;
copy->group = channel_offset;
copy->size_written = size_written;
inst->insert_before(block, copy);
}
} else {
dst = horiz_offset(inst->dst, channel_offset);
}
linst->dst = dst;
/* Compute split source regions */
for (int i = 0; i < 3; i++) {
if (linst->src[i].file == BAD_FILE)
continue;
if (!is_uniform(linst->src[i]))
linst->src[i] = horiz_offset(linst->src[i], channel_offset);
}
inst->insert_before(block, linst);
/* If we used a temporary to store the result of the split
* instruction, copy the result to the original destination
*/
if (needs_temp) {
vec4_instruction *mov =
MOV(offset(inst->dst, lowered_width, n), src_reg(dst));
mov->exec_size = lowered_width;
mov->group = channel_offset;
mov->size_written = size_written;
mov->predicate = inst->predicate;
inst->insert_before(block, mov);
}
}
inst->remove(block);
progress = true;
}
if (progress)
invalidate_live_intervals();
return progress;
}
i965/vec4: add a scalarization pass for double-precision instructions The hardware only supports 32-bit swizzles, which means that we can only access directly channels XY of a DF making access to channels ZW more difficult, specially considering the various regioning restrictions imposed by the hardware. The combination of both things makes handling ramdom swizzles on DF operands rather difficult, as there are many combinations that can't be represented at all, at least not without some work and some level of instruction splitting depending on the case. Writemasks are 64-bit in general, however XY and ZW writemasks also work in 32-bit, which means these writemasks can't be represented natively, adding to the complexity. For now, we decided to try and simplify things as much as possible to avoid dealing with all this from the get go by adding a scalarization pass that runs after the main optimization loop. By fully scalarizing DF instructions in align16 we avoid most of the complexity introduced by the aforementioned hardware restrictions and we have an easier path to an initial fully functional version for the vector backend in Haswell and IvyBridge. Later, we can improve the implementation so we don't necessarily scalarize everything, iteratively adding more complexity and building on top of a framework that is already working. Curro drafted some ideas for how this could be done here: https://bugs.freedesktop.org/show_bug.cgi?id=92760#c82 v2: - Use a copy constructor for the scalar instructions so we copy all relevant instructions fields from the original instruction. v3: Fix indention in one switch (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 09:20:51 +02:00
static brw_predicate
scalarize_predicate(brw_predicate predicate, unsigned writemask)
{
if (predicate != BRW_PREDICATE_NORMAL)
return predicate;
switch (writemask) {
case WRITEMASK_X:
return BRW_PREDICATE_ALIGN16_REPLICATE_X;
case WRITEMASK_Y:
return BRW_PREDICATE_ALIGN16_REPLICATE_Y;
case WRITEMASK_Z:
return BRW_PREDICATE_ALIGN16_REPLICATE_Z;
case WRITEMASK_W:
return BRW_PREDICATE_ALIGN16_REPLICATE_W;
default:
unreachable("invalid writemask");
}
}
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
/* Gen7 has a hardware decompression bug that we can exploit to represent
* handful of additional swizzles natively.
*/
static bool
is_gen7_supported_64bit_swizzle(vec4_instruction *inst, unsigned arg)
{
switch (inst->src[arg].swizzle) {
case BRW_SWIZZLE_XXXX:
case BRW_SWIZZLE_YYYY:
case BRW_SWIZZLE_ZZZZ:
case BRW_SWIZZLE_WWWW:
case BRW_SWIZZLE_XYXY:
case BRW_SWIZZLE_YXYX:
case BRW_SWIZZLE_ZWZW:
case BRW_SWIZZLE_WZWZ:
return true;
default:
return false;
}
}
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
/* 64-bit sources use regions with a width of 2. These 2 elements in each row
* can be addressed using 32-bit swizzles (which is what the hardware supports)
* but it also means that the swizzle we apply on the first two components of a
* dvec4 is coupled with the swizzle we use for the last 2. In other words,
* only some specific swizzle combinations can be natively supported.
*
* FIXME: we can go an step further and implement even more swizzle
* variations using only partial scalarization.
*
* For more details see:
* https://bugs.freedesktop.org/show_bug.cgi?id=92760#c82
*/
bool
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
vec4_visitor::is_supported_64bit_region(vec4_instruction *inst, unsigned arg)
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
{
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
const src_reg &src = inst->src[arg];
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
assert(type_sz(src.type) == 8);
/* Uniform regions have a vstride=0. Because we use 2-wide rows with
* 64-bit regions it means that we cannot access components Z/W, so
* return false for any such case. Interleaved attributes will also be
* mapped to GRF registers with a vstride of 0, so apply the same
* treatment.
*/
if ((is_uniform(src) ||
(stage_uses_interleaved_attributes(stage, prog_data->dispatch_mode) &&
src.file == ATTR)) &&
(brw_mask_for_swizzle(src.swizzle) & 12))
return false;
switch (src.swizzle) {
case BRW_SWIZZLE_XYZW:
case BRW_SWIZZLE_XXZZ:
case BRW_SWIZZLE_YYWW:
case BRW_SWIZZLE_YXWZ:
return true;
default:
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
return devinfo->gen == 7 && is_gen7_supported_64bit_swizzle(inst, arg);
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
}
}
i965/vec4: add a scalarization pass for double-precision instructions The hardware only supports 32-bit swizzles, which means that we can only access directly channels XY of a DF making access to channels ZW more difficult, specially considering the various regioning restrictions imposed by the hardware. The combination of both things makes handling ramdom swizzles on DF operands rather difficult, as there are many combinations that can't be represented at all, at least not without some work and some level of instruction splitting depending on the case. Writemasks are 64-bit in general, however XY and ZW writemasks also work in 32-bit, which means these writemasks can't be represented natively, adding to the complexity. For now, we decided to try and simplify things as much as possible to avoid dealing with all this from the get go by adding a scalarization pass that runs after the main optimization loop. By fully scalarizing DF instructions in align16 we avoid most of the complexity introduced by the aforementioned hardware restrictions and we have an easier path to an initial fully functional version for the vector backend in Haswell and IvyBridge. Later, we can improve the implementation so we don't necessarily scalarize everything, iteratively adding more complexity and building on top of a framework that is already working. Curro drafted some ideas for how this could be done here: https://bugs.freedesktop.org/show_bug.cgi?id=92760#c82 v2: - Use a copy constructor for the scalar instructions so we copy all relevant instructions fields from the original instruction. v3: Fix indention in one switch (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 09:20:51 +02:00
bool
vec4_visitor::scalarize_df()
{
bool progress = false;
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
/* Skip DF instructions that operate in Align1 mode */
if (is_align1_df(inst))
continue;
/* Check if this is a double-precision instruction */
bool is_double = type_sz(inst->dst.type) == 8;
for (int arg = 0; !is_double && arg < 3; arg++) {
is_double = inst->src[arg].file != BAD_FILE &&
type_sz(inst->src[arg].type) == 8;
}
if (!is_double)
continue;
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
/* Skip the lowering for specific regioning scenarios that we can
* support natively.
*/
bool skip_lowering = true;
/* XY and ZW writemasks operate in 32-bit, which means that they don't
* have a native 64-bit representation and they should always be split.
*/
if (inst->dst.writemask == WRITEMASK_XY ||
inst->dst.writemask == WRITEMASK_ZW) {
skip_lowering = false;
} else {
for (unsigned i = 0; i < 3; i++) {
if (inst->src[i].file == BAD_FILE || type_sz(inst->src[i].type) < 8)
continue;
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
skip_lowering = skip_lowering && is_supported_64bit_region(inst, i);
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
}
}
if (skip_lowering)
continue;
i965/vec4: add a scalarization pass for double-precision instructions The hardware only supports 32-bit swizzles, which means that we can only access directly channels XY of a DF making access to channels ZW more difficult, specially considering the various regioning restrictions imposed by the hardware. The combination of both things makes handling ramdom swizzles on DF operands rather difficult, as there are many combinations that can't be represented at all, at least not without some work and some level of instruction splitting depending on the case. Writemasks are 64-bit in general, however XY and ZW writemasks also work in 32-bit, which means these writemasks can't be represented natively, adding to the complexity. For now, we decided to try and simplify things as much as possible to avoid dealing with all this from the get go by adding a scalarization pass that runs after the main optimization loop. By fully scalarizing DF instructions in align16 we avoid most of the complexity introduced by the aforementioned hardware restrictions and we have an easier path to an initial fully functional version for the vector backend in Haswell and IvyBridge. Later, we can improve the implementation so we don't necessarily scalarize everything, iteratively adding more complexity and building on top of a framework that is already working. Curro drafted some ideas for how this could be done here: https://bugs.freedesktop.org/show_bug.cgi?id=92760#c82 v2: - Use a copy constructor for the scalar instructions so we copy all relevant instructions fields from the original instruction. v3: Fix indention in one switch (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 09:20:51 +02:00
/* Generate scalar instructions for each enabled channel */
for (unsigned chan = 0; chan < 4; chan++) {
unsigned chan_mask = 1 << chan;
if (!(inst->dst.writemask & chan_mask))
continue;
vec4_instruction *scalar_inst = new(mem_ctx) vec4_instruction(*inst);
for (unsigned i = 0; i < 3; i++) {
unsigned swz = BRW_GET_SWZ(inst->src[i].swizzle, chan);
scalar_inst->src[i].swizzle = BRW_SWIZZLE4(swz, swz, swz, swz);
}
scalar_inst->dst.writemask = chan_mask;
if (inst->predicate != BRW_PREDICATE_NONE) {
scalar_inst->predicate =
scalarize_predicate(inst->predicate, chan_mask);
}
inst->insert_before(block, scalar_inst);
}
inst->remove(block);
progress = true;
}
if (progress)
invalidate_live_intervals();
return progress;
}
i965/vec4: Lower 64-bit MAD The previous patch made sure that we do not generate MAD instructions for any NIR's 64-bit ffma, but there is nothing preventing i965 from producing MAD instructions as a result of lowerings or optimization passes. This patch makes sure that any 64-bit MAD produced inside the driver after translating from NIR is also converted to MUL+ADD before we generate code. v2: - Use a copy constructor to copy all relevant instruction fields from the original mad into the add and mul instructions v3: - Rename the lowering and fix commit log (Matt) Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-08 11:04:34 +02:00
bool
vec4_visitor::lower_64bit_mad_to_mul_add()
{
bool progress = false;
foreach_block_and_inst_safe(block, vec4_instruction, inst, cfg) {
if (inst->opcode != BRW_OPCODE_MAD)
continue;
if (type_sz(inst->dst.type) != 8)
continue;
dst_reg mul_dst = dst_reg(this, glsl_type::dvec4_type);
/* Use the copy constructor so we copy all relevant instruction fields
* from the original mad into the add and mul instructions
*/
vec4_instruction *mul = new(mem_ctx) vec4_instruction(*inst);
mul->opcode = BRW_OPCODE_MUL;
mul->dst = mul_dst;
mul->src[0] = inst->src[1];
mul->src[1] = inst->src[2];
mul->src[2].file = BAD_FILE;
vec4_instruction *add = new(mem_ctx) vec4_instruction(*inst);
add->opcode = BRW_OPCODE_ADD;
add->src[0] = src_reg(mul_dst);
add->src[1] = inst->src[0];
add->src[2].file = BAD_FILE;
inst->insert_before(block, mul);
inst->insert_before(block, add);
inst->remove(block);
progress = true;
}
if (progress)
invalidate_live_intervals();
return progress;
}
i965/vec4: translate 64-bit swizzles to 32-bit The hardware can only operate with 32-bit swizzles, which is a rather limiting restriction. However, the idea is not to expose this to the optimization passes, which would be a mess to deal with. Instead, we let the bulk of the vec4 backend ignore this fact and we fix the swizzles right at codegen time. At the moment the pass only needs to handle single value swizzles thanks to the scalarization pass that runs before it. Notice that this only works for X/Y swizzles. We will add support for Z/W swizzles in the next patch, since they need a bit more work. v2 (Sam): - Do not expand swizzle of 64-bit immediate values. v3: - Do this after translation to hardware registers instead of doing it right before so we don't need the force_vstride0 flag (Curro). - Squashed patch that included FIXED_GRF in the list of register files that need this translation (Iago). - Remove swizzle assignments for VGRF and UNIFORM files in convert_to_hw_regs(), they will be set by apply_logical_swizzle() (Iago). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 11:01:27 +02:00
/* The align16 hardware can only do 32-bit swizzle channels, so we need to
* translate the logical 64-bit swizzle channels that we use in the Vec4 IR
* to 32-bit swizzle channels in hardware registers.
*
* @inst and @arg identify the original vec4 IR source operand we need to
* translate the swizzle for and @hw_reg is the hardware register where we
* will write the hardware swizzle to use.
*
* This pass assumes that Align16/DF instructions have been fully scalarized
* previously so there is just one 64-bit swizzle channel to deal with for any
* given Vec4 IR source.
*/
void
vec4_visitor::apply_logical_swizzle(struct brw_reg *hw_reg,
vec4_instruction *inst, int arg)
{
src_reg reg = inst->src[arg];
if (reg.file == BAD_FILE || reg.file == BRW_IMMEDIATE_VALUE)
return;
/* If this is not a 64-bit operand or this is a scalar instruction we don't
* need to do anything about the swizzles.
*/
if(type_sz(reg.type) < 8 || is_align1_df(inst)) {
hw_reg->swizzle = reg.swizzle;
return;
}
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
/* Take the 64-bit logical swizzle channel and translate it to 32-bit */
assert(brw_is_single_value_swizzle(reg.swizzle) ||
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
is_supported_64bit_region(inst, arg));
i965/vec4: translate 64-bit swizzles to 32-bit The hardware can only operate with 32-bit swizzles, which is a rather limiting restriction. However, the idea is not to expose this to the optimization passes, which would be a mess to deal with. Instead, we let the bulk of the vec4 backend ignore this fact and we fix the swizzles right at codegen time. At the moment the pass only needs to handle single value swizzles thanks to the scalarization pass that runs before it. Notice that this only works for X/Y swizzles. We will add support for Z/W swizzles in the next patch, since they need a bit more work. v2 (Sam): - Do not expand swizzle of 64-bit immediate values. v3: - Do this after translation to hardware registers instead of doing it right before so we don't need the force_vstride0 flag (Curro). - Squashed patch that included FIXED_GRF in the list of register files that need this translation (Iago). - Remove swizzle assignments for VGRF and UNIFORM files in convert_to_hw_regs(), they will be set by apply_logical_swizzle() (Iago). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 11:01:27 +02:00
i965/vec4: fix register width for DF VGRF and UNIFORM On gen7, the swizzles used in DF align16 instructions works for element size of 32 bits, so we can address only 2 consecutive DFs. As we assumed that in the rest of the code and prepare the instructions for this (scalarize_df()), we need to set it to two again. However, for DF align1 instructions, a width of 2 is wrong as we are not reading the data we want. For example, an uniform would have a region of <0, 2, 1> so it would repeat the first 2 DFs, when we wanted to access to the first 4. This patch sets the default one to 4 and then modifies the width of align16 instruction's DF sources when we translate the logical swizzle to the physical one. v2: - Remove conditional (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Cc: "17.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-04-25 12:18:17 +02:00
/* Apply the region <2, 2, 1> for GRF or <0, 2, 1> for uniforms, as align16
* HW can only do 32-bit swizzle channels.
*/
hw_reg->width = BRW_WIDTH_2;
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
if (is_supported_64bit_region(inst, arg) &&
!is_gen7_supported_64bit_swizzle(inst, arg)) {
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
/* Supported 64-bit swizzles are those such that their first two
* components, when expanded to 32-bit swizzles, match the semantics
* of the original 64-bit swizzle with 2-wide row regioning.
*/
unsigned swizzle0 = BRW_GET_SWZ(reg.swizzle, 0);
unsigned swizzle1 = BRW_GET_SWZ(reg.swizzle, 1);
hw_reg->swizzle = BRW_SWIZZLE4(swizzle0 * 2, swizzle0 * 2 + 1,
swizzle1 * 2, swizzle1 * 2 + 1);
} else {
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
/* If we got here then we have one of the following:
*
* 1. An unsupported swizzle, which should be single-value thanks to the
* scalarization pass.
*
* 2. A gen7 supported swizzle. These can be single-value or double-value
* swizzles. If the latter, they are never cross-dvec2 channels. For
* these we always need to activate the gen7 vstride=0 exploit.
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
*/
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
unsigned swizzle0 = BRW_GET_SWZ(reg.swizzle, 0);
unsigned swizzle1 = BRW_GET_SWZ(reg.swizzle, 1);
assert((swizzle0 < 2) == (swizzle1 < 2));
i965/vec4: implement access to DF source components Z/W The general idea is that with 32-bit swizzles we cannot address DF components Z/W directly, so instead we select the region that starts at the the 16B offset into the register and use X/Y swizzles. The above, however, has the caveat that we can't do that without violating register region restrictions unless we probably do some sort of SIMD splitting. Alternatively, we can accomplish what we need without SIMD splitting by exploiting the gen7 hardware decompression bug for instructions with a vstride=0. For example, an instruction like this: mov(8) r2.x:DF r0.2<0>xyzw:DF Activates the hardware bug and produces this region: Component: x0 y0 z0 w0 x1 y1 z1 w1 Register: r0.2 r0.3 r0.2 r0.3 r1.2 r1.3 r1.2 r1.3 Where r0.2 and r0.3 are r0.z:DF for the first vertex of the SIMD4x2 execution and r1.2 and r1.3 are the same for the second vertex. Using this to our advantage we can select r0.z:DF by doing r0.2<0,2,1>.xyxy and r0.w by doing r0.2<0,2,1>.zwzw without needing to split the instruction. Of course, this only works for gen7, but that is the only hardware platform were we implement align16/fp64 at the moment. v2: Adapted to the fact that we now do this after converting to hardware registers (Iago) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-18 13:43:00 +02:00
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
/* To gain access to Z/W components we need to select the second half
* of the register and then use a X/Y swizzle to select Z/W respectively.
*/
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
if (swizzle0 >= 2) {
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
*hw_reg = suboffset(*hw_reg, 2);
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
swizzle0 -= 2;
swizzle1 -= 2;
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
}
i965/vec4: implement access to DF source components Z/W The general idea is that with 32-bit swizzles we cannot address DF components Z/W directly, so instead we select the region that starts at the the 16B offset into the register and use X/Y swizzles. The above, however, has the caveat that we can't do that without violating register region restrictions unless we probably do some sort of SIMD splitting. Alternatively, we can accomplish what we need without SIMD splitting by exploiting the gen7 hardware decompression bug for instructions with a vstride=0. For example, an instruction like this: mov(8) r2.x:DF r0.2<0>xyzw:DF Activates the hardware bug and produces this region: Component: x0 y0 z0 w0 x1 y1 z1 w1 Register: r0.2 r0.3 r0.2 r0.3 r1.2 r1.3 r1.2 r1.3 Where r0.2 and r0.3 are r0.z:DF for the first vertex of the SIMD4x2 execution and r1.2 and r1.3 are the same for the second vertex. Using this to our advantage we can select r0.z:DF by doing r0.2<0,2,1>.xyxy and r0.w by doing r0.2<0,2,1>.zwzw without needing to split the instruction. Of course, this only works for gen7, but that is the only hardware platform were we implement align16/fp64 at the moment. v2: Adapted to the fact that we now do this after converting to hardware registers (Iago) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-18 13:43:00 +02:00
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
/* All gen7-specific supported swizzles require the vstride=0 exploit */
if (devinfo->gen == 7 && is_gen7_supported_64bit_swizzle(inst, arg))
hw_reg->vstride = BRW_VERTICAL_STRIDE_0;
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
/* Any 64-bit source with an offset at 16B is intended to address the
* second half of a register and needs a vertical stride of 0 so we:
*
* 1. Don't violate register region restrictions.
* 2. Activate the gen7 instruction decompresion bug exploit when
* execsize > 4
*/
if (hw_reg->subnr % REG_SIZE == 16) {
assert(devinfo->gen == 7);
hw_reg->vstride = BRW_VERTICAL_STRIDE_0;
}
i965/vec4: implement access to DF source components Z/W The general idea is that with 32-bit swizzles we cannot address DF components Z/W directly, so instead we select the region that starts at the the 16B offset into the register and use X/Y swizzles. The above, however, has the caveat that we can't do that without violating register region restrictions unless we probably do some sort of SIMD splitting. Alternatively, we can accomplish what we need without SIMD splitting by exploiting the gen7 hardware decompression bug for instructions with a vstride=0. For example, an instruction like this: mov(8) r2.x:DF r0.2<0>xyzw:DF Activates the hardware bug and produces this region: Component: x0 y0 z0 w0 x1 y1 z1 w1 Register: r0.2 r0.3 r0.2 r0.3 r1.2 r1.3 r1.2 r1.3 Where r0.2 and r0.3 are r0.z:DF for the first vertex of the SIMD4x2 execution and r1.2 and r1.3 are the same for the second vertex. Using this to our advantage we can select r0.z:DF by doing r0.2<0,2,1>.xyxy and r0.w by doing r0.2<0,2,1>.zwzw without needing to split the instruction. Of course, this only works for gen7, but that is the only hardware platform were we implement align16/fp64 at the moment. v2: Adapted to the fact that we now do this after converting to hardware registers (Iago) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-18 13:43:00 +02:00
i965/vec4/scalarize_df: support more swizzles via vstride=0 By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-18 11:15:56 +02:00
hw_reg->swizzle = BRW_SWIZZLE4(swizzle0 * 2, swizzle0 * 2 + 1,
swizzle1 * 2, swizzle1 * 2 + 1);
i965/vec4/scalarize_df: do not scalarize swizzles that we can support natively Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-19 09:28:04 +02:00
}
i965/vec4: translate 64-bit swizzles to 32-bit The hardware can only operate with 32-bit swizzles, which is a rather limiting restriction. However, the idea is not to expose this to the optimization passes, which would be a mess to deal with. Instead, we let the bulk of the vec4 backend ignore this fact and we fix the swizzles right at codegen time. At the moment the pass only needs to handle single value swizzles thanks to the scalarization pass that runs before it. Notice that this only works for X/Y swizzles. We will add support for Z/W swizzles in the next patch, since they need a bit more work. v2 (Sam): - Do not expand swizzle of 64-bit immediate values. v3: - Do this after translation to hardware registers instead of doing it right before so we don't need the force_vstride0 flag (Curro). - Squashed patch that included FIXED_GRF in the list of register files that need this translation (Iago). - Remove swizzle assignments for VGRF and UNIFORM files in convert_to_hw_regs(), they will be set by apply_logical_swizzle() (Iago). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 11:01:27 +02:00
}
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
bool
i965: Move legacy clip plane handling to vec4_vs_visitor. This is now only used for the vertex shader, so it makes sense to get it out of any paths run by the geometry shader. Instead of passing the gl_clip_plane array into the run() method (which is shared among all subclasses), we add it as a vec4_vs_visitor constructor parameter. This eliminates the bogus NULL parameter in the GS case. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-08-27 23:49:03 -07:00
vec4_visitor::run()
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
{
i965: Pull calls to get_shader_time_index out of the visitor Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-06-19 15:40:09 -07:00
if (shader_time_index >= 0)
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
emit_shader_time_begin();
i965/vs: Make some vec4_visitor functions virtual. This patch makes the following vec4_visitor functions virtual, since they will need to be implemented differently for vertex and geometry shaders. Some of the functions are renamed to reflect their generic purpose, rather than their VS-specific behaviour: - setup_attributes - emit_attribute_fixups (renamed to emit_prolog) - emit_vertex_program_code (renamed to emit_program_code) - emit_urb_writes (renamed to emit_thread_end) Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-02-17 08:05:52 -08:00
emit_prolog();
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
i965/vec4: Always use NIR GLSL IR vs. NIR shader-db results for vec4 programs on i965: total instructions in shared programs: 1499328 -> 1388354 (-7.40%) instructions in affected programs: 1245199 -> 1134225 (-8.91%) helped: 7469 HURT: 2440 GLSL IR vs. NIR shader-db results for vec4 programs on G4x: total instructions in shared programs: 1436799 -> 1325825 (-7.72%) instructions in affected programs: 1205599 -> 1094625 (-9.20%) helped: 7469 HURT: 2440 GLSL IR vs. NIR shader-db results for vec4 programs on Iron Lake: total instructions in shared programs: 1436654 -> 1325682 (-7.72%) instructions in affected programs: 1205503 -> 1094531 (-9.21%) helped: 7468 HURT: 2440 GLSL IR vs. NIR shader-db results for vec4 programs on Sandy Bridge: total instructions in shared programs: 2016249 -> 1787033 (-11.37%) instructions in affected programs: 1850547 -> 1621331 (-12.39%) helped: 14856 HURT: 1481 GLSL IR vs. NIR shader-db results for vec4 programs on Ivy Bridge: total instructions in shared programs: 1848027 -> 1648216 (-10.81%) instructions in affected programs: 1660279 -> 1460468 (-12.03%) helped: 14668 HURT: 1369 GLSL IR vs. NIR shader-db results for vec4 programs on Bay Trail: total instructions in shared programs: 1848027 -> 1648216 (-10.81%) instructions in affected programs: 1660279 -> 1460468 (-12.03%) helped: 14668 HURT: 1369 GLSL IR vs. NIR shader-db results for vec4 programs on Haswell: total instructions in shared programs: 1848027 -> 1648216 (-10.81%) instructions in affected programs: 1660279 -> 1460468 (-12.03%) helped: 14668 HURT: 1369 I also ran our full suite of benchmarks on a Haswell and had the following statistically significant (according to ministat) changes: Test master-glsl master-nir diff bench_OglGeomPoint 461.556 463.006 1.450 bench_OglTerrainFlyInst 184.484 187.574 3.090 bench_OglTerrainPanInst 132.412 136.307 3.895 bench_OglTexFilterAniso 19.653 19.645 -0.008 bench_OglTexFilterTri 58.333 58.009 -0.324 bench_OglVSInstancing 65.049 65.327 0.278 bench_trexoff 69.474 69.694 0.220 bench_valley 40.708 41.125 0.417 v2 (Jason Ekstrand): - Remove more uses of NirOptions as a switch - New shader-db numbers - Added benchmark numbers Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-21 10:42:19 -07:00
emit_nir_code();
if (failed)
return false;
i965: Don't leak the IR annotation into later instructions. After walking our IR instructions (Mesa or GLSL), we don't want to also mark the start of the FB/URB writes or whatever as being that IR. This can end up being misleading when the end of the IR visit got copy propagated out to a later instruction in the URB writes. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-29 16:51:13 -08:00
base_ir = NULL;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
i965/vs: Make some vec4_visitor functions virtual. This patch makes the following vec4_visitor functions virtual, since they will need to be implemented differently for vertex and geometry shaders. Some of the functions are renamed to reflect their generic purpose, rather than their VS-specific behaviour: - setup_attributes - emit_attribute_fixups (renamed to emit_prolog) - emit_vertex_program_code (renamed to emit_program_code) - emit_urb_writes (renamed to emit_thread_end) Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-02-17 08:05:52 -08:00
emit_thread_end();
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
i965: Remove now unneeded calls to calculate_cfg(). Now that nothing invalidates the CFG, we can calculate_cfg() immediately after emit_fb_writes()/emit_thread_end() and never again. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-09-01 10:58:11 -07:00
calculate_cfg();
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
/* Before any optimization, push array accesses out to scratch
* space where we need them to be. This pass may allocate new
* virtual GRFs, so we want to do it early. It also makes sure
* that we have reladdr computations available for CSE, since we'll
* often do repeated subexpressions for those.
*/
i965/vec4: Always use NIR GLSL IR vs. NIR shader-db results for vec4 programs on i965: total instructions in shared programs: 1499328 -> 1388354 (-7.40%) instructions in affected programs: 1245199 -> 1134225 (-8.91%) helped: 7469 HURT: 2440 GLSL IR vs. NIR shader-db results for vec4 programs on G4x: total instructions in shared programs: 1436799 -> 1325825 (-7.72%) instructions in affected programs: 1205599 -> 1094625 (-9.20%) helped: 7469 HURT: 2440 GLSL IR vs. NIR shader-db results for vec4 programs on Iron Lake: total instructions in shared programs: 1436654 -> 1325682 (-7.72%) instructions in affected programs: 1205503 -> 1094531 (-9.21%) helped: 7468 HURT: 2440 GLSL IR vs. NIR shader-db results for vec4 programs on Sandy Bridge: total instructions in shared programs: 2016249 -> 1787033 (-11.37%) instructions in affected programs: 1850547 -> 1621331 (-12.39%) helped: 14856 HURT: 1481 GLSL IR vs. NIR shader-db results for vec4 programs on Ivy Bridge: total instructions in shared programs: 1848027 -> 1648216 (-10.81%) instructions in affected programs: 1660279 -> 1460468 (-12.03%) helped: 14668 HURT: 1369 GLSL IR vs. NIR shader-db results for vec4 programs on Bay Trail: total instructions in shared programs: 1848027 -> 1648216 (-10.81%) instructions in affected programs: 1660279 -> 1460468 (-12.03%) helped: 14668 HURT: 1369 GLSL IR vs. NIR shader-db results for vec4 programs on Haswell: total instructions in shared programs: 1848027 -> 1648216 (-10.81%) instructions in affected programs: 1660279 -> 1460468 (-12.03%) helped: 14668 HURT: 1369 I also ran our full suite of benchmarks on a Haswell and had the following statistically significant (according to ministat) changes: Test master-glsl master-nir diff bench_OglGeomPoint 461.556 463.006 1.450 bench_OglTerrainFlyInst 184.484 187.574 3.090 bench_OglTerrainPanInst 132.412 136.307 3.895 bench_OglTexFilterAniso 19.653 19.645 -0.008 bench_OglTexFilterTri 58.333 58.009 -0.324 bench_OglVSInstancing 65.049 65.327 0.278 bench_trexoff 69.474 69.694 0.220 bench_valley 40.708 41.125 0.417 v2 (Jason Ekstrand): - Remove more uses of NirOptions as a switch - New shader-db numbers - Added benchmark numbers Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-21 10:42:19 -07:00
move_grf_array_access_to_scratch();
move_uniform_array_access_to_pull_constants();
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
pack_uniform_registers();
move_push_constants_to_pull_constants();
split_virtual_grfs();
i965: Show opt_vector_float() and later passes in INTEL_DEBUG=optimizer. In order to support calling opt_vector_float() inside a condition, this patch makes OPT() a statement expression: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html We've used that elsewhere already. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-31 16:47:25 -08:00
#define OPT(pass, args...) ({ \
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
pass_num++; \
bool this_progress = pass(args); \
\
if (unlikely(INTEL_DEBUG & DEBUG_OPTIMIZER) && this_progress) { \
char filename[64]; \
i965/vec4: Use nir info instead of pulling things out of [shader_]prog Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-01 15:22:23 -07:00
snprintf(filename, 64, "%s-%s-%02d-%02d-" #pass, \
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
stage_abbrev, nir->info.name, iteration, pass_num); \
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
\
i965: Rename backend_visitor to backend_shader The backend_shader class really is a representation of a shader. The fact that it inherits from ir_visitor is somewhat immaterial. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-05-20 09:44:01 -07:00
backend_shader::dump_instructions(filename); \
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
} \
\
progress = progress || this_progress; \
i965: Show opt_vector_float() and later passes in INTEL_DEBUG=optimizer. In order to support calling opt_vector_float() inside a condition, this patch makes OPT() a statement expression: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html We've used that elsewhere already. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-31 16:47:25 -08:00
this_progress; \
})
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
if (unlikely(INTEL_DEBUG & DEBUG_OPTIMIZER)) {
char filename[64];
i965: Rename optimizer debug 00 filename This allows ls, and scripts to get the file names in the correct order of optimization. Signed-off-by: Ben Widawsky <benjamin.widawsky@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-02-11 10:55:48 -08:00
snprintf(filename, 64, "%s-%s-00-00-start",
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
stage_abbrev, nir->info.name);
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
i965: Rename backend_visitor to backend_shader The backend_shader class really is a representation of a shader. The fact that it inherits from ir_visitor is somewhat immaterial. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-05-20 09:44:01 -07:00
backend_shader::dump_instructions(filename);
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
}
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
bool progress;
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
int iteration = 0;
i965: Show opt_vector_float() and later passes in INTEL_DEBUG=optimizer. In order to support calling opt_vector_float() inside a condition, this patch makes OPT() a statement expression: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html We've used that elsewhere already. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-31 16:47:25 -08:00
int pass_num = 0;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
do {
progress = false;
i965: Show opt_vector_float() and later passes in INTEL_DEBUG=optimizer. In order to support calling opt_vector_float() inside a condition, this patch makes OPT() a statement expression: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html We've used that elsewhere already. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-31 16:47:25 -08:00
pass_num = 0;
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
iteration++;
i965: Generalize predicated break pass for use in vec4 backend. instructions in affected programs: 44204 -> 43762 (-1.00%) helped: 221 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-10-02 20:30:41 -07:00
OPT(opt_predicated_break, this);
i965/vec4: Add a pass to reduce swizzles. total instructions in shared programs: 4344280 -> 4288033 (-1.29%) instructions in affected programs: 397468 -> 341221 (-14.15%) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-17 15:13:54 -07:00
OPT(opt_reduce_swizzle);
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
OPT(dead_code_eliminate);
OPT(dead_control_flow_eliminate, this);
OPT(opt_copy_propagation);
i965/vec4: adding vec4_cmod_propagation optimization vec4 port of fs_cmod_propagation. Shader-db results (no vec4 grepping): total instructions in shared programs: 6240413 -> 6235841 (-0.07%) instructions in affected programs: 401933 -> 397361 (-1.14%) total loops in shared programs: 1979 -> 1979 (0.00%) helped: 2265 HURT: 0 v2: remove extra space and combine two if blocks, as suggested by Matt Turner v3: add condition check to bail out if current inst and inst being scanned has different writemask, as pointed by Matt Turner v3: updated shader-db numbers v4: remove block from foreach_inst_in_block_*_starting_from after commit 801f151917fedb13c5c6e96281a18d833dd6901f Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-09-28 17:00:19 +02:00
OPT(opt_cmod_propagation);
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
OPT(opt_cse);
i965/vec4: Call opt_algebraic after opt_cse. The next patch adds an algebraic optimization for the pattern sqrt a, b rcp c, a and turns it into sqrt a, b rsq c, b but many vertex shaders do a = sqrt(b); var1 /= a; var2 /= a; which generates sqrt a, b rcp c, a rcp d, a If we apply the algebraic optimization before CSE, we'll end up with sqrt a, b rsq c, b rcp d, a Applying CSE combines the RCP instructions, preventing this from happening. No shader-db changes. Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-09-27 10:34:07 -07:00
OPT(opt_algebraic);
i965: Port INTEL_DEBUG=optimizer to the vec4 backend. Largely via copy and paste. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-21 20:06:23 -07:00
OPT(opt_register_coalesce);
i965: Perform basic optimizations on the FIND_LIVE_CHANNEL opcode. v2: Save some CPU cycles by doing 'return progress' rather than 'depth++' in the discard jump special case. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-20 20:25:04 +02:00
OPT(eliminate_find_live_channel);
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
} while (progress);
i965: Show opt_vector_float() and later passes in INTEL_DEBUG=optimizer. In order to support calling opt_vector_float() inside a condition, this patch makes OPT() a statement expression: https://gcc.gnu.org/onlinedocs/gcc/Statement-Exprs.html We've used that elsewhere already. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-12-31 16:47:25 -08:00
pass_num = 0;
if (OPT(opt_vector_float)) {
OPT(opt_cse);
OPT(opt_copy_propagation, false);
OPT(opt_copy_propagation, true);
OPT(dead_code_eliminate);
i965/vec4: Do CSE, copy propagation, and DCE after opt_vector_float(). total instructions in shared programs: 5869005 -> 5868220 (-0.01%) instructions in affected programs: 70208 -> 69423 (-1.12%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-12-20 13:17:00 -08:00
}
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
i965: Lower min/max after optimization on Gen4/5. Gen4/5's SEL instruction cannot use conditional modifiers, so min/max are implemented as CMP + SEL. Handling that after optimization lets us CSE more. On Ironlake: total instructions in shared programs: 6426035 -> 6422753 (-0.05%) instructions in affected programs: 326604 -> 323322 (-1.00%) helped: 1411 total cycles in shared programs: 129184700 -> 129101586 (-0.06%) cycles in affected programs: 18950290 -> 18867176 (-0.44%) helped: 2419 HURT: 328 Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-02-11 12:27:02 -08:00
if (devinfo->gen <= 5 && OPT(lower_minmax)) {
OPT(opt_cmod_propagation);
OPT(opt_cse);
OPT(opt_copy_propagation);
OPT(dead_code_eliminate);
i965/vec4: add a SIMD lowering pass Generally, instructions in Align16 mode only ever write to a single register and don't need any form of SIMD splitting, that's why we have never had a SIMD splitting pass in the vec4 backend. However, double-precision instructions typically write 2 registers and in some cases they run into certain hardware bugs and limitations that we need to work around by splitting the instructions so we only write to 1 register at a time. This patch implements a SIMD splitting pass similar to the one in the scalar backend. Because we only use double-precision instructions in Align16 mode in gen7 (gen8+ is fully scalar and gens < 7 do not implement fp64) the pass should be a no-op on any other generation. For now the pass only handles the gen7 restriction where any instruction that writes 2 registers also needs to read 2 registers. This affects double-precision instructions reading uniforms, for example. Later patches will extend the lowering pass adding a few more cases. v2: - Move the simd lowering pass after the main optimization loop and run copy-propagation and dce if it reports progress (Curro) - Compute number of registers written instead of fixing it to 1 (Iago) - Use group from backend_instruction (Iago) - Drop assertion that checked that we only split 8-wide instructions into 4-wide. (Curro) - Don't assume that instructions can only be 8-wide, we might want to use 16-wide instructions in the future too (Curro) - Wrap gen7 workarounds in a conditional to ease adding workarounds for other gens in the future (Curro) - Handle dst/src overlap hazard (Curro) - Use the horiz_offset() helper to simplify the implementation (Curro) - Drop the assertion that checks that each split instruction writes exactly one register (Curro) - Use the copy constructor to generate split instructions with all the relevant fields initialized to the values in the original instruction instead of copying only a handful of them manually (Curro) v3 (Iago): - When copying to a temporary, allocate the number of registers required for the copy based on the size written of the lowered instruction instead of assuming that all lowered instructions produce single-register writes - Adapt to changes in offset() Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-08-29 10:41:45 +02:00
}
if (OPT(lower_simd_width)) {
OPT(opt_copy_propagation);
OPT(dead_code_eliminate);
i965: Lower min/max after optimization on Gen4/5. Gen4/5's SEL instruction cannot use conditional modifiers, so min/max are implemented as CMP + SEL. Handling that after optimization lets us CSE more. On Ironlake: total instructions in shared programs: 6426035 -> 6422753 (-0.05%) instructions in affected programs: 326604 -> 323322 (-1.00%) helped: 1411 total cycles in shared programs: 129184700 -> 129101586 (-0.06%) cycles in affected programs: 18950290 -> 18867176 (-0.44%) helped: 2419 HURT: 328 Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-02-11 12:27:02 -08:00
}
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
if (failed)
return false;
i965/vec4: Lower 64-bit MAD The previous patch made sure that we do not generate MAD instructions for any NIR's 64-bit ffma, but there is nothing preventing i965 from producing MAD instructions as a result of lowerings or optimization passes. This patch makes sure that any 64-bit MAD produced inside the driver after translating from NIR is also converted to MUL+ADD before we generate code. v2: - Use a copy constructor to copy all relevant instruction fields from the original mad into the add and mul instructions v3: - Rename the lowering and fix commit log (Matt) Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-08 11:04:34 +02:00
OPT(lower_64bit_mad_to_mul_add);
i965/vec4: run scalarize_df() after spilling Spilling of 64-bit data requires data shuffling for the corresponding scratch read/write messages. This produces unsupported swizzle regions and writemasks that we need to scalarize. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-09-01 12:01:02 +02:00
/* Run this before payload setup because tesselation shaders
* rely on it to prevent cross dvec2 regioning on DF attributes
* that are setup so that XY are on the second half of register and
* ZW are in the first half of the next.
*/
i965/vec4: add a scalarization pass for double-precision instructions The hardware only supports 32-bit swizzles, which means that we can only access directly channels XY of a DF making access to channels ZW more difficult, specially considering the various regioning restrictions imposed by the hardware. The combination of both things makes handling ramdom swizzles on DF operands rather difficult, as there are many combinations that can't be represented at all, at least not without some work and some level of instruction splitting depending on the case. Writemasks are 64-bit in general, however XY and ZW writemasks also work in 32-bit, which means these writemasks can't be represented natively, adding to the complexity. For now, we decided to try and simplify things as much as possible to avoid dealing with all this from the get go by adding a scalarization pass that runs after the main optimization loop. By fully scalarizing DF instructions in align16 we avoid most of the complexity introduced by the aforementioned hardware restrictions and we have an easier path to an initial fully functional version for the vector backend in Haswell and IvyBridge. Later, we can improve the implementation so we don't necessarily scalarize everything, iteratively adding more complexity and building on top of a framework that is already working. Curro drafted some ideas for how this could be done here: https://bugs.freedesktop.org/show_bug.cgi?id=92760#c82 v2: - Use a copy constructor for the scalar instructions so we copy all relevant instructions fields from the original instruction. v3: Fix indention in one switch (Matt) Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-24 09:20:51 +02:00
OPT(scalarize_df);
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
setup_payload();
i965: Add a debug option for spilling everything in vec4 code Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-07-23 11:11:53 +02:00
if (unlikely(INTEL_DEBUG & DEBUG_SPILL_VEC4)) {
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
/* Debug of register spilling: Go spill everything. */
i965: Factor out virtual GRF allocation to a separate object. Right now virtual GRF book-keeping and allocation is performed in each visitor class separately (among other hundred different things), leading to duplicated logic in each visitor and preventing layering as it forces any code that manipulates i965 IR and needs to allocate virtual registers to depend on the specific visitor that happens to be used to translate from GLSL IR. v2: Use realloc()/free() to allocate VGRF book-keeping arrays (Connor). Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-10 15:51:34 +02:00
const int grf_count = alloc.count;
float spill_costs[alloc.count];
bool no_spill[alloc.count];
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
evaluate_spill_costs(spill_costs, no_spill);
for (int i = 0; i < grf_count; i++) {
if (no_spill[i])
continue;
spill_reg(i);
}
i965/vec4: run scalarize_df() after spilling Spilling of 64-bit data requires data shuffling for the corresponding scratch read/write messages. This produces unsupported swizzle regions and writemasks that we need to scalarize. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-09-01 12:01:02 +02:00
/* We want to run this after spilling because 64-bit (un)spills need to
* emit code to shuffle 64-bit data for the 32-bit scratch read/write
* messages that can produce unsupported 64-bit swizzle regions.
*/
OPT(scalarize_df);
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
}
i965/vec4: Move perf_debug about register spilling into the visitor. This patch makes us only issue the performance warning about register spilling if we actually spilled registers. We also use scratch space for indirect addressing and the like. This is basically commit c51163b0cf7aff0375b1a5ea4cb3da9d9e164044 for the vec4 backend. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-06-28 20:45:47 -07:00
bool allocated_without_spills = reg_allocate();
if (!allocated_without_spills) {
compiler->shader_perf_log(log_data,
"%s shader triggered register spilling. "
"Try reducing the number of live vec4 values "
"to improve performance.\n",
stage_name);
while (!reg_allocate()) {
if (failed)
return false;
}
i965/vec4: run scalarize_df() after spilling Spilling of 64-bit data requires data shuffling for the corresponding scratch read/write messages. This produces unsupported swizzle regions and writemasks that we need to scalarize. Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-09-01 12:01:02 +02:00
/* We want to run this after spilling because 64-bit (un)spills need to
* emit code to shuffle 64-bit data for the 32-bit scratch read/write
* messages that can produce unsupported 64-bit swizzle regions.
*/
OPT(scalarize_df);
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
}
i965/vs: Add instruction scheduling. While this is ignorant of dependency control, it's still good for a 0.39% +/- 0.08% performance improvement on GLBenchmark 2.7 (n=548) v2: Rewrite as a subclass of the base class for the FS instruction scheduler, inheriting the same latency information. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2012-11-30 16:13:34 -08:00
opt_schedule_instructions();
i965/vs: Add a pass to set dependency control fields on instructions. This is a more aggressive version of the old brw_optimize() path. Reduces cycles spent in the vertex shader on minecraft by 18.6% +/- 10.0% (n=15). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 18:29:34 -08:00
opt_set_dependency_control();
i965/vec4: Convert src_reg/dst_reg to brw_reg at the end of the visitor. This patch makes the visitor convert registers to the HW_REG file at the very end, after register allocation, post-RA scheduling, and dependency control flagging. After that, everything is in fixed brw_regs. This simplifies the code generator, as it can just use the hardware registers rather than having to interpret our abstract files. In particular, interpreting the UNIFORM file meant reading prog_data to figure out where push constants are supposed to start. Having the part of the code that performs register allocation also translate everything to hardware registers seems sensible. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 16:04:15 -07:00
convert_to_hw_regs();
i965/vec4: Move c->last_scratch into vec4_visitor. Nothing outside of vec4_visitor uses it, so we may as well keep it internal. Commit db9c915abcc5ad78d2d11d0e732f04cc94631350 for the vec4 backend. (The empty class will be going away soon.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-06-28 21:02:15 -07:00
if (last_scratch > 0) {
i965/vec4: Move total_scratch calculation into the visitor. This is more consistent with how we do it in the FS backend, and reduces a tiny bit of duplication. It'll also allow for a bit more tidying. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-06-28 20:55:25 -07:00
prog_data->base.total_scratch =
i965/vec4: Move c->last_scratch into vec4_visitor. Nothing outside of vec4_visitor uses it, so we may as well keep it internal. Commit db9c915abcc5ad78d2d11d0e732f04cc94631350 for the vec4 backend. (The empty class will be going away soon.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-06-28 21:02:15 -07:00
brw_get_scratch_size(last_scratch * REG_SIZE);
i965/vec4: Move total_scratch calculation into the visitor. This is more consistent with how we do it in the FS backend, and reduces a tiny bit of duplication. It'll also allow for a bit more tidying. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-06-28 20:55:25 -07:00
}
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
return !failed;
}
i965/vs: Copy the live intervals calculation over from the FS. This is a rather pessimistic calculation, since it doesn't distinguish individual channels of a vec4, or elements of an array, but should be a minimum start for register allocation.
2011-08-16 15:09:48 -07:00
} /* namespace brw */
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
extern "C" {
i965/vs: Move uses of brw_compile from do_vs_prog to brw_vs_emit. The brw_compile structure is closely tied to the Gen4-7 hardware encoding. However, do_vs_prog is very generic: it just calls out to get a compiled program and then uploads it. This isn't ultimately where we want it, but it's a step in the right direction: it's now closer to the code generator. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 23:59:32 -08:00
/**
* Compile a vertex shader.
*
* Returns the final assembly and the program's size.
*/
const unsigned *
i965: Rename brw_foo_emit to brw_compile_foo Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 16:20:34 -07:00
brw_compile_vs(const struct brw_compiler *compiler, void *log_data,
void *mem_ctx,
const struct brw_vs_prog_key *key,
struct brw_vs_prog_data *prog_data,
i965: Move postprocess_nir to codegen time This allows us to insert NIR passes between initial NIR compilation and optimization (link time) and actual backend code-gen. In particular, it will allow us to do shader variants in NIR and share some of that shader variant code between backends. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-11-11 10:04:43 -08:00
const nir_shader *src_shader,
i965: Rename brw_foo_emit to brw_compile_foo Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 16:20:34 -07:00
gl_clip_plane *clip_planes,
bool use_legacy_snorm_formula,
int shader_time_index,
unsigned *final_assembly_size,
char **error_str)
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
{
i965: Make an is_scalar boolean in brw_compile_vs(). Shorter than compiler->scalar_stage[MESA_SHADER_VERTEX], which can help with line-wrapping. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-13 15:23:48 -08:00
const bool is_scalar = compiler->scalar_stage[MESA_SHADER_VERTEX];
i965: Move postprocess_nir to codegen time This allows us to insert NIR passes between initial NIR compilation and optimization (link time) and actual backend code-gen. In particular, it will allow us to do shader variants in NIR and share some of that shader variant code between backends. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-11-11 10:04:43 -08:00
nir_shader *shader = nir_shader_clone(mem_ctx, src_shader);
nir: pass compiler rather than devinfo to functions that call nir_optimize Later we will pass compiler to nir_optimise to be used by the loop unroll pass. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-09-15 12:20:38 +10:00
shader = brw_nir_apply_sampler_key(shader, compiler, &key->tex, is_scalar);
i965: Move postprocess_nir to codegen time This allows us to insert NIR passes between initial NIR compilation and optimization (link time) and actual backend code-gen. In particular, it will allow us to do shader variants in NIR and share some of that shader variant code between backends. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-11-11 10:04:43 -08:00
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
const unsigned *assembly = NULL;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
i965/vec4: Set VERT_BIT_EDGEFLAG based on the VUE map We also add a nice little comment to make it more clear exactly what happens with the edge flag copy. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-09 12:34:10 -07:00
if (prog_data->base.vue_map.varying_to_slot[VARYING_SLOT_EDGE] != -1) {
/* If the output VUE map contains VARYING_SLOT_EDGE then we need to copy
* the edge flag from VERT_ATTRIB_EDGEFLAG. This will be done
* automatically by brw_vec4_visitor::emit_urb_slot but we need to
* ensure that prog_data->inputs_read is accurate.
i965/vec4: Use NIR remapping for VS attributes The NIR pass already handles remapping system values to attributes for us so we delete the system value code as part of the conversion. We also change nir_lower_vs_inputs to take an explicit inputs_read bitmask and pass in the inputs_read from prog_data instead from pulling it out of NIR. This is because the version in prog_data may get EDGEFLAG added to it on some old platforms. Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-03 17:40:54 -07:00
*
* In order to make late NIR passes aware of the change, we actually
* whack shader->info.inputs_read instead. This is safe because we just
* made a copy of the shader.
i965/vec4: Set VERT_BIT_EDGEFLAG based on the VUE map We also add a nice little comment to make it more clear exactly what happens with the edge flag copy. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-09 12:34:10 -07:00
*/
assert(!is_scalar);
assert(key->copy_edgeflag);
i965/vec4: Use NIR remapping for VS attributes The NIR pass already handles remapping system values to attributes for us so we delete the system value code as part of the conversion. We also change nir_lower_vs_inputs to take an explicit inputs_read bitmask and pass in the inputs_read from prog_data instead from pulling it out of NIR. This is because the version in prog_data may get EDGEFLAG added to it on some old platforms. Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-03 17:40:54 -07:00
shader->info.inputs_read |= VERT_BIT_EDGEFLAG;
i965/vec4: Set VERT_BIT_EDGEFLAG based on the VUE map We also add a nice little comment to make it more clear exactly what happens with the edge flag copy. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-09 12:34:10 -07:00
}
intel/compiler/vs: Move inputs_read handling to generic code Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-04 14:50:20 -07:00
prog_data->inputs_read = shader->info.inputs_read;
prog_data->double_inputs_read = shader->info.double_inputs_read;
i965/vec4: Use NIR remapping for VS attributes The NIR pass already handles remapping system values to attributes for us so we delete the system value code as part of the conversion. We also change nir_lower_vs_inputs to take an explicit inputs_read bitmask and pass in the inputs_read from prog_data instead from pulling it out of NIR. This is because the version in prog_data may get EDGEFLAG added to it on some old platforms. Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-03 17:40:54 -07:00
brw_nir_lower_vs_inputs(shader, use_legacy_snorm_formula,
key->gl_attrib_wa_flags);
brw_nir_lower_vue_outputs(shader, is_scalar);
shader = brw_postprocess_nir(shader, compiler, is_scalar);
i965: Store a clip_distance_mask field similar to cull_distance_mask. This isn't useful for legacy GL, but will be used in Vulkan. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-10-03 23:41:09 -07:00
prog_data->base.clip_distance_mask =
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
((1 << shader->info.clip_distance_array_size) - 1);
i965: Use shader_info for brw_vue_prog_data::cull_distance_mask. This also allows us to move it from a GL specific location to a part of the compiler shared by both GL and Vulkan. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-10-03 23:36:30 -07:00
prog_data->base.cull_distance_mask =
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
((1 << shader->info.cull_distance_array_size) - 1) <<
shader->info.clip_distance_array_size;
i965: Use shader_info for brw_vue_prog_data::cull_distance_mask. This also allows us to move it from a GL specific location to a part of the compiler shared by both GL and Vulkan. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-10-03 23:36:30 -07:00
nir/i965: use two slots from inputs_read for dvec3/dvec4 vertex input attributes So far, input_reads was a bitmap tracking which vertex input locations were being used. In OpenGL, an attribute bigger than a vec4 (like a dvec3 or dvec4) consumes just one location, any other small attribute. So we mark the proper bit in inputs_read, and also the same bit in double_inputs_read if the attribute is a dvec3/dvec4. But in Vulkan, this is slightly different: a dvec3/dvec4 attribute consumes two locations, not just one. And hence two bits would be marked in inputs_read for the same vertex input attribute. To avoid handling two different situations in NIR, we just choose the latest one: in OpenGL, when creating NIR from GLSL/IR, any dvec3/dvec4 vertex input attribute is marked with two bits in the inputs_read bitmap (and also in the double_inputs_read), and following attributes are adjusted accordingly. As example, if in our GLSL/IR shader we have three attributes: layout(location = 0) vec3 attr0; layout(location = 1) dvec4 attr1; layout(location = 2) dvec3 attr2; then in our NIR shader we put attr0 in location 0, attr1 in locations 1 and 2, and attr2 in location 3 and 4. Checking carefully, basically we are using slots rather than locations in NIR. When emitting the vertices, we do a inverse map to know the corresponding location for each slot. v2 (Jason): - use two slots from inputs_read for dvec3/dvec4 NIR from GLSL/IR. v3 (Jason): - Fix commit log error. - Use ladder ifs and fix braces. - elements_double is divisible by 2, don't need DIV_ROUND_UP(). - Use if ladder instead of a switch. - Add comment about hardware restriction in 64bit vertex attributes. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-12-16 10:24:43 +01:00
unsigned nr_attribute_slots = _mesa_bitcount_64(prog_data->inputs_read);
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
/* gl_VertexID and gl_InstanceID are system values, but arrive via an
* incoming vertex attribute. So, add an extra slot.
*/
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
if (shader->info.system_values_read &
i965: Add support for gl_BaseVertexARB and gl_BaseInstanceARB We already have gl_BaseVertexARB in the .x component of the SGVS vec4 and plug gl_BaseInstanceARB into the last free component (.y). Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2015-12-10 12:24:50 -08:00
(BITFIELD64_BIT(SYSTEM_VALUE_BASE_VERTEX) |
BITFIELD64_BIT(SYSTEM_VALUE_BASE_INSTANCE) |
BITFIELD64_BIT(SYSTEM_VALUE_VERTEX_ID_ZERO_BASE) |
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
BITFIELD64_BIT(SYSTEM_VALUE_INSTANCE_ID))) {
nir/i965: use two slots from inputs_read for dvec3/dvec4 vertex input attributes So far, input_reads was a bitmap tracking which vertex input locations were being used. In OpenGL, an attribute bigger than a vec4 (like a dvec3 or dvec4) consumes just one location, any other small attribute. So we mark the proper bit in inputs_read, and also the same bit in double_inputs_read if the attribute is a dvec3/dvec4. But in Vulkan, this is slightly different: a dvec3/dvec4 attribute consumes two locations, not just one. And hence two bits would be marked in inputs_read for the same vertex input attribute. To avoid handling two different situations in NIR, we just choose the latest one: in OpenGL, when creating NIR from GLSL/IR, any dvec3/dvec4 vertex input attribute is marked with two bits in the inputs_read bitmap (and also in the double_inputs_read), and following attributes are adjusted accordingly. As example, if in our GLSL/IR shader we have three attributes: layout(location = 0) vec3 attr0; layout(location = 1) dvec4 attr1; layout(location = 2) dvec3 attr2; then in our NIR shader we put attr0 in location 0, attr1 in locations 1 and 2, and attr2 in location 3 and 4. Checking carefully, basically we are using slots rather than locations in NIR. When emitting the vertices, we do a inverse map to know the corresponding location for each slot. v2 (Jason): - use two slots from inputs_read for dvec3/dvec4 NIR from GLSL/IR. v3 (Jason): - Fix commit log error. - Use ladder ifs and fix braces. - elements_double is divisible by 2, don't need DIV_ROUND_UP(). - Use if ladder instead of a switch. - Add comment about hardware restriction in 64bit vertex attributes. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-12-16 10:24:43 +01:00
nr_attribute_slots++;
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
}
i965/vs: Set uses_vertexid and friends from brw_compile_vs Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-03 17:24:43 -07:00
if (shader->info.system_values_read &
BITFIELD64_BIT(SYSTEM_VALUE_BASE_VERTEX))
prog_data->uses_basevertex = true;
if (shader->info.system_values_read &
BITFIELD64_BIT(SYSTEM_VALUE_BASE_INSTANCE))
prog_data->uses_baseinstance = true;
if (shader->info.system_values_read &
BITFIELD64_BIT(SYSTEM_VALUE_VERTEX_ID_ZERO_BASE))
prog_data->uses_vertexid = true;
if (shader->info.system_values_read &
BITFIELD64_BIT(SYSTEM_VALUE_INSTANCE_ID))
prog_data->uses_instanceid = true;
i965: Add support for gl_DrawIDARB and enable extension We have to break open a new vec4 for gl_DrawIDARB. We've used up all space in the vec4 we use for SGVS and gl_DrawIDARB has to come from its own separate vertex buffer anyway. This is because we point the vb for base vertex and base instance into the draw parameter BO for indirect draw calls, but the draw id is generated by mesa in a different buffer. Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2015-12-10 12:27:38 -08:00
/* gl_DrawID has its very own vec4 */
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
if (shader->info.system_values_read &
nir/i965/anv/radv/gallium: make shader info a pointer When restoring something from shader cache we won't have and don't want to create a nir_shader this change detaches the two. There are other advantages such as being able to reuse the shader info populated by GLSL IR. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-10-13 11:41:23 +11:00
BITFIELD64_BIT(SYSTEM_VALUE_DRAW_ID)) {
i965/vs: Set uses_vertexid and friends from brw_compile_vs Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-03 17:24:43 -07:00
prog_data->uses_drawid = true;
nir/i965: use two slots from inputs_read for dvec3/dvec4 vertex input attributes So far, input_reads was a bitmap tracking which vertex input locations were being used. In OpenGL, an attribute bigger than a vec4 (like a dvec3 or dvec4) consumes just one location, any other small attribute. So we mark the proper bit in inputs_read, and also the same bit in double_inputs_read if the attribute is a dvec3/dvec4. But in Vulkan, this is slightly different: a dvec3/dvec4 attribute consumes two locations, not just one. And hence two bits would be marked in inputs_read for the same vertex input attribute. To avoid handling two different situations in NIR, we just choose the latest one: in OpenGL, when creating NIR from GLSL/IR, any dvec3/dvec4 vertex input attribute is marked with two bits in the inputs_read bitmap (and also in the double_inputs_read), and following attributes are adjusted accordingly. As example, if in our GLSL/IR shader we have three attributes: layout(location = 0) vec3 attr0; layout(location = 1) dvec4 attr1; layout(location = 2) dvec3 attr2; then in our NIR shader we put attr0 in location 0, attr1 in locations 1 and 2, and attr2 in location 3 and 4. Checking carefully, basically we are using slots rather than locations in NIR. When emitting the vertices, we do a inverse map to know the corresponding location for each slot. v2 (Jason): - use two slots from inputs_read for dvec3/dvec4 NIR from GLSL/IR. v3 (Jason): - Fix commit log error. - Use ladder ifs and fix braces. - elements_double is divisible by 2, don't need DIV_ROUND_UP(). - Use if ladder instead of a switch. - Add comment about hardware restriction in 64bit vertex attributes. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-12-16 10:24:43 +01:00
nr_attribute_slots++;
i965: Add support for gl_DrawIDARB and enable extension We have to break open a new vec4 for gl_DrawIDARB. We've used up all space in the vec4 we use for SGVS and gl_DrawIDARB has to come from its own separate vertex buffer anyway. This is because we point the vb for base vertex and base instance into the draw parameter BO for indirect draw calls, but the draw id is generated by mesa in a different buffer. Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2015-12-10 12:27:38 -08:00
}
nir/i965: use two slots from inputs_read for dvec3/dvec4 vertex input attributes So far, input_reads was a bitmap tracking which vertex input locations were being used. In OpenGL, an attribute bigger than a vec4 (like a dvec3 or dvec4) consumes just one location, any other small attribute. So we mark the proper bit in inputs_read, and also the same bit in double_inputs_read if the attribute is a dvec3/dvec4. But in Vulkan, this is slightly different: a dvec3/dvec4 attribute consumes two locations, not just one. And hence two bits would be marked in inputs_read for the same vertex input attribute. To avoid handling two different situations in NIR, we just choose the latest one: in OpenGL, when creating NIR from GLSL/IR, any dvec3/dvec4 vertex input attribute is marked with two bits in the inputs_read bitmap (and also in the double_inputs_read), and following attributes are adjusted accordingly. As example, if in our GLSL/IR shader we have three attributes: layout(location = 0) vec3 attr0; layout(location = 1) dvec4 attr1; layout(location = 2) dvec3 attr2; then in our NIR shader we put attr0 in location 0, attr1 in locations 1 and 2, and attr2 in location 3 and 4. Checking carefully, basically we are using slots rather than locations in NIR. When emitting the vertices, we do a inverse map to know the corresponding location for each slot. v2 (Jason): - use two slots from inputs_read for dvec3/dvec4 NIR from GLSL/IR. v3 (Jason): - Fix commit log error. - Use ladder ifs and fix braces. - elements_double is divisible by 2, don't need DIV_ROUND_UP(). - Use if ladder instead of a switch. - Add comment about hardware restriction in 64bit vertex attributes. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-12-16 10:24:43 +01:00
unsigned nr_attributes = nr_attribute_slots -
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
DIV_ROUND_UP(_mesa_bitcount_64(shader->info.double_inputs_read), 2);
i965/vec4: use attribute slots to calculate URB read length Do not use total attributes because a dvec3/dvec4 attribute requires two slots. So rather use total attribute slots. v2: do not use loop to calculate required attribute slots (Kenneth Graunke) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-04 12:47:57 +02:00
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
/* The 3DSTATE_VS documentation lists the lower bound on "Vertex URB Entry
* Read Length" as 1 in vec4 mode, and 0 in SIMD8 mode. Empirically, in
* vec4 mode, the hardware appears to wedge unless we read something.
*/
i965: Make an is_scalar boolean in brw_compile_vs(). Shorter than compiler->scalar_stage[MESA_SHADER_VERTEX], which can help with line-wrapping. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-13 15:23:48 -08:00
if (is_scalar)
i965/vec4: use attribute slots to calculate URB read length Do not use total attributes because a dvec3/dvec4 attribute requires two slots. So rather use total attribute slots. v2: do not use loop to calculate required attribute slots (Kenneth Graunke) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-04 12:47:57 +02:00
prog_data->base.urb_read_length =
DIV_ROUND_UP(nr_attribute_slots, 2);
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
else
i965/vec4: use attribute slots to calculate URB read length Do not use total attributes because a dvec3/dvec4 attribute requires two slots. So rather use total attribute slots. v2: do not use loop to calculate required attribute slots (Kenneth Graunke) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-04 12:47:57 +02:00
prog_data->base.urb_read_length =
DIV_ROUND_UP(MAX2(nr_attribute_slots, 1), 2);
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
prog_data->nr_attributes = nr_attributes;
i965/fs: calculate first non-payload GRF using attrib slots When computing where the first non-payload GRF starts, we can't rely on the number of attributes, as each attribute can be using 1 or 2 slots depending on whether they are a dvec3/4 or other. Instead, we need to use the number of slots used by the attributes. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-04 12:47:57 +02:00
prog_data->nr_attribute_slots = nr_attribute_slots;
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
/* Since vertex shaders reuse the same VUE entry for inputs and outputs
* (overwriting the original contents), we need to make sure the size is
* the larger of the two.
*/
const unsigned vue_entries =
i965/vec4: use attribute slots to calculate URB read length Do not use total attributes because a dvec3/dvec4 attribute requires two slots. So rather use total attribute slots. v2: do not use loop to calculate required attribute slots (Kenneth Graunke) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-04 12:47:57 +02:00
MAX2(nr_attribute_slots, (unsigned)prog_data->base.vue_map.num_slots);
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
i965/cnl: Make URB {VS, GS, HS, DS} sizes non multiple of 3 v1: By Ben Widawsky <benjamin.widawsky@intel.com> v2: v1 had an assert only for VS. Add the restriction for GS, HS and DS as well and make sure the allocated sizes are not multiple of 3. v3: Move the entry_size checks in to compiler code (Ken) Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-01-05 08:41:39 -08:00
if (compiler->devinfo->gen == 6) {
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
prog_data->base.urb_entry_size = DIV_ROUND_UP(vue_entries, 8);
i965/cnl: Make URB {VS, GS, HS, DS} sizes non multiple of 3 v1: By Ben Widawsky <benjamin.widawsky@intel.com> v2: v1 had an assert only for VS. Add the restriction for GS, HS and DS as well and make sure the allocated sizes are not multiple of 3. v3: Move the entry_size checks in to compiler code (Ken) Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-01-05 08:41:39 -08:00
} else {
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
prog_data->base.urb_entry_size = DIV_ROUND_UP(vue_entries, 4);
i965/cnl: Make URB {VS, GS, HS, DS} sizes non multiple of 3 v1: By Ben Widawsky <benjamin.widawsky@intel.com> v2: v1 had an assert only for VS. Add the restriction for GS, HS and DS as well and make sure the allocated sizes are not multiple of 3. v3: Move the entry_size checks in to compiler code (Ken) Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-01-05 08:41:39 -08:00
/* On Cannonlake software shall not program an allocation size that
* specifies a size that is a multiple of 3 64B (512-bit) cachelines.
*/
if (compiler->devinfo->gen == 10 &&
prog_data->base.urb_entry_size % 3 == 0)
prog_data->base.urb_entry_size++;
}
i965/vs: Move URB entry_size and read_length calculations to compile_vs Reviewed-By: Eduardo Lima Mitev <elima@igalia.com>
2015-10-15 11:39:06 -07:00
i965: Print VS output VUE map in Vulkan too. We need to move this to the shared layer. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>
2017-01-04 17:52:38 -08:00
if (INTEL_DEBUG & DEBUG_VS) {
fprintf(stderr, "VS Output ");
brw_print_vue_map(stderr, &prog_data->base.vue_map);
}
i965: Make an is_scalar boolean in brw_compile_vs(). Shorter than compiler->scalar_stage[MESA_SHADER_VERTEX], which can help with line-wrapping. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-13 15:23:48 -08:00
if (is_scalar) {
i965: Create a shader_dispatch_mode enum to replace VS/GS fields. We used to store the GS dispatch mode in brw_gs_prog_data while separately storing the VS dispatch mode in brw_vue_prog_data::simd8. This patch introduces an enum to represent all possible dispatch modes, and stores it in brw_vue_prog_data::dispatch_mode, unifying the two. Based on a suggestion by Matt Turner. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ben Widawsky <ben@bwidawsk.net>
2015-03-11 21:18:42 -07:00
prog_data->base.dispatch_mode = DISPATCH_MODE_SIMD8;
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
fs_visitor v(compiler, log_data, mem_ctx, key, &prog_data->base.base,
i965/shader: Get rid of the shader, prog, and shader_prog fields Unfortunately, we can't get rid of them entirely. The FS backend still needs gl_program for handling TEXTURE_RECTANGLE. The GS vec4 backend still needs gl_shader_program for handling transfom feedback. However, the VS needs neither and we can substantially reduce the amount they are used. One day we will be free from their tyranny. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-01 15:21:57 -07:00
NULL, /* prog; Only used for TEXTURE_RECTANGLE on gen < 8 */
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
shader, 8, shader_time_index);
if (!v.run_vs(clip_planes)) {
if (error_str)
*error_str = ralloc_strdup(mem_ctx, v.fail_msg);
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
return NULL;
i965: NULL check prog on shader compilation failure. Also change if (shader) to if (prog) for consistency. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-04-11 09:55:42 -07:00
}
i965/vs: Print error if vertex shader fails to compile. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-04-11 09:57:23 -07:00
i965/fs: Stop setting dispatch_grf_start_reg from the visitor Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-04-28 14:20:36 -07:00
prog_data->base.base.dispatch_grf_start_reg = v.payload.num_regs;
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
fs_generator g(compiler, log_data, mem_ctx, (void *) key,
&prog_data->base.base, v.promoted_constants,
i965/fs/generator: Take an actual shader stage rather than a string Cc: "11.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-14 20:27:51 -08:00
v.runtime_check_aads_emit, MESA_SHADER_VERTEX);
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
if (INTEL_DEBUG & DEBUG_VS) {
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
const char *debug_name =
ralloc_asprintf(mem_ctx, "%s vertex shader %s",
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
shader->info.label ? shader->info.label :
nir/i965/anv/radv/gallium: make shader info a pointer When restoring something from shader cache we won't have and don't want to create a nir_shader this change detaches the two. There are other advantages such as being able to reuse the shader info populated by GLSL IR. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-10-13 11:41:23 +11:00
"unnamed",
nir: Embed the shader_info in the nir_shader again Commit e1af20f18a86f52a9640faf2d4ff8a71b0a4fa9b changed the shader_info from being embedded into being just a pointer. The idea was that sharing the shader_info between NIR and GLSL would be easier if it were a pointer pointing to the same shader_info struct. This, however, has caused a few problems: 1) There are many things which generate NIR without GLSL. This means we have to support both NIR shaders which come from GLSL and ones that don't and need to have an info elsewhere. 2) The solution to (1) raises all sorts of ownership issues which have to be resolved with ralloc_parent checks. 3) Ever since 00620782c92100d77c660f9783504c6d80fa1d58, we've been using nir_gather_info to fill out the final shader_info. Thanks to cloning and the above ownership issues, the nir_shader::info may not point back to the gl_shader anymore and so we have to do a copy of the shader_info from NIR back to GLSL anyway. All of these issues go away if we just embed the shader_info in the nir_shader. There's a little downside of having to copy it back after calling nir_gather_info but, as explained above, we have to do that anyway. Acked-by: Timothy Arceri <tarceri@itsqueeze.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-05-08 09:20:21 -07:00
shader->info.name);
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
g.enable_debug(debug_name);
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
}
g.generate_code(v.cfg, 8);
assembly = g.get_assembly(final_assembly_size);
i965: Include codegen time in the INTEL_DEBUG=perf stall detection. In the VS case, we were missing the entire compile time in the stall detection! Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 12:55:50 -08:00
}
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
if (!assembly) {
i965: Create a shader_dispatch_mode enum to replace VS/GS fields. We used to store the GS dispatch mode in brw_gs_prog_data while separately storing the VS dispatch mode in brw_vue_prog_data::simd8. This patch introduces an enum to represent all possible dispatch modes, and stores it in brw_vue_prog_data::dispatch_mode, unifying the two. Based on a suggestion by Matt Turner. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ben Widawsky <ben@bwidawsk.net>
2015-03-11 21:18:42 -07:00
prog_data->base.dispatch_mode = DISPATCH_MODE_4X2_DUAL_OBJECT;
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
vec4_vs_visitor v(compiler, log_data, key, prog_data,
shader, clip_planes, mem_ctx,
shader_time_index, use_legacy_snorm_formula);
i965: Move legacy clip plane handling to vec4_vs_visitor. This is now only used for the vertex shader, so it makes sense to get it out of any paths run by the geometry shader. Instead of passing the gl_clip_plane array into the run() method (which is shared among all subclasses), we add it as a vec4_vs_visitor constructor parameter. This eliminates the bogus NULL parameter in the GS case. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-08-27 23:49:03 -07:00
if (!v.run()) {
i965/vs: Rework vs_emit to take a nir_shader and a brw_compiler This commit removes all dependence on GL state by getting rid of the brw_context parameter and the GL data structures. v2 (Jason Ekstrand): - Patch use_legacy_snorm_formula through as a function argument rather than trying to go through the shader key. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-08 13:53:33 -07:00
if (error_str)
*error_str = ralloc_strdup(mem_ctx, v.fail_msg);
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
return NULL;
}
i965/vec4: Wrap vec4_generator in a C function. vec4_generator is a class for convenience, but only exports a single method as its public API. It makes much more sense to just export a single function. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-10-22 15:01:27 -07:00
assembly = brw_vec4_generate_assembly(compiler, log_data, mem_ctx,
shader, &prog_data->base, v.cfg,
final_assembly_size);
i965: Generate vs code using scalar backend for BDW+ With everything in place, we can now use the scalar backend compiler for vertex shaders on BDW+. We make scalar vertex shaders the default on BDW+ but add a new vec4vs debug option to force the vec4 backend. No piglit regressions. Performance impact is minimal, I see a ~1.5 improvement on the T-Rex GLBenchmark case, but in general it's in the noise. Some of our internal synthetic, vs bounded benchmarks show great improvement, 20%-40% in some cases, but real-world cases are mostly unaffected. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-20 23:29:41 -07:00
}
i965: Include codegen time in the INTEL_DEBUG=perf stall detection. In the VS case, we were missing the entire compile time in the stall detection! Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-11-30 12:55:50 -08:00
i965: Create a new vec4 backend for Broadwell. This replaces the old vec4_generator backend. v2: Port to use the C-based instruction representation. Also, remove Geometry Shader offset hacks - the visitor will handle those instead of this code. v3: Texturing fixes (including adding textureGather support). v4: Pass brw_context to gen8_instruction functions as required. v5: Add SHADER_OPCODE_TXF_MCS support; port DUAL_INSTANCED gs fixes (caught by Eric). Simplify ADDC/SUBB handling; add comments to gen8_set_dp_message calls (suggested by Matt). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2012-12-06 22:37:34 -08:00
return assembly;
i965/vs: Move some functions from brw_vec4_emit.cpp to brw_vec4.cpp. This leaves only the final code generation stage in brw_vec4_emit.cpp, moving the payload setup, run(), and brw_vs_emit functions to brw_vec4.cpp. The fragment shader backend puts these functions in brw_fs.cpp, so this patch also helps with consistency. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-11-26 21:46:27 -08:00
}
} /* extern "C" */
Reference in a new issue Copy permalink