fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2025-12-22 20:00:10 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
mesa/src/intel/compiler/brw_eu_emit.c

3717 lines
134 KiB
C
Raw Normal View History

Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/*
Copyright (C) Intel Corp. 2006. All Rights Reserved.
s/Tungsten Graphics/VMware/ Tungsten Graphics Inc. was acquired by VMware Inc. in 2008. Leaving the old copyright name is creating unnecessary confusion, hence this change. This was the sed script I used: $ cat tg2vmw.sed # Run as: # # git reset --hard HEAD && find include scons src -type f -not -name 'sed*' -print0 | xargs -0 sed -i -f tg2vmw.sed # # Rename copyrights s/Tungsten Gra\(ph\|hp\)ics,\? [iI]nc\.\?\(, Cedar Park\)\?\(, Austin\)\?\(, \(Texas\|TX\)\)\?\.\?/VMware, Inc./g /Copyright/s/Tungsten Graphics\(,\? [iI]nc\.\)\?\(, Cedar Park\)\?\(, Austin\)\?\(, \(Texas\|TX\)\)\?\.\?/VMware, Inc./ s/TUNGSTEN GRAPHICS/VMWARE/g # Rename emails s/alanh@tungstengraphics.com/alanh@vmware.com/ s/jens@tungstengraphics.com/jowen@vmware.com/g s/jrfonseca-at-tungstengraphics-dot-com/jfonseca-at-vmware-dot-com/ s/jrfonseca\?@tungstengraphics.com/jfonseca@vmware.com/g s/keithw\?@tungstengraphics.com/keithw@vmware.com/g s/michel@tungstengraphics.com/daenzer@vmware.com/g s/thomas-at-tungstengraphics-dot-com/thellstom-at-vmware-dot-com/ s/zack@tungstengraphics.com/zackr@vmware.com/ # Remove dead links s@Tungsten Graphics (http://www.tungstengraphics.com)@Tungsten Graphics@g # C string src/gallium/state_trackers/vega/api_misc.c s/"Tungsten Graphics, Inc"/"VMware, Inc"/ Reviewed-by: Brian Paul <brianp@vmware.com>
2014-01-17 16:27:50 +00:00
Intel funded Tungsten Graphics to
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
develop this 3D driver.
i965: Drop trailing whitespace from files shared with intel-gpu-tools. Performed via s/ *$//g. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:39:03 -08:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
i965: Drop trailing whitespace from files shared with intel-gpu-tools. Performed via s/ *$//g. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:39:03 -08:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
The above copyright notice and this permission notice (including the
next paragraph) shall be included in all copies or substantial
portions of the Software.
i965: Drop trailing whitespace from files shared with intel-gpu-tools. Performed via s/ *$//g. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:39:03 -08:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
i965: Drop trailing whitespace from files shared with intel-gpu-tools. Performed via s/ *$//g. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:39:03 -08:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
**********************************************************************/
/*
* Authors:
s/Tungsten Graphics/VMware/ Tungsten Graphics Inc. was acquired by VMware Inc. in 2008. Leaving the old copyright name is creating unnecessary confusion, hence this change. This was the sed script I used: $ cat tg2vmw.sed # Run as: # # git reset --hard HEAD && find include scons src -type f -not -name 'sed*' -print0 | xargs -0 sed -i -f tg2vmw.sed # # Rename copyrights s/Tungsten Gra\(ph\|hp\)ics,\? [iI]nc\.\?\(, Cedar Park\)\?\(, Austin\)\?\(, \(Texas\|TX\)\)\?\.\?/VMware, Inc./g /Copyright/s/Tungsten Graphics\(,\? [iI]nc\.\)\?\(, Cedar Park\)\?\(, Austin\)\?\(, \(Texas\|TX\)\)\?\.\?/VMware, Inc./ s/TUNGSTEN GRAPHICS/VMWARE/g # Rename emails s/alanh@tungstengraphics.com/alanh@vmware.com/ s/jens@tungstengraphics.com/jowen@vmware.com/g s/jrfonseca-at-tungstengraphics-dot-com/jfonseca-at-vmware-dot-com/ s/jrfonseca\?@tungstengraphics.com/jfonseca@vmware.com/g s/keithw\?@tungstengraphics.com/keithw@vmware.com/g s/michel@tungstengraphics.com/daenzer@vmware.com/g s/thomas-at-tungstengraphics-dot-com/thellstom-at-vmware-dot-com/ s/zack@tungstengraphics.com/zackr@vmware.com/ # Remove dead links s@Tungsten Graphics (http://www.tungstengraphics.com)@Tungsten Graphics@g # C string src/gallium/state_trackers/vega/api_misc.c s/"Tungsten Graphics, Inc"/"VMware, Inc"/ Reviewed-by: Brian Paul <brianp@vmware.com>
2014-01-17 16:27:50 +00:00
* Keith Whitwell <keithw@vmware.com>
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
*/
i965: Drop trailing whitespace from files shared with intel-gpu-tools. Performed via s/ *$//g. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:39:03 -08:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: split EU defines to brw_eu_defines.h Split out the EU defines from the 'generic' ones, as the former are more compiler oriented. With a later commit we'll move brw_eu_defines.h alongside the compiler infra to src/intel/. Pulling all the defines in there seems overzealous. Some defines are used by both i965 and the i965 compiler. Those are moved to brw_eu_defines.h, and annotated accordingly. The i965 users were updated to have the extre include to indicate that. With future work we might provide a better, split but for now this seems reasonable. Cc: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Emil Velikov <emil.velikov@collabora.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-03-09 00:44:29 +00:00
#include "brw_eu_defines.h"
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
#include "brw_eu.h"
util: Move ralloc to a new src/util directory. For a long time, we've wanted a place to put utility code which isn't directly tied to Mesa or Gallium internals. This patch creates a new src/util directory for exactly that purpose, and builds the contents as libmesautil.la. ralloc seemed like a good first candidate. These days, it's directly used by mesa/main, i965, i915, and r300g, so keeping it in src/glsl didn't make much sense. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> v2 (Jason Ekstrand): More realloc uses and some scons fixes Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Marek Olšák <marek.olsak@amd.com>
2014-02-24 23:39:14 -08:00
#include "util/ralloc.h"
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
/**
* Prior to Sandybridge, the SEND instruction accepted non-MRF source
* registers, implicitly moving the operand to a message register.
*
* On Sandybridge, this is no longer the case. This function performs the
* explicit move; it should be called before emitting a SEND instruction.
*/
i965/vs: Add support for pull constant loads for uniform arrays. v2: reworked the instruction emit and made use of gen6_resolve_implied_move, from Ken's review
2011-08-22 10:35:24 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
gen6_resolve_implied_move(struct brw_codegen *p,
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
struct brw_reg *src,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_reg_nr)
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Remove the context field from brw_compiler Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 14:13:58 -07:00
if (devinfo->gen < 6)
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
return;
i965: Make gen6_resolve_implied_move a no-op for MRF sources. Attempting to move an MRF to a MRF is not only pointless, it will fail because MRFs are read-only, resulting in garbage in your register. If we already set up a MRF source, there's nothing to resolve anyway. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-14 19:49:54 -08:00
if (src->file == BRW_MESSAGE_REGISTER_FILE)
return;
i965: Fix null register use in Sandybridge implied move resolution. Fixes regressions caused by commit 9a21bc6401, namely GPU hangs when running gnome-shell or compiz (Mesa bugs #35820 and #35853). I incorrectly refactored the case that dealt with ARF_NULL; even in that case, the source register needs to be changed to the MRF. NOTE: This is a candidate for the 7.10 branch (if 9a21bc6401 is cherry-picked, take this one too).
2011-04-03 00:57:30 -07:00
if (src->file != BRW_ARCHITECTURE_REGISTER_FILE || src->nr != BRW_ARF_NULL) {
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
assert(devinfo->gen < 12);
i965: Fix null register use in Sandybridge implied move resolution. Fixes regressions caused by commit 9a21bc6401, namely GPU hangs when running gnome-shell or compiz (Mesa bugs #35820 and #35853). I incorrectly refactored the case that dealt with ARF_NULL; even in that case, the source register needs to be changed to the MRF. NOTE: This is a candidate for the 7.10 branch (if 9a21bc6401 is cherry-picked, take this one too).
2011-04-03 00:57:30 -07:00
brw_push_insn_state(p);
i965: Replace guess_execution_size with something simpler. guess_execution_size() does two things: 1. Cope with small destination registers. 2. Cope with SIMD8 vs SIMD16 mode. This patch replaces the first with a simple if block in brw_set_dest: if the destination register width is less than 8, you probably want the execution size to match. (I didn't put this in the 3src block because it doesn't seem to matter.) Since only the FS compiler cares about SIMD16 mode, it's easy to just set the default execution size there. This pattern was already been proven in the Gen8+ generator, but we didn't port it back to the existing generator when we combined the two. This is based on a patch from Ken from about a year ago. I've rebased it and and fixed a few bugs. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 12:40:34 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_8);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 16:57:02 -07:00
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
i965: Fix null register use in Sandybridge implied move resolution. Fixes regressions caused by commit 9a21bc6401, namely GPU hangs when running gnome-shell or compiz (Mesa bugs #35820 and #35853). I incorrectly refactored the case that dealt with ARF_NULL; even in that case, the source register needs to be changed to the MRF. NOTE: This is a candidate for the 7.10 branch (if 9a21bc6401 is cherry-picked, take this one too).
2011-04-03 00:57:30 -07:00
brw_MOV(p, retype(brw_message_reg(msg_reg_nr), BRW_REGISTER_TYPE_UD),
retype(*src, BRW_REGISTER_TYPE_UD));
brw_pop_insn_state(p);
}
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
*src = brw_message_reg(msg_reg_nr);
}
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
gen7_convert_mrf_to_grf(struct brw_codegen *p, struct brw_reg *reg)
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
{
i965: Cite the Ivybridge PRM for why the fake MRF range is what it is. The exact text is in the public docs, so we should cite those. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-07-10 13:16:13 -07:00
/* From the Ivybridge PRM, Volume 4 Part 3, page 218 ("send"):
i965: Bump Ivybridge's fake MRF range to g112-127 instead of g111-126. When I originally implemented the hack to use GRFs 111+ as fake MRFs, I did so purely to avoid rewriting all the code that dealt with MRFs. However, it turns out that a similar hack is actually required. Newly discovered language in the BSpec indicates that SEND instructions with EOT set "should" use g112-g127 as their source registers. Based on assertions in the simulator, this is actually a requirement on certain platforms. Since we're faking MRFs already, we may as well use the officially sanctioned range. My guess is that we avoided this issue because we seldom use m0: URB writes in the new VS backend start at m1, and RT writes in the new FS backend start at m2. NOTE: This is a candidate for stable release branches. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2012-01-17 08:08:25 -08:00
* "The send with EOT should use register space R112-R127 for <src>. This is
* to enable loading of a new thread into the same slot while the message
* with EOT for current thread is pending dispatch."
*
* Since we're pretending to have 16 MRFs anyway, we may as well use the
* registers required for messages with EOT.
*/
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Remove the context field from brw_compiler Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 14:13:58 -07:00
if (devinfo->gen >= 7 && reg->file == BRW_MESSAGE_REGISTER_FILE) {
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
reg->file = BRW_GENERAL_REGISTER_FILE;
i965/fs: Fix rendering corruption in unigine tropics. We were allocating registers into the MRF hack region, resulting in sparkly renering in a few of the scenes. We could do better allocation by making an MRF class, having MRFs conflict with the corresponding GRFs, and tracking the live intervals of the "MRF"s and setting up the conflicts. But this is way easier for the moment. NOTE: This is a candidate for the 8.0 branch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-01-27 12:54:11 -08:00
reg->nr += GEN7_MRF_HACK_START;
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
}
}
i965: Make some EU emit code for DP read/write messages non-static. We keep building these strange interfaces for DP read/write where there's a helper function with some partially-specific, partially-general controls, which is used in exactly one place in code generation. Making these public will let us set up those instructions in the one place they're to be generated.
2011-08-07 13:16:06 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_set_dest(struct brw_codegen *p, brw_inst *inst, struct brw_reg dest)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
i965: Move MRF register asserts out of brw_reg.h In a later patch we will make BRW_MAX_MRF return a different value depending on the hardware generation, but it is inconvenient to add a gen parameter to the brw_reg functions only for the assertions, so move these to places where we have the hardware generation available. Ken suggested to add the asserts to brw_set_src0 and brw_set_dest since that would make sure that we catch all uses of MRF registers, even those coming from modules that generate native code directly, like blorp. Unfortunately, this is very late in the process which can make things harder to debug, so add asserts to the generator as well. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-16 09:08:19 +02:00
if (dest.file == BRW_MESSAGE_REGISTER_FILE)
i965: Use BRW_MRF_COMPR4 macro in more places. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 10:23:12 -08:00
assert((dest.nr & ~BRW_MRF_COMPR4) < BRW_MAX_MRF(devinfo->gen));
intel/compiler: Avoid false positive assertions A follow on patch will move the 'nr' field to the union containing the immediate field, so prepare by checking that we're only testing these assertions if the .file is correct. The assertions with != ARF were kind of silly to begin with because the <128 check is specifically only for things in the GRF. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-12-10 11:42:44 -08:00
else if (dest.file == BRW_GENERAL_REGISTER_FILE)
i965: add some register number assertions Haven't seen failures yet, but if/when there are, more investigation will be done.
2009-03-13 09:17:08 -06:00
assert(dest.nr < 128);
intel/compiler: Don't change hstride if not needed Alignment requirements may have changed the horizontal stride already, so don't set it if not required to avoid breaking said requirements. Fixes several tests such as dEQP-VK.subgroups.vote.graphics.subgroupallequal_int8_t Signed-off-by: Iván Briano <ivan.briano@intel.com> Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-10-23 09:18:03 -07:00
/* The hardware has a restriction where a destination of size Byte with
* a stride of 1 is only allowed for a packed byte MOV. For any other
* instruction, the stride must be at least 2, even when the destination
* is the NULL register.
intel/eu: force stride of 2 on NULL register for Byte instructions The hardware only allows a stride of 1 on a Byte destination for raw byte MOV instructions. This is required even when the destination is the NULL register. Rather than making sure that we emit a proper NULL:B destination every time we need one, just fix it at emission time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2018-11-07 12:08:02 +01:00
*/
if (dest.file == BRW_ARCHITECTURE_REGISTER_FILE &&
dest.nr == BRW_ARF_NULL &&
intel/compiler: Don't change hstride if not needed Alignment requirements may have changed the horizontal stride already, so don't set it if not required to avoid breaking said requirements. Fixes several tests such as dEQP-VK.subgroups.vote.graphics.subgroupallequal_int8_t Signed-off-by: Iván Briano <ivan.briano@intel.com> Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-10-23 09:18:03 -07:00
type_sz(dest.type) == 1 &&
dest.hstride == BRW_HORIZONTAL_STRIDE_1) {
intel/eu: force stride of 2 on NULL register for Byte instructions The hardware only allows a stride of 1 on a Byte destination for raw byte MOV instructions. This is required even when the destination is the NULL register. Rather than making sure that we emit a proper NULL:B destination every time we need one, just fix it at emission time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2018-11-07 12:08:02 +01:00
dest.hstride = BRW_HORIZONTAL_STRIDE_2;
}
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
gen7_convert_mrf_to_grf(p, &dest);
intel/eu/gen12: Codegen pathological SEND source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 18:13:42 -07:00
if (devinfo->gen >= 12 &&
(brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SEND ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDC)) {
assert(dest.file == BRW_GENERAL_REGISTER_FILE ||
dest.file == BRW_ARCHITECTURE_REGISTER_FILE);
assert(dest.address_mode == BRW_ADDRESS_DIRECT);
assert(dest.subnr == 0);
assert(brw_inst_exec_size(devinfo, inst) == BRW_EXECUTE_1 ||
(dest.hstride == BRW_HORIZONTAL_STRIDE_1 &&
dest.vstride == dest.width + 1));
assert(!dest.negate && !dest.abs);
brw_inst_set_dst_reg_file(devinfo, inst, dest.file);
brw_inst_set_dst_da_reg_nr(devinfo, inst, dest.nr);
} else if (brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDS ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDSC) {
assert(devinfo->gen < 12);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
assert(dest.file == BRW_GENERAL_REGISTER_FILE ||
dest.file == BRW_ARCHITECTURE_REGISTER_FILE);
assert(dest.address_mode == BRW_ADDRESS_DIRECT);
assert(dest.subnr % 16 == 0);
assert(dest.hstride == BRW_HORIZONTAL_STRIDE_1 &&
dest.vstride == dest.width + 1);
assert(!dest.negate && !dest.abs);
brw_inst_set_dst_da_reg_nr(devinfo, inst, dest.nr);
brw_inst_set_dst_da16_subreg_nr(devinfo, inst, dest.subnr / 16);
brw_inst_set_send_dst_reg_file(devinfo, inst, dest.file);
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_dst_file_type(devinfo, inst, dest.file, dest.type);
brw_inst_set_dst_address_mode(devinfo, inst, dest.address_mode);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
if (dest.address_mode == BRW_ADDRESS_DIRECT) {
brw_inst_set_dst_da_reg_nr(devinfo, inst, dest.nr);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
brw_inst_set_dst_da1_subreg_nr(devinfo, inst, dest.subnr);
if (dest.hstride == BRW_HORIZONTAL_STRIDE_0)
dest.hstride = BRW_HORIZONTAL_STRIDE_1;
brw_inst_set_dst_hstride(devinfo, inst, dest.hstride);
} else {
brw_inst_set_dst_da16_subreg_nr(devinfo, inst, dest.subnr / 16);
brw_inst_set_da16_writemask(devinfo, inst, dest.writemask);
if (dest.file == BRW_GENERAL_REGISTER_FILE ||
dest.file == BRW_MESSAGE_REGISTER_FILE) {
assert(dest.writemask != 0);
}
/* From the Ivybridge PRM, Vol 4, Part 3, Section 5.2.4.1:
* Although Dst.HorzStride is a don't care for Align16, HW needs
* this to be programmed as "01".
*/
brw_inst_set_dst_hstride(devinfo, inst, 1);
i965: Fix writemask != 0 assertions on Sandybridge. This fixes myriads of regressions since commit 169f9c030c16d1247a3a7629 ("i965: Add an assertion that writemask != NULL for non-ARFs."). On Sandybridge, our control flow handling (such as brw_IF) does: brw_set_dest(p, insn, brw_imm_w(0)); insn->bits1.branch_gen6.jump_count = 0; This results in a IMM destination with zero for the writemask. IMM destinations are rather bizarre, but the code has been working for ages, so I'm loathe to change it. Fixes glxgears on Sandybridge. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-17 11:54:05 -07:00
}
i965: Fix else and brace placement in brw_eu_emit.c. I'm making a lot of changes to this area, and I figured I may as well not conflate these trivial changes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-04 16:55:59 -07:00
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_dst_ia_subreg_nr(devinfo, inst, dest.subnr);
/* These are different sizes in align1 vs align16:
*/
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
brw_inst_set_dst_ia1_addr_imm(devinfo, inst,
dest.indirect_offset);
if (dest.hstride == BRW_HORIZONTAL_STRIDE_0)
dest.hstride = BRW_HORIZONTAL_STRIDE_1;
brw_inst_set_dst_hstride(devinfo, inst, dest.hstride);
} else {
brw_inst_set_dst_ia16_addr_imm(devinfo, inst,
dest.indirect_offset);
/* even ignored in da16, still need to set as '01' */
brw_inst_set_dst_hstride(devinfo, inst, 1);
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
}
i965: Replace guess_execution_size with something simpler. guess_execution_size() does two things: 1. Cope with small destination registers. 2. Cope with SIMD8 vs SIMD16 mode. This patch replaces the first with a simple if block in brw_set_dest: if the destination register width is less than 8, you probably want the execution size to match. (I didn't put this in the 3src block because it doesn't seem to matter.) Since only the FS compiler cares about SIMD16 mode, it's easy to just set the default execution size there. This pattern was already been proven in the Gen8+ generator, but we didn't port it back to the existing generator when we combined the two. This is based on a patch from Ken from about a year ago. I've rebased it and and fixed a few bugs. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 12:40:34 -07:00
/* Generators should set a default exec_size of either 8 (SIMD4x2 or SIMD8)
* or 16 (SIMD16), as that's normally correct. However, when dealing with
intel/eu: Make automatic exec sizes a configurable option We have had a feature in codegen for some time that tries to automatically infer the execution size of an instruction from the width of its destination. For things such as fixed function GS, clipper, and SF programs, this is very useful because they tend to have lots of hand-rolled register setup and trying to specify the exec size all the time would be prohibitive. For things that come from a higher-level IR, however, it's easier to just set the right size all the time and the automatic exec sizes can, in fact, cause problems. This commit makes it optional while enabling it by default. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-31 09:41:22 -07:00
* small registers, it can be useful for us to automatically reduce it to
* match the register size.
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
*/
intel/eu: Make automatic exec sizes a configurable option We have had a feature in codegen for some time that tries to automatically infer the execution size of an instruction from the width of its destination. For things such as fixed function GS, clipper, and SF programs, this is very useful because they tend to have lots of hand-rolled register setup and trying to specify the exec size all the time would be prohibitive. For things that come from a higher-level IR, however, it's easier to just set the right size all the time and the automatic exec sizes can, in fact, cause problems. This commit makes it optional while enabling it by default. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-31 09:41:22 -07:00
if (p->automatic_exec_sizes) {
/*
* In platforms that support fp64 we can emit instructions with a width
* of 4 that need two SIMD8 registers and an exec_size of 8 or 16. In
* these cases we need to make sure that these instructions have their
* exec sizes set properly when they are emitted and we can't rely on
* this code to fix it.
*/
bool fix_exec_size;
if (devinfo->gen >= 6)
fix_exec_size = dest.width < BRW_EXECUTE_4;
else
fix_exec_size = dest.width < BRW_EXECUTE_8;
if (fix_exec_size)
brw_inst_set_exec_size(devinfo, inst, dest.width);
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Make some EU emit code for DP read/write messages non-static. We keep building these strange interfaces for DP read/write where there's a helper function with some partially-specific, partially-general controls, which is used in exactly one place in code generation. Making these public will let us set up those instructions in the one place they're to be generated.
2011-08-07 13:16:06 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_set_src0(struct brw_codegen *p, brw_inst *inst, struct brw_reg reg)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965/gen7: Add some safety checks for send messages from GRFs.
2012-11-09 11:38:14 -08:00
i965: Move MRF register asserts out of brw_reg.h In a later patch we will make BRW_MAX_MRF return a different value depending on the hardware generation, but it is inconvenient to add a gen parameter to the brw_reg functions only for the assertions, so move these to places where we have the hardware generation available. Ken suggested to add the asserts to brw_set_src0 and brw_set_dest since that would make sure that we catch all uses of MRF registers, even those coming from modules that generate native code directly, like blorp. Unfortunately, this is very late in the process which can make things harder to debug, so add asserts to the generator as well. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-16 09:08:19 +02:00
if (reg.file == BRW_MESSAGE_REGISTER_FILE)
i965: Use BRW_MRF_COMPR4 macro in more places. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 10:23:12 -08:00
assert((reg.nr & ~BRW_MRF_COMPR4) < BRW_MAX_MRF(devinfo->gen));
intel/compiler: Avoid false positive assertions A follow on patch will move the 'nr' field to the union containing the immediate field, so prepare by checking that we're only testing these assertions if the .file is correct. The assertions with != ARF were kind of silly to begin with because the <128 check is specifically only for things in the GRF. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-12-10 11:42:44 -08:00
else if (reg.file == BRW_GENERAL_REGISTER_FILE)
i965: add some register number assertions Haven't seen failures yet, but if/when there are, more investigation will be done.
2009-03-13 09:17:08 -06:00
assert(reg.nr < 128);
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
gen7_convert_mrf_to_grf(p, &reg);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
if (devinfo->gen >= 6 &&
(brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SEND ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDC ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDS ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDSC)) {
i965/gen7: Add some safety checks for send messages from GRFs.
2012-11-09 11:38:14 -08:00
/* Any source modifiers or regions will be ignored, since this just
* identifies the MRF/GRF to start reading the message contents from.
* Check for some likely failures.
*/
assert(!reg.negate);
assert(!reg.abs);
assert(reg.address_mode == BRW_ADDRESS_DIRECT);
}
intel/eu/gen12: Codegen pathological SEND source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 18:13:42 -07:00
if (devinfo->gen >= 12 &&
(brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SEND ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDC)) {
assert(reg.file != BRW_IMMEDIATE_VALUE);
assert(reg.address_mode == BRW_ADDRESS_DIRECT);
assert(reg.subnr == 0);
intel/compiler: Relax SENDS regioning assertions The next commit fixes a mistake in the assembler and ends up running afoul of this assertion. Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5956>
2020-07-15 15:12:57 -07:00
assert(has_scalar_region(reg) ||
intel/eu/gen12: Codegen pathological SEND source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 18:13:42 -07:00
(reg.hstride == BRW_HORIZONTAL_STRIDE_1 &&
reg.vstride == reg.width + 1));
assert(!reg.negate && !reg.abs);
brw_inst_set_send_src0_reg_file(devinfo, inst, reg.file);
brw_inst_set_src0_da_reg_nr(devinfo, inst, reg.nr);
} else if (brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDS ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDSC) {
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
assert(reg.file == BRW_GENERAL_REGISTER_FILE);
assert(reg.address_mode == BRW_ADDRESS_DIRECT);
assert(reg.subnr % 16 == 0);
intel/compiler: Relax SENDS regioning assertions The next commit fixes a mistake in the assembler and ends up running afoul of this assertion. Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5956>
2020-07-15 15:12:57 -07:00
assert(has_scalar_region(reg) ||
(reg.hstride == BRW_HORIZONTAL_STRIDE_1 &&
reg.vstride == reg.width + 1));
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
assert(!reg.negate && !reg.abs);
brw_inst_set_src0_da_reg_nr(devinfo, inst, reg.nr);
brw_inst_set_src0_da16_subreg_nr(devinfo, inst, reg.subnr / 16);
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_src0_file_type(devinfo, inst, reg.file, reg.type);
brw_inst_set_src0_abs(devinfo, inst, reg.abs);
brw_inst_set_src0_negate(devinfo, inst, reg.negate);
brw_inst_set_src0_address_mode(devinfo, inst, reg.address_mode);
if (reg.file == BRW_IMMEDIATE_VALUE) {
if (reg.type == BRW_REGISTER_TYPE_DF ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_DIM)
brw_inst_set_imm_df(devinfo, inst, reg.df);
else if (reg.type == BRW_REGISTER_TYPE_UQ ||
reg.type == BRW_REGISTER_TYPE_Q)
brw_inst_set_imm_uq(devinfo, inst, reg.u64);
else
brw_inst_set_imm_ud(devinfo, inst, reg.ud);
intel/eu/gen12: Fix codegen of immediate source regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 17:52:54 -07:00
if (devinfo->gen < 12 && type_sz(reg.type) < 8) {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_src1_reg_file(devinfo, inst,
BRW_ARCHITECTURE_REGISTER_FILE);
brw_inst_set_src1_reg_hw_type(devinfo, inst,
brw_inst_src0_reg_hw_type(devinfo, inst));
}
i965: Fix else and brace placement in brw_eu_emit.c. I'm making a lot of changes to this area, and I figured I may as well not conflate these trivial changes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-04 16:55:59 -07:00
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
if (reg.address_mode == BRW_ADDRESS_DIRECT) {
brw_inst_set_src0_da_reg_nr(devinfo, inst, reg.nr);
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
brw_inst_set_src0_da1_subreg_nr(devinfo, inst, reg.subnr);
} else {
brw_inst_set_src0_da16_subreg_nr(devinfo, inst, reg.subnr / 16);
}
} else {
brw_inst_set_src0_ia_subreg_nr(devinfo, inst, reg.subnr);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
brw_inst_set_src0_ia1_addr_imm(devinfo, inst, reg.indirect_offset);
} else {
brw_inst_set_src0_ia16_addr_imm(devinfo, inst, reg.indirect_offset);
}
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
if (reg.width == BRW_WIDTH_1 &&
brw_inst_exec_size(devinfo, inst) == BRW_EXECUTE_1) {
brw_inst_set_src0_hstride(devinfo, inst, BRW_HORIZONTAL_STRIDE_0);
brw_inst_set_src0_width(devinfo, inst, BRW_WIDTH_1);
brw_inst_set_src0_vstride(devinfo, inst, BRW_VERTICAL_STRIDE_0);
} else {
brw_inst_set_src0_hstride(devinfo, inst, reg.hstride);
brw_inst_set_src0_width(devinfo, inst, reg.width);
brw_inst_set_src0_vstride(devinfo, inst, reg.vstride);
}
i965: Use correct VertStride on align16 instructions. In commit c35fa7a, we changed the "width" of DF source registers to 2, which is conceptually fine. Unfortunately a VertStride of 2 is not allowed by align16 instructions on IVB/BYT, and the regular VertStride of 4 works fine in any case. See generated_tests/spec/arb_gpu_shader_fp64/execution/built-in-functions/vs-round-double.shader_test for example: cmp.ge.f0(8) g18<1>DF g1<0>.xyxyDF -g8<2>DF { align16 1Q }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed cmp.ge.f0(8) g19<1>DF g1<0>.xyxyDF -g9<2>DF { align16 2N }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed v2: - Add spec quote (Curro). - Change the condition to only BRW_VERTICAL_STRIDE_2 (Curro) Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-01-20 13:35:33 -08:00
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_src0_da16_swiz_x(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_X));
brw_inst_set_src0_da16_swiz_y(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_Y));
brw_inst_set_src0_da16_swiz_z(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_Z));
brw_inst_set_src0_da16_swiz_w(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_W));
if (reg.vstride == BRW_VERTICAL_STRIDE_8) {
/* This is an oddity of the fact we're using the same
* descriptions for registers in align_16 as align_1:
*/
brw_inst_set_src0_vstride(devinfo, inst, BRW_VERTICAL_STRIDE_4);
} else if (devinfo->gen == 7 && !devinfo->is_haswell &&
reg.type == BRW_REGISTER_TYPE_DF &&
reg.vstride == BRW_VERTICAL_STRIDE_2) {
/* From SNB PRM:
*
* "For Align16 access mode, only encodings of 0000 and 0011
* are allowed. Other codes are reserved."
*
* Presumably the DevSNB behavior applies to IVB as well.
*/
brw_inst_set_src0_vstride(devinfo, inst, BRW_VERTICAL_STRIDE_4);
} else {
brw_inst_set_src0_vstride(devinfo, inst, reg.vstride);
}
i965: Use correct VertStride on align16 instructions. In commit c35fa7a, we changed the "width" of DF source registers to 2, which is conceptually fine. Unfortunately a VertStride of 2 is not allowed by align16 instructions on IVB/BYT, and the regular VertStride of 4 works fine in any case. See generated_tests/spec/arb_gpu_shader_fp64/execution/built-in-functions/vs-round-double.shader_test for example: cmp.ge.f0(8) g18<1>DF g1<0>.xyxyDF -g8<2>DF { align16 1Q }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed cmp.ge.f0(8) g19<1>DF g1<0>.xyxyDF -g9<2>DF { align16 2N }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed v2: - Add spec quote (Curro). - Change the condition to only BRW_VERTICAL_STRIDE_2 (Curro) Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-01-20 13:35:33 -08:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
}
}
i965: Reformat brw_set_src1 so it can be easily found with grep.
2014-05-02 14:49:24 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_set_src1(struct brw_codegen *p, brw_inst *inst, struct brw_reg reg)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
fix a couple of cases where a message reg is used as an instruction source.
2006-09-01 14:18:06 +00:00
intel/compiler: Avoid false positive assertions A follow on patch will move the 'nr' field to the union containing the immediate field, so prepare by checking that we're only testing these assertions if the .file is correct. The assertions with != ARF were kind of silly to begin with because the <128 check is specifically only for things in the GRF. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-12-10 11:42:44 -08:00
if (reg.file == BRW_GENERAL_REGISTER_FILE)
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
assert(reg.nr < 128);
i965: add some register number assertions Haven't seen failures yet, but if/when there are, more investigation will be done.
2009-03-13 09:17:08 -06:00
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
if (brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDS ||
intel/eu/gen12: Codegen pathological SEND source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 18:13:42 -07:00
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDSC ||
(devinfo->gen >= 12 &&
(brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SEND ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDC))) {
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
assert(reg.file == BRW_GENERAL_REGISTER_FILE ||
reg.file == BRW_ARCHITECTURE_REGISTER_FILE);
assert(reg.address_mode == BRW_ADDRESS_DIRECT);
assert(reg.subnr == 0);
intel/compiler: Relax SENDS regioning assertions The next commit fixes a mistake in the assembler and ends up running afoul of this assertion. Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5956>
2020-07-15 15:12:57 -07:00
assert(has_scalar_region(reg) ||
intel/eu/gen12: Codegen pathological SEND source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 18:13:42 -07:00
(reg.hstride == BRW_HORIZONTAL_STRIDE_1 &&
reg.vstride == reg.width + 1));
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
assert(!reg.negate && !reg.abs);
brw_inst_set_send_src1_reg_nr(devinfo, inst, reg.nr);
brw_inst_set_send_src1_reg_file(devinfo, inst, reg.file);
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
/* From the IVB PRM Vol. 4, Pt. 3, Section 3.3.3.5:
*
* "Accumulator registers may be accessed explicitly as src0
* operands only."
*/
assert(reg.file != BRW_ARCHITECTURE_REGISTER_FILE ||
reg.nr != BRW_ARF_ACCUMULATOR);
i965/emit: Add assertions for accumulator restrictions Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-10 16:19:22 -07:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
gen7_convert_mrf_to_grf(p, &reg);
assert(reg.file != BRW_MESSAGE_REGISTER_FILE);
i965: Mad hacks to avoid using MRFs on Ivybridge. Ivybridge's SEND instruction uses GRFs instead of MRFs. Unfortunately, a lot of our code explicitly uses MRFs, and rewriting it would take a fair bit of effort. In the meantime, use a hack: - Change brw_set_dest, brw_set_src0, and brw_set_src1 to implicitly convert any MRFs into the top 16 GRFs. - Enable gen6_resolve_implied_move on Ivybridge: Moving g0 to m0 actually moves it to g111 thanks to the previous hack. It remains to officially reserve these registers so the allocator doesn't try to reuse them. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-09 00:32:46 -07:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_src1_file_type(devinfo, inst, reg.file, reg.type);
brw_inst_set_src1_abs(devinfo, inst, reg.abs);
brw_inst_set_src1_negate(devinfo, inst, reg.negate);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
/* Only src1 can be immediate in two-argument instructions.
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
*/
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
assert(brw_inst_src0_reg_file(devinfo, inst) != BRW_IMMEDIATE_VALUE);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
if (reg.file == BRW_IMMEDIATE_VALUE) {
/* two-argument instructions can only use 32-bit immediates */
assert(type_sz(reg.type) < 8);
brw_inst_set_imm_ud(devinfo, inst, reg.ud);
i965: Fix else and brace placement in brw_eu_emit.c. I'm making a lot of changes to this area, and I figured I may as well not conflate these trivial changes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-04 16:55:59 -07:00
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
/* This is a hardware restriction, which may or may not be lifted
* in the future:
*/
assert (reg.address_mode == BRW_ADDRESS_DIRECT);
/* assert (reg.file == BRW_GENERAL_REGISTER_FILE); */
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_src1_da_reg_nr(devinfo, inst, reg.nr);
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
brw_inst_set_src1_da1_subreg_nr(devinfo, inst, reg.subnr);
} else {
brw_inst_set_src1_da16_subreg_nr(devinfo, inst, reg.subnr / 16);
}
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
if (reg.width == BRW_WIDTH_1 &&
brw_inst_exec_size(devinfo, inst) == BRW_EXECUTE_1) {
brw_inst_set_src1_hstride(devinfo, inst, BRW_HORIZONTAL_STRIDE_0);
brw_inst_set_src1_width(devinfo, inst, BRW_WIDTH_1);
brw_inst_set_src1_vstride(devinfo, inst, BRW_VERTICAL_STRIDE_0);
} else {
brw_inst_set_src1_hstride(devinfo, inst, reg.hstride);
brw_inst_set_src1_width(devinfo, inst, reg.width);
brw_inst_set_src1_vstride(devinfo, inst, reg.vstride);
}
i965: Use correct VertStride on align16 instructions. In commit c35fa7a, we changed the "width" of DF source registers to 2, which is conceptually fine. Unfortunately a VertStride of 2 is not allowed by align16 instructions on IVB/BYT, and the regular VertStride of 4 works fine in any case. See generated_tests/spec/arb_gpu_shader_fp64/execution/built-in-functions/vs-round-double.shader_test for example: cmp.ge.f0(8) g18<1>DF g1<0>.xyxyDF -g8<2>DF { align16 1Q }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed cmp.ge.f0(8) g19<1>DF g1<0>.xyxyDF -g9<2>DF { align16 2N }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed v2: - Add spec quote (Curro). - Change the condition to only BRW_VERTICAL_STRIDE_2 (Curro) Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-01-20 13:35:33 -08:00
} else {
intel/inst: Indent some code We're about to add some more if cases so let's have the giant re-indent in it's own patch to make review easier. Acked-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 17:40:32 -06:00
brw_inst_set_src1_da16_swiz_x(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_X));
brw_inst_set_src1_da16_swiz_y(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_Y));
brw_inst_set_src1_da16_swiz_z(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_Z));
brw_inst_set_src1_da16_swiz_w(devinfo, inst,
BRW_GET_SWZ(reg.swizzle, BRW_CHANNEL_W));
if (reg.vstride == BRW_VERTICAL_STRIDE_8) {
/* This is an oddity of the fact we're using the same
* descriptions for registers in align_16 as align_1:
*/
brw_inst_set_src1_vstride(devinfo, inst, BRW_VERTICAL_STRIDE_4);
} else if (devinfo->gen == 7 && !devinfo->is_haswell &&
reg.type == BRW_REGISTER_TYPE_DF &&
reg.vstride == BRW_VERTICAL_STRIDE_2) {
/* From SNB PRM:
*
* "For Align16 access mode, only encodings of 0000 and 0011
* are allowed. Other codes are reserved."
*
* Presumably the DevSNB behavior applies to IVB as well.
*/
brw_inst_set_src1_vstride(devinfo, inst, BRW_VERTICAL_STRIDE_4);
} else {
brw_inst_set_src1_vstride(devinfo, inst, reg.vstride);
}
i965: Use correct VertStride on align16 instructions. In commit c35fa7a, we changed the "width" of DF source registers to 2, which is conceptually fine. Unfortunately a VertStride of 2 is not allowed by align16 instructions on IVB/BYT, and the regular VertStride of 4 works fine in any case. See generated_tests/spec/arb_gpu_shader_fp64/execution/built-in-functions/vs-round-double.shader_test for example: cmp.ge.f0(8) g18<1>DF g1<0>.xyxyDF -g8<2>DF { align16 1Q }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed cmp.ge.f0(8) g19<1>DF g1<0>.xyxyDF -g9<2>DF { align16 2N }; ERROR: In Align16 mode, only VertStride of 0 or 4 is allowed v2: - Add spec quote (Curro). - Change the condition to only BRW_VERTICAL_STRIDE_2 (Curro) Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-01-20 13:35:33 -08:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
}
}
intel/eu: Define helper to specify the descriptor immediates of a SEND instruction. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 13:48:42 -07:00
/**
* Specify the descriptor and extended descriptor immediate for a SEND(C)
* message instruction.
*/
void
brw_set_desc_ex(struct brw_codegen *p, brw_inst *inst,
unsigned desc, unsigned ex_desc)
{
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Assert that the instruction is send-like in brw_set_desc_ex(). Constructing a descriptor in-place as part of the immediate of an ALU instruction is no longer supported. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-03 13:20:45 -07:00
assert(brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SEND ||
brw_inst_opcode(devinfo, inst) == BRW_OPCODE_SENDC);
intel/eu/gen12: Codegen SEND descriptor regions correctly. The SEND instruction is now four-source. The descriptor is no longer part of source 1, so avoid touching it to avoid corruption while initializing the descriptor. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-09-03 12:18:38 -07:00
if (devinfo->gen < 12)
brw_inst_set_src1_file_type(devinfo, inst,
BRW_IMMEDIATE_VALUE, BRW_REGISTER_TYPE_UD);
intel/eu: Define helper to specify the descriptor immediates of a SEND instruction. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 13:48:42 -07:00
brw_inst_set_send_desc(devinfo, inst, desc);
intel/eu: Assert that the instruction is send-like in brw_set_desc_ex(). Constructing a descriptor in-place as part of the immediate of an ALU instruction is no longer supported. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-03 13:20:45 -07:00
if (devinfo->gen >= 9)
intel/eu: Define helper to specify the descriptor immediates of a SEND instruction. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 13:48:42 -07:00
brw_inst_set_send_ex_desc(devinfo, inst, ex_desc);
}
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
static void brw_set_math_message( struct brw_codegen *p,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *inst,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned function,
unsigned integer_type,
intel: Convert from GLboolean to 'bool' from stdbool.h. I initially produced the patch using this bash command: for file in {intel,i915,i965}/*.{c,cpp,h}; do [ ! -h $file ] && sed -i 's/GLboolean/bool/g' $file && sed -i 's/GL_TRUE/true/g' $file && sed -i 's/GL_FALSE/false/g' $file; done Then I manually added #include <stdbool.h> to fix compilation errors, and converted a few functions back to GLboolean that were used in core Mesa's function pointer table to avoid "incompatible pointer" warnings. Finally, I cleaned up some whitespace issues introduced by the change. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Chad Versace <chad@chad-versace.us> Acked-by: Paul Berry <stereotype441@gmail.com>
2011-10-07 12:26:50 -07:00
bool low_precision,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned dataType )
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Fix message and response length calculations for INT DIV. Both POW and INT DIV need a message length of 2; previously, we only checked for POW. Also, BRW_MATH_FUNCTION_INT_DIV_QUOTIENT_AND_REMAINDER has a response length of 2; previously, we only checked for SINCOS. We don't use this message, but in case we ever decide to, we may as well fix it now. While we're at it, just move these computations into brw_set_math_message, since they're entirely based on the function. This fixes it for both brw_math and the old backend's brw_math_16. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Tested-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-28 17:37:52 -07:00
unsigned msg_length;
unsigned response_length;
/* Infer message length from the function */
switch (function) {
case BRW_MATH_FUNCTION_POW:
case BRW_MATH_FUNCTION_INT_DIV_QUOTIENT:
case BRW_MATH_FUNCTION_INT_DIV_REMAINDER:
case BRW_MATH_FUNCTION_INT_DIV_QUOTIENT_AND_REMAINDER:
msg_length = 2;
break;
default:
msg_length = 1;
break;
}
/* Infer response length from the function */
switch (function) {
case BRW_MATH_FUNCTION_SINCOS:
case BRW_MATH_FUNCTION_INT_DIV_QUOTIENT_AND_REMAINDER:
response_length = 2;
break;
default:
response_length = 1;
break;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_set_desc(p, inst, brw_message_desc(
devinfo, msg_length, response_length, false));
i965: Drop the confusing saturate argument to math instruction setup. This was ridiculous. We were ignoring the inst->header.saturate flag in the case of math and only math. On gen4, we would leave inst->header.saturate in place if it happened to be set, which would end up being applied to the implicit mov and thus trash the first argument. On gen6, we would overwrite inst->header.saturate with the saturate flag from the argument, which was not set appropriately in brw_vec4_emit.cpp, and was only not a bug due to our incompetence at coalescing saturate moves. By ripping the argument out and making saturate work just like all the other brw_eu_emit.c code generation, we can avoid both these classes of bugs. Fixes piglit fog-modes, and the new specific fs-saturate-exp2 case. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=48628 NOTE: This is a candidate for the 8.0 branch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-08-06 14:59:39 -07:00
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_inst_set_sfid(devinfo, inst, BRW_SFID_MATH);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_math_msg_function(devinfo, inst, function);
brw_inst_set_math_msg_signed_int(devinfo, inst, integer_type);
brw_inst_set_math_msg_precision(devinfo, inst, low_precision);
brw_inst_set_math_msg_saturate(devinfo, inst, brw_inst_saturate(devinfo, inst));
brw_inst_set_math_msg_data_type(devinfo, inst, dataType);
brw_inst_set_saturate(devinfo, inst, 0);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
static void brw_set_ff_sync_message(struct brw_codegen *p,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn,
intel: Convert from GLboolean to 'bool' from stdbool.h. I initially produced the patch using this bash command: for file in {intel,i915,i965}/*.{c,cpp,h}; do [ ! -h $file ] && sed -i 's/GLboolean/bool/g' $file && sed -i 's/GL_TRUE/true/g' $file && sed -i 's/GL_FALSE/false/g' $file; done Then I manually added #include <stdbool.h> to fix compilation errors, and converted a few functions back to GLboolean that were used in core Mesa's function pointer table to avoid "incompatible pointer" warnings. Finally, I cleaned up some whitespace issues introduced by the change. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Chad Versace <chad@chad-versace.us> Acked-by: Paul Berry <stereotype441@gmail.com>
2011-10-07 12:26:50 -07:00
bool allocate,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned response_length,
intel: Convert from GLboolean to 'bool' from stdbool.h. I initially produced the patch using this bash command: for file in {intel,i915,i965}/*.{c,cpp,h}; do [ ! -h $file ] && sed -i 's/GLboolean/bool/g' $file && sed -i 's/GL_TRUE/true/g' $file && sed -i 's/GL_FALSE/false/g' $file; done Then I manually added #include <stdbool.h> to fix compilation errors, and converted a few functions back to GLboolean that were used in core Mesa's function pointer table to avoid "incompatible pointer" warnings. Finally, I cleaned up some whitespace issues introduced by the change. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Chad Versace <chad@chad-versace.us> Acked-by: Paul Berry <stereotype441@gmail.com>
2011-10-07 12:26:50 -07:00
bool end_of_thread)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_set_desc(p, insn, brw_message_desc(
devinfo, 1, response_length, true));
brw_inst_set_sfid(devinfo, insn, BRW_SFID_URB);
brw_inst_set_eot(devinfo, insn, end_of_thread);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_urb_opcode(devinfo, insn, 1); /* FF_SYNC */
brw_inst_set_urb_allocate(devinfo, insn, allocate);
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
/* The following fields are not used by FF_SYNC: */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_urb_global_offset(devinfo, insn, 0);
brw_inst_set_urb_swizzle_control(devinfo, insn, 0);
brw_inst_set_urb_used(devinfo, insn, 0);
brw_inst_set_urb_complete(devinfo, insn, 0);
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
static void brw_set_urb_message( struct brw_codegen *p,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn,
i965: Allow C++ type safety in the use of enum brw_urb_write_flags. (From a suggestion by Francisco Jerez) If an enum represents a bitfield of flags, e.g.: enum E { A = 1, B = 2, C = 4, D = 8, }; then C++ normally prohibits statements like this: enum E x = A | B; because A and B are implicitly converted to ints before OR-ing them, and an int can't be stored in an enum without a type cast. C, on the other hand, allows an int to be implicitly converted to an enum without casting. In the past we've dealt with this situation by storing flag bitfields as ints. This avoids ugly casting at the expense of some type safety that C++ would normally have offered (e.g. we get no warning if we accidentally use the wrong enum type). However, we can get the best of both worlds if we override the | operator. The ugly casting is confined to the operator overload, and we still get the benefit of C++ making sure we don't use the wrong enum type. v2: Remove unnecessary comment and unnecessary use of "enum" keyword. Use static_cast. Reviewed-by: Chad Versace <chad.versace@linux.intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2013-08-23 13:19:19 -07:00
enum brw_urb_write_flags flags,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_length,
unsigned response_length,
unsigned offset,
unsigned swizzle_control )
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Fix inconsistent indentation in brw_eu_emit.c. Most of these functions used three spaces for the first level of indentation, but four spaces for the next level. One used tabs and then three spaces. Some used 3/4 in a then block but 3/3 in the else block. Normally I try to avoid field days like this, but since the functions were so inconsistent, even internally, it was making it difficult to edit without introducing spurious whitespace changes. So, just get it over with. git diff -b shows 0 lines changed. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2011-10-07 11:59:06 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen < 7 || swizzle_control != BRW_URB_SWIZZLE_TRANSPOSE);
assert(devinfo->gen < 7 || !(flags & BRW_URB_WRITE_ALLOCATE));
assert(devinfo->gen >= 7 || !(flags & BRW_URB_WRITE_PER_SLOT_OFFSET));
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_set_desc(p, insn, brw_message_desc(
devinfo, msg_length, response_length, true));
brw_inst_set_sfid(devinfo, insn, BRW_SFID_URB);
brw_inst_set_eot(devinfo, insn, !!(flags & BRW_URB_WRITE_EOT));
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
if (flags & BRW_URB_WRITE_OWORD) {
assert(msg_length == 2); /* header + one OWORD of data */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_urb_opcode(devinfo, insn, BRW_URB_OPCODE_WRITE_OWORD);
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_urb_opcode(devinfo, insn, BRW_URB_OPCODE_WRITE_HWORD);
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_urb_global_offset(devinfo, insn, offset);
brw_inst_set_urb_swizzle_control(devinfo, insn, swizzle_control);
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 8) {
brw_inst_set_urb_complete(devinfo, insn, !!(flags & BRW_URB_WRITE_COMPLETE));
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 7) {
brw_inst_set_urb_allocate(devinfo, insn, !!(flags & BRW_URB_WRITE_ALLOCATE));
brw_inst_set_urb_used(devinfo, insn, !(flags & BRW_URB_WRITE_UNUSED));
i965: Fix inconsistent indentation in brw_eu_emit.c. Most of these functions used three spaces for the first level of indentation, but four spaces for the next level. One used tabs and then three spaces. Some used 3/4 in a then block but 3/3 in the else block. Normally I try to avoid field days like this, but since the functions were so inconsistent, even internally, it was making it difficult to edit without introducing spurious whitespace changes. So, just get it over with. git diff -b shows 0 lines changed. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2011-10-07 11:59:06 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_urb_per_slot_offset(devinfo, insn,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
!!(flags & BRW_URB_WRITE_PER_SLOT_OFFSET));
i965: Fix inconsistent indentation in brw_eu_emit.c. Most of these functions used three spaces for the first level of indentation, but four spaces for the next level. One used tabs and then three spaces. Some used 3/4 in a then block but 3/3 in the else block. Normally I try to avoid field days like this, but since the functions were so inconsistent, even internally, it was making it difficult to edit without introducing spurious whitespace changes. So, just get it over with. git diff -b shows 0 lines changed. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2011-10-07 11:59:06 -07:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
gen7_set_dp_scratch_message(struct brw_codegen *p,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *inst,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
bool write,
bool dword,
bool invalidate_after_read,
unsigned num_regs,
unsigned addr_offset,
unsigned mlen,
unsigned rlen,
bool header_present)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
assert(num_regs == 1 || num_regs == 2 || num_regs == 4 ||
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
(devinfo->gen >= 8 && num_regs == 8));
replace _mesa_logbase2 with util_logbase2 Reviewed-by: Marek Olšák <marek.olsak@amd.com> Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com> Reviewed-by: Matt Turner <mattst88@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3024>
2019-12-06 09:20:09 -08:00
const unsigned block_size = (devinfo->gen >= 8 ? util_logbase2(num_regs) :
i965/eu: Fix Gen7+ DP scratch message size calculation on Gen7. Gen7 hardware expects the block size field in the message descriptor to be the number of registers minus one instead of the log2 of the number of registers. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 16:03:33 -07:00
num_regs - 1);
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_set_desc(p, inst, brw_message_desc(
devinfo, mlen, rlen, header_present));
brw_inst_set_sfid(devinfo, inst, GEN7_SFID_DATAPORT_DATA_CACHE);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_dp_category(devinfo, inst, 1); /* Scratch Block Read/Write msgs */
brw_inst_set_scratch_read_write(devinfo, inst, write);
brw_inst_set_scratch_type(devinfo, inst, dword);
brw_inst_set_scratch_invalidate_after_read(devinfo, inst, invalidate_after_read);
i965/eu: Fix Gen7+ DP scratch message size calculation on Gen7. Gen7 hardware expects the block size field in the message descriptor to be the number of registers minus one instead of the log2 of the number of registers. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 16:03:33 -07:00
brw_inst_set_scratch_block_size(devinfo, inst, block_size);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_scratch_addr_offset(devinfo, inst, addr_offset);
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Copy fields manually in brw_next_insn Instead of doing a memcpy, this moves us to start with a blank instruction (memset to zero) and copy the fields over one at a time. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 13:45:57 -07:00
static void
brw_inst_set_state(const struct gen_device_info *devinfo,
brw_inst *insn,
const struct brw_insn_state *state)
{
brw_inst_set_exec_size(devinfo, insn, state->exec_size);
brw_inst_set_group(devinfo, insn, state->group);
brw_inst_set_compression(devinfo, insn, state->compressed);
brw_inst_set_access_mode(devinfo, insn, state->access_mode);
brw_inst_set_mask_control(devinfo, insn, state->mask_control);
intel/eu/gen12: Add tracking of default SWSB state to the current brw_codegen instruction. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-11-09 14:13:36 -08:00
if (devinfo->gen >= 12)
brw_inst_set_swsb(devinfo, insn, tgl_swsb_encode(state->swsb));
intel/eu: Copy fields manually in brw_next_insn Instead of doing a memcpy, this moves us to start with a blank instruction (memset to zero) and copy the fields over one at a time. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 13:45:57 -07:00
brw_inst_set_saturate(devinfo, insn, state->saturate);
brw_inst_set_pred_control(devinfo, insn, state->predicate);
brw_inst_set_pred_inv(devinfo, insn, state->pred_inv);
intel/eu: Set flag [sub]register number differently for 3src Prior to gen8, the flag [sub]register number is in a different spot on 3src instructions than on other instructions. Starting with Broadwell, they made it consistent. This commit fixes bugs that occur when a conditional modifier gets propagated into a 3src instruction such as a MAD. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 15:28:36 -07:00
if (is_3src(devinfo, brw_inst_opcode(devinfo, insn)) &&
state->access_mode == BRW_ALIGN_16) {
brw_inst_set_3src_a16_flag_subreg_nr(devinfo, insn, state->flag_subreg % 2);
if (devinfo->gen >= 7)
brw_inst_set_3src_a16_flag_reg_nr(devinfo, insn, state->flag_subreg / 2);
} else {
brw_inst_set_flag_subreg_nr(devinfo, insn, state->flag_subreg % 2);
if (devinfo->gen >= 7)
brw_inst_set_flag_reg_nr(devinfo, insn, state->flag_subreg / 2);
}
intel/eu: Copy fields manually in brw_next_insn Instead of doing a memcpy, this moves us to start with a blank instruction (memset to zero) and copy the fields over one at a time. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 13:45:57 -07:00
if (devinfo->gen >= 6)
brw_inst_set_acc_wr_control(devinfo, insn, state->acc_wr_control);
}
intel/eu: Add some new helpers Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-08-07 21:59:12 -05:00
static brw_inst *
brw_append_insns(struct brw_codegen *p, unsigned nr_insn, unsigned align)
{
assert(util_is_power_of_two_or_zero(sizeof(brw_inst)));
assert(util_is_power_of_two_or_zero(align));
const unsigned align_insn = MAX2(align / sizeof(brw_inst), 1);
const unsigned start_insn = ALIGN(p->nr_insn, align_insn);
const unsigned new_nr_insn = start_insn + nr_insn;
if (p->store_size < new_nr_insn) {
p->store_size = util_next_power_of_two(new_nr_insn * sizeof(brw_inst));
p->store = reralloc(p->mem_ctx, p->store, brw_inst, p->store_size);
}
/* Memset any padding due to alignment to 0. We don't want to be hashing
* or caching a bunch of random bits we got from a memory allocation.
*/
if (p->nr_insn < start_insn) {
memset(&p->store[p->nr_insn], 0,
(start_insn - p->nr_insn) * sizeof(brw_inst));
}
assert(p->next_insn_offset == p->nr_insn * sizeof(brw_inst));
p->nr_insn = new_nr_insn;
p->next_insn_offset = new_nr_insn * sizeof(brw_inst);
return &p->store[start_insn];
}
void
brw_realign(struct brw_codegen *p, unsigned align)
{
brw_append_insns(p, 0, align);
}
int
brw_append_data(struct brw_codegen *p, void *data,
unsigned size, unsigned align)
{
unsigned nr_insn = DIV_ROUND_UP(size, sizeof(brw_inst));
void *dst = brw_append_insns(p, nr_insn, align);
memcpy(dst, data, size);
/* If it's not a whole number of instructions, memset the end */
if (size < nr_insn * sizeof(brw_inst))
memset(dst + size, 0, nr_insn * sizeof(brw_inst) - size);
return dst - (void *)p->store;
}
i965: Make some EU emit code for DP read/write messages non-static. We keep building these strange interfaces for DP read/write where there's a helper function with some partially-specific, partially-general controls, which is used in exactly one place in code generation. Making these public will let us set up those instructions in the one place they're to be generated.
2011-08-07 13:16:06 -07:00
#define next_insn brw_next_insn
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_next_insn(struct brw_codegen *p, unsigned opcode)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Add some new helpers Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-08-07 21:59:12 -05:00
brw_inst *insn = brw_append_insns(p, 1, sizeof(brw_inst));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Copy fields manually in brw_next_insn Instead of doing a memcpy, this moves us to start with a blank instruction (memset to zero) and copy the fields over one at a time. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 13:45:57 -07:00
memset(insn, 0, sizeof(*insn));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_opcode(devinfo, insn, opcode);
intel/eu: Copy fields manually in brw_next_insn Instead of doing a memcpy, this moves us to start with a blank instruction (memset to zero) and copy the fields over one at a time. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 13:45:57 -07:00
/* Apply the default instruction state */
intel/eu: Switch to a logical state stack Instead of the state stack that's based on copying a dummy instruction around, we start using a logical stack of brw_insn_states. This uses a bit less memory and is way less conceptually bogus. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:37:35 -07:00
brw_inst_set_state(devinfo, insn, p->current);
intel/eu: Copy fields manually in brw_next_insn Instead of doing a memcpy, this moves us to start with a blank instruction (memset to zero) and copy the fields over one at a time. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 13:45:57 -07:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
return insn;
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
static brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_alu1(struct brw_codegen *p, unsigned opcode,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
struct brw_reg dest, struct brw_reg src)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, opcode);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
return insn;
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
static brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_alu2(struct brw_codegen *p, unsigned opcode,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
struct brw_reg dest, struct brw_reg src0, struct brw_reg src1)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
i965/eu: add support for DF immediates v2 (Sam): - Remove 'however' from the comment (Topi) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-08-03 14:47:36 -07:00
/* 64-bit immediates are only supported on 1-src instructions */
assert(src0.file != BRW_IMMEDIATE_VALUE || type_sz(src0.type) <= 4);
assert(src1.file != BRW_IMMEDIATE_VALUE || type_sz(src1.type) <= 4);
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, opcode);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, src1);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
return insn;
}
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
static int
get_3src_subreg_nr(struct brw_reg reg)
{
i965: Move 3-src subnr swizzle handling into the vec4 backend. While most align16 instructions only support a SubRegNum of 0 or 4 (using swizzling to control the other channels), 3-src instructions actually support arbitrary SubRegNums. When the RepCtrl bit is set, we believe it ignores the swizzle and uses the equivalent of a <0,1,0> region from the subnr. In the past, we adopted a vec4-centric approach of specifying subnr of 0 or 4 and a swizzle, then having brw_eu_emit.c convert that to a proper SubRegNum. This isn't a great fit for the scalar backend, where we don't set swizzles at all, and happily set subnrs in the range [0, 7]. This patch changes brw_eu_emit.c to use subnr and swizzle directly, relying on the higher levels to set them sensibly. This should fix problems where scalar sources get copy propagated into 3-src instructions in the FS backend. I've only observed this with TES push model inputs, but I suppose it could happen in other cases. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-12-31 12:47:19 -08:00
/* Normally, SubRegNum is in bytes (0..31). However, 3-src instructions
* use 32-bit units (components 0..7). Since they only support F/D/UD
* types, this doesn't lose any flexibility, but uses fewer bits.
*/
return reg.subnr / 4;
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
}
i965/fs: Add/use functions to convert to 3src_align1 vstride/hstride Some cases weren't handled, such as stride 4 which is needed for 64-bit operations. Presumably fixes the assertion failure mentioned in commit 2d0457203871 (Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+") but who can really say since the commit neglected to list any of them! Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-12-11 11:44:48 -08:00
static enum gen10_align1_3src_vertical_stride
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
to_3src_align1_vstride(const struct gen_device_info *devinfo,
enum brw_vertical_stride vstride)
i965/fs: Add/use functions to convert to 3src_align1 vstride/hstride Some cases weren't handled, such as stride 4 which is needed for 64-bit operations. Presumably fixes the assertion failure mentioned in commit 2d0457203871 (Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+") but who can really say since the commit neglected to list any of them! Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-12-11 11:44:48 -08:00
{
switch (vstride) {
case BRW_VERTICAL_STRIDE_0:
return BRW_ALIGN1_3SRC_VERTICAL_STRIDE_0;
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
case BRW_VERTICAL_STRIDE_1:
assert(devinfo->gen >= 12);
return BRW_ALIGN1_3SRC_VERTICAL_STRIDE_1;
i965/fs: Add/use functions to convert to 3src_align1 vstride/hstride Some cases weren't handled, such as stride 4 which is needed for 64-bit operations. Presumably fixes the assertion failure mentioned in commit 2d0457203871 (Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+") but who can really say since the commit neglected to list any of them! Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-12-11 11:44:48 -08:00
case BRW_VERTICAL_STRIDE_2:
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
assert(devinfo->gen < 12);
i965/fs: Add/use functions to convert to 3src_align1 vstride/hstride Some cases weren't handled, such as stride 4 which is needed for 64-bit operations. Presumably fixes the assertion failure mentioned in commit 2d0457203871 (Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+") but who can really say since the commit neglected to list any of them! Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-12-11 11:44:48 -08:00
return BRW_ALIGN1_3SRC_VERTICAL_STRIDE_2;
case BRW_VERTICAL_STRIDE_4:
return BRW_ALIGN1_3SRC_VERTICAL_STRIDE_4;
case BRW_VERTICAL_STRIDE_8:
case BRW_VERTICAL_STRIDE_16:
return BRW_ALIGN1_3SRC_VERTICAL_STRIDE_8;
default:
unreachable("invalid vstride");
}
}
static enum gen10_align1_3src_src_horizontal_stride
to_3src_align1_hstride(enum brw_horizontal_stride hstride)
{
switch (hstride) {
case BRW_HORIZONTAL_STRIDE_0:
return BRW_ALIGN1_3SRC_SRC_HORIZONTAL_STRIDE_0;
case BRW_HORIZONTAL_STRIDE_1:
return BRW_ALIGN1_3SRC_SRC_HORIZONTAL_STRIDE_1;
case BRW_HORIZONTAL_STRIDE_2:
return BRW_ALIGN1_3SRC_SRC_HORIZONTAL_STRIDE_2;
case BRW_HORIZONTAL_STRIDE_4:
return BRW_ALIGN1_3SRC_SRC_HORIZONTAL_STRIDE_4;
default:
unreachable("invalid hstride");
}
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
static brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_alu3(struct brw_codegen *p, unsigned opcode, struct brw_reg dest,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
struct brw_reg src0, struct brw_reg src1, struct brw_reg src2)
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *inst = next_insn(p, opcode);
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
gen7_convert_mrf_to_grf(p, &dest);
assert(dest.nr < 128);
intel/compiler: Add Immediate support for 3 source instruction On Gen >= 10, Either src0 or src2 can use 16-bit immediate value, but not both. Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-07-25 18:28:06 -07:00
if (devinfo->gen >= 10)
assert(!(src0.file == BRW_IMMEDIATE_VALUE &&
src2.file == BRW_IMMEDIATE_VALUE));
intel/compiler: Fix assertions in brw_alu3 v2: Fix assertion for src1 (Ian Romanick) Fixes: 3b967e17 (intel/compiler: Avoid false positive assertions) Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Suggested-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-04-15 23:26:47 -07:00
assert(src0.file == BRW_IMMEDIATE_VALUE || src0.nr < 128);
assert(src1.file != BRW_IMMEDIATE_VALUE && src1.nr < 128);
assert(src2.file == BRW_IMMEDIATE_VALUE || src2.nr < 128);
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
assert(dest.address_mode == BRW_ADDRESS_DIRECT);
assert(src0.address_mode == BRW_ADDRESS_DIRECT);
assert(src1.address_mode == BRW_ADDRESS_DIRECT);
assert(src2.address_mode == BRW_ADDRESS_DIRECT);
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
if (brw_inst_access_mode(devinfo, inst) == BRW_ALIGN_1) {
assert(dest.file == BRW_GENERAL_REGISTER_FILE ||
dest.file == BRW_ARCHITECTURE_REGISTER_FILE);
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
if (devinfo->gen >= 12) {
brw_inst_set_3src_a1_dst_reg_file(devinfo, inst, dest.file);
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
brw_inst_set_3src_dst_reg_nr(devinfo, inst, dest.nr);
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
} else {
if (dest.file == BRW_ARCHITECTURE_REGISTER_FILE) {
brw_inst_set_3src_a1_dst_reg_file(devinfo, inst,
BRW_ALIGN1_3SRC_ACCUMULATOR);
brw_inst_set_3src_dst_reg_nr(devinfo, inst, BRW_ARF_ACCUMULATOR);
} else {
brw_inst_set_3src_a1_dst_reg_file(devinfo, inst,
BRW_ALIGN1_3SRC_GENERAL_REGISTER_FILE);
brw_inst_set_3src_dst_reg_nr(devinfo, inst, dest.nr);
}
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
}
brw_inst_set_3src_a1_dst_subreg_nr(devinfo, inst, dest.subnr / 8);
brw_inst_set_3src_a1_dst_hstride(devinfo, inst, BRW_ALIGN1_3SRC_DST_HORIZONTAL_STRIDE_1);
if (brw_reg_type_is_floating_point(dest.type)) {
brw_inst_set_3src_a1_exec_type(devinfo, inst,
BRW_ALIGN1_3SRC_EXEC_TYPE_FLOAT);
} else {
brw_inst_set_3src_a1_exec_type(devinfo, inst,
BRW_ALIGN1_3SRC_EXEC_TYPE_INT);
}
brw_inst_set_3src_a1_dst_type(devinfo, inst, dest.type);
brw_inst_set_3src_a1_src0_type(devinfo, inst, src0.type);
brw_inst_set_3src_a1_src1_type(devinfo, inst, src1.type);
brw_inst_set_3src_a1_src2_type(devinfo, inst, src2.type);
intel/compiler: Add Immediate support for 3 source instruction On Gen >= 10, Either src0 or src2 can use 16-bit immediate value, but not both. Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-07-25 18:28:06 -07:00
if (src0.file == BRW_IMMEDIATE_VALUE) {
brw_inst_set_3src_a1_src0_imm(devinfo, inst, src0.ud);
} else {
brw_inst_set_3src_a1_src0_vstride(
devinfo, inst, to_3src_align1_vstride(devinfo, src0.vstride));
brw_inst_set_3src_a1_src0_hstride(devinfo, inst,
to_3src_align1_hstride(src0.hstride));
brw_inst_set_3src_a1_src0_subreg_nr(devinfo, inst, src0.subnr);
if (src0.type == BRW_REGISTER_TYPE_NF) {
brw_inst_set_3src_src0_reg_nr(devinfo, inst, BRW_ARF_ACCUMULATOR);
} else {
brw_inst_set_3src_src0_reg_nr(devinfo, inst, src0.nr);
}
brw_inst_set_3src_src0_abs(devinfo, inst, src0.abs);
brw_inst_set_3src_src0_negate(devinfo, inst, src0.negate);
}
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
brw_inst_set_3src_a1_src1_vstride(
devinfo, inst, to_3src_align1_vstride(devinfo, src1.vstride));
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
brw_inst_set_3src_a1_src1_hstride(devinfo, inst,
i965/fs: Add/use functions to convert to 3src_align1 vstride/hstride Some cases weren't handled, such as stride 4 which is needed for 64-bit operations. Presumably fixes the assertion failure mentioned in commit 2d0457203871 (Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+") but who can really say since the commit neglected to list any of them! Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-12-11 11:44:48 -08:00
to_3src_align1_hstride(src1.hstride));
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
brw_inst_set_3src_a1_src1_subreg_nr(devinfo, inst, src1.subnr);
if (src1.file == BRW_ARCHITECTURE_REGISTER_FILE) {
brw_inst_set_3src_src1_reg_nr(devinfo, inst, BRW_ARF_ACCUMULATOR);
} else {
brw_inst_set_3src_src1_reg_nr(devinfo, inst, src1.nr);
}
brw_inst_set_3src_src1_abs(devinfo, inst, src1.abs);
brw_inst_set_3src_src1_negate(devinfo, inst, src1.negate);
intel/compiler: Add Immediate support for 3 source instruction On Gen >= 10, Either src0 or src2 can use 16-bit immediate value, but not both. Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-07-25 18:28:06 -07:00
if (src2.file == BRW_IMMEDIATE_VALUE) {
brw_inst_set_3src_a1_src2_imm(devinfo, inst, src2.ud);
} else {
brw_inst_set_3src_a1_src2_hstride(devinfo, inst,
to_3src_align1_hstride(src2.hstride));
/* no vstride on src2 */
brw_inst_set_3src_a1_src2_subreg_nr(devinfo, inst, src2.subnr);
brw_inst_set_3src_src2_reg_nr(devinfo, inst, src2.nr);
brw_inst_set_3src_src2_abs(devinfo, inst, src2.abs);
brw_inst_set_3src_src2_negate(devinfo, inst, src2.negate);
}
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
assert(src0.file == BRW_GENERAL_REGISTER_FILE ||
intel/compiler: Add Gen11+ native float type This new type exposes the additional precision offered by the accumulator register and will be used in the next patch to implement the functionality of the PLN instruction using a pair of MAD instructions. One weird thing to note: align1 ternary instructions may only have an accumulator in the dst or src1 normally, but when src0's type is :NF the accumulator is read. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 11:03:19 -07:00
src0.file == BRW_IMMEDIATE_VALUE ||
(src0.file == BRW_ARCHITECTURE_REGISTER_FILE &&
src0.type == BRW_REGISTER_TYPE_NF));
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
assert(src1.file == BRW_GENERAL_REGISTER_FILE ||
src1.file == BRW_ARCHITECTURE_REGISTER_FILE);
assert(src2.file == BRW_GENERAL_REGISTER_FILE ||
src2.file == BRW_IMMEDIATE_VALUE);
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
if (devinfo->gen >= 12) {
intel/compiler: Set bits according to source file On Gen >= 12, if src0 or src2 holds immediate value, we need set src[0/2]_is_imm bits instead of register file. Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-04-19 13:37:17 -07:00
if (src0.file == BRW_IMMEDIATE_VALUE) {
brw_inst_set_3src_a1_src0_is_imm(devinfo, inst, 1);
} else {
brw_inst_set_3src_a1_src0_reg_file(devinfo, inst, src0.file);
}
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
brw_inst_set_3src_a1_src1_reg_file(devinfo, inst, src1.file);
intel/compiler: Set bits according to source file On Gen >= 12, if src0 or src2 holds immediate value, we need set src[0/2]_is_imm bits instead of register file. Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-04-19 13:37:17 -07:00
if (src2.file == BRW_IMMEDIATE_VALUE) {
brw_inst_set_3src_a1_src2_is_imm(devinfo, inst, 1);
} else {
brw_inst_set_3src_a1_src2_reg_file(devinfo, inst, src2.file);
}
intel/eu/gen12: Codegen three-source instruction source and destination regions. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
} else {
brw_inst_set_3src_a1_src0_reg_file(devinfo, inst,
src0.file == BRW_GENERAL_REGISTER_FILE ?
BRW_ALIGN1_3SRC_GENERAL_REGISTER_FILE :
BRW_ALIGN1_3SRC_IMMEDIATE_VALUE);
brw_inst_set_3src_a1_src1_reg_file(devinfo, inst,
src1.file == BRW_GENERAL_REGISTER_FILE ?
BRW_ALIGN1_3SRC_GENERAL_REGISTER_FILE :
BRW_ALIGN1_3SRC_ACCUMULATOR);
brw_inst_set_3src_a1_src2_reg_file(devinfo, inst,
src2.file == BRW_GENERAL_REGISTER_FILE ?
BRW_ALIGN1_3SRC_GENERAL_REGISTER_FILE :
BRW_ALIGN1_3SRC_IMMEDIATE_VALUE);
}
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
} else {
assert(dest.file == BRW_GENERAL_REGISTER_FILE ||
dest.file == BRW_MESSAGE_REGISTER_FILE);
assert(dest.type == BRW_REGISTER_TYPE_F ||
dest.type == BRW_REGISTER_TYPE_DF ||
dest.type == BRW_REGISTER_TYPE_D ||
intel/compiler: allow half-float on 3-source instructions since gen8 Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-22 08:17:17 +02:00
dest.type == BRW_REGISTER_TYPE_UD ||
(dest.type == BRW_REGISTER_TYPE_HF && devinfo->gen >= 8));
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
if (devinfo->gen == 6) {
brw_inst_set_3src_a16_dst_reg_file(devinfo, inst,
dest.file == BRW_MESSAGE_REGISTER_FILE);
}
brw_inst_set_3src_dst_reg_nr(devinfo, inst, dest.nr);
intel/eu: Set the right subnr for ALIGN16 destinations Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5596>
2019-10-17 09:54:02 -05:00
brw_inst_set_3src_a16_dst_subreg_nr(devinfo, inst, dest.subnr / 4);
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
brw_inst_set_3src_a16_dst_writemask(devinfo, inst, dest.writemask);
assert(src0.file == BRW_GENERAL_REGISTER_FILE);
brw_inst_set_3src_a16_src0_swizzle(devinfo, inst, src0.swizzle);
brw_inst_set_3src_a16_src0_subreg_nr(devinfo, inst, get_3src_subreg_nr(src0));
brw_inst_set_3src_src0_reg_nr(devinfo, inst, src0.nr);
brw_inst_set_3src_src0_abs(devinfo, inst, src0.abs);
brw_inst_set_3src_src0_negate(devinfo, inst, src0.negate);
brw_inst_set_3src_a16_src0_rep_ctrl(devinfo, inst,
src0.vstride == BRW_VERTICAL_STRIDE_0);
assert(src1.file == BRW_GENERAL_REGISTER_FILE);
brw_inst_set_3src_a16_src1_swizzle(devinfo, inst, src1.swizzle);
brw_inst_set_3src_a16_src1_subreg_nr(devinfo, inst, get_3src_subreg_nr(src1));
brw_inst_set_3src_src1_reg_nr(devinfo, inst, src1.nr);
brw_inst_set_3src_src1_abs(devinfo, inst, src1.abs);
brw_inst_set_3src_src1_negate(devinfo, inst, src1.negate);
brw_inst_set_3src_a16_src1_rep_ctrl(devinfo, inst,
src1.vstride == BRW_VERTICAL_STRIDE_0);
assert(src2.file == BRW_GENERAL_REGISTER_FILE);
brw_inst_set_3src_a16_src2_swizzle(devinfo, inst, src2.swizzle);
brw_inst_set_3src_a16_src2_subreg_nr(devinfo, inst, get_3src_subreg_nr(src2));
brw_inst_set_3src_src2_reg_nr(devinfo, inst, src2.nr);
brw_inst_set_3src_src2_abs(devinfo, inst, src2.abs);
brw_inst_set_3src_src2_negate(devinfo, inst, src2.negate);
brw_inst_set_3src_a16_src2_rep_ctrl(devinfo, inst,
src2.vstride == BRW_VERTICAL_STRIDE_0);
if (devinfo->gen >= 7) {
/* Set both the source and destination types based on dest.type,
* ignoring the source register types. The MAD and LRP emitters ensure
* that all four types are float. The BFE and BFI2 emitters, however,
* may send us mixed D and UD types and want us to ignore that and use
* the destination type.
*/
brw_inst_set_3src_a16_src_type(devinfo, inst, dest.type);
brw_inst_set_3src_a16_dst_type(devinfo, inst, dest.type);
intel/compiler: set correct precision fields for 3-source float instructions Source0 and Destination extract the floating-point precision automatically from the SrcType and DstType instruction fields respectively when they are set to types :F or :HF. For Source1 and Source2 operands, we use the new 1-bit fields Src1Type and Src2Type, where 0 means normal precision and 1 means half-precision. Since we always use the type of the destination for all operands when we emit 3-source instructions, we only need set Src1Type and Src2Type to 1 when we are emitting a half-precision instruction. v2: - Set the bit separately for each source based on its type so we can do mixed floating-point mode in the future (Topi). v3: - Use regular citation style for the comment referencing the PRM (Matt). - Decided not to add asserts in the emission code to check that only mixed HF/F types are used since such checks would break negative tests for brw_eu_validate.c (Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-22 08:17:38 +02:00
/* From the Bspec, 3D Media GPGPU, Instruction fields, srcType:
*
* "Three source instructions can use operands with mixed-mode
* precision. When SrcType field is set to :f or :hf it defines
* precision for source 0 only, and fields Src1Type and Src2Type
* define precision for other source operands:
*
* 0b = :f. Single precision Float (32-bit).
* 1b = :hf. Half precision Float (16-bit)."
*/
if (src1.type == BRW_REGISTER_TYPE_HF)
brw_inst_set_3src_a16_src1_type(devinfo, inst, 1);
if (src2.type == BRW_REGISTER_TYPE_HF)
brw_inst_set_3src_a16_src2_type(devinfo, inst, 1);
i965: Add align1 ternary instruction emission support Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-06-14 14:49:52 -07:00
}
i965/gen7: Set src/dst types for 3-src instructions. Also update asserts to allow BFE and BFI2, which take (unsigned) doubleword arguments. v2: Allow BRW_REGISTER_TYPE_UD for src1 and src2 as well. Assert that src2.type (instead of src0.type) matches dest.type since it's the primary argument and src0 and src1 might correctly have different types. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v1]
2013-04-17 12:23:54 -07:00
}
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
return inst;
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/***********************************************************************
* Convenience routines.
*/
#define ALU1(OP) \
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_inst *brw_##OP(struct brw_codegen *p, \
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg dest, \
struct brw_reg src0) \
{ \
return brw_alu1(p, BRW_OPCODE_##OP, dest, src0); \
}
#define ALU2(OP) \
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_inst *brw_##OP(struct brw_codegen *p, \
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg dest, \
struct brw_reg src0, \
struct brw_reg src1) \
{ \
return brw_alu2(p, BRW_OPCODE_##OP, dest, src0, src1); \
}
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
#define ALU3(OP) \
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_inst *brw_##OP(struct brw_codegen *p, \
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
struct brw_reg dest, \
struct brw_reg src0, \
struct brw_reg src1, \
struct brw_reg src2) \
intel/compiler: Set swizzle to BRW_SWIZZLE_XXXX for scalar region When RepCtrl is set, the swizzle field is ignored by the hardware. In order to ensure a 1-to-1 correspondence between the human-readable disassembly and the binary instruction encoding always set the swizzle to XXXX (all zeros) when it is unused due to RepCtrl Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-12-08 21:50:36 -08:00
{ \
if (p->current->access_mode == BRW_ALIGN_16) { \
if (src0.vstride == BRW_VERTICAL_STRIDE_0) \
src0.swizzle = BRW_SWIZZLE_XXXX; \
if (src1.vstride == BRW_VERTICAL_STRIDE_0) \
src1.swizzle = BRW_SWIZZLE_XXXX; \
if (src2.vstride == BRW_VERTICAL_STRIDE_0) \
src2.swizzle = BRW_SWIZZLE_XXXX; \
} \
i965: Add support for the MAD opcode on gen6+. v2: Fix MRF handling on gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1)
2010-03-22 10:05:42 -07:00
return brw_alu3(p, BRW_OPCODE_##OP, dest, src0, src1, src2); \
}
i965: Add back strict type assertions for MAD and LRP. Commit 526ffdfc033ab01cf133cb7e8290c65d12ccc9be relaxed the type assertions in brw_alu3 to allow D/UD types (required by BFE and BFI2). This lost us the strict type checking for MAD and LRP, which require all four types to be float. This patch adds a new ALU3F wrapper which checks these once again. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2013-06-13 14:55:18 -07:00
#define ALU3F(OP) \
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_inst *brw_##OP(struct brw_codegen *p, \
i965: Add back strict type assertions for MAD and LRP. Commit 526ffdfc033ab01cf133cb7e8290c65d12ccc9be relaxed the type assertions in brw_alu3 to allow D/UD types (required by BFE and BFI2). This lost us the strict type checking for MAD and LRP, which require all four types to be float. This patch adds a new ALU3F wrapper which checks these once again. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2013-06-13 14:55:18 -07:00
struct brw_reg dest, \
struct brw_reg src0, \
struct brw_reg src1, \
struct brw_reg src2) \
{ \
i965/eu: Allow 3-src float ops with doubles v2: - set 3src_src_type for BRW_REGISTER_TYPE_DF (Connor) Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2014-10-20 11:46:37 +03:00
assert(dest.type == BRW_REGISTER_TYPE_F || \
dest.type == BRW_REGISTER_TYPE_DF); \
if (dest.type == BRW_REGISTER_TYPE_F) { \
assert(src0.type == BRW_REGISTER_TYPE_F); \
assert(src1.type == BRW_REGISTER_TYPE_F); \
assert(src2.type == BRW_REGISTER_TYPE_F); \
} else if (dest.type == BRW_REGISTER_TYPE_DF) { \
assert(src0.type == BRW_REGISTER_TYPE_DF); \
assert(src1.type == BRW_REGISTER_TYPE_DF); \
assert(src2.type == BRW_REGISTER_TYPE_DF); \
intel/compiler: Set swizzle to BRW_SWIZZLE_XXXX for scalar region When RepCtrl is set, the swizzle field is ignored by the hardware. In order to ensure a 1-to-1 correspondence between the human-readable disassembly and the binary instruction encoding always set the swizzle to XXXX (all zeros) when it is unused due to RepCtrl Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-12-08 21:50:36 -08:00
} \
\
if (p->current->access_mode == BRW_ALIGN_16) { \
if (src0.vstride == BRW_VERTICAL_STRIDE_0) \
src0.swizzle = BRW_SWIZZLE_XXXX; \
if (src1.vstride == BRW_VERTICAL_STRIDE_0) \
src1.swizzle = BRW_SWIZZLE_XXXX; \
if (src2.vstride == BRW_VERTICAL_STRIDE_0) \
src2.swizzle = BRW_SWIZZLE_XXXX; \
i965/eu: Allow 3-src float ops with doubles v2: - set 3src_src_type for BRW_REGISTER_TYPE_DF (Connor) Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2014-10-20 11:46:37 +03:00
} \
i965: Add back strict type assertions for MAD and LRP. Commit 526ffdfc033ab01cf133cb7e8290c65d12ccc9be relaxed the type assertions in brw_alu3 to allow D/UD types (required by BFE and BFI2). This lost us the strict type checking for MAD and LRP, which require all four types to be float. This patch adds a new ALU3F wrapper which checks these once again. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2013-06-13 14:55:18 -07:00
return brw_alu3(p, BRW_OPCODE_##OP, dest, src0, src1, src2); \
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
ALU2(SEL)
ALU1(NOT)
ALU2(AND)
ALU2(OR)
ALU2(XOR)
ALU2(SHR)
ALU2(SHL)
i965: enable the emission of the DIM instruction v2 (Matt): - Take a DF source argument for the DIM instruction emission in the visitors. - Indentation. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-07 08:38:22 +02:00
ALU1(DIM)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
ALU2(ASR)
intel/compiler: Enable the emission of ROR/ROL instructions v2: 1) Drop changes for vec4 backend as on Gen11+ we don't support align16 mode (Matt Turner) Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2019-05-29 11:43:30 -07:00
ALU2(ROL)
ALU2(ROR)
i965/fs: Add infrastructure for generating CSEL instructions. v2 (idr): Don't allow CSEL with a non-float src2. v3 (idr): Add CSEL to fs_inst::flags_written. Suggested by Matt. v4 (idr): Only set BRW_ALIGN_16 on Gen < 10 (suggested by Matt). Don't reset the access mode afterwards (suggested by Samuel and Matt). Add support for CSEL not modifying the flags to more places (requested by Matt). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> [v3] Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-22 20:12:17 -08:00
ALU3(CSEL)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
ALU1(FRC)
ALU1(RNDD)
intel/compiler: Move Gen4/5 rounding to visitor Gen4/5's rounding instructions operate differently than later Gens'. They all return the floor of the input and the "Round-increment" conditional modifier answers whether the result should be incremented by 1.0 to get the appropriate result for the operation (and thus its behavior is determined by the round opcode; e.g., RNDZ vs RNDE). Since this requires a second instruciton (a predicated ADD) that consumes the result of the round instruction, the round instruction cannot write its result directly to the (write-only) message registers. By emitting the ADD in the generator, the backend thinks it's safe to store the round's result directly to the message register file. To avoid this, we move the emission of the ADD instruction to the NIR translator so that the backend has the information it needs. I suspect this also fixes code generated for RNDZ.SAT but since Gen4/5 don't support GLSL 1.30 which adds the trunc() function, I couldn't write a piglit test to confirm. My thinking is that if x=-0.5: sat(trunc(-0.5)) = 0.0 But on Gen4/5 where sat(trunc(x)) is implemented as rndz.r.f0 result, x // result = floor(x) // set f0 if increment needed (+f0) add result, result, 1.0 // fixup so result = trunc(x) then putting saturate on both instructions will give the wrong result. floor(-0.5) = -1.0 sat(floor(-0.5)) = 0.0 // +1 increment would be needed since floor(-0.5) != trunc(-0.5) sat(sat(floor(-0.5)) + 1.0) = 1.0 Fixes: 6f394343b1f ("nir/algebraic: i2f(f2i()) -> trunc()") Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2355 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3459> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3459>
2020-01-16 11:17:14 -08:00
ALU1(RNDE)
intel/eu: Add the RNDU opcode We don't want to use it on gen5 and earlier because only RNDD can be done with a single instruction and we can implement RNDU(x) as -RNDD(-x) so it's better to just do that when we have the instruction. On gen6 and above, we may as well just use the right instruction. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5596>
2019-08-22 11:15:50 -05:00
ALU1(RNDU)
intel/compiler: Move Gen4/5 rounding to visitor Gen4/5's rounding instructions operate differently than later Gens'. They all return the floor of the input and the "Round-increment" conditional modifier answers whether the result should be incremented by 1.0 to get the appropriate result for the operation (and thus its behavior is determined by the round opcode; e.g., RNDZ vs RNDE). Since this requires a second instruciton (a predicated ADD) that consumes the result of the round instruction, the round instruction cannot write its result directly to the (write-only) message registers. By emitting the ADD in the generator, the backend thinks it's safe to store the round's result directly to the message register file. To avoid this, we move the emission of the ADD instruction to the NIR translator so that the backend has the information it needs. I suspect this also fixes code generated for RNDZ.SAT but since Gen4/5 don't support GLSL 1.30 which adds the trunc() function, I couldn't write a piglit test to confirm. My thinking is that if x=-0.5: sat(trunc(-0.5)) = 0.0 But on Gen4/5 where sat(trunc(x)) is implemented as rndz.r.f0 result, x // result = floor(x) // set f0 if increment needed (+f0) add result, result, 1.0 // fixup so result = trunc(x) then putting saturate on both instructions will give the wrong result. floor(-0.5) = -1.0 sat(floor(-0.5)) = 0.0 // +1 increment would be needed since floor(-0.5) != trunc(-0.5) sat(sat(floor(-0.5)) + 1.0) = 1.0 Fixes: 6f394343b1f ("nir/algebraic: i2f(f2i()) -> trunc()") Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2355 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3459> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3459>
2020-01-16 11:17:14 -08:00
ALU1(RNDZ)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
ALU2(MAC)
ALU2(MACH)
ALU1(LZD)
ALU2(DP4)
ALU2(DPH)
ALU2(DP3)
ALU2(DP2)
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 14:47:19 -07:00
ALU3(MAD)
i965: Add back strict type assertions for MAD and LRP. Commit 526ffdfc033ab01cf133cb7e8290c65d12ccc9be relaxed the type assertions in brw_alu3 to allow D/UD types (required by BFE and BFI2). This lost us the strict type checking for MAD and LRP, which require all four types to be float. This patch adds a new ALU3F wrapper which checks these once again. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2013-06-13 14:55:18 -07:00
ALU3F(LRP)
i965: Add support for emitting and disassembling bit instructions. Specifically bfe - for bitfieldExtract() bfi1 and bfi2 - for bitfieldInsert() bfrev - for bitfieldReverse() cbit - for bitCount() fbh - for findMSB() fbl - for findLSB() Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2013-04-09 17:56:19 -07:00
ALU1(BFREV)
ALU3(BFE)
ALU2(BFI1)
ALU3(BFI2)
ALU1(FBH)
ALU1(FBL)
ALU1(CBIT)
i965: Generate code for ir_binop_carry and ir_binop_borrow. Using the ADDC and SUBB instructions on Gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-09-19 13:01:08 -07:00
ALU2(ADDC)
ALU2(SUBB)
i965: Correctly emit the RNDZ instruction. Simply using RNDU, RNDZ, or RNDE does not produce the desired result. Rather, the RND* instructions place a value in the destination register that may be 1 less than the correct answer. They can also set per-channel "increment bits" in a flag register, which, if set, mean dest needs to be incremented by 1. A second instruction - a predicated add - completes the job. Notably, RNDD always produces the correct answer in a single instruction. Fixes piglit test glsl-fs-trunc.
2010-10-14 11:40:19 -07:00
i965: Use source region <1,2,0> when converting to DF. Doing so allows us to use a single MOV in VEC4_OPCODE_TO_DOUBLE instead of two. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2017-01-12 18:05:58 -08:00
brw_inst *
brw_MOV(struct brw_codegen *p, struct brw_reg dest, struct brw_reg src0)
{
const struct gen_device_info *devinfo = p->devinfo;
/* When converting F->DF on IVB/BYT, every odd source channel is ignored.
intel/eu/gen7: Fix brw_MOV() with DF destination and strided source. I triggered this bug while prototyping code for a future platform on IVB. Could be a problem today though if a strided move is copy-propagated into a type-converting move with DF destination. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-12-07 14:05:52 -08:00
* To avoid the problems that causes, we use an <X,2,0> source region to
* read each element twice.
i965: Use source region <1,2,0> when converting to DF. Doing so allows us to use a single MOV in VEC4_OPCODE_TO_DOUBLE instead of two. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2017-01-12 18:05:58 -08:00
*/
if (devinfo->gen == 7 && !devinfo->is_haswell &&
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_get_default_access_mode(p) == BRW_ALIGN_1 &&
i965: Use source region <1,2,0> when converting to DF. Doing so allows us to use a single MOV in VEC4_OPCODE_TO_DOUBLE instead of two. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2017-01-12 18:05:58 -08:00
dest.type == BRW_REGISTER_TYPE_DF &&
(src0.type == BRW_REGISTER_TYPE_F ||
src0.type == BRW_REGISTER_TYPE_D ||
src0.type == BRW_REGISTER_TYPE_UD) &&
!has_scalar_region(src0)) {
intel/eu/gen7: Fix brw_MOV() with DF destination and strided source. I triggered this bug while prototyping code for a future platform on IVB. Could be a problem today though if a strided move is copy-propagated into a type-converting move with DF destination. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-12-07 14:05:52 -08:00
assert(src0.vstride == src0.width + src0.hstride);
src0.vstride = src0.hstride;
i965: Use source region <1,2,0> when converting to DF. Doing so allows us to use a single MOV in VEC4_OPCODE_TO_DOUBLE instead of two. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2017-01-12 18:05:58 -08:00
src0.width = BRW_WIDTH_2;
src0.hstride = BRW_HORIZONTAL_STRIDE_0;
}
return brw_alu1(p, BRW_OPCODE_MOV, dest, src0);
}
i965: Correctly emit the RNDZ instruction. Simply using RNDU, RNDZ, or RNDE does not produce the desired result. Rather, the RND* instructions place a value in the destination register that may be 1 less than the correct answer. They can also set per-channel "increment bits" in a flag register, which, if set, mean dest needs to be incremented by 1. A second instruction - a predicated add - completes the job. Notably, RNDD always produces the correct answer in a single instruction. Fixes piglit test glsl-fs-trunc.
2010-10-14 11:40:19 -07:00
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_ADD(struct brw_codegen *p, struct brw_reg dest,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
struct brw_reg src0, struct brw_reg src1)
i965: Add some validation on BRW_OPCODE_MUL and ADD's arguments. Now that we're playing with other types in brw_fs.cpp, it's easy to trip over issues like these.
2010-09-04 21:28:04 -07:00
{
/* 6.2.2: add */
if (src0.type == BRW_REGISTER_TYPE_F ||
(src0.file == BRW_IMMEDIATE_VALUE &&
src0.type == BRW_REGISTER_TYPE_VF)) {
assert(src1.type != BRW_REGISTER_TYPE_UD);
assert(src1.type != BRW_REGISTER_TYPE_D);
}
if (src1.type == BRW_REGISTER_TYPE_F ||
(src1.file == BRW_IMMEDIATE_VALUE &&
src1.type == BRW_REGISTER_TYPE_VF)) {
assert(src0.type != BRW_REGISTER_TYPE_UD);
assert(src0.type != BRW_REGISTER_TYPE_D);
}
return brw_alu2(p, BRW_OPCODE_ADD, dest, src0, src1);
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_AVG(struct brw_codegen *p, struct brw_reg dest,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
struct brw_reg src0, struct brw_reg src1)
i965: Add support for AVG instruction. From the Ivy Bridge PRM, Vol4 Part3 p152: "The avg instruction performs component-wise integer average of src0 and src1 and stores the results in dst. An integer average uses integer upward rounding. It is equivalent to increment one to the addition of src0 and src1 and then apply an arithmetic right shift to this intermediate value." Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-07-07 08:28:46 -07:00
{
assert(dest.type == src0.type);
assert(src0.type == src1.type);
switch (src0.type) {
case BRW_REGISTER_TYPE_B:
case BRW_REGISTER_TYPE_UB:
case BRW_REGISTER_TYPE_W:
case BRW_REGISTER_TYPE_UW:
case BRW_REGISTER_TYPE_D:
case BRW_REGISTER_TYPE_UD:
break;
default:
i965: Use unreachable() instead of unconditional assert(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-29 14:54:01 -07:00
unreachable("Bad type for brw_AVG");
i965: Add support for AVG instruction. From the Ivy Bridge PRM, Vol4 Part3 p152: "The avg instruction performs component-wise integer average of src0 and src1 and stores the results in dst. An integer average uses integer upward rounding. It is equivalent to increment one to the addition of src0 and src1 and then apply an arithmetic right shift to this intermediate value." Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2012-07-07 08:28:46 -07:00
}
return brw_alu2(p, BRW_OPCODE_AVG, dest, src0, src1);
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_MUL(struct brw_codegen *p, struct brw_reg dest,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
struct brw_reg src0, struct brw_reg src1)
i965: Add some validation on BRW_OPCODE_MUL and ADD's arguments. Now that we're playing with other types in brw_fs.cpp, it's easy to trip over issues like these.
2010-09-04 21:28:04 -07:00
{
/* 6.32.38: mul */
if (src0.type == BRW_REGISTER_TYPE_D ||
src0.type == BRW_REGISTER_TYPE_UD ||
src1.type == BRW_REGISTER_TYPE_D ||
src1.type == BRW_REGISTER_TYPE_UD) {
assert(dest.type != BRW_REGISTER_TYPE_F);
}
if (src0.type == BRW_REGISTER_TYPE_F ||
(src0.file == BRW_IMMEDIATE_VALUE &&
src0.type == BRW_REGISTER_TYPE_VF)) {
assert(src1.type != BRW_REGISTER_TYPE_UD);
assert(src1.type != BRW_REGISTER_TYPE_D);
}
if (src1.type == BRW_REGISTER_TYPE_F ||
(src1.file == BRW_IMMEDIATE_VALUE &&
src1.type == BRW_REGISTER_TYPE_VF)) {
assert(src0.type != BRW_REGISTER_TYPE_UD);
assert(src0.type != BRW_REGISTER_TYPE_D);
}
assert(src0.file != BRW_ARCHITECTURE_REGISTER_FILE ||
src0.nr != BRW_ARF_ACCUMULATOR);
assert(src1.file != BRW_ARCHITECTURE_REGISTER_FILE ||
src1.nr != BRW_ARF_ACCUMULATOR);
return brw_alu2(p, BRW_OPCODE_MUL, dest, src0, src1);
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Set the region of LINE's src0 to <0,1,0>. The PRMs say that <src0> region must be a replicated scalar (with HorzStride = VertStride = 0). but apparently that doesn't actually apply to all generations. I did notice when implementing the optimization later in this series that G45 and ILK needed this regioning. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-18 23:14:44 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_LINE(struct brw_codegen *p, struct brw_reg dest,
i965: Set the region of LINE's src0 to <0,1,0>. The PRMs say that <src0> region must be a replicated scalar (with HorzStride = VertStride = 0). but apparently that doesn't actually apply to all generations. I did notice when implementing the optimization later in this series that G45 and ILK needed this regioning. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-18 23:14:44 -07:00
struct brw_reg src0, struct brw_reg src1)
{
src0.vstride = BRW_VERTICAL_STRIDE_0;
src0.width = BRW_WIDTH_1;
src0.hstride = BRW_HORIZONTAL_STRIDE_0;
return brw_alu2(p, BRW_OPCODE_LINE, dest, src0, src1);
}
i965/fs: Manually set source regioning on PLN instructions. Like LINE (commit 92346db0), src0 must have a scalar region. Setting src1's region to <8,8,1> lets us pass a properly sized combined delta_xy argument in a few commits without getting a bogus <16,16,1> region. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-06 21:46:54 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_PLN(struct brw_codegen *p, struct brw_reg dest,
i965/fs: Manually set source regioning on PLN instructions. Like LINE (commit 92346db0), src0 must have a scalar region. Setting src1's region to <8,8,1> lets us pass a properly sized combined delta_xy argument in a few commits without getting a bogus <16,16,1> region. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-06 21:46:54 -07:00
struct brw_reg src0, struct brw_reg src1)
{
src0.vstride = BRW_VERTICAL_STRIDE_0;
src0.width = BRW_WIDTH_1;
src0.hstride = BRW_HORIZONTAL_STRIDE_0;
src1.vstride = BRW_VERTICAL_STRIDE_8;
src1.width = BRW_WIDTH_8;
src1.hstride = BRW_HORIZONTAL_STRIDE_1;
return brw_alu2(p, BRW_OPCODE_PLN, dest, src0, src1);
}
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_F32TO16(struct brw_codegen *p, struct brw_reg dst, struct brw_reg src)
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
const bool align16 = brw_get_default_access_mode(p) == BRW_ALIGN_16;
i965/gen8: Fix F32TO16 in vec4 mode if the source and destination registers alias. We cannot zero out the destination register if it overlaps with the source. Use an Align1 instruction instead to zero out the high 16 bits after the conversion to half float. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:08:47 +02:00
/* The F32TO16 instruction doesn't support 32-bit destination types in
* Align1 mode, and neither does the Gen8 implementation in terms of a
* converting MOV. Gen7 does zero out the high 16 bits in Align16 mode as
* an undocumented feature.
*/
const bool needs_zero_fill = (dst.type == BRW_REGISTER_TYPE_UD &&
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
(!align16 || devinfo->gen >= 8));
i965/gen8: Fix F32TO16 in vec4 mode if the source and destination registers alias. We cannot zero out the destination register if it overlaps with the source. Use an Align1 instruction instead to zero out the high 16 bits after the conversion to half float. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:08:47 +02:00
brw_inst *inst;
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
if (align16) {
assert(dst.type == BRW_REGISTER_TYPE_UD);
} else {
i965: Handle F16TO32/F32TO16 with dword src/dst consistently on both back-ends. Due to the way it's implemented in hardware, the F16TO32/F32TO16 instructions require the source/destination register to be of some 16-bit type in Align1 mode, while they require it to be some 32-bit type in Align16 mode (and as an undocumented feature the high 16 bits of the destination register are zeroed out in the case of the F32TO16 instruction on Gen7). Make their behaviour consistent so you can specify a 32 bit register type as source or destination and get predictable results in the most significant bits no matter what access mode is being used. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:24:47 +02:00
assert(dst.type == BRW_REGISTER_TYPE_UD ||
dst.type == BRW_REGISTER_TYPE_W ||
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
dst.type == BRW_REGISTER_TYPE_UW ||
dst.type == BRW_REGISTER_TYPE_HF);
}
i965/gen8: Fix F32TO16 in vec4 mode if the source and destination registers alias. We cannot zero out the destination register if it overlaps with the source. Use an Align1 instruction instead to zero out the high 16 bits after the conversion to half float. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:08:47 +02:00
brw_push_insn_state(p);
if (needs_zero_fill) {
brw_set_default_access_mode(p, BRW_ALIGN_1);
dst = spread(retype(dst, BRW_REGISTER_TYPE_W), 2);
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 8) {
i965/gen8: Fix F32TO16 in vec4 mode if the source and destination registers alias. We cannot zero out the destination register if it overlaps with the source. Use an Align1 instruction instead to zero out the high 16 bits after the conversion to half float. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:08:47 +02:00
inst = brw_MOV(p, retype(dst, BRW_REGISTER_TYPE_HF), src);
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen == 7);
i965/gen8: Fix F32TO16 in vec4 mode if the source and destination registers alias. We cannot zero out the destination register if it overlaps with the source. Use an Align1 instruction instead to zero out the high 16 bits after the conversion to half float. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:08:47 +02:00
inst = brw_alu1(p, BRW_OPCODE_F32TO16, dst, src);
}
if (needs_zero_fill) {
intel/eu/gen12: Don't set DD control, it's gone. A future lowering pass will simulate the same behavior originally provided by NoDDChk/NoDDClr at the IR level by using appropriate SWSB annotations. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:36 -08:00
if (devinfo->gen < 12)
brw_inst_set_no_dd_clear(devinfo, inst, true);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_null());
i965: Use W-typed immediate in brw_F32TO16(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-11-20 00:10:54 -08:00
inst = brw_MOV(p, suboffset(dst, 1), brw_imm_w(0));
intel/eu/gen12: Don't set DD control, it's gone. A future lowering pass will simulate the same behavior originally provided by NoDDChk/NoDDClr at the IR level by using appropriate SWSB annotations. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:36 -08:00
if (devinfo->gen < 12)
brw_inst_set_no_dd_check(devinfo, inst, true);
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
}
i965/gen8: Fix F32TO16 in vec4 mode if the source and destination registers alias. We cannot zero out the destination register if it overlaps with the source. Use an Align1 instruction instead to zero out the high 16 bits after the conversion to half float. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:08:47 +02:00
brw_pop_insn_state(p);
return inst;
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
}
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_F16TO32(struct brw_codegen *p, struct brw_reg dst, struct brw_reg src)
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
bool align16 = brw_get_default_access_mode(p) == BRW_ALIGN_16;
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
if (align16) {
assert(src.type == BRW_REGISTER_TYPE_UD);
} else {
i965: Handle F16TO32/F32TO16 with dword src/dst consistently on both back-ends. Due to the way it's implemented in hardware, the F16TO32/F32TO16 instructions require the source/destination register to be of some 16-bit type in Align1 mode, while they require it to be some 32-bit type in Align16 mode (and as an undocumented feature the high 16 bits of the destination register are zeroed out in the case of the F32TO16 instruction on Gen7). Make their behaviour consistent so you can specify a 32 bit register type as source or destination and get predictable results in the most significant bits no matter what access mode is being used. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 18:24:47 +02:00
/* From the Ivybridge PRM, Vol4, Part3, Section 6.26 f16to32:
*
* Because this instruction does not have a 16-bit floating-point
* type, the source data type must be Word (W). The destination type
* must be F (Float).
*/
if (src.type == BRW_REGISTER_TYPE_UD)
src = spread(retype(src, BRW_REGISTER_TYPE_W), 2);
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
assert(src.type == BRW_REGISTER_TYPE_W ||
src.type == BRW_REGISTER_TYPE_UW ||
src.type == BRW_REGISTER_TYPE_HF);
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 8) {
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
return brw_MOV(p, dst, retype(src, BRW_REGISTER_TYPE_HF));
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen == 7);
i965/eu: Emulate F32TO16 and F16TO32 on Broadwell. When we combine the Gen4-7 and Gen8+ generators, we'll need to handle half float packing/unpacking functions somehow. The Gen8+ generator code today just emulates the behavior of the Gen7 F32TO16/F16TO32 instructions, including the align16 mode bugs. Rather than messing with fs_generator/vec4_generator, I decided to just emulate the instructions at the brw_eu_emit.c layer. v2: Change gen >= 7 asserts to gen == 7 (suggested by Chris Forbes). Fix regressions on Haswell in VS tests due to type assertions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-28 16:08:39 -07:00
return brw_alu1(p, BRW_OPCODE_F16TO32, dst, src);
}
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_NOP(struct brw_codegen *p)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_NOP);
i965: Emit proper NOPs. The PRMs for HSW and newer say that other than the opcode and DebugCtrl bits of the instruction word, the rest must be zero. By zeroing the instruction word manually, we avoid using any of the state inherited through brw_codegen. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=96959 Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2016-12-03 20:14:55 -08:00
memset(insn, 0, sizeof(*insn));
brw_inst_set_opcode(p->devinfo, insn, BRW_OPCODE_NOP);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
intel/fs/gen12: Add codegen support for the SYNC instruction. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-09-03 17:51:17 -07:00
void brw_SYNC(struct brw_codegen *p, enum tgl_sync_function func)
{
brw_inst *insn = next_insn(p, BRW_OPCODE_SYNC);
brw_inst_set_cond_modifier(p->devinfo, insn, func);
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/***********************************************************************
* Comparisons, if/else/endif
*/
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_JMPI(struct brw_codegen *p, struct brw_reg index,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
unsigned predicate_control)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Remove the dst and src0 parameters from brw_JMPI. In all cases, we set both dst and src0 to brw_ip_reg(). This is no accident: according to the ISA reference, both are required to be the IP register. So, we may as well drop the parameters. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-27 22:45:16 -07:00
struct brw_reg ip = brw_ip_reg();
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *inst = brw_alu2(p, BRW_OPCODE_JMPI, ip, ip, index);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Use EXECUTE_1 for JMPI The PRM says "The execution size must be 1." In 73137997e23ff6c11, the execution size was set to 1 when it should have been BRW_EXECUTE_1 (which maps to 0). Later, in dc2d3a7f5c217a7cee9, JMPI was used for line AA on gen6 and earlier and we started manually stomping the exeution size to BRW_EXECUTE_1 in the generator. This commit fixes the original bug and makes brw_JMPI just do the right thing. Reviewed-by: Matt Turner <mattst88@gmail.com> Fixes: 73137997e23ff6c1145d036315d1a9ad96651281
2017-08-31 11:42:00 -07:00
brw_inst_set_exec_size(devinfo, inst, BRW_EXECUTE_1);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, inst, BRW_COMPRESSION_NONE);
brw_inst_set_mask_control(devinfo, inst, BRW_MASK_DISABLE);
brw_inst_set_pred_control(devinfo, inst, predicate_control);
i965: fixes for JMPI 1. the data type of <src1> (JMPI offset) must be D 2. execution size must be 1 3. NoMask 4. instruction compression isn't allowed.
2009-07-02 16:32:19 +08:00
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
return inst;
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
push_if_stack(struct brw_codegen *p, brw_inst *inst)
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
{
i965: let the if_stack just store the instruction index If dynamic instruction store size is enabled, while after the brw_IF/ELSE() and before the brw_ENDIF() function, the eu instruction store base address(p->store) may change. Thus let if_stack just store the instruction index. This is somehow more flexible and safe than store the instruction memory address. Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-12-21 14:51:59 +08:00
p->if_stack[p->if_stack_depth] = inst - p->store;
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
p->if_stack_depth++;
if (p->if_stack_array_size <= p->if_stack_depth) {
p->if_stack_array_size *= 2;
i965: let the if_stack just store the instruction index If dynamic instruction store size is enabled, while after the brw_IF/ELSE() and before the brw_ENDIF() function, the eu instruction store base address(p->store) may change. Thus let if_stack just store the instruction index. This is somehow more flexible and safe than store the instruction memory address. Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-12-21 14:51:59 +08:00
p->if_stack = reralloc(p->mem_ctx, p->if_stack, int,
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
p->if_stack_array_size);
}
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
static brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
pop_if_stack(struct brw_codegen *p)
i965: let the if_stack just store the instruction index If dynamic instruction store size is enabled, while after the brw_IF/ELSE() and before the brw_ENDIF() function, the eu instruction store base address(p->store) may change. Thus let if_stack just store the instruction index. This is somehow more flexible and safe than store the instruction memory address. Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-12-21 14:51:59 +08:00
{
p->if_stack_depth--;
return &p->store[p->if_stack[p->if_stack_depth]];
}
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
push_loop_stack(struct brw_codegen *p, brw_inst *inst)
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
{
i965: fix invalid memory write I noticed some heap corruption running virgl tests, and valgrind helped me to track it down to the following error: ==29272== Invalid write of size 4 ==29272== at 0x90283D4: push_loop_stack (brw_eu_emit.c:1307) ==29272== by 0x9029A7D: brw_DO (brw_eu_emit.c:1750) ==29272== by 0x90554B0: fs_generator::generate_code(cfg_t const*, int) (brw_fs_generator.cpp:1999) ==29272== by 0x904491F: brw_compile_fs (brw_fs.cpp:5685) ==29272== by 0x8FC5DC5: brw_codegen_wm_prog (brw_wm.c:137) ==29272== by 0x8FC7663: brw_fs_precompile (brw_wm.c:638) ==29272== by 0x8FA4040: brw_shader_precompile(gl_context*, gl_shader_program*) (brw_link.cpp:51) ==29272== by 0x8FA4A9A: brw_link_shader (brw_link.cpp:260) ==29272== by 0x8DEF751: _mesa_glsl_link_shader (ir_to_mesa.cpp:3006) ==29272== by 0x8C84325: _mesa_link_program (shaderapi.c:1042) ==29272== by 0x8C851D7: _mesa_LinkProgram (shaderapi.c:1515) ==29272== by 0x4E4B8E8: add_shader_program (vrend_renderer.c:880) ==29272== Address 0xf2f3cb0 is 0 bytes after a block of size 112 alloc'd ==29272== at 0x4C2AA98: calloc (vg_replace_malloc.c:711) ==29272== by 0x8ED11F7: ralloc_size (ralloc.c:113) ==29272== by 0x8ED1282: rzalloc_size (ralloc.c:134) ==29272== by 0x8ED14C0: rzalloc_array_size (ralloc.c:196) ==29272== by 0x9019C7B: brw_init_codegen (brw_eu.c:291) ==29272== by 0x904F565: fs_generator::fs_generator(brw_compiler const*, void*, void*, void const*, brw_stage_prog_data*, unsigned int, bool, gl_shader_stage) (brw_fs_generator.cpp:124) ==29272== by 0x9044883: brw_compile_fs (brw_fs.cpp:5675) ==29272== by 0x8FC5DC5: brw_codegen_wm_prog (brw_wm.c:137) ==29272== by 0x8FC7663: brw_fs_precompile (brw_wm.c:638) ==29272== by 0x8FA4040: brw_shader_precompile(gl_context*, gl_shader_program*) (brw_link.cpp:51) ==29272== by 0x8FA4A9A: brw_link_shader (brw_link.cpp:260) ==29272== by 0x8DEF751: _mesa_glsl_link_shader (ir_to_mesa.cpp:3006) if_depth_in_loop is an array of size p->loop_stack_array_size, and push_loop_stack() will access if_depth_in_loop[p->loop_stack_depth+1], thus the condition to grow the array should be p->loop_stack_array_size <= (p->loop_stack_depth + 1) (it's currently off by 2...) This can be reproduced by running the following test with virgl test server: LIBGL_ALWAYS_SOFTWARE=y GALLIUM_DRIVER=virpipe bin/shader_runner ./tests/shaders/glsl-fs-unroll-explosion.shader_test -auto Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-03-18 20:01:07 +01:00
if (p->loop_stack_array_size <= (p->loop_stack_depth + 1)) {
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
p->loop_stack_array_size *= 2;
p->loop_stack = reralloc(p->mem_ctx, p->loop_stack, int,
p->loop_stack_array_size);
i965: Don't make consumers of brw_CONT/brw_WHILE track if depth in loop. The codegen backends all had this same tracking, so just do it at the EU level. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:44:41 -08:00
p->if_depth_in_loop = reralloc(p->mem_ctx, p->if_depth_in_loop, int,
p->loop_stack_array_size);
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
}
p->loop_stack[p->loop_stack_depth] = inst - p->store;
p->loop_stack_depth++;
i965: Don't make consumers of brw_CONT/brw_WHILE track if depth in loop. The codegen backends all had this same tracking, so just do it at the EU level. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:44:41 -08:00
p->if_depth_in_loop[p->loop_stack_depth] = 0;
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
static brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
get_inner_do_insn(struct brw_codegen *p)
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
{
return &p->store[p->loop_stack[p->loop_stack_depth - 1]];
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/* EU takes the value from the flag register and pushes it onto some
* sort of a stack (presumably merging with any flag value already on
* the stack). Within an if block, the flags at the top of the stack
* control execution on each channel of the unit, eg. on each of the
* 16 pixel values in our wm programs.
*
* When the matching 'else' instruction is reached (presumably by
* countdown of the instruction count patched in by our ELSE/ENDIF
Fix a few typos Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-04-22 11:33:17 +01:00
* functions), the relevant flags are inverted.
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
*
* When the matching 'endif' instruction is reached, the flags are
* popped off. If the stack is now empty, normal execution resumes.
*/
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_IF(struct brw_codegen *p, unsigned execute_size)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
insn = next_insn(p, BRW_OPCODE_IF);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/* Override the defaults for this instruction:
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_ip_reg());
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d(0x0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen == 6) {
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_imm_w(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, insn, 0);
i965 gen6+: Use 1-wide null operands for IF instructions The Sandy Bridge PRM, volume 4, part 2, section 5.3.10 ("5.3.10 Register Region Restrictions") contains the following restriction on the execution size and operand width of instructions: "3. ExecSize must be equal to or greater than Width." When emitting an IF instruction in single program flow mode on Gen6+, we use an ExecSize of 1, therefore the Width of each operand must also be 1. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-12-05 13:52:16 -08:00
brw_set_src0(p, insn, vec1(retype(brw_null_reg(), BRW_REGISTER_TYPE_D)));
brw_set_src1(p, insn, vec1(retype(brw_null_reg(), BRW_REGISTER_TYPE_D)));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen == 7) {
i965 gen6+: Use 1-wide null operands for IF instructions The Sandy Bridge PRM, volume 4, part 2, section 5.3.10 ("5.3.10 Register Region Restrictions") contains the following restriction on the execution size and operand width of instructions: "3. ExecSize must be equal to or greater than Width." When emitting an IF instruction in single program flow mode on Gen6+, we use an ExecSize of 1, therefore the Width of each operand must also be 1. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-12-05 13:52:16 -08:00
brw_set_dest(p, insn, vec1(retype(brw_null_reg(), BRW_REGISTER_TYPE_D)));
brw_set_src0(p, insn, vec1(retype(brw_null_reg(), BRW_REGISTER_TYPE_D)));
i965: Emit IF/ELSE/ENDIF/WHILE JIP with type W on Gen7 IvyBridge and Haswell PRM say that the JIP should be emitted with type W but we were using UD. The previous implementation did not show adverse effects, but IMHO it is safer to follow the specification thoroughly. Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Antia Puentes <apuentes@igalia.com>
2015-03-12 13:59:17 +01:00
brw_set_src1(p, insn, brw_imm_w(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, 0);
brw_inst_set_uip(devinfo, insn, 0);
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
} else {
brw_set_dest(p, insn, vec1(retype(brw_null_reg(), BRW_REGISTER_TYPE_D)));
intel/eu/gen12: Codegen control flow instructions correctly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
if (devinfo->gen < 12)
brw_set_src0(p, insn, brw_imm_d(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, 0);
brw_inst_set_uip(devinfo, insn, 0);
i965: Fix up IF/ELSE/ENDIF for gen6. The jump delta is now in the part of the instruction where the destination fields used to be, and the src args are ignored (or not, for the new non-predicated IF that we don't use yet).
2010-10-04 15:08:03 -07:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_exec_size(devinfo, insn, execute_size);
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
brw_inst_set_pred_control(devinfo, insn, BRW_PREDICATE_NORMAL);
brw_inst_set_mask_control(devinfo, insn, BRW_MASK_ENABLE);
if (!p->single_program_flow && devinfo->gen < 6)
brw_inst_set_thread_control(devinfo, insn, BRW_THREAD_SWITCH);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
push_if_stack(p, insn);
i965: Don't make consumers of brw_CONT/brw_WHILE track if depth in loop. The codegen backends all had this same tracking, so just do it at the EU level. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:44:41 -08:00
p->if_depth_in_loop[p->loop_stack_depth]++;
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
return insn;
}
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
/* This function is only used for gen6-style IF instructions with an
* embedded comparison (conditional modifier). It is not used on gen7.
*/
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
gen6_IF(struct brw_codegen *p, enum brw_conditional_mod conditional,
i965: Rename brw_(IF|CONT)_gen6 functions to gen6_(IF|CONT).
2011-03-16 00:00:09 -07:00
struct brw_reg src0, struct brw_reg src1)
i965: Add EU emit support for gen6's new IF instruction with comparison.
2010-10-19 12:55:04 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Add EU emit support for gen6's new IF instruction with comparison.
2010-10-19 12:55:04 -07:00
insn = next_insn(p, BRW_OPCODE_IF);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_imm_w(0));
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_inst_set_exec_size(devinfo, insn, brw_get_default_exec_size(p));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, insn, 0);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, src1);
i965: Add EU emit support for gen6's new IF instruction with comparison.
2010-10-19 12:55:04 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_qtr_control(devinfo, insn) == BRW_COMPRESSION_NONE);
assert(brw_inst_pred_control(devinfo, insn) == BRW_PREDICATE_NONE);
brw_inst_set_cond_modifier(devinfo, insn, conditional);
i965: Add EU emit support for gen6's new IF instruction with comparison.
2010-10-19 12:55:04 -07:00
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
push_if_stack(p, insn);
i965: Add EU emit support for gen6's new IF instruction with comparison.
2010-10-19 12:55:04 -07:00
return insn;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/**
* In single-program-flow (SPF) mode, convert IF and ELSE into ADDs.
*/
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
convert_IF_ELSE_to_ADD(struct brw_codegen *p,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *if_inst, brw_inst *else_inst)
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* The next instruction (where the ENDIF would be, if it existed) */
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *next_inst = &p->store[p->nr_insn];
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
assert(p->single_program_flow);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(if_inst != NULL && brw_inst_opcode(devinfo, if_inst) == BRW_OPCODE_IF);
assert(else_inst == NULL || brw_inst_opcode(devinfo, else_inst) == BRW_OPCODE_ELSE);
assert(brw_inst_exec_size(devinfo, if_inst) == BRW_EXECUTE_1);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* Convert IF to an ADD instruction that moves the instruction pointer
* to the first instruction of the ELSE block. If there is no ELSE
* block, point to where ENDIF would be. Reverse the predicate.
*
* There's no need to execute an ENDIF since we don't need to do any
* stack operations, and if we're currently executing, we just want to
* continue normally.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_opcode(devinfo, if_inst, BRW_OPCODE_ADD);
brw_inst_set_pred_inv(devinfo, if_inst, true);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
if (else_inst != NULL) {
/* Convert ELSE to an ADD instruction that points where the ENDIF
* would be.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_opcode(devinfo, else_inst, BRW_OPCODE_ADD);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_imm_ud(devinfo, if_inst, (else_inst - if_inst + 1) * 16);
brw_inst_set_imm_ud(devinfo, else_inst, (next_inst - else_inst) * 16);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_imm_ud(devinfo, if_inst, (next_inst - if_inst) * 16);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
}
}
/**
* Patch IF and ELSE instructions with appropriate jump targets.
*/
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
patch_IF_ELSE(struct brw_codegen *p,
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *if_inst, brw_inst *else_inst, brw_inst *endif_inst)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: the offset of any branch/jump instruction is in unit of 64bits on IGDNG
2009-07-15 10:40:16 +08:00
i965: Only convert if/else to conditional adds prior to Gen6. Normally when outputting instructions in SPF (single program flow) mode, we convert IF and ELSE instructions to conditional ADD instructions applied to the IP register. On platforms prior to Gen6, flow control instructions cause an implied thread switch, so this is a significant savings. However, according to the SandyBridge PRM (Volume 4 part 2, p79): [Errata DevSNB{WA}] - When SPF is ON, IP may not be updated by non-flow control instructions. So we have to disable this optimization on Gen6. On later platforms, there is no significant benefit to converting flow control instructions to ADDs, so for the sake of consistency, this patch disables the optimization on later platforms too. The reason we never noticed this problem before is that so far we haven't needed to use SPF mode on Gen6. However, later patches in this series will introduce a Gen6 GS program which uses SPF mode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-11-24 21:41:07 -08:00
/* We shouldn't be patching IF and ELSE instructions in single program flow
* mode when gen < 6, because in single program flow mode on those
* platforms, we convert flow control instructions to conditional ADDs that
* operate on IP (see brw_ENDIF).
*
* However, on Gen6, writing to IP doesn't work in single program flow mode
* (see the SandyBridge PRM, Volume 4 part 2, p79: "When SPF is ON, IP may
* not be updated by non-flow control instructions."). And on later
* platforms, there is no significant benefit to converting control flow
* instructions to conditional ADDs. So we do patch IF and ELSE
* instructions in single program flow mode on those platforms.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6)
i965: Only convert if/else to conditional adds prior to Gen6. Normally when outputting instructions in SPF (single program flow) mode, we convert IF and ELSE instructions to conditional ADD instructions applied to the IP register. On platforms prior to Gen6, flow control instructions cause an implied thread switch, so this is a significant savings. However, according to the SandyBridge PRM (Volume 4 part 2, p79): [Errata DevSNB{WA}] - When SPF is ON, IP may not be updated by non-flow control instructions. So we have to disable this optimization on Gen6. On later platforms, there is no significant benefit to converting flow control instructions to ADDs, so for the sake of consistency, this patch disables the optimization on later platforms too. The reason we never noticed this problem before is that so far we haven't needed to use SPF mode on Gen6. However, later patches in this series will introduce a Gen6 GS program which uses SPF mode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-11-24 21:41:07 -08:00
assert(!p->single_program_flow);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(if_inst != NULL && brw_inst_opcode(devinfo, if_inst) == BRW_OPCODE_IF);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
assert(endif_inst != NULL);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(else_inst == NULL || brw_inst_opcode(devinfo, else_inst) == BRW_OPCODE_ELSE);
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
unsigned br = brw_jump_scale(devinfo);
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_opcode(devinfo, endif_inst) == BRW_OPCODE_ENDIF);
brw_inst_set_exec_size(devinfo, endif_inst, brw_inst_exec_size(devinfo, if_inst));
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
if (else_inst == NULL) {
/* Patch IF -> ENDIF */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* Turn it into an IFF, which means no mask stack operations for
* all-false and jumping past the ENDIF.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_opcode(devinfo, if_inst, BRW_OPCODE_IFF);
brw_inst_set_gen4_jump_count(devinfo, if_inst,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
br * (endif_inst - if_inst + 1));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen4_pop_count(devinfo, if_inst, 0);
} else if (devinfo->gen == 6) {
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* As of gen6, there is no IFF and IF must point to the ENDIF. */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, if_inst, br*(endif_inst - if_inst));
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_uip(devinfo, if_inst, br * (endif_inst - if_inst));
brw_inst_set_jip(devinfo, if_inst, br * (endif_inst - if_inst));
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
}
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_exec_size(devinfo, else_inst, brw_inst_exec_size(devinfo, if_inst));
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* Patch IF -> ELSE */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
brw_inst_set_gen4_jump_count(devinfo, if_inst,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
br * (else_inst - if_inst));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen4_pop_count(devinfo, if_inst, 0);
} else if (devinfo->gen == 6) {
brw_inst_set_gen6_jump_count(devinfo, if_inst,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
br * (else_inst - if_inst + 1));
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
}
/* Patch ELSE -> ENDIF */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* BRW_OPCODE_ELSE pre-gen6 should point just past the
* matching ENDIF.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen4_jump_count(devinfo, else_inst,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
br * (endif_inst - else_inst + 1));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen4_pop_count(devinfo, else_inst, 1);
} else if (devinfo->gen == 6) {
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* BRW_OPCODE_ELSE on gen6 should point to the matching ENDIF. */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, else_inst,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
br * (endif_inst - else_inst));
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
} else {
/* The IF instruction's JIP should point just past the ELSE */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, if_inst, br * (else_inst - if_inst + 1));
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
/* The IF instruction's UIP and ELSE's JIP should point to ENDIF */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_uip(devinfo, if_inst, br * (endif_inst - if_inst));
brw_inst_set_jip(devinfo, else_inst, br * (endif_inst - else_inst));
if (devinfo->gen >= 8) {
i965/eu: Set UIP on ELSE instructions on Broadwell. Broadwell adds UIP on ELSE instructions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-30 08:05:42 -07:00
/* Since we don't set branch_ctrl, the ELSE's JIP and UIP both
* should point to ENDIF.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_uip(devinfo, else_inst, br * (endif_inst - else_inst));
i965/eu: Set UIP on ELSE instructions on Broadwell. Broadwell adds UIP on ELSE instructions. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-30 08:05:42 -07:00
}
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
}
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
}
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
}
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_ELSE(struct brw_codegen *p)
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
insn = next_insn(p, BRW_OPCODE_ELSE);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_ip_reg());
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d(0x0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen == 6) {
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_imm_w(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, insn, 0);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src1(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen == 7) {
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Emit IF/ELSE/ENDIF/WHILE JIP with type W on Gen7 IvyBridge and Haswell PRM say that the JIP should be emitted with type W but we were using UD. The previous implementation did not show adverse effects, but IMHO it is safer to follow the specification thoroughly. Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Antia Puentes <apuentes@igalia.com>
2015-03-12 13:59:17 +01:00
brw_set_src1(p, insn, brw_imm_w(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, 0);
brw_inst_set_uip(devinfo, insn, 0);
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
} else {
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
intel/eu/gen12: Codegen control flow instructions correctly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
if (devinfo->gen < 12)
brw_set_src0(p, insn, brw_imm_d(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, 0);
brw_inst_set_uip(devinfo, insn, 0);
i965: Fix up IF/ELSE/ENDIF for gen6. The jump delta is now in the part of the instruction where the destination fields used to be, and the src args are ignored (or not, for the new non-predicated IF that we don't use yet).
2010-10-04 15:08:03 -07:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
brw_inst_set_mask_control(devinfo, insn, BRW_MASK_ENABLE);
if (!p->single_program_flow && devinfo->gen < 6)
brw_inst_set_thread_control(devinfo, insn, BRW_THREAD_SWITCH);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
push_if_stack(p, insn);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Move IF stack handling into the EU abstraction layer/brw_compile. This hides the IF stack and back-patching of IF/ELSE instructions from each of the code generators, greatly simplifying the interface. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 12:25:18 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_ENDIF(struct brw_codegen *p)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = NULL;
brw_inst *else_inst = NULL;
brw_inst *if_inst = NULL;
brw_inst *tmp;
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
bool emit_endif = true;
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
i965: Only convert if/else to conditional adds prior to Gen6. Normally when outputting instructions in SPF (single program flow) mode, we convert IF and ELSE instructions to conditional ADD instructions applied to the IP register. On platforms prior to Gen6, flow control instructions cause an implied thread switch, so this is a significant savings. However, according to the SandyBridge PRM (Volume 4 part 2, p79): [Errata DevSNB{WA}] - When SPF is ON, IP may not be updated by non-flow control instructions. So we have to disable this optimization on Gen6. On later platforms, there is no significant benefit to converting flow control instructions to ADDs, so for the sake of consistency, this patch disables the optimization on later platforms too. The reason we never noticed this problem before is that so far we haven't needed to use SPF mode on Gen6. However, later patches in this series will introduce a Gen6 GS program which uses SPF mode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-11-24 21:41:07 -08:00
/* In single program flow mode, we can express IF and ELSE instructions
* equivalently as ADD instructions that operate on IP. On platforms prior
* to Gen6, flow control instructions cause an implied thread switch, so
* this is a significant savings.
*
* However, on Gen6, writing to IP doesn't work in single program flow mode
* (see the SandyBridge PRM, Volume 4 part 2, p79: "When SPF is ON, IP may
* not be updated by non-flow control instructions."). And on later
* platforms, there is no significant benefit to converting control flow
* instructions to conditional ADDs. So we only do this trick on Gen4 and
* Gen5.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6 && p->single_program_flow)
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
emit_endif = false;
/*
Fix a few typos Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-04-22 11:33:17 +01:00
* A single next_insn() may change the base address of instruction store
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
* memory(p->store), so call it first before referencing the instruction
* store pointer from an index
*/
if (emit_endif)
insn = next_insn(p, BRW_OPCODE_ENDIF);
/* Pop the IF and (optional) ELSE instructions from the stack */
p->if_depth_in_loop[p->loop_stack_depth]--;
tmp = pop_if_stack(p);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (brw_inst_opcode(devinfo, tmp) == BRW_OPCODE_ELSE) {
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
else_inst = tmp;
tmp = pop_if_stack(p);
}
if_inst = tmp;
if (!emit_endif) {
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* ENDIF is useless; don't bother emitting it. */
convert_IF_ELSE_to_ADD(p, if_inst, else_inst);
return;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
i965/gen4-5: Set ENDIF dst and src0 fields to the null register. The hardware docs don't mention explicitly what these fields should be, but I've verified experimentally on ILK that using a GRF as destination causes the register to be corrupted when the execution size of an ENDIF instruction is higher than 8 -- and because the destination we were using was g0, eventually a hang. Fixes some 150 piglit tests on Gen4-5 when forced to run shaders with if conditionals 16-wide, e.g. shaders/glsl-fs-sampler-numbering-3. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-06 18:23:57 +03:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
brw_set_src1(p, insn, brw_imm_d(0x0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen == 6) {
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
brw_set_dest(p, insn, brw_imm_w(0));
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src1(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen == 7) {
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Emit IF/ELSE/ENDIF/WHILE JIP with type W on Gen7 IvyBridge and Haswell PRM say that the JIP should be emitted with type W but we were using UD. The previous implementation did not show adverse effects, but IMHO it is safer to follow the specification thoroughly. Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Antia Puentes <apuentes@igalia.com>
2015-03-12 13:59:17 +01:00
brw_set_src1(p, insn, brw_imm_w(0));
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
} else {
brw_set_src0(p, insn, brw_imm_d(0));
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
}
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
brw_inst_set_mask_control(devinfo, insn, BRW_MASK_ENABLE);
if (devinfo->gen < 6)
brw_inst_set_thread_control(devinfo, insn, BRW_THREAD_SWITCH);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
/* Also pop item off the stack in the endif instruction: */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
brw_inst_set_gen4_jump_count(devinfo, insn, 0);
brw_inst_set_gen4_pop_count(devinfo, insn, 1);
} else if (devinfo->gen == 6) {
brw_inst_set_gen6_jump_count(devinfo, insn, 2);
i965: Add support for IF/ELSE/ENDIF control flow on Ivybridge. Ivybridge's IF instruction doesn't support conditional modifiers. It also introduces UIP, which must point to the ENDIF instruction. ELSE and ENDIF remain the same except that JIP moves from dst to src1. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-03-15 23:53:40 -07:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, 2);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Rework IF/ELSE jump target back-patching. The primary motivation for this is to better support Ivybridge control flow. Ivybridge IF instructions need to point to the first instruction of the ELSE block -and- the ENDIF instruction; the existing code only supported back-patching one instruction ago. A second goal is to simplify and centralize the back-patching, hopefully clarifying the code somewhat. Previously, brw_ELSE back-patched the IF instruction, and brw_ENDIF back-patched the previous instruction (IF or ELSE). With this patch, brw_ENDIF is responsible for patching both the IF and (optional) ELSE. To support this, the control flow stack (if_stack) maintains pointers to both the IF and ELSE instructions. Unfortunately, in single program flow (SPF) mode, both were emitted as ADD instructions, and thus indistinguishable. To remedy this, this patch simply emits IF and ELSE, rather than ADDs; brw_ENDIF will convert them to ADDs (the SPF version of back-patching). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-16 13:40:00 -07:00
patch_IF_ELSE(p, if_inst, else_inst, insn);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_BREAK(struct brw_codegen *p)
support branch and loop in pixel shader most of the sample working with some small modification
2007-06-21 10:22:28 +08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
support branch and loop in pixel shader most of the sample working with some small modification
2007-06-21 10:22:28 +08:00
insn = next_insn(p, BRW_OPCODE_BREAK);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 8) {
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src0(p, insn, brw_imm_d(0x0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
} else if (devinfo->gen >= 6) {
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src1(p, insn, brw_imm_d(0x0));
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
} else {
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_ip_reg());
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d(0x0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen4_pop_count(devinfo, insn,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
p->if_depth_in_loop[p->loop_stack_depth]);
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_inst_set_exec_size(devinfo, insn, brw_get_default_exec_size(p));
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
support continue, fix conditional
2007-09-29 15:00:52 +08:00
return insn;
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_CONT(struct brw_codegen *p)
i965: Add support for gen6 CONTINUE instruction emit. At this point, piglit tests for fragment shader loops are working.
2010-12-01 14:02:14 -08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Add support for gen6 CONTINUE instruction emit. At this point, piglit tests for fragment shader loops are working.
2010-12-01 14:02:14 -08:00
insn = next_insn(p, BRW_OPCODE_CONTINUE);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_ip_reg());
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 8) {
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
brw_set_src0(p, insn, brw_imm_d(0x0));
} else {
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d(0x0));
}
i965: Add support for gen6 CONTINUE instruction emit. At this point, piglit tests for fragment shader loops are working.
2010-12-01 14:02:14 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6) {
brw_inst_set_gen4_pop_count(devinfo, insn,
i965/eu: Merge brw_CONT and gen6_CONT. The only difference is setting PopCount on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-04 14:26:26 -07:00
p->if_depth_in_loop[p->loop_stack_depth]);
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_inst_set_exec_size(devinfo, insn, brw_get_default_exec_size(p));
support branch and loop in pixel shader most of the sample working with some small modification
2007-06-21 10:22:28 +08:00
return insn;
}
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
intel/fs: Emit HALT for discard on Gen4-5 Using HALT to immediately jump to the end of the shader is required to implement GL_EXT_gpu_shader4 and OpenGL 3.0. However, vanilla OpenGL 1.2 doesn't forbid it and it likely makes something somewhere faster. We should be consistent and implement the same discard behavior on all hardware if we can. The rules for HALT on Gen4-5 are a bit different from Gen6+. On the older hardware, there is no stack for HALT; instead it's up to software to save and restore mask registers. However, there's no real saving needed since we only use HALT to jump to the end of the program where we're about about to do our FB writes. All we need to do is reset AMask to DMask, the value it was initialized to at the start of the thread. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5244>
2020-04-25 14:59:30 -05:00
brw_HALT(struct brw_codegen *p)
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
insn = next_insn(p, BRW_OPCODE_HALT);
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
intel/fs: Emit HALT for discard on Gen4-5 Using HALT to immediately jump to the end of the shader is required to implement GL_EXT_gpu_shader4 and OpenGL 3.0. However, vanilla OpenGL 1.2 doesn't forbid it and it likely makes something somewhere faster. We should be consistent and implement the same discard behavior on all hardware if we can. The rules for HALT on Gen4-5 are a bit different from Gen6+. On the older hardware, there is no stack for HALT; instead it's up to software to save and restore mask registers. However, there's no real saving needed since we only use HALT to jump to the end of the program where we're about about to do our FB writes. All we need to do is reset AMask to DMask, the value it was initialized to at the start of the thread. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5244>
2020-04-25 14:59:30 -05:00
if (devinfo->gen < 6) {
/* From the Gen4 PRM:
*
* "IP register must be put (for example, by the assembler) at <dst>
* and <src0> locations.
*/
brw_set_dest(p, insn, brw_ip_reg());
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d(0x0)); /* exitcode updated later. */
} else if (devinfo->gen < 8) {
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src1(p, insn, brw_imm_d(0x0)); /* UIP and JIP, updated later. */
intel/eu/gen12: Codegen control flow instructions correctly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
} else if (devinfo->gen < 12) {
brw_set_src0(p, insn, brw_imm_d(0x0));
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
}
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 15:29:27 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_inst_set_exec_size(devinfo, insn, brw_get_default_exec_size(p));
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
return insn;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/* DO/WHILE loop:
i965: Add support for gen6 DO/WHILE ISA emit. There's no more DO since there's no more mask stack, and WHILE has been shuffled like IF was.
2010-12-01 10:45:52 -08:00
*
* The DO/WHILE is just an unterminated loop -- break or continue are
* used for control within the loop. We have a few ways they can be
* done.
*
* For uniform control flow, the WHILE is just a jump, so ADD ip, ip,
* jip and no DO instruction.
*
* For non-uniform control flow pre-gen6, there's a DO instruction to
* push the mask, and a WHILE to jump back, and BREAK to get out and
* pop the mask.
*
* For gen6, there's no more mask stack, so no need for DO. WHILE
* just points back to the first instruction of the loop.
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
*/
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_DO(struct brw_codegen *p, unsigned execute_size)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add support for gen6 DO/WHILE ISA emit. There's no more DO since there's no more mask stack, and WHILE has been shuffled like IF was.
2010-12-01 10:45:52 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6 || p->single_program_flow) {
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
push_loop_stack(p, &p->store[p->nr_insn]);
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
return &p->store[p->nr_insn];
} else {
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_DO);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Don't make consumers of brw_DO()/brw_WHILE() track loop start. This is a similar cleanup to what we did for brw_IF(), brw_ELSE(), brw_ENDIF() handling. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:13:32 -08:00
push_loop_stack(p, insn);
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
/* Override the defaults for this instruction:
*/
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_null_reg());
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_null_reg());
brw_set_src1(p, insn, brw_null_reg());
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
brw_inst_set_exec_size(devinfo, insn, execute_size);
brw_inst_set_pred_control(devinfo, insn, BRW_PREDICATE_NONE);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
return insn;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
/**
* For pre-gen6, we patch BREAK/CONT instructions to point at the WHILE
* instruction here.
*
* For gen6+, see brw_set_uip_jip(), which doesn't care so much about the loop
* nesting, since it can always just point to the end of the block/current loop.
*/
static void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_patch_break_cont(struct brw_codegen *p, brw_inst *while_inst)
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *do_inst = get_inner_do_insn(p);
brw_inst *inst;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
unsigned br = brw_jump_scale(devinfo);
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen < 6);
i965/eu: Make it clear that brw_patch_break_count only runs on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-30 09:22:27 -07:00
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
for (inst = while_inst - 1; inst != do_inst; inst--) {
/* If the jump count is != 0, that means that this instruction has already
* been patched because it's part of a loop inside of the one we're
* patching.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (brw_inst_opcode(devinfo, inst) == BRW_OPCODE_BREAK &&
brw_inst_gen4_jump_count(devinfo, inst) == 0) {
brw_inst_set_gen4_jump_count(devinfo, inst, br*((while_inst - inst) + 1));
} else if (brw_inst_opcode(devinfo, inst) == BRW_OPCODE_CONTINUE &&
brw_inst_gen4_jump_count(devinfo, inst) == 0) {
brw_inst_set_gen4_jump_count(devinfo, inst, br * (while_inst - inst));
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
}
}
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_WHILE(struct brw_codegen *p)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn, *do_insn;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
unsigned br = brw_jump_scale(devinfo);
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
insn = next_insn(p, BRW_OPCODE_WHILE);
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
do_insn = get_inner_do_insn(p);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 8) {
i965/eu: Set src0 file to IMM on Gen8+ flow control instructions. According to the documentation, we need to set the source 0 register type to IMM for flow control instructinos that have both JIP and UIP. Out of paranoia, just make all flow control instructions use IMM; there's no benefit to using ARF anyway, and it could trouble that's difficult to diagnose. See commit 9584959123b0453cf5313722357e3abb9f736aa7, which did the analogous change in the gen8_generator code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:10:55 -07:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
intel/eu/gen12: Codegen control flow instructions correctly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-09 14:13:35 -08:00
if (devinfo->gen < 12)
brw_set_src0(p, insn, brw_imm_d(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, br * (do_insn - insn));
} else if (devinfo->gen == 7) {
i965/eu: Refactor brw_WHILE to share a bit more code on Gen6+. We're going to add a Gen8+ case shortly, which would need to duplicate this code again. Instead, share it. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:06:36 -07:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
i965: Emit IF/ELSE/ENDIF/WHILE JIP with type W on Gen7 IvyBridge and Haswell PRM say that the JIP should be emitted with type W but we were using UD. The previous implementation did not show adverse effects, but IMHO it is safer to follow the specification thoroughly. Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Antia Puentes <apuentes@igalia.com>
2015-03-12 13:59:17 +01:00
brw_set_src1(p, insn, brw_imm_w(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, br * (do_insn - insn));
i965/eu: Refactor brw_WHILE to share a bit more code on Gen6+. We're going to add a Gen8+ case shortly, which would need to duplicate this code again. Instead, share it. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:06:36 -07:00
} else {
brw_set_dest(p, insn, brw_imm_w(0));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, insn, br * (do_insn - insn));
i965/eu: Refactor brw_WHILE to share a bit more code on Gen6+. We're going to add a Gen8+ case shortly, which would need to duplicate this code again. Instead, share it. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-10 07:06:36 -07:00
brw_set_src0(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
brw_set_src1(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_D));
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_inst_set_exec_size(devinfo, insn, brw_get_default_exec_size(p));
i965/eu: Stop using p->compressed to specify the exec size of control flow instructions. p->compressed won't work for SIMD32, we should just be using the execution size value specified via p->current instead. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 19:17:31 -07:00
i965: Add support for gen6 DO/WHILE ISA emit. There's no more DO since there's no more mask stack, and WHILE has been shuffled like IF was.
2010-12-01 10:45:52 -08:00
} else {
if (p->single_program_flow) {
insn = next_insn(p, BRW_OPCODE_ADD);
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
do_insn = get_inner_do_insn(p);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_ip_reg());
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d((do_insn - insn) * 16));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_exec_size(devinfo, insn, BRW_EXECUTE_1);
i965: Add support for gen6 DO/WHILE ISA emit. There's no more DO since there's no more mask stack, and WHILE has been shuffled like IF was.
2010-12-01 10:45:52 -08:00
} else {
insn = next_insn(p, BRW_OPCODE_WHILE);
i965: call next_insn() before referencing a instruction by index A single next_insn may change the base address of instruction store memory(p->store), so call it first before referencing the instruction store pointer from an index. This the final prepare work to enable the dynamic store size. v2: comments from Ken, define emit_endif as bool type Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-12-21 15:32:02 +08:00
do_insn = get_inner_do_insn(p);
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_opcode(devinfo, do_insn) == BRW_OPCODE_DO);
i965: Avoid branch instructions while in single program flow mode. There is an errata for Broadwater that threads don't have the instruction/loop mask stacks initialized on thread spawn. In single program flow mode, those stacks are not writable, so we can't initialize them. However, they do get read during ELSE and ENDIF instructions. So, instead, replace branch instructions in single program flow mode with predicated jumps (ADD to the ip register), avoiding use of the more complicated branch instructions that may fail. This is also a minor optimization as no ENDIF equivalent is necessary. Signed-off-by: Keith Packard <keithp@neko.keithp.com>
2006-10-18 00:24:01 -07:00
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, brw_ip_reg());
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_ip_reg());
brw_set_src1(p, insn, brw_imm_d(0));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_exec_size(devinfo, insn, brw_inst_exec_size(devinfo, do_insn));
brw_inst_set_gen4_jump_count(devinfo, insn, br * (do_insn - insn + 1));
brw_inst_set_gen4_pop_count(devinfo, insn, 0);
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
brw_patch_break_cont(p, insn);
i965: Add support for gen6 DO/WHILE ISA emit. There's no more DO since there's no more mask stack, and WHILE has been shuffled like IF was.
2010-12-01 10:45:52 -08:00
}
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_qtr_control(devinfo, insn, BRW_COMPRESSION_NONE);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 12:30:03 -08:00
p->loop_stack_depth--;
support branch and loop in pixel shader most of the sample working with some small modification
2007-06-21 10:22:28 +08:00
return insn;
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
Revert "i965: Move brw_land_fwd_jump() to compilation unit of its use." This reverts commit f3cb2e6ed7059b22752a6b7d7a98c07ba6b5552e. brw_land_fwd_jump() is convenient wherever we produce JMPI instructions and we will use JMPI to implement framebuffer writes that involve line antialiasing in gen < 6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 15:03:07 +02:00
/* FORWARD JUMPS:
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_land_fwd_jump(struct brw_codegen *p, int jmp_insn_idx)
Revert "i965: Move brw_land_fwd_jump() to compilation unit of its use." This reverts commit f3cb2e6ed7059b22752a6b7d7a98c07ba6b5552e. brw_land_fwd_jump() is convenient wherever we produce JMPI instructions and we will use JMPI to implement framebuffer writes that involve line antialiasing in gen < 6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 15:03:07 +02:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *jmp_insn = &p->store[jmp_insn_idx];
Revert "i965: Move brw_land_fwd_jump() to compilation unit of its use." This reverts commit f3cb2e6ed7059b22752a6b7d7a98c07ba6b5552e. brw_land_fwd_jump() is convenient wherever we produce JMPI instructions and we will use JMPI to implement framebuffer writes that involve line antialiasing in gen < 6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 15:03:07 +02:00
unsigned jmpi = 1;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 5)
Revert "i965: Move brw_land_fwd_jump() to compilation unit of its use." This reverts commit f3cb2e6ed7059b22752a6b7d7a98c07ba6b5552e. brw_land_fwd_jump() is convenient wherever we produce JMPI instructions and we will use JMPI to implement framebuffer writes that involve line antialiasing in gen < 6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 15:03:07 +02:00
jmpi = 2;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_opcode(devinfo, jmp_insn) == BRW_OPCODE_JMPI);
assert(brw_inst_src1_reg_file(devinfo, jmp_insn) == BRW_IMMEDIATE_VALUE);
Revert "i965: Move brw_land_fwd_jump() to compilation unit of its use." This reverts commit f3cb2e6ed7059b22752a6b7d7a98c07ba6b5552e. brw_land_fwd_jump() is convenient wherever we produce JMPI instructions and we will use JMPI to implement framebuffer writes that involve line antialiasing in gen < 6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 15:03:07 +02:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen4_jump_count(devinfo, jmp_insn,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
jmpi * (p->nr_insn - jmp_insn_idx - 1));
Revert "i965: Move brw_land_fwd_jump() to compilation unit of its use." This reverts commit f3cb2e6ed7059b22752a6b7d7a98c07ba6b5552e. brw_land_fwd_jump() is convenient wherever we produce JMPI instructions and we will use JMPI to implement framebuffer writes that involve line antialiasing in gen < 6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 15:03:07 +02:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/* To integrate with the above, it makes sense that the comparison
* instruction should populate the flag register. It might be simpler
* just to use the flag reg for most WM tasks?
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_CMP(struct brw_codegen *p,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned conditional,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg src0,
struct brw_reg src1)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_CMP);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_cond_modifier(devinfo, insn, conditional);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, src1);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Implement work-around for CMP with null dest on Haswell. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-04-15 14:59:09 -07:00
/* Item WaCMPInstNullDstForcesThreadSwitch in the Haswell Bspec workarounds
* page says:
* "Any CMP instruction with a null destination must use a {switch}."
i965: Apply CMP NULL {Switch} work-around to other Gen7s. Listed in the restrictions section of CMP, but not on the work-arounds page. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-04-21 00:18:11 -07:00
*
* It also applies to other Gen7 platforms (IVB, BYT) even though it isn't
* mentioned on their work-arounds pages.
i965: Implement work-around for CMP with null dest on Haswell. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-04-15 14:59:09 -07:00
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen == 7) {
i965: Implement work-around for CMP with null dest on Haswell. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-04-15 14:59:09 -07:00
if (dest.file == BRW_ARCHITECTURE_REGISTER_FILE &&
dest.nr == BRW_ARF_NULL) {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_thread_control(devinfo, insn, BRW_THREAD_SWITCH);
i965: Implement work-around for CMP with null dest on Haswell. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-04-15 14:59:09 -07:00
}
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
intel/compiler: Enable the ability to emit CMPN instructions v2: Move checks to the EU validator. Suggested by Jason. Fixes: 2f2c00c7279 ("i965: Lower min/max after optimization on Gen4/5.") Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9027>
2021-02-13 14:11:30 -08:00
void brw_CMPN(struct brw_codegen *p,
struct brw_reg dest,
unsigned conditional,
struct brw_reg src0,
struct brw_reg src1)
{
const struct gen_device_info *devinfo = p->devinfo;
brw_inst *insn = next_insn(p, BRW_OPCODE_CMPN);
brw_inst_set_cond_modifier(devinfo, insn, conditional);
brw_set_dest(p, insn, dest);
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, src1);
/* Page 166 of the Ivy Bridge PRM Volume 4 part 3 (Execution Unit ISA)
* says:
*
* If the destination is the null register, the {Switch} instruction
* option must be used.
*
* Page 77 of the Haswell PRM Volume 2b contains the same text.
*/
if (devinfo->gen == 7) {
if (dest.file == BRW_ARCHITECTURE_REGISTER_FILE &&
dest.nr == BRW_ARF_NULL) {
brw_inst_set_thread_control(devinfo, insn, BRW_THREAD_SWITCH);
}
}
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/***********************************************************************
* Helpers for the various SEND message types:
*/
i965: new and updated comments
2009-03-31 10:49:41 -06:00
/** Extended math function, float[8].
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void gen4_math(struct brw_codegen *p,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned function,
unsigned msg_reg_nr,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg src,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned precision )
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_SEND);
i965: Use BRW_MATH_DATA_SCALAR when source regioning is scalar. Notice the mistaken (but harmless) argument swapping in brw_math_invert(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-29 21:10:32 -07:00
unsigned data_type;
i965: Extract scalar region checking logic There are currently 2 users of this functionality. I have 2 more users coming up, and having a simple function makes the results much cleaner. The existing interface semantics was proposed by Matt. v2 (Ken): Rename to region_matches()/has_scalar_region(). Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-12-22 19:29:22 -08:00
if (has_scalar_region(src)) {
i965: Use BRW_MATH_DATA_SCALAR when source regioning is scalar. Notice the mistaken (but harmless) argument swapping in brw_math_invert(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-29 21:10:32 -07:00
data_type = BRW_MATH_DATA_SCALAR;
} else {
data_type = BRW_MATH_DATA_VECTOR;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen < 6);
i965: Add SNB math opcode support. This is untested at this point.
2010-02-24 09:16:19 -08:00
i965: Split Gen4-5 and Gen6+ MATH instruction emitters. Our existing functions, brw_math and brw_math2, had unclear roles: Gen4-5 used brw_math for both unary and binary math functions; it never used brw_math2. Since operands are already in message registers, this is reasonable. Gen6+ used brw_math for unary math functions, and brw_math2 for binary math functions, duplicating a lot of code. The only real difference was that brw_math used brw_null_reg() for src1. This patch improves brw_math2's assertions to allow both unary and binary operations, renames it to gen6_math(), and drops the Gen6+ code out of brw_math(). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-07 01:56:12 -07:00
/* Example code doesn't set predicate_control for send
* instructions.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_pred_control(devinfo, insn, 0);
brw_inst_set_base_mrf(devinfo, insn, msg_reg_nr);
i965: Fix message and response length calculations for INT DIV. Both POW and INT DIV need a message length of 2; previously, we only checked for POW. Also, BRW_MATH_FUNCTION_INT_DIV_QUOTIENT_AND_REMAINDER has a response length of 2; previously, we only checked for SINCOS. We don't use this message, but in case we ever decide to, we may as well fix it now. While we're at it, just move these computations into brw_set_math_message, since they're entirely based on the function. This fixes it for both brw_math and the old backend's brw_math_16. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Tested-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-28 17:37:52 -07:00
i965: Split Gen4-5 and Gen6+ MATH instruction emitters. Our existing functions, brw_math and brw_math2, had unclear roles: Gen4-5 used brw_math for both unary and binary math functions; it never used brw_math2. Since operands are already in message registers, this is reasonable. Gen6+ used brw_math for unary math functions, and brw_math2 for binary math functions, duplicating a lot of code. The only real difference was that brw_math used brw_null_reg() for src1. This patch improves brw_math2's assertions to allow both unary and binary operations, renames it to gen6_math(), and drops the Gen6+ code out of brw_math(). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-07 01:56:12 -07:00
brw_set_dest(p, insn, dest);
brw_set_src0(p, insn, src);
brw_set_math_message(p,
insn,
function,
src.type == BRW_REGISTER_TYPE_D,
precision,
data_type);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void gen6_math(struct brw_codegen *p,
i965: Add support for POW in gen6 FS. Fixes glsl-algebraic-pow-2 in brw_wm_glsl.c mode.
2010-08-22 01:33:57 -07:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned function,
i965: Add support for POW in gen6 FS. Fixes glsl-algebraic-pow-2 in brw_wm_glsl.c mode.
2010-08-22 01:33:57 -07:00
struct brw_reg src0,
struct brw_reg src1)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_MATH);
i965: Add support for POW in gen6 FS. Fixes glsl-algebraic-pow-2 in brw_wm_glsl.c mode.
2010-08-22 01:33:57 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen >= 6);
i965: Split Gen4-5 and Gen6+ MATH instruction emitters. Our existing functions, brw_math and brw_math2, had unclear roles: Gen4-5 used brw_math for both unary and binary math functions; it never used brw_math2. Since operands are already in message registers, this is reasonable. Gen6+ used brw_math for unary math functions, and brw_math2 for binary math functions, duplicating a lot of code. The only real difference was that brw_math used brw_null_reg() for src1. This patch improves brw_math2's assertions to allow both unary and binary operations, renames it to gen6_math(), and drops the Gen6+ code out of brw_math(). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-07 01:56:12 -07:00
i965/gen7: Relax restrictions on fake MRFs Gen6 has write-only MRF registers, and for ease of implementation we paritition off 16 general purposes registers to act as MRFs on Gen7. Knowing that our Gen7 MRFs are actually GRFs, we can do things we can't do with real MRFs: - read from them; - return values directly to them from a send instruction; and - compute directly to them with math instructions. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-02-11 11:06:13 -08:00
assert(dest.file == BRW_GENERAL_REGISTER_FILE ||
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
(devinfo->gen >= 7 && dest.file == BRW_MESSAGE_REGISTER_FILE));
i965: Add a couple of checks for gen6 math instruction limits.
2010-10-11 13:30:12 -07:00
assert(dest.hstride == BRW_HORIZONTAL_STRIDE_1);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen == 6) {
Revert "i965/fs: Allow scalar source regions on SNB math instructions." This reverts commit c1107cec44ab030c7fcc97c67baa12df1cc9d7b5. Apparently the hardware spec text I quoted in the commit message was outright lying about scalar source math being supported on SNB, the hardware seems to load 32 contiguous bits of data for each channel regardless of the regioning mode. Fixes regressions in the following CTS tests (which we didn't catch early due to CTS being temporarily disabled in our CI system): es2-cts.gtf.gl.atan.atan_vec3_frag_xvary es2-cts.gtf.gl.cos.cos_vec2_frag_xvary es2-cts.gtf.gl.atan.atan_vec2_frag_xvary es2-cts.gtf.gl.pow.pow_vec2_frag_xvary_yconsthalf es2-cts.gtf.gl.cos.cos_float_frag_xvary es2-cts.gtf.gl.pow.pow_float_frag_xvary_yconsthalf es2-cts.gtf.gl.atan.atan_vec3_frag_xvaryyvary es2-cts.gtf.gl.pow.pow_vec3_frag_xvary_yconsthalf es2-cts.gtf.gl.cos.cos_vec3_frag_xvary es2-cts.gtf.gl.atan.atan_vec2_frag_xvaryyvary Cc: mesa-stable@lists.freedesktop.org Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=96346 Reported-by: Mark Janes <mark.a.janes@intel.com> Acked-by: Matt Turner <mattst88@gmail.com>
2016-06-03 12:32:15 -07:00
assert(src0.hstride == BRW_HORIZONTAL_STRIDE_1);
assert(src1.hstride == BRW_HORIZONTAL_STRIDE_1);
i965: Enable faster workaround-free math on Ivybridge. According to the documentation, Ivybridge's math instruction works in SIMD16 mode for the fragment shader, and no longer forbids align16 mode for the vertex shader. The documentation claims that SIMD16 mode isn't supported for INT DIV, but empirical evidence shows that it works fine. Presumably the note is trying to warn us that the variant that returns both quotient and remainder in (dst, dst + 1) doesn't work in SIMD16 mode since dst + 1 would be sechalf(dst), trashing half your results. Since we don't use that variant, we don't care and can just enable SIMD16 everywhere. The documentation also still claims that source modifiers and conditional modifiers aren't supported, but empirical evidence and study of the simulator both show that they work just fine. Goodbye workarounds. Math just works now. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-10-18 12:24:47 -07:00
}
i965: Add a couple of checks for gen6 math instruction limits.
2010-10-11 13:30:12 -07:00
i965: Fix assertions about register types for INT DIV in brw_math. BRW_MATH_FUNCTION_REMAINDER was missing. Also, it seems worthwhile to assert that INT DIV's arguments are signed/unsigned integers. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Tested-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-28 17:37:51 -07:00
if (function == BRW_MATH_FUNCTION_INT_DIV_QUOTIENT ||
function == BRW_MATH_FUNCTION_INT_DIV_REMAINDER ||
function == BRW_MATH_FUNCTION_INT_DIV_QUOTIENT_AND_REMAINDER) {
assert(src0.type != BRW_REGISTER_TYPE_F);
assert(src1.type != BRW_REGISTER_TYPE_F);
i965/eu: Allow math on immediates on Broadwell. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-07-11 15:54:11 -07:00
assert(src1.file == BRW_GENERAL_REGISTER_FILE ||
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
(devinfo->gen >= 8 && src1.file == BRW_IMMEDIATE_VALUE));
i965: Fix assertions about register types for INT DIV in brw_math. BRW_MATH_FUNCTION_REMAINDER was missing. Also, it seems worthwhile to assert that INT DIV's arguments are signed/unsigned integers. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Tested-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-28 17:37:51 -07:00
} else {
intel/compiler: handle extended math restrictions for half-float Extended math with half-float operands is only supported since gen9, but it is limited to SIMD8. In gen8 we lower it to 32-bit. v2: quashed together the following patches (Jason): - intel/compiler: allow extended math functions with HF operands - intel/compiler: lower 16-bit extended math to 32-bit prior to gen9 - intel/compiler: extended Math is limited to SIMD8 on half-float Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> (allow extended math functions with HF operands, extended Math is limited to SIMD8 on half-float)
2018-04-26 10:26:22 +02:00
assert(src0.type == BRW_REGISTER_TYPE_F ||
(src0.type == BRW_REGISTER_TYPE_HF && devinfo->gen >= 9));
assert(src1.type == BRW_REGISTER_TYPE_F ||
(src1.type == BRW_REGISTER_TYPE_HF && devinfo->gen >= 9));
i965: Add a couple of checks for gen6 math instruction limits.
2010-10-11 13:30:12 -07:00
}
i965: Enable faster workaround-free math on Ivybridge. According to the documentation, Ivybridge's math instruction works in SIMD16 mode for the fragment shader, and no longer forbids align16 mode for the vertex shader. The documentation claims that SIMD16 mode isn't supported for INT DIV, but empirical evidence shows that it works fine. Presumably the note is trying to warn us that the variant that returns both quotient and remainder in (dst, dst + 1) doesn't work in SIMD16 mode since dst + 1 would be sechalf(dst), trashing half your results. Since we don't use that variant, we don't care and can just enable SIMD16 everywhere. The documentation also still claims that source modifiers and conditional modifiers aren't supported, but empirical evidence and study of the simulator both show that they work just fine. Goodbye workarounds. Math just works now. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-10-18 12:24:47 -07:00
/* Source modifiers are ignored for extended math instructions on Gen6. */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen == 6) {
i965: Enable faster workaround-free math on Ivybridge. According to the documentation, Ivybridge's math instruction works in SIMD16 mode for the fragment shader, and no longer forbids align16 mode for the vertex shader. The documentation claims that SIMD16 mode isn't supported for INT DIV, but empirical evidence shows that it works fine. Presumably the note is trying to warn us that the variant that returns both quotient and remainder in (dst, dst + 1) doesn't work in SIMD16 mode since dst + 1 would be sechalf(dst), trashing half your results. Since we don't use that variant, we don't care and can just enable SIMD16 everywhere. The documentation also still claims that source modifiers and conditional modifiers aren't supported, but empirical evidence and study of the simulator both show that they work just fine. Goodbye workarounds. Math just works now. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-10-18 12:24:47 -07:00
assert(!src0.negate);
assert(!src0.abs);
assert(!src1.negate);
assert(!src1.abs);
}
i965: Work around gen6 ignoring source modifiers on math instructions. With the change of extended math from having the arguments moved into mrfs and handed off through message passing to being directly hooked up to the EU, it looks like the piece for doing source modifiers (negate and abs) was left out. Fixes: fog-modes glean/fp1-ARB_fog_exp test glean/fp1-ARB_fog_exp2 test glean/fp1-Computed fog exp test glean/fp1-Computed fog exp2 test ext_fog_coord-modes
2010-12-07 14:50:50 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_math_function(devinfo, insn, function);
i965: Add support for POW in gen6 FS. Fixes glsl-algebraic-pow-2 in brw_wm_glsl.c mode.
2010-08-22 01:33:57 -07:00
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, src1);
i965: Add support for POW in gen6 FS. Fixes glsl-algebraic-pow-2 in brw_wm_glsl.c mode.
2010-08-22 01:33:57 -07:00
}
i965/gen9+: Switch thread scratch space to non-coherent stateless access. The thread scratch space is thread-local so using the full IA-coherent stateless surface index (255 since Gen8) is unnecessary and potentially expensive. On Gen8 and early steppings of Gen9 this is not a functional change because the kernel already sets bit 4 of HDC_CHICKEN0 which overrides all HDC memory access to be non-coherent in order to workaround a hardware bug. This happens to fix a full system hang when running any spilling code on a pre-production SKL GT4e machine I have on my desk (forcing all HDC access to non-coherent from the kernel up to stepping F0 might be a good idea though regardless of this patch), and improves performance of the OglPSBump2 SynMark benchmark run with INTEL_DEBUG=spill_fs by 33% (11 runs, 5% significance) on a production SKL GT2 (on which HDC IA-coherency is apparently functional so it wouldn't make sense to disable globally). Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-11-23 19:18:26 +02:00
/**
* Return the right surface index to access the thread scratch space using
* stateless dataport messages.
*/
unsigned
brw_scratch_surface_idx(const struct brw_codegen *p)
{
/* The scratch space is thread-local so IA coherency is unnecessary. */
if (p->devinfo->gen >= 8)
return GEN8_BTI_STATELESS_NON_COHERENT;
else
return BRW_BTI_STATELESS;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: new and updated comments
2009-03-31 10:49:41 -06:00
/**
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
* Write a block of OWORDs (half a GRF each) from the scratch buffer,
* using a constant offset per channel.
*
* The offset must be aligned to oword size (16 bytes). Used for
* register spilling.
i965: new and updated comments
2009-03-31 10:49:41 -06:00
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_oword_block_write_scratch(struct brw_codegen *p,
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
struct brw_reg mrf,
int num_regs,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned offset)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Let the caller of brw_set_dp_write/read_message control the target cache. brw_set_dp_read_message already had a target_cache argument, but its interpretation was rather convoluted (on Gen6 the render cache was used if the caller asked for it, otherwise it was ignored using the sampler cache instead), and the constant cache wasn't representable at all. brw_set_dp_write_message used the data cache on Gen7+ except for RENDER_TARGET_WRITE messages, in which case it would use the render cache. On Gen6 the render cache was always used. Instead of the above, provide the shared unit SFID that the caller expects will be used. Makes no functional changes. v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:36:16 +03:00
const unsigned target_cache =
(devinfo->gen >= 7 ? GEN7_SFID_DATAPORT_DATA_CACHE :
devinfo->gen >= 6 ? GEN6_SFID_DATAPORT_RENDER_CACHE :
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
BRW_SFID_DATAPORT_WRITE);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
i965/fs: Implement scratch reads and writes of 4 GRFs at a time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 15:47:39 -07:00
uint32_t msg_type;
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6)
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
offset /= 16;
i965: Don't emit register spill offsets directly into g0. g0 is used by others, and is expected to be left exactly as it was dispatched to us. So manually move g0 into our message reg when spilling/unspilling and update the offset in the MRF. Fixes failures in texture sampling after having spilled a register.
2010-10-21 14:40:49 -07:00
mrf = retype(mrf, BRW_REGISTER_TYPE_UD);
i965/fs: Implement scratch reads and writes of 4 GRFs at a time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 15:47:39 -07:00
const unsigned mlen = 1 + num_regs;
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
i965: Don't emit register spill offsets directly into g0. g0 is used by others, and is expected to be left exactly as it was dispatched to us. So manually move g0 into our message reg when spilling/unspilling and update the offset in the MRF. Fixes failures in texture sampling after having spilled a register.
2010-10-21 14:40:49 -07:00
/* Set up the message header. This is g0, with g0.2 filled with
* the offset. We don't want to leave our offset around in g0 or
* it'll screw up texture samples, so set it up inside the message
* reg.
*/
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
brw_push_insn_state(p);
i965: Replace guess_execution_size with something simpler. guess_execution_size() does two things: 1. Cope with small destination registers. 2. Cope with SIMD8 vs SIMD16 mode. This patch replaces the first with a simple if block in brw_set_dest: if the destination register width is less than 8, you probably want the execution size to match. (I didn't put this in the 3src block because it doesn't seem to matter.) Since only the FS compiler cares about SIMD16 mode, it's easy to just set the default execution size there. This pattern was already been proven in the Gen8+ generator, but we didn't port it back to the existing generator when we combined the two. This is based on a patch from Ken from about a year ago. I've rebased it and and fixed a few bugs. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 12:40:34 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_8);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 16:57:02 -07:00
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Don't emit register spill offsets directly into g0. g0 is used by others, and is expected to be left exactly as it was dispatched to us. So manually move g0 into our message reg when spilling/unspilling and update the offset in the MRF. Fixes failures in texture sampling after having spilled a register.
2010-10-21 14:40:49 -07:00
brw_MOV(p, mrf, retype(brw_vec8_grf(0, 0), BRW_REGISTER_TYPE_UD));
i965: new and updated comments
2009-03-31 10:49:41 -06:00
/* set message header global offset field (reg 0, element 2) */
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_null());
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
brw_MOV(p,
i965: Don't emit register spill offsets directly into g0. g0 is used by others, and is expected to be left exactly as it was dispatched to us. So manually move g0 into our message reg when spilling/unspilling and update the offset in the MRF. Fixes failures in texture sampling after having spilled a register.
2010-10-21 14:40:49 -07:00
retype(brw_vec1_reg(BRW_MESSAGE_REGISTER_FILE,
mrf.nr,
2), BRW_REGISTER_TYPE_UD),
brw_imm_ud(offset));
i965: fix fetching constants from constant buffer in glsl path the driver used to overwrite grf0 then use implicit move by send instruction to move contents of grf0 to mrf1. However, we must not overwrite grf0 since it's still used later for fb write. Instead, do the move directly do mrf1 (we could use implicit move from another grf reg to mrf1 but since we need a mov to encode the data anyway it doesn't seem to make sense). I think the dp_READ/WRITE_16 functions may suffer from the same issue. While here also remove unnecessary msg_reg_nr parameter from the dataport functions since always message register 1 is used.
2009-06-26 20:38:07 +02:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
brw_pop_insn_state(p);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
{
i965: Set the send commit bit on register spills as required pre-gen6. Otherwise, the subsequent read may not get the written value.
2010-07-21 13:07:12 -07:00
struct brw_reg dest;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_SEND);
i965: Set the send commit bit on register spills as required pre-gen6. Otherwise, the subsequent read may not get the written value.
2010-07-21 13:07:12 -07:00
int send_commit_msg;
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
struct brw_reg src_header = retype(brw_vec8_grf(0, 0),
BRW_REGISTER_TYPE_UW);
i965: Set the send commit bit on register spills as required pre-gen6. Otherwise, the subsequent read may not get the written value.
2010-07-21 13:07:12 -07:00
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
brw_inst_set_sfid(devinfo, insn, target_cache);
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 15:29:27 -07:00
brw_inst_set_compression(devinfo, insn, false);
if (brw_inst_exec_size(devinfo, insn) >= 16)
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
src_header = vec16(src_header);
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 15:29:27 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_pred_control(devinfo, insn) == BRW_PREDICATE_NONE);
if (devinfo->gen < 6)
brw_inst_set_base_mrf(devinfo, insn, mrf.nr);
i965: Set the send commit bit on register spills as required pre-gen6. Otherwise, the subsequent read may not get the written value.
2010-07-21 13:07:12 -07:00
/* Until gen6, writes followed by reads from the same location
* are not guaranteed to be ordered unless write_commit is set.
* If set, then a no-op write is issued to the destination
* register to set a dependency, and a read from the destination
* can be used to ensure the ordering.
*
* For gen6, only writes between different threads need ordering
* protection. Our use of DP writes is all about register
* spilling within a thread.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i956: Set the execution size correctly for scratch space writes. Otherwise, the second half isn't written, and we end up reading back black. Fixes the remaining junk drawn in glsl-max-varyings, and will likely help with a number of large real-world shaders.
2010-07-21 15:54:53 -07:00
dest = retype(vec16(brw_null_reg()), BRW_REGISTER_TYPE_UW);
i965: Set the send commit bit on register spills as required pre-gen6. Otherwise, the subsequent read may not get the written value.
2010-07-21 13:07:12 -07:00
send_commit_msg = 0;
} else {
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
dest = src_header;
i965: Set the send commit bit on register spills as required pre-gen6. Otherwise, the subsequent read may not get the written value.
2010-07-21 13:07:12 -07:00
send_commit_msg = 1;
}
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, mrf);
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
} else {
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_null_reg());
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6)
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
msg_type = GEN6_DATAPORT_WRITE_MESSAGE_OWORD_BLOCK_WRITE;
else
msg_type = BRW_DATAPORT_WRITE_MESSAGE_OWORD_BLOCK_WRITE;
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
brw_set_desc(p, insn,
brw_message_desc(devinfo, mlen, send_commit_msg, true) |
brw_dp_write_desc(devinfo, brw_scratch_surface_idx(p),
BRW_DATAPORT_OWORD_BLOCK_DWORDS(num_regs * 8),
msg_type, 0, /* not a render target */
send_commit_msg));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
}
i965: new and updated comments
2009-03-31 10:49:41 -06:00
/**
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
* Read a block of owords (half a GRF each) from the scratch buffer
* using a constant index per channel.
*
* Offset must be aligned to oword size (16 bytes). Used for register
* spilling.
i965: new and updated comments
2009-03-31 10:49:41 -06:00
*/
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_oword_block_read_scratch(struct brw_codegen *p,
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
struct brw_reg dest,
struct brw_reg mrf,
int num_regs,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned offset)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6)
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
offset /= 16;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (p->devinfo->gen >= 7) {
i965: Use the spill destination for the message header on GEN >= 7 Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2014-10-24 12:22:04 -07:00
/* On gen 7 and above, we no longer have message registers and we can
* send from any register we want. By using the destination register
* for the message, we guarantee that the implied message write won't
* accidentally overwrite anything. This has been a problem because
* the MRF registers and source for the final FB write are both fixed
* and may overlap.
*/
mrf = retype(dest, BRW_REGISTER_TYPE_UD);
} else {
mrf = retype(mrf, BRW_REGISTER_TYPE_UD);
}
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
dest = retype(dest, BRW_REGISTER_TYPE_UW);
i965/fs: Implement scratch reads and writes of 4 GRFs at a time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 15:47:39 -07:00
const unsigned rlen = num_regs;
i965: Let the caller of brw_set_dp_write/read_message control the target cache. brw_set_dp_read_message already had a target_cache argument, but its interpretation was rather convoluted (on Gen6 the render cache was used if the caller asked for it, otherwise it was ignored using the sampler cache instead), and the constant cache wasn't representable at all. brw_set_dp_write_message used the data cache on Gen7+ except for RENDER_TARGET_WRITE messages, in which case it would use the render cache. On Gen6 the render cache was always used. Instead of the above, provide the shared unit SFID that the caller expects will be used. Makes no functional changes. v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:36:16 +03:00
const unsigned target_cache =
(devinfo->gen >= 7 ? GEN7_SFID_DATAPORT_DATA_CACHE :
devinfo->gen >= 6 ? GEN6_SFID_DATAPORT_RENDER_CACHE :
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
BRW_SFID_DATAPORT_READ);
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
brw_push_insn_state(p);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
i965: Replace guess_execution_size with something simpler. guess_execution_size() does two things: 1. Cope with small destination registers. 2. Cope with SIMD8 vs SIMD16 mode. This patch replaces the first with a simple if block in brw_set_dest: if the destination register width is less than 8, you probably want the execution size to match. (I didn't put this in the 3src block because it doesn't seem to matter.) Since only the FS compiler cares about SIMD16 mode, it's easy to just set the default execution size there. This pattern was already been proven in the Gen8+ generator, but we didn't port it back to the existing generator when we combined the two. This is based on a patch from Ken from about a year ago. I've rebased it and and fixed a few bugs. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 12:40:34 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_8);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 16:57:02 -07:00
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Don't emit register spill offsets directly into g0. g0 is used by others, and is expected to be left exactly as it was dispatched to us. So manually move g0 into our message reg when spilling/unspilling and update the offset in the MRF. Fixes failures in texture sampling after having spilled a register.
2010-10-21 14:40:49 -07:00
brw_MOV(p, mrf, retype(brw_vec8_grf(0, 0), BRW_REGISTER_TYPE_UD));
i965: new and updated comments
2009-03-31 10:49:41 -06:00
/* set message header global offset field (reg 0, element 2) */
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_null());
i965: Use the spill destination for the message header on GEN >= 7 Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2014-10-24 12:22:04 -07:00
brw_MOV(p, get_element_ud(mrf, 2), brw_imm_ud(offset));
i965: fix fetching constants from constant buffer in glsl path the driver used to overwrite grf0 then use implicit move by send instruction to move contents of grf0 to mrf1. However, we must not overwrite grf0 since it's still used later for fb write. Instead, do the move directly do mrf1 (we could use implicit move from another grf reg to mrf1 but since we need a mov to encode the data anyway it doesn't seem to make sense). I think the dp_READ/WRITE_16 functions may suffer from the same issue. While here also remove unnecessary msg_reg_nr parameter from the dataport functions since always message register 1 is used.
2009-06-26 20:38:07 +02:00
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
brw_pop_insn_state(p);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
{
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_SEND);
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
brw_inst_set_sfid(devinfo, insn, target_cache);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_pred_control(devinfo, insn) == 0);
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 15:29:27 -07:00
brw_inst_set_compression(devinfo, insn, false);
i965: Add support for register spilling. It can be tested with if (0) replaced with if (1) to force spilling for all virtual GRFs. Some simple tests work, but large texturing tests fail.
2010-10-19 09:25:51 -07:00
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest); /* UW? */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, mrf);
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
} else {
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_null_reg());
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_base_mrf(devinfo, insn, mrf.nr);
i965/fs: Add gen6 register spilling support. Most of this is code movement to get the scratch space allocated in a shared location. Other than that, the only real changes are that the old oword block messages now operate on oword-aligned areas (with new messages for unaligned access, which we don't do), and that the caching control is in the SFID part of the descriptor instead of message control. Fixes glsl-fs-convolution-1.
2011-04-14 19:36:28 -07:00
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
brw_set_desc(p, insn,
brw_message_desc(devinfo, 1, rlen, true) |
brw_dp_read_desc(devinfo, brw_scratch_surface_idx(p),
BRW_DATAPORT_OWORD_BLOCK_DWORDS(num_regs * 8),
BRW_DATAPORT_READ_MESSAGE_OWORD_BLOCK_READ,
BRW_DATAPORT_READ_TARGET_RENDER_CACHE));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
}
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:51:22 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
gen7_block_read_scratch(struct brw_codegen *p,
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:51:22 -07:00
struct brw_reg dest,
int num_regs,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned offset)
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:51:22 -07:00
{
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_SEND);
i965: fix unused variable warning in gen7_block_read_scratch() Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-10-04 11:15:05 +11:00
assert(brw_inst_pred_control(p->devinfo, insn) == BRW_PREDICATE_NONE);
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:51:22 -07:00
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
brw_set_dest(p, insn, retype(dest, BRW_REGISTER_TYPE_UW));
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:51:22 -07:00
/* The HW requires that the header is present; this is to get the g0.5
* scratch offset.
*/
brw_set_src0(p, insn, brw_vec8_grf(0, 0));
/* According to the docs, offset is "A 12-bit HWord offset into the memory
* Immediate Memory buffer as specified by binding table 0xFF." An HWORD
* is 32 bytes, which happens to be the size of a register.
*/
offset /= REG_SIZE;
assert(offset < (1 << 12));
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
gen7_set_dp_scratch_message(p, insn,
false, /* scratch read */
false, /* OWords */
false, /* invalidate after read */
num_regs,
offset,
1, /* mlen: just g0 */
num_regs, /* rlen */
true); /* header present */
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 11:51:22 -07:00
}
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
/**
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-08 20:05:18 -08:00
* Read float[4] vectors from the data port constant cache.
i965: checkpoint commit: VS constant buffers Hook up a constant buffer, binding table, etc for the VS unit. This will allow using large constant buffers with vertex shaders. The new code is disabled at this time (use_const_buffer=FALSE).
2009-04-14 11:08:42 -06:00
* Location (in buffer) should be a multiple of 16.
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
* Used for fetching shader constants.
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_oword_block_read(struct brw_codegen *p,
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
struct brw_reg dest,
struct brw_reg mrf,
uint32_t offset,
uint32_t bind_table_index)
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Let the caller of brw_set_dp_write/read_message control the target cache. brw_set_dp_read_message already had a target_cache argument, but its interpretation was rather convoluted (on Gen6 the render cache was used if the caller asked for it, otherwise it was ignored using the sampler cache instead), and the constant cache wasn't representable at all. brw_set_dp_write_message used the data cache on Gen7+ except for RENDER_TARGET_WRITE messages, in which case it would use the render cache. On Gen6 the render cache was always used. Instead of the above, provide the shared unit SFID that the caller expects will be used. Makes no functional changes. v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:36:16 +03:00
const unsigned target_cache =
i965/fs: Switch to the constant cache for uniform pull constants. This reverts to using the oword block read messages for uniform pull constant loads, as used to be the case until 4c1fdae0a01b3f92ec03b61aac1d3df5. There are two important differences though: Now the L3 cacheability bits are set up correctly for UBOs (since 11f5d8a5d4fbb861ec161f68593e429cbd65d1cd), and we target the constant cache instead of the data cache. The latter used to get no L3 way allocation on boot on all platforms that existed at the time, so oword read messages wouldn't get cached on L3 regardless of the MOCS bits, what probably explains the apparent slowness of oword fetches. Constant cache loads seem to perform better than SIMD4x2 sampler loads in a number of cases, they alleviate some of the cache thrashing caused by the competition with textures for the L1/L2 sampler caches, and they allow fetching up to 128B worth of constants with a single oword fetch message. Note that IVB devices suffer from a hardware bug that leads to serialization of L3 read requests overlapping the same cacheline as result of a (on IVB buggy) mechanism of the L3 to preserve coherency. Since read requests for matching cachelines from any L3 client are not pipelined, throughput may decrease in cases where there are no non-overlapping requests left in the queue that can be processed between them. This situation should be relatively uncommon as long as we make sure that we don't use the 1/2 oword messages in cases where the shader intends to read from any other location of the same cacheline at some other point. This is generally a good idea anyway on all generations because using the 1 and 2 oword messages is expected to waste bandwidth since the minimum L3 request size for the DC is exactly 4 owords (i.e. one cacheline). A future commit will have this effect. I haven't been able to find any real-world example where this would still result in a regression on IVB, but if someone happens to find one it shouldn't be too difficult to add an IVB-specific check to have it fall back to the sampler cache for pull constant loads. Note that on SKL+ this change has the additional benefit of reducing the register footprint of pull constant loads. The following table summarizes the effect of the whole series on several shader-db stats: Total instructions Total cycles BWR: 4571248 -> 4568342 (-0.06%) 123375740 -> 123373296 (-0.00%) ELK: 3989020 -> 3985402 (-0.09%) 98757068 -> 98754058 (-0.00%) ILK: 6383591 -> 6376787 (-0.11%) 143649910 -> 143648914 (-0.00%) SNB: 7528395 -> 7501446 (-0.36%) 103503796 -> 102460370 (-1.01%) IVB: 6949221 -> 6943317 (-0.08%) 60592262 -> 60584422 (-0.01%) HSW: 6409753 -> 6403702 (-0.09%) 60609070 -> 60604414 (-0.01%) BDW: 8043467 -> 7976364 (-0.83%) 68427730 -> 68483042 (0.08%) CHV: 8045019 -> 7977916 (-0.83%) 68297426 -> 68352756 (0.08%) SKL: 8204037 -> 7939086 (-3.23%) 66583900 -> 65624378 (-1.44%) Lost->Gained Total spills Total fills BWR: 5 -> 5 1488 -> 1488 (0.00%) 1957 -> 1957 (0.00%) ELK: 5 -> 5 1489 -> 1489 (0.00%) 1958 -> 1958 (0.00%) ILK: 1 -> 4 1449 -> 1449 (0.00%) 1921 -> 1921 (0.00%) SNB: 0 -> 0 549 -> 549 (0.00%) 52 -> 52 (0.00%) IVB: 13 -> 3 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) HSW: 11 -> 0 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) BDW: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) CHV: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) SKL: 0 -> 120 1269 -> 375 (-70.45%) 1563 -> 690 (-55.85%) v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-10-26 14:25:06 -07:00
(devinfo->gen >= 6 ? GEN6_SFID_DATAPORT_CONSTANT_CACHE :
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
BRW_SFID_DATAPORT_READ);
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
const unsigned exec_size = 1 << brw_get_default_exec_size(p);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
i965: Add support for constant buffer loads on gen6. Fixes glsl-fs-uniform-array-5.
2010-10-26 13:17:54 -07:00
/* On newer hardware, offset is in units of owords. */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6)
i965: Add support for constant buffer loads on gen6. Fixes glsl-fs-uniform-array-5.
2010-10-26 13:17:54 -07:00
offset /= 16;
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
mrf = retype(mrf, BRW_REGISTER_TYPE_UD);
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
brw_push_insn_state(p);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 16:57:02 -07:00
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-08 20:05:18 -08:00
brw_push_insn_state(p);
brw_set_default_exec_size(p, BRW_EXECUTE_8);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
brw_MOV(p, mrf, retype(brw_vec8_grf(0, 0), BRW_REGISTER_TYPE_UD));
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
/* set message header global offset field (reg 0, element 2) */
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_null());
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
brw_MOV(p,
retype(brw_vec1_reg(BRW_MESSAGE_REGISTER_FILE,
mrf.nr,
2), BRW_REGISTER_TYPE_UD),
brw_imm_ud(offset));
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-08 20:05:18 -08:00
brw_pop_insn_state(p);
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_SEND);
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
brw_inst_set_sfid(devinfo, insn, target_cache);
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
/* cast dest to a uword[8] vector */
dest = retype(vec8(dest), BRW_REGISTER_TYPE_UW);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, mrf);
i965: Add support for constant buffer loads on gen6. Fixes glsl-fs-uniform-array-5.
2010-10-26 13:17:54 -07:00
} else {
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, brw_null_reg());
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_base_mrf(devinfo, insn, mrf.nr);
i965: Add support for constant buffer loads on gen6. Fixes glsl-fs-uniform-array-5.
2010-10-26 13:17:54 -07:00
}
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
brw_set_desc(p, insn,
brw_message_desc(devinfo, 1, DIV_ROUND_UP(exec_size, 8), true) |
brw_dp_read_desc(devinfo, bind_table_index,
BRW_DATAPORT_OWORD_BLOCK_DWORDS(exec_size),
BRW_DATAPORT_READ_MESSAGE_OWORD_BLOCK_READ,
BRW_DATAPORT_READ_TARGET_DATA_CACHE));
i965: Add support for pull constants to the new FS backend. Fixes glsl-fs-uniform-array-5, but not 6 which fails in ir_to_mesa.
2010-10-22 12:57:00 -07:00
brw_pop_insn_state(p);
i965: added new brw_dp_READ_4() function Used to read float[4] vectors from the constant buffer/surface.
2009-03-31 10:50:55 -06:00
}
intel/eu: Return new instruction to caller from brw_fb_WRITE(). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 14:16:12 -08:00
brw_inst *
brw_fb_WRITE(struct brw_codegen *p,
struct brw_reg payload,
struct brw_reg implied_header,
unsigned msg_control,
unsigned binding_table_index,
unsigned msg_length,
unsigned response_length,
bool eot,
bool last_render_target,
bool header_present)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Let the caller of brw_set_dp_write/read_message control the target cache. brw_set_dp_read_message already had a target_cache argument, but its interpretation was rather convoluted (on Gen6 the render cache was used if the caller asked for it, otherwise it was ignored using the sampler cache instead), and the constant cache wasn't representable at all. brw_set_dp_write_message used the data cache on Gen7+ except for RENDER_TARGET_WRITE messages, in which case it would use the render cache. On Gen6 the render cache was always used. Instead of the above, provide the shared unit SFID that the caller expects will be used. Makes no functional changes. v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:36:16 +03:00
const unsigned target_cache =
(devinfo->gen >= 6 ? GEN6_SFID_DATAPORT_RENDER_CACHE :
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
BRW_SFID_DATAPORT_WRITE);
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_type;
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 15:16:20 -07:00
struct brw_reg dest, src0;
i965: Move the destination reg setup for 8/16 wide to the emit code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-03-14 10:51:19 -07:00
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
if (brw_get_default_exec_size(p) >= BRW_EXECUTE_16)
i965: Move the destination reg setup for 8/16 wide to the emit code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-03-14 10:51:19 -07:00
dest = retype(vec16(brw_null_reg()), BRW_REGISTER_TYPE_UW);
else
dest = retype(vec8(brw_null_reg()), BRW_REGISTER_TYPE_UW);
i965: Add support for FB writes on Sandybridge.
2010-08-20 15:02:19 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i965: Use SENDC on the first render target write on gen6. This is apparently required, as the thread will be initiated while it still has dependencies, and this is what waits for those to be resolved before writing color.
2010-10-26 09:35:34 -07:00
insn = next_insn(p, BRW_OPCODE_SENDC);
} else {
insn = next_insn(p, BRW_OPCODE_SEND);
}
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
brw_inst_set_sfid(devinfo, insn, target_cache);
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 15:29:27 -07:00
brw_inst_set_compression(devinfo, insn, false);
i965: Add support for FB writes on Sandybridge.
2010-08-20 15:02:19 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 6) {
i965: Fix inconsistent indentation in brw_eu_emit.c. Most of these functions used three spaces for the first level of indentation, but four spaces for the next level. One used tabs and then three spaces. Some used 3/4 in a then block but 3/3 in the else block. Normally I try to avoid field days like this, but since the functions were so inconsistent, even internally, it was making it difficult to edit without introducing spurious whitespace changes. So, just get it over with. git diff -b shows 0 lines changed. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2011-10-07 11:59:06 -07:00
/* headerless version, just submit color payload */
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 15:16:20 -07:00
src0 = payload;
i965: Add support for FB writes on Sandybridge.
2010-08-20 15:02:19 -07:00
i965: Fix inconsistent indentation in brw_eu_emit.c. Most of these functions used three spaces for the first level of indentation, but four spaces for the next level. One used tabs and then three spaces. Some used 3/4 in a then block but 3/3 in the else block. Normally I try to avoid field days like this, but since the functions were so inconsistent, even internally, it was making it difficult to edit without introducing spurious whitespace changes. So, just get it over with. git diff -b shows 0 lines changed. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2011-10-07 11:59:06 -07:00
msg_type = GEN6_DATAPORT_WRITE_MESSAGE_RENDER_TARGET_WRITE;
i965: Add support for FB writes on Sandybridge.
2010-08-20 15:02:19 -07:00
} else {
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 15:16:20 -07:00
assert(payload.file == BRW_MESSAGE_REGISTER_FILE);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_base_mrf(devinfo, insn, payload.nr);
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 15:16:20 -07:00
src0 = implied_header;
i965: Add support for FB writes on Sandybridge.
2010-08-20 15:02:19 -07:00
i965: Also use the SIMD8 FB writes for SIMD8 mode on non-SNB.
2010-08-20 15:32:17 -07:00
msg_type = BRW_DATAPORT_WRITE_MESSAGE_RENDER_TARGET_WRITE;
i965: Add support for FB writes on Sandybridge.
2010-08-20 15:02:19 -07:00
}
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
brw_set_desc(p, insn,
brw_message_desc(devinfo, msg_length, response_length,
header_present) |
brw_dp_write_desc(devinfo, binding_table_index, msg_control,
msg_type, last_render_target,
0 /* send_commit_msg */));
brw_inst_set_eot(devinfo, insn, eot);
intel/eu: Return new instruction to caller from brw_fb_WRITE(). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 14:16:12 -08:00
return insn;
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965/eu: Add codegen support for the Gen9+ render target read message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-21 19:13:55 -07:00
brw_inst *
gen9_fb_READ(struct brw_codegen *p,
struct brw_reg dst,
struct brw_reg payload,
unsigned binding_table_index,
unsigned msg_length,
unsigned response_length,
bool per_sample)
{
intel: Rename brw_get_device_name/info to gen_get_device_name/info Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-25 16:22:58 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965/eu: Add codegen support for the Gen9+ render target read message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-21 19:13:55 -07:00
assert(devinfo->gen >= 9);
const unsigned msg_subtype =
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_get_default_exec_size(p) == BRW_EXECUTE_16 ? 0 : 1;
i965/eu: Add codegen support for the Gen9+ render target read message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-21 19:13:55 -07:00
brw_inst *insn = next_insn(p, BRW_OPCODE_SENDC);
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
brw_inst_set_sfid(devinfo, insn, GEN6_SFID_DATAPORT_RENDER_CACHE);
i965/eu: Add codegen support for the Gen9+ render target read message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-21 19:13:55 -07:00
brw_set_dest(p, insn, dst);
brw_set_src0(p, insn, payload);
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 10:50:20 -07:00
brw_set_desc(
p, insn,
brw_message_desc(devinfo, msg_length, response_length, true) |
brw_dp_read_desc(devinfo, binding_table_index,
per_sample << 5 | msg_subtype,
GEN9_DATAPORT_RC_RENDER_TARGET_READ,
BRW_DATAPORT_READ_TARGET_RENDER_CACHE));
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
brw_inst_set_rt_slot_group(devinfo, insn, brw_get_default_group(p) / 16);
i965/eu: Add codegen support for the Gen9+ render target read message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-21 19:13:55 -07:00
return insn;
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: comments for brw_SAMPLE()
2009-03-24 18:05:53 -06:00
/**
* Texture sample instruction.
* Note: the msg_type plus msg_length values determine exactly what kind
* of sampling operation is performed. See volume 4, page 161 of docs.
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_SAMPLE(struct brw_codegen *p,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_reg_nr,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg src0,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned binding_table_index,
unsigned sampler,
unsigned msg_type,
unsigned response_length,
unsigned msg_length,
unsigned header_present,
unsigned simd_mode,
unsigned return_format)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Fix sampler on sandybridge Sandybridge has not much change on texture sampler with Ironlake.
2010-09-17 14:17:06 +08:00
i965/fs: Convert gen7 to using GRFs for texture messages. Looking at Lightsmark's shaders, the way we used MRFs (or in gen7's case, GRFs) was bad in a couple of ways. One was that it prevented compute-to-MRF for the common case of a texcoord that gets used exactly once, but where the texcoord setup all gets emitted before the texture calls (such as when it's a bare fragment shader input, which gets interpolated before processing main()). Another was that it introduced a bunch of dependencies that constrained scheduling, and forced waits for texture operations to be done before they are required. For example, we can now move the compute-to-MRF interpolation for the second texture send down after the first send. The downside is that this generally prevents remove_duplicate_mrf_writes() from doing anything, whereas previously it avoided work for the case of sampling from the same texcoord twice. However, I suspect that most of the win that originally justified that code was in avoiding the WAR stall on the first send, which this patch also avoids, rather than the small cost of the extra instruction. We see instruction count regressions in shaders in unigine, yofrankie, savage2, hon, and gstreamer. Improves GLB2.7 performance by 0.633628% +/- 0.491809% (n=121/125, avg of ~66fps, outliers below 61 dropped). Improves openarena performance by 1.01092% +/- 0.66897% (n=425). No significant difference on Lightsmark (n=44). v2: Squash in the fix for register unspilling for send-from-GRF, fixing a segfault in lightsmark. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Matt Turner <mattst88@gmail.com>
2013-10-09 17:17:59 -07:00
if (msg_reg_nr != -1)
gen6_resolve_implied_move(p, &src0, msg_reg_nr);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Remove writemask support from brw_SAMPLE(). The code was rather broken for non-XYZW on 8-wide, but all of our callers were using XYZW anyway. For my experiments with using writemask on texturing, I've been using manual header setup in the compiler backends, since we want to actually know what registers are written for optimization and register allocation. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-02-05 16:21:07 -08:00
insn = next_insn(p, BRW_OPCODE_SEND);
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:15:15 -07:00
brw_inst_set_sfid(devinfo, insn, BRW_SFID_SAMPLER);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_pred_control(devinfo, insn, BRW_PREDICATE_NONE); /* XXX */
i965: make BRW_COMPRESSION_2NDHALF valid for brw_SAMPLE SIMD8 sampler messages are allowed in SIMD16 mode, and they could not work without BRW_COMPRESSION_2NDHALF. Later PRMs (gen5 and later) do not explicitly state whether BRW_COMPRESSION_2NDHALF is allowed, but they do have examples using send with SecHalf. It should be safe to assume SecHalf is valid. Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-27 15:23:56 +08:00
/* From the 965 PRM (volume 4, part 1, section 14.2.41):
*
* "Instruction compression is not allowed for this instruction (that
* is, send). The hardware behavior is undefined if this instruction is
* set as compressed. However, compress control can be set to "SecHalf"
* to affect the EMask generation."
*
* No similar wording is found in later PRMs, but there are examples
* utilizing send with SecHalf. More importantly, SIMD8 sampler messages
* are allowed in SIMD16 mode and they could not work without SecHalf. For
* these reasons, we allow BRW_COMPRESSION_2NDHALF here.
*/
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 15:29:27 -07:00
brw_inst_set_compression(devinfo, insn, false);
i965: make BRW_COMPRESSION_2NDHALF valid for brw_SAMPLE SIMD8 sampler messages are allowed in SIMD16 mode, and they could not work without BRW_COMPRESSION_2NDHALF. Later PRMs (gen5 and later) do not explicitly state whether BRW_COMPRESSION_2NDHALF is allowed, but they do have examples using send with SecHalf. It should be safe to assume SecHalf is valid. Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-27 15:23:56 +08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6)
brw_inst_set_base_mrf(devinfo, insn, msg_reg_nr);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Remove writemask support from brw_SAMPLE(). The code was rather broken for non-XYZW on 8-wide, but all of our callers were using XYZW anyway. For my experiments with using writemask on texturing, I've been using manual header setup in the compiler backends, since we want to actually know what registers are written for optimization and register allocation. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-02-05 16:21:07 -08:00
brw_set_dest(p, insn, dest);
brw_set_src0(p, insn, src0);
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:15:15 -07:00
brw_set_desc(p, insn,
brw_message_desc(devinfo, msg_length, response_length,
header_present) |
brw_sampler_desc(devinfo, binding_table_index, sampler,
msg_type, simd_mode, return_format));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
}
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
/* Adjust the message header's sampler state pointer to
* select the correct group of 16 samplers.
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_adjust_sampler_state_pointer(struct brw_codegen *p,
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
struct brw_reg header,
i965/emit: Do the sampler index adjustment directly in header.0.3 Prior to this commit, the adjust_sampler_state_pointer function took an extra register that it could use as scratch space. The usual candidate was the destination of the sampler instruction. However, if that register ever aliased anything important such as the sampler index, this would scratch all over important data. Fortunately, the calculation is such that we can just do it in place and we don't need the scratch space at all. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-01-22 13:46:44 -08:00
struct brw_reg sampler_index)
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
{
/* The "Sampler Index" field can only store values between 0 and 15.
* However, we can add an offset to the "Sampler State Pointer"
* field, effectively selecting a different set of 16 samplers.
*
* The "Sampler State Pointer" needs to be aligned to a 32-byte
* offset, and each sampler state is only 16-bytes, so we can't
* exclusively use the offset - we have to use both.
*/
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
if (sampler_index.file == BRW_IMMEDIATE_VALUE) {
const int sampler_state_size = 16; /* 16 bytes */
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-22 19:41:30 -07:00
uint32_t sampler = sampler_index.ud;
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
if (sampler >= 16) {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->is_haswell || devinfo->gen >= 8);
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
brw_ADD(p,
get_element_ud(header, 3),
get_element_ud(brw_vec8_grf(0, 0), 3),
brw_imm_ud(16 * (sampler / 16) * sampler_state_size));
}
} else {
i965: Generalize sampler state pointer mangling for non-const For now, assume that the addressed sampler can be in any of the 16-sampler banks. If we preserved range information this far, we could avoid emitting these instructions if the sampler were known to be contained within one bank. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:39:44 +12:00
/* Non-const sampler array indexing case */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 8 && !devinfo->is_haswell) {
i965: Generalize sampler state pointer mangling for non-const For now, assume that the addressed sampler can be in any of the 16-sampler banks. If we preserved range information this far, we could avoid emitting these instructions if the sampler were known to be contained within one bank. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:39:44 +12:00
return;
}
i965/emit: Do the sampler index adjustment directly in header.0.3 Prior to this commit, the adjust_sampler_state_pointer function took an extra register that it could use as scratch space. The usual candidate was the destination of the sampler instruction. However, if that register ever aliased anything important such as the sampler index, this would scratch all over important data. Fortunately, the calculation is such that we can just do it in place and we don't need the scratch space at all. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-01-22 13:46:44 -08:00
struct brw_reg temp = get_element_ud(header, 3);
i965: Generalize sampler state pointer mangling for non-const For now, assume that the addressed sampler can be in any of the 16-sampler banks. If we preserved range information this far, we could avoid emitting these instructions if the sampler were known to be contained within one bank. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:39:44 +12:00
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_push_insn_state(p);
i965: Fix sampler state pointer adjustment for nonconst samplers This started hitting an assertion recently. Only affects Haswell (Ivybridge doesn't support this meddling with the sampler state pointer, and ARB_gpu_shader5 is not enabled yet on Broadwell) 14 Piglits crash->pass. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-05 06:41:13 +13:00
brw_AND(p, temp, get_element_ud(sampler_index, 0), brw_imm_ud(0x0f0));
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_regdist(1));
i965: Generalize sampler state pointer mangling for non-const For now, assume that the addressed sampler can be in any of the 16-sampler banks. If we preserved range information this far, we could avoid emitting these instructions if the sampler were known to be contained within one bank. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:39:44 +12:00
brw_SHL(p, temp, temp, brw_imm_ud(4));
brw_ADD(p,
get_element_ud(header, 3),
get_element_ud(brw_vec8_grf(0, 0), 3),
temp);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_pop_insn_state(p);
i965: Extract helper function for surface state pointer adjustment Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 11:49:31 +12:00
}
}
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
/* All these variables are pretty confusing - we might be better off
* using bitmasks and macros for this, in the old style. Or perhaps
* just having the caller instantiate the fields in dword3 itself.
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_urb_WRITE(struct brw_codegen *p,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_reg_nr,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
struct brw_reg src0,
i965: Allow C++ type safety in the use of enum brw_urb_write_flags. (From a suggestion by Francisco Jerez) If an enum represents a bitfield of flags, e.g.: enum E { A = 1, B = 2, C = 4, D = 8, }; then C++ normally prohibits statements like this: enum E x = A | B; because A and B are implicitly converted to ints before OR-ing them, and an int can't be stored in an enum without a type cast. C, on the other hand, allows an int to be implicitly converted to an enum without casting. In the past we've dealt with this situation by storing flag bitfields as ints. This avoids ugly casting at the expense of some type safety that C++ would normally have offered (e.g. we get no warning if we accidentally use the wrong enum type). However, we can get the best of both worlds if we override the | operator. The ugly casting is confined to the operator overload, and we still get the benefit of C++ making sure we don't use the wrong enum type. v2: Remove unnecessary comment and unnecessary use of "enum" keyword. Use static_cast. Reviewed-by: Chad Versace <chad.versace@linux.intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2013-08-23 13:19:19 -07:00
enum brw_urb_write_flags flags,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_length,
unsigned response_length,
unsigned offset,
unsigned swizzle)
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: Try to hook up the Sandybridge URB_WRITE SEND message. My units still hang when doing this if the VS is enabled.
2010-02-25 18:35:08 -08:00
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
gen6_resolve_implied_move(p, &src0, msg_reg_nr);
i965: Try to hook up the Sandybridge URB_WRITE SEND message. My units still hang when doing this if the VS is enabled.
2010-02-25 18:35:08 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 7 && !(flags & BRW_URB_WRITE_USE_CHANNEL_MASKS)) {
i965: Enable channel masks in Ivybridge's URB_WRITE_HWORD header. This shouldn't be done using MRFs, but until I have a proper solution for dealing with MRFs, this allows my hack to keep working. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-26 17:24:38 -07:00
/* Enable Channel Masks in the URB_WRITE_HWORD message header */
i965/gen7: Use align1 mode to set URB_WRITE_HWORD channel enables. Makes the new vertex shader backend work on Ivybridge. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-18 02:15:56 -07:00
brw_push_insn_state(p);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 16:57:02 -07:00
brw_set_default_access_mode(p, BRW_ALIGN_1);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
Revert "i965/vec4: Use MOV, not OR, to set URB write channel mask bits." This reverts commit af13cf609f4257768ad8b80be8cec7f2e6ca8c81, which appears to cause huge performance problems on Ivybridge. I'd missed that the FFTID bits are in the low byte. The documentation doesn't indicate that the URB write message header actually wants FFTID - it just labels those bits as "Reserved." But it appears necessary. This does slightly more than revert the original change: originally, Broadwell had separate code generation, which used MOV, and this patch only changed it for Gen4-7. Now that both are unified, reverting this also makes Broadwell use OR. Which should be fine. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-14 12:22:16 -07:00
brw_OR(p, retype(brw_vec1_reg(BRW_MESSAGE_REGISTER_FILE, msg_reg_nr, 5),
BRW_REGISTER_TYPE_UD),
retype(brw_vec1_grf(0, 5), BRW_REGISTER_TYPE_UD),
brw_imm_ud(0xff00));
i965/gen7: Use align1 mode to set URB_WRITE_HWORD channel enables. Makes the new vertex shader backend work on Ivybridge. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-18 02:15:56 -07:00
brw_pop_insn_state(p);
i965: Enable channel masks in Ivybridge's URB_WRITE_HWORD header. This shouldn't be done using MRFs, but until I have a proper solution for dealing with MRFs, this allows my hack to keep working. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-26 17:24:38 -07:00
}
i965: Try to hook up the Sandybridge URB_WRITE SEND message. My units still hang when doing this if the VS is enabled.
2010-02-25 18:35:08 -08:00
insn = next_insn(p, BRW_OPCODE_SEND);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Turn BRW_MAX_MRF into a macro that accepts a hardware generation There are some bug reports about shaders failing to compile in gen6 because MRF 14 is used when we need to spill. For example: https://bugs.freedesktop.org/show_bug.cgi?id=86469 https://bugs.freedesktop.org/show_bug.cgi?id=90631 Discussion in bugzilla pointed to the fact that gen6 might actually have 24 MRF registers available instead of 16, so we could use other MRF registers and avoid these conflicts (we still need to investigate why some shaders need up to MRF 14 anyway, since this is not expected). Notice that the hardware docs are not clear about this fact: SNB PRM Vol4 Part2's "Table 5-4. MRF Registers Available in Device Hardware" says "Number per Thread" - "24 registers" However, SNB PRM Vol4 Part1, 1.6.1 Message Register File (MRF) says: "Normal threads should construct their messages in m1..m15. (...) Regardless of actual hardware implementation, the thread should not assume th at MRF addresses above m15 wrap to legal MRF registers." Therefore experimentation was necessary to evaluate if we had these extra MRF registers available or not. This was tested in gen6 using MRF registers 21..23 for spilling and doing a full piglit run (all.py) forcing spilling of everything on the FS backend. It was also tested by doing spilling of everything on both the FS and the VS backends with a piglit run of shader.py. In both cases no regressions were observed. In fact, many of these tests where helped in the cases where we forced spilling, since that triggered the same underlying problem described in the bug reports. Here are some results using INTEL_DEBUG=spill_fs,spill_vec4 for a shader.py run on gen6 hardware: Using MRFs 13..15 for spilling: crash: 2, fail: 113, pass: 6621, skip: 5461 Using MRFs 21..23 for spilling: crash: 2, fail: 12, pass: 6722, skip: 5461 This patch sets the ground for later patches to implement spilling using MRF registers 21..23 in gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-15 16:00:26 +02:00
assert(msg_length < BRW_MAX_MRF(devinfo->gen));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, brw_imm_d(0));
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6)
brw_inst_set_base_mrf(devinfo, insn, msg_reg_nr);
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_urb_message(p,
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
insn,
i965: Combine 4 boolean args of brw_urb_WRITE into a flags bitfield. The arguments to brw_urb_WRITE() were getting pretty unwieldy, and we have to add more flags to support geometry shaders anyhow. Also plumb these flags through brw_clip_emit_vue(), brw_set_urb_message(), and the vec4_instruction class. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2013-08-10 21:13:33 -07:00
flags,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
msg_length,
i965: Drop trailing whitespace from files shared with intel-gpu-tools. Performed via s/ *$//g. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:39:03 -08:00
response_length,
Add Intel i965G/Q DRI driver. This driver comes from Tungsten Graphics, with a few further modifications by Intel.
2006-08-09 19:14:05 +00:00
offset,
swizzle);
}
intel/eu: Get rid of the return value of brw_send_indirect_message(). The return value is not used anymore. This allows simplifying the code slightly, and in addition it should frustrate anybody's attempts to continue using the obsolete piecemeal approach to construct a message descriptor in combination with brw_send_indirect_message(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:08:18 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_send_indirect_message(struct brw_codegen *p,
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
unsigned sfid,
struct brw_reg dst,
struct brw_reg payload,
intel/eu: Provide desc immediate argument up front to brw_send_indirect_message(). The current approach of returning a setup instruction where additional descriptor fields can be specified is still supported in order to keep things working, but it will be removed later in this series. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:07:31 -07:00
struct brw_reg desc,
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
unsigned desc_imm,
bool eot)
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Fix invalid memory accesses after resizing brw_codegen's store table Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-10-28 10:58:09 -07:00
struct brw_inst *send;
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
i965: Set dest type to UW for several send messages Without this, on SIMD 16 the send instruction destination will appear to write more than one destination register, causing the simulator to report an error. Of course, the send instruction can actually write more than one destination register regardless of the type set for the destination, so this is a bit strange. Suggested-by: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-01-31 18:28:42 -08:00
dst = retype(dst, BRW_REGISTER_TYPE_UW);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
assert(desc.type == BRW_REGISTER_TYPE_UD);
if (desc.file == BRW_IMMEDIATE_VALUE) {
i965: Fix invalid memory accesses after resizing brw_codegen's store table Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2015-10-28 10:58:09 -07:00
send = next_insn(p, BRW_OPCODE_SEND);
intel/compiler: Fix src0/desc setter ordering src0 vstride and type overlap with bits of the extended descriptor. brw_set_desc() also sets the extended descriptor to 0. So by setting the descriptor, then setting src0, we were accidentally setting a bunch of extended descriptor bits unintentionally. When using this infrastructure for framebuffer writes (in a future patch), this ended up setting the extended descriptor bit 20, which is "Null Render Target" on Icelake, causing nothing to be written to the framebuffer. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-08-26 15:21:40 -07:00
brw_set_src0(p, send, retype(payload, BRW_REGISTER_TYPE_UD));
intel/eu: Provide desc immediate argument up front to brw_send_indirect_message(). The current approach of returning a setup instruction where additional descriptor fields can be specified is still supported in order to keep things working, but it will be removed later in this series. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:07:31 -07:00
brw_set_desc(p, send, desc.ud | desc_imm);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
} else {
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
struct brw_reg addr = retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD);
brw_push_insn_state(p);
brw_set_default_access_mode(p, BRW_ALIGN_1);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
/* Load the indirect descriptor to an address register using OR so the
intel/eu: Get rid of the return value of brw_send_indirect_message(). The return value is not used anymore. This allows simplifying the code slightly, and in addition it should frustrate anybody's attempts to continue using the obsolete piecemeal approach to construct a message descriptor in combination with brw_send_indirect_message(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:08:18 -07:00
* caller can specify additional descriptor bits with the desc_imm
* immediate.
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
*/
intel/eu: Provide desc immediate argument up front to brw_send_indirect_message(). The current approach of returning a setup instruction where additional descriptor fields can be specified is still supported in order to keep things working, but it will be removed later in this series. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 15:07:31 -07:00
brw_OR(p, addr, desc, brw_imm_ud(desc_imm));
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
brw_pop_insn_state(p);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
send = next_insn(p, BRW_OPCODE_SEND);
intel/compiler: Fix src0/desc setter ordering src0 vstride and type overlap with bits of the extended descriptor. brw_set_desc() also sets the extended descriptor to 0. So by setting the descriptor, then setting src0, we were accidentally setting a bunch of extended descriptor bits unintentionally. When using this infrastructure for framebuffer writes (in a future patch), this ended up setting the extended descriptor bit 20, which is "Null Render Target" on Icelake, causing nothing to be written to the framebuffer. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-08-26 15:21:40 -07:00
brw_set_src0(p, send, retype(payload, BRW_REGISTER_TYPE_UD));
intel/eu/gen12: Codegen SEND descriptor regions correctly. The SEND instruction is now four-source. The descriptor is no longer part of source 1, so avoid touching it to avoid corruption while initializing the descriptor. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-09-03 12:18:38 -07:00
if (devinfo->gen >= 12)
brw_inst_set_send_sel_reg32_desc(devinfo, send, true);
else
brw_set_src1(p, send, addr);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
}
brw_set_dest(p, send, dst);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_sfid(devinfo, send, sfid);
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
brw_inst_set_eot(devinfo, send, eot);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 15:44:24 +02:00
}
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
void
brw_send_indirect_split_message(struct brw_codegen *p,
unsigned sfid,
struct brw_reg dst,
struct brw_reg payload0,
struct brw_reg payload1,
struct brw_reg desc,
unsigned desc_imm,
struct brw_reg ex_desc,
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
unsigned ex_desc_imm,
bool eot)
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
{
const struct gen_device_info *devinfo = p->devinfo;
struct brw_inst *send;
dst = retype(dst, BRW_REGISTER_TYPE_UW);
assert(desc.type == BRW_REGISTER_TYPE_UD);
if (desc.file == BRW_IMMEDIATE_VALUE) {
desc.ud |= desc_imm;
} else {
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
struct brw_reg addr = retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD);
brw_push_insn_state(p);
brw_set_default_access_mode(p, BRW_ALIGN_1);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_exec_size(p, BRW_EXECUTE_1);
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
/* Load the indirect descriptor to an address register using OR so the
* caller can specify additional descriptor bits with the desc_imm
* immediate.
*/
brw_OR(p, addr, desc, brw_imm_ud(desc_imm));
brw_pop_insn_state(p);
desc = addr;
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
}
intel/compiler: Handle bits 15:12 in brw_send_indirect_split_message() Annoyingly, these bits exist in some extended message descriptors (in particular render target writes), but they don't have any corresponding bits in the ISA encoding. So we can't use an immediate and have to fall back to an indirect extended descriptor. Thanks to Jason Ekstrand for reminding me that you can still set these bits via an indirect descriptor, even if they don't exist in the ISA. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-08-25 23:43:29 -07:00
if (ex_desc.file == BRW_IMMEDIATE_VALUE &&
intel/fs: Add an ex_desc field to fs_inst for SHADER_OPCODE_SEND I meant to do this years ago when I first added SHADER_OPCODE_SEND. At the time, the only use for the extended descriptor was bindless handles which were always one thing and never non-constant. However, it doesn't actually require any extra instructions because we have to OR in ex_mlen anyway. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8748>
2021-01-27 15:28:24 -06:00
(devinfo->gen >= 12 ||
((ex_desc.ud | ex_desc_imm) & INTEL_MASK(15, 12)) == 0)) {
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
ex_desc.ud |= ex_desc_imm;
} else {
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
struct brw_reg addr = retype(brw_address_reg(2), BRW_REGISTER_TYPE_UD);
brw_push_insn_state(p);
brw_set_default_access_mode(p, BRW_ALIGN_1);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_exec_size(p, BRW_EXECUTE_1);
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
/* Load the indirect extended descriptor to an address register using OR
* so the caller can specify additional descriptor bits with the
* desc_imm immediate.
*
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
* Even though the instruction dispatcher always pulls the SFID and EOT
* fields from the instruction itself, actual external unit which
* processes the message gets the SFID and EOT from the extended
* descriptor which comes from the address register. If we don't OR
* those two bits in, the external unit may get confused and hang.
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
*/
intel/compiler: Handle bits 15:12 in brw_send_indirect_split_message() Annoyingly, these bits exist in some extended message descriptors (in particular render target writes), but they don't have any corresponding bits in the ISA encoding. So we can't use an immediate and have to fall back to an indirect extended descriptor. Thanks to Jason Ekstrand for reminding me that you can still set these bits via an indirect descriptor, even if they don't exist in the ISA. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-08-25 23:43:29 -07:00
unsigned imm_part = ex_desc_imm | sfid | eot << 5;
if (ex_desc.file == BRW_IMMEDIATE_VALUE) {
intel/fs: Don't unnecessarily fall back to indirect sends on Gen12 The instruction encoding for SENDS changed on Gen12 and it now supports embedding the entire extended message descriptor in the instruction if it's an immediate. Stop falling back to doing an indirect SEND just because we had something in [15:12] of ex_desc.ud. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3547>
2020-01-22 22:54:20 -06:00
/* ex_desc bits 15:12 don't exist in the instruction encoding prior
* to Gen12, so we may have fallen back to an indirect extended
* descriptor.
intel/compiler: Handle bits 15:12 in brw_send_indirect_split_message() Annoyingly, these bits exist in some extended message descriptors (in particular render target writes), but they don't have any corresponding bits in the ISA encoding. So we can't use an immediate and have to fall back to an indirect extended descriptor. Thanks to Jason Ekstrand for reminding me that you can still set these bits via an indirect descriptor, even if they don't exist in the ISA. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2019-08-25 23:43:29 -07:00
*/
brw_MOV(p, addr, brw_imm_ud(ex_desc.ud | imm_part));
} else {
brw_OR(p, addr, ex_desc, brw_imm_ud(imm_part));
}
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
brw_pop_insn_state(p);
ex_desc = addr;
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
}
intel/eu/gen12: Use SEND instruction for split sends. The new SEND instruction behaves like the former SENDS instruction. The original single-payload SEND instruction is gone. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-08-25 18:12:35 -07:00
send = next_insn(p, devinfo->gen >= 12 ? BRW_OPCODE_SEND : BRW_OPCODE_SENDS);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
brw_set_dest(p, send, dst);
brw_set_src0(p, send, retype(payload0, BRW_REGISTER_TYPE_UD));
brw_set_src1(p, send, retype(payload1, BRW_REGISTER_TYPE_UD));
if (desc.file == BRW_IMMEDIATE_VALUE) {
brw_inst_set_send_sel_reg32_desc(devinfo, send, 0);
brw_inst_set_send_desc(devinfo, send, desc.ud);
} else {
assert(desc.file == BRW_ARCHITECTURE_REGISTER_FILE);
assert(desc.nr == BRW_ARF_ADDRESS);
assert(desc.subnr == 0);
brw_inst_set_send_sel_reg32_desc(devinfo, send, 1);
}
if (ex_desc.file == BRW_IMMEDIATE_VALUE) {
brw_inst_set_send_sel_reg32_ex_desc(devinfo, send, 0);
intel/eu: Split brw_inst ex_desc accessors for SEND(C) vs. SENDS(C). The brw_inst opcode accessors are going away in one of the following commits. We could potentially replace them with the new helpers that do opcode remapping, but that would lead to a circular dependency between brw_inst.h and brw_eu.h. This way we also avoid ordering issues that can cause the semantics of the ex_desc accessors to change depending on whether the ex_desc field is set after or before the opcode instruction field. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-02-05 23:22:06 -08:00
brw_inst_set_sends_ex_desc(devinfo, send, ex_desc.ud);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
} else {
assert(ex_desc.file == BRW_ARCHITECTURE_REGISTER_FILE);
assert(ex_desc.nr == BRW_ARF_ADDRESS);
assert((ex_desc.subnr & 0x3) == 0);
brw_inst_set_send_sel_reg32_ex_desc(devinfo, send, 1);
brw_inst_set_send_ex_desc_ia_subreg_nr(devinfo, send, ex_desc.subnr >> 2);
}
brw_inst_set_sfid(devinfo, send, sfid);
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
brw_inst_set_eot(devinfo, send, eot);
intel/eu: Add support for the SENDS[C] messages Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-15 15:17:06 -06:00
}
intel/eu: Get rid of the return value of brw_send_indirect_surface_message(). All users of brw_send_indirect_surface_message() should be providing a full descriptor immediate up front by now, this isn't necessary anymore. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-03 03:30:50 -07:00
static void
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
brw_send_indirect_surface_message(struct brw_codegen *p,
unsigned sfid,
struct brw_reg dst,
struct brw_reg payload,
struct brw_reg surface,
intel/eu: Provide single descriptor argument to brw_send_indirect_surface_message(). Instead of the current message_len, response_len and header_present arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:19:49 -07:00
unsigned desc_imm)
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
{
if (surface.file != BRW_IMMEDIATE_VALUE) {
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
const struct tgl_swsb swsb = brw_get_default_swsb(p);
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
struct brw_reg addr = retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD);
brw_push_insn_state(p);
brw_set_default_access_mode(p, BRW_ALIGN_1);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_src_dep(swsb));
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
/* Mask out invalid bits from the surface index to avoid hangs e.g. when
* some surface array is accessed out of bounds.
*/
intel/eu: Get rid of the return value of brw_send_indirect_surface_message(). All users of brw_send_indirect_surface_message() should be providing a full descriptor immediate up front by now, this isn't necessary anymore. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-03 03:30:50 -07:00
brw_AND(p, addr,
suboffset(vec1(retype(surface, BRW_REGISTER_TYPE_UD)),
BRW_GET_SWZ(surface.swizzle, 0)),
brw_imm_ud(0xff));
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
brw_pop_insn_state(p);
surface = addr;
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_dst_dep(swsb, 1));
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
}
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
brw_send_indirect_message(p, sfid, dst, payload, surface, desc_imm, false);
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
}
i965: Make a "does this while jump before our instruction?" helper. I need to use this in an additional place. Cc: mesa-stable@lists.freedesktop.org Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-14 23:53:19 -07:00
static bool
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
while_jumps_before_offset(const struct gen_device_info *devinfo,
i965: Make a "does this while jump before our instruction?" helper. I need to use this in an additional place. Cc: mesa-stable@lists.freedesktop.org Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-14 23:53:19 -07:00
brw_inst *insn, int while_offset, int start_offset)
{
int scale = 16 / brw_jump_scale(devinfo);
int jip = devinfo->gen == 6 ? brw_inst_gen6_jump_count(devinfo, insn)
: brw_inst_jip(devinfo, insn);
i965: add assert to while_jumps_before_offset() jip should always be negative here as its the result of do instruction - while instruction. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-01-26 13:50:42 +11:00
assert(jip < 0);
i965: Make a "does this while jump before our instruction?" helper. I need to use this in an additional place. Cc: mesa-stable@lists.freedesktop.org Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-14 23:53:19 -07:00
return while_offset + jip * scale <= start_offset;
}
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
static int
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_find_next_block_end(struct brw_codegen *p, int start_offset)
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
{
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
int offset;
i965: Prepare the break/cont uip/jip setting for compacted instructions. The first cut at instruction compaction won't compact things that would change control flow jump distances, but we do need to still be able to walk the instruction stream, which involves jumping by 8 or 16 bytes between instructions. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-02-03 12:05:05 +01:00
void *store = p->store;
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
i965: Fix JIP to properly skip over unrelated control flow. We've apparently always been botching JIP for sequences such as: do cmp.f0.0 ... (+f0.0) break ... if ... else ... endif ... while Normally, UIP is supposed to point to the final destination of the jump, while in nested control flow, JIP is supposed to point to the end of the current nesting level. It essentially bounces out of the current nested control flow, to an instruction that has a JIP which bounces out another level, and so on. In the above example, when setting JIP for the BREAK, we call brw_find_next_block_end(), which begins a search after the BREAK for the next ENDIF, ELSE, WHILE, or HALT. It ignores the IF and finds the ELSE, setting JIP there. This makes no sense at all. The break is supposed to skip over the whole if/else/endif block entirely. They have a sibling relationship, not a nesting relationship. This patch fixes brw_find_next_block_end() to track depth as it does its search, and ignore anything not at depth 0. So when it sees the IF, it ignores everything until after the ENDIF. That way, it finds the end of the right block. I noticed this while reading some assembly code. We believe jumping earlier is harmless, but makes the EU walk through a bunch of disabled instructions for no reason. I noticed that GLBenchmark Manhattan had a shader that contained a BREAK with a bogus JIP, but didn't measure any performance improvement (it's likely miniscule, if there is any). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-17 18:24:11 -08:00
int depth = 0;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
for (offset = next_offset(devinfo, store, start_offset);
i965: Pass brw into next_offset(). The new brw_inst API is going to require a brw pointer in order to access fields (so it can do generation checks). Plumb it in now. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-07 21:15:59 -07:00
offset < p->next_insn_offset;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
offset = next_offset(devinfo, store, offset)) {
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = store + offset;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
switch (brw_inst_opcode(devinfo, insn)) {
i965: Fix JIP to properly skip over unrelated control flow. We've apparently always been botching JIP for sequences such as: do cmp.f0.0 ... (+f0.0) break ... if ... else ... endif ... while Normally, UIP is supposed to point to the final destination of the jump, while in nested control flow, JIP is supposed to point to the end of the current nesting level. It essentially bounces out of the current nested control flow, to an instruction that has a JIP which bounces out another level, and so on. In the above example, when setting JIP for the BREAK, we call brw_find_next_block_end(), which begins a search after the BREAK for the next ENDIF, ELSE, WHILE, or HALT. It ignores the IF and finds the ELSE, setting JIP there. This makes no sense at all. The break is supposed to skip over the whole if/else/endif block entirely. They have a sibling relationship, not a nesting relationship. This patch fixes brw_find_next_block_end() to track depth as it does its search, and ignore anything not at depth 0. So when it sees the IF, it ignores everything until after the ENDIF. That way, it finds the end of the right block. I noticed this while reading some assembly code. We believe jumping earlier is harmless, but makes the EU walk through a bunch of disabled instructions for no reason. I noticed that GLBenchmark Manhattan had a shader that contained a BREAK with a bogus JIP, but didn't measure any performance improvement (it's likely miniscule, if there is any). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-17 18:24:11 -08:00
case BRW_OPCODE_IF:
depth++;
break;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
case BRW_OPCODE_ENDIF:
i965: Fix JIP to properly skip over unrelated control flow. We've apparently always been botching JIP for sequences such as: do cmp.f0.0 ... (+f0.0) break ... if ... else ... endif ... while Normally, UIP is supposed to point to the final destination of the jump, while in nested control flow, JIP is supposed to point to the end of the current nesting level. It essentially bounces out of the current nested control flow, to an instruction that has a JIP which bounces out another level, and so on. In the above example, when setting JIP for the BREAK, we call brw_find_next_block_end(), which begins a search after the BREAK for the next ENDIF, ELSE, WHILE, or HALT. It ignores the IF and finds the ELSE, setting JIP there. This makes no sense at all. The break is supposed to skip over the whole if/else/endif block entirely. They have a sibling relationship, not a nesting relationship. This patch fixes brw_find_next_block_end() to track depth as it does its search, and ignore anything not at depth 0. So when it sees the IF, it ignores everything until after the ENDIF. That way, it finds the end of the right block. I noticed this while reading some assembly code. We believe jumping earlier is harmless, but makes the EU walk through a bunch of disabled instructions for no reason. I noticed that GLBenchmark Manhattan had a shader that contained a BREAK with a bogus JIP, but didn't measure any performance improvement (it's likely miniscule, if there is any). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-17 18:24:11 -08:00
if (depth == 0)
return offset;
depth--;
break;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
case BRW_OPCODE_WHILE:
i965: Fix JIP to skip over sibling do...while loops. We've apparently always been botching JIP for sequences such as: do cmp.f0.0 ... (+f0.0) break ... do ... while ... while Because the "do" instruction doesn't actually exist, the inner "while" is at the same depth as the "break". brw_find_next_block_end() thus mistook the inner "while" as the end of the loop containing the "break", and set the "break" to point to the wrong place. Only "while" instructions that jump before our instruction are relevant. We need to ignore the rest, as they're sibling control flow nodes (or children, but this was already handled by the depth == 0 check). See also commit 1ac1581f3889d5f7e6e231c05651f44fbd80f0b6. This prevents channel masks from being screwed up, and fixes GPU hangs(*) in dEQP-GLES31.functional.shaders.multisample_interpolation. interpolate_at_sample.centroid_qualified.multisample_texture_16. The test ended up executing code with no channels enabled, and that code contained FIND_LIVE_CHANNEL, which returned 8 (out of range for a SIMD8 program), which then was used in indirect GRF addressing, which randomly got a boolean value (0xFFFFFFFF), interpreted it as a sample ID, OR'd it into an indirect send message descriptor, which corrupted the message length, sending a pixel interpolator message with mlen 15, which is illegal. Whew :) (*) Technically, the test doesn't GPU hang currently, but only because another bug prevents it from issuing pixel interpolator messages entirely...with that fixed, it hangs. Cc: mesa-stable@lists.freedesktop.org Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-14 23:54:48 -07:00
/* If the while doesn't jump before our instruction, it's the end
* of a sibling do...while loop. Ignore it.
*/
if (!while_jumps_before_offset(devinfo, insn, offset, start_offset))
continue;
i965: Mark fallthrough in switch statement. Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Eric Engestrom <eric.engestrom@imgtec.com>
2016-05-24 12:23:00 -07:00
/* fallthrough */
i965: Fix JIP to skip over sibling do...while loops. We've apparently always been botching JIP for sequences such as: do cmp.f0.0 ... (+f0.0) break ... do ... while ... while Because the "do" instruction doesn't actually exist, the inner "while" is at the same depth as the "break". brw_find_next_block_end() thus mistook the inner "while" as the end of the loop containing the "break", and set the "break" to point to the wrong place. Only "while" instructions that jump before our instruction are relevant. We need to ignore the rest, as they're sibling control flow nodes (or children, but this was already handled by the depth == 0 check). See also commit 1ac1581f3889d5f7e6e231c05651f44fbd80f0b6. This prevents channel masks from being screwed up, and fixes GPU hangs(*) in dEQP-GLES31.functional.shaders.multisample_interpolation. interpolate_at_sample.centroid_qualified.multisample_texture_16. The test ended up executing code with no channels enabled, and that code contained FIND_LIVE_CHANNEL, which returned 8 (out of range for a SIMD8 program), which then was used in indirect GRF addressing, which randomly got a boolean value (0xFFFFFFFF), interpreted it as a sample ID, OR'd it into an indirect send message descriptor, which corrupted the message length, sending a pixel interpolator message with mlen 15, which is illegal. Whew :) (*) Technically, the test doesn't GPU hang currently, but only because another bug prevents it from issuing pixel interpolator messages entirely...with that fixed, it hangs. Cc: mesa-stable@lists.freedesktop.org Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-14 23:54:48 -07:00
case BRW_OPCODE_ELSE:
i965: Also consider HALTs a potential block end. The final halt of the fragment shader turns off the remaining channels, then jumps such that everything is turned back on. So, we can have our last ENDIF of the shader point at that directly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-12 12:47:50 -08:00
case BRW_OPCODE_HALT:
i965: Fix JIP to properly skip over unrelated control flow. We've apparently always been botching JIP for sequences such as: do cmp.f0.0 ... (+f0.0) break ... if ... else ... endif ... while Normally, UIP is supposed to point to the final destination of the jump, while in nested control flow, JIP is supposed to point to the end of the current nesting level. It essentially bounces out of the current nested control flow, to an instruction that has a JIP which bounces out another level, and so on. In the above example, when setting JIP for the BREAK, we call brw_find_next_block_end(), which begins a search after the BREAK for the next ENDIF, ELSE, WHILE, or HALT. It ignores the IF and finds the ELSE, setting JIP there. This makes no sense at all. The break is supposed to skip over the whole if/else/endif block entirely. They have a sibling relationship, not a nesting relationship. This patch fixes brw_find_next_block_end() to track depth as it does its search, and ignore anything not at depth 0. So when it sees the IF, it ignores everything until after the ENDIF. That way, it finds the end of the right block. I noticed this while reading some assembly code. We believe jumping earlier is harmless, but makes the EU walk through a bunch of disabled instructions for no reason. I noticed that GLBenchmark Manhattan had a shader that contained a BREAK with a bogus JIP, but didn't measure any performance improvement (it's likely miniscule, if there is any). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-17 18:24:11 -08:00
if (depth == 0)
return offset;
intel/eu: Encode and decode native instruction opcodes from/to IR opcodes. Change brw_inst_set_opcode() and brw_inst_opcode() to call brw_opcode_encode/decode() transparently in order to translate between hardware and IR opcodes, and update the EU compaction code in order to do the same as needed, so we can eventually drop the one-to-one correspondence between hardware and IR opcodes. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-01-23 19:23:20 -08:00
default:
break;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
}
}
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
return 0;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
}
/* There is no DO instruction on gen6, so to find the end of the loop
* we have to see if the loop is jumping back before our start
* instruction.
*/
static int
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_find_loop_end(struct brw_codegen *p, int start_offset)
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
int offset;
i965: Prepare the break/cont uip/jip setting for compacted instructions. The first cut at instruction compaction won't compact things that would change control flow jump distances, but we do need to still be able to walk the instruction stream, which involves jumping by 8 or 16 bytes between instructions. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-02-03 12:05:05 +01:00
void *store = p->store;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(devinfo->gen >= 6);
i965/eu: Make it clear that brw_find_loop_end only runs on Gen6+. It has Gen6+ knowledge baked in, and indeed is only called for Gen6+, but it wasn't immediately obvious that this was the case. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-30 08:06:43 -07:00
i965: Prepare the break/cont uip/jip setting for compacted instructions. The first cut at instruction compaction won't compact things that would change control flow jump distances, but we do need to still be able to walk the instruction stream, which involves jumping by 8 or 16 bytes between instructions. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-02-03 12:05:05 +01:00
/* Always start after the instruction (such as a WHILE) we're trying to fix
* up.
*/
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
for (offset = next_offset(devinfo, store, start_offset);
i965: Pass brw into next_offset(). The new brw_inst API is going to require a brw pointer in order to access fields (so it can do generation checks). Plumb it in now. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-07 21:15:59 -07:00
offset < p->next_insn_offset;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
offset = next_offset(devinfo, store, offset)) {
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = store + offset;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (brw_inst_opcode(devinfo, insn) == BRW_OPCODE_WHILE) {
i965: Make a "does this while jump before our instruction?" helper. I need to use this in an additional place. Cc: mesa-stable@lists.freedesktop.org Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-14 23:53:19 -07:00
if (while_jumps_before_offset(devinfo, insn, offset, start_offset))
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
return offset;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
}
}
assert(!"not reached");
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
return start_offset;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
}
/* After program generation, go back and update the UIP and JIP of
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
* BREAK, CONT, and HALT instructions to their correct locations.
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
*/
void
i965: Pass start_offset to brw_set_uip_jip(). Without this, we would pass over the instructions in the SIMD8 program (which is located earlier in the buffer) when brw_set_uip_jip() is called to handle the SIMD16 program. The assertion about compacted control flow was bogus: halt, cont, break cannot be compacted because they have both JIP and UIP. Instead, we should never see a compacted instruction in this code at all. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-08-29 15:57:41 -07:00
brw_set_uip_jip(struct brw_codegen *p, int start_offset)
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
int offset;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
int br = brw_jump_scale(devinfo);
i965/eu: Refactor jump distance scaling to use a helper function. Different generations of hardware measure jump distances in different units. Previously, every function that needed to set a jump target open coded this scaling, or made a hardcoded assumption (i.e. just used 2). Most functions start with the number of instructions to jump, and scale up to the hardware-specific value. So, I made the function match that. Others start with a byte offset, and divide by a constant (8) to obtain the jump distance. This is actually 16 / 2 (the jump scale for Gen5-7). v2: Make the helper a static inline defined in brw_eu.h, instead of an actual function in brw_eu_emit.c (as suggested by Matt). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-30 08:00:25 -07:00
int scale = 16 / br;
i965: Prepare the break/cont uip/jip setting for compacted instructions. The first cut at instruction compaction won't compact things that would change control flow jump distances, but we do need to still be able to walk the instruction stream, which involves jumping by 8 or 16 bytes between instructions. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-02-03 12:05:05 +01:00
void *store = p->store;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6)
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
return;
i965: Pass start_offset to brw_set_uip_jip(). Without this, we would pass over the instructions in the SIMD8 program (which is located earlier in the buffer) when brw_set_uip_jip() is called to handle the SIMD16 program. The assertion about compacted control flow was bogus: halt, cont, break cannot be compacted because they have both JIP and UIP. Instead, we should never see a compacted instruction in this code at all. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-08-29 15:57:41 -07:00
for (offset = start_offset; offset < p->next_insn_offset; offset += 16) {
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn = store + offset;
i965: Pass start_offset to brw_set_uip_jip(). Without this, we would pass over the instructions in the SIMD8 program (which is located earlier in the buffer) when brw_set_uip_jip() is called to handle the SIMD16 program. The assertion about compacted control flow was bogus: halt, cont, break cannot be compacted because they have both JIP and UIP. Instead, we should never see a compacted instruction in this code at all. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-08-29 15:57:41 -07:00
assert(brw_inst_cmpt_control(devinfo, insn) == 0);
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
int block_end_offset = brw_find_next_block_end(p, offset);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
switch (brw_inst_opcode(devinfo, insn)) {
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
case BRW_OPCODE_BREAK:
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
assert(block_end_offset != 0);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, (block_end_offset - offset) / scale);
i965: Add support for loops on Ivybridge. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-30 01:17:52 -07:00
/* Gen7 UIP points to WHILE; Gen6 points just after it */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_uip(devinfo, insn,
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
(brw_find_loop_end(p, offset) - offset +
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
(devinfo->gen == 6 ? 16 : 0)) / scale);
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
break;
case BRW_OPCODE_CONTINUE:
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
assert(block_end_offset != 0);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, (block_end_offset - offset) / scale);
brw_inst_set_uip(devinfo, insn,
i965: Convert brw_eu_emit.c to the new brw_inst API. v2: - Fix IF -> ELSE patching on Sandybridge. - Don't set base_mrf on Gen6+ in OWord Block Read functions. (Although - the old code did this universally, it shouldn't have - the field - doesn't exist on Gen6+ and just got overwritten by the SFID anyway.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-04 17:08:57 -07:00
(brw_find_loop_end(p, offset) - offset) / scale);
i965: Make the CONT instruction point to the WHILE instruction. This fixes piglit test glsl-fs-loop-continue.shader_test on Ivybridge. According to the documentation, the CONT instruction's UIP field should point to the WHILE instruction on both Sandybridge and Ivybridge. The previous code made UIP point to the implicit DO instruction, which seems incorrect. I'm not sure how it could have worked on Sandybridge. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-04-30 01:30:55 -07:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_uip(devinfo, insn) != 0);
assert(brw_inst_jip(devinfo, insn) != 0);
i965: Add break statement at end of BRW_OPCODE_CONTINUE case. Fixes missing break in switch defect reported by Coverity. Signed-off-by: Vinson Lee <vlee@freedesktop.org> Reviewed-by: Paul Berry <stereotype441@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-01-01 17:02:38 -08:00
break;
i965: Jump to the end of the next outer conditional block on ENDIFs. From the Ivybridge PRM, Volume 4, Part 3, section 6.24 (page 172): "The endif instruction is also used to hop out of nested conditionals by jumping to the end of the next outer conditional block when all channels are disabled." Also: "Pseudocode: Evaluate(WrEn); if ( WrEn == 0 ) { // all channels false Jump(IP + JIP); }" First, ENDIF re-enables any channels that were disabled because they didn't match the conditional. If any channels are active, it proceeds to the next instruction (IP + 16). However, if they're all disabled, there's no point in walking through all of the instructions that have no effect---it can jump to the next instruction that might re-enable some channels (an ELSE, ENDIF, or WHILE). Previously, we always set JIP on ENDIF instructions to 2 (which is measured in 8-byte units). This made it do Jump(IP + 16), which just meant it would go to the next instruction even if all channels were off. It turns out that walking over instructions while all the channels are disabled like this is worse than just instruction dispatch overhead: if there are texturing messages, it still costs a couple hundred cycles to not-actually-read from the texture results. This patch finds the next instruction that could re-enable channels and sets JIP accordingly. Reviewed-by: Eric Anholt <eric@anholt.net>
2012-12-12 02:20:05 -08:00
i965: Set JumpCount, not JIP, on ENDIF on Gen 6. Despite what the Sandybridge PRM says, ENDIF has Jump Count in <dst>, not JIP in <src1>. (The same mistake appears about WHILE as well). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-28 13:34:22 -07:00
case BRW_OPCODE_ENDIF: {
int32_t jump = (block_end_offset == 0) ?
1 * br : (block_end_offset - offset) / scale;
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen >= 7)
brw_inst_set_jip(devinfo, insn, jump);
i965: Jump to the end of the next outer conditional block on ENDIFs. From the Ivybridge PRM, Volume 4, Part 3, section 6.24 (page 172): "The endif instruction is also used to hop out of nested conditionals by jumping to the end of the next outer conditional block when all channels are disabled." Also: "Pseudocode: Evaluate(WrEn); if ( WrEn == 0 ) { // all channels false Jump(IP + JIP); }" First, ENDIF re-enables any channels that were disabled because they didn't match the conditional. If any channels are active, it proceeds to the next instruction (IP + 16). However, if they're all disabled, there's no point in walking through all of the instructions that have no effect---it can jump to the next instruction that might re-enable some channels (an ELSE, ENDIF, or WHILE). Previously, we always set JIP on ENDIF instructions to 2 (which is measured in 8-byte units). This made it do Jump(IP + 16), which just meant it would go to the next instruction even if all channels were off. It turns out that walking over instructions while all the channels are disabled like this is worse than just instruction dispatch overhead: if there are texturing messages, it still costs a couple hundred cycles to not-actually-read from the texture results. This patch finds the next instruction that could re-enable channels and sets JIP accordingly. Reviewed-by: Eric Anholt <eric@anholt.net>
2012-12-12 02:20:05 -08:00
else
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_gen6_jump_count(devinfo, insn, jump);
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
break;
i965: Set JumpCount, not JIP, on ENDIF on Gen 6. Despite what the Sandybridge PRM says, ENDIF has Jump Count in <dst>, not JIP in <src1>. (The same mistake appears about WHILE as well). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-28 13:34:22 -07:00
}
i965: Jump to the end of the next outer conditional block on ENDIFs. From the Ivybridge PRM, Volume 4, Part 3, section 6.24 (page 172): "The endif instruction is also used to hop out of nested conditionals by jumping to the end of the next outer conditional block when all channels are disabled." Also: "Pseudocode: Evaluate(WrEn); if ( WrEn == 0 ) { // all channels false Jump(IP + JIP); }" First, ENDIF re-enables any channels that were disabled because they didn't match the conditional. If any channels are active, it proceeds to the next instruction (IP + 16). However, if they're all disabled, there's no point in walking through all of the instructions that have no effect---it can jump to the next instruction that might re-enable some channels (an ELSE, ENDIF, or WHILE). Previously, we always set JIP on ENDIF instructions to 2 (which is measured in 8-byte units). This made it do Jump(IP + 16), which just meant it would go to the next instruction even if all channels were off. It turns out that walking over instructions while all the channels are disabled like this is worse than just instruction dispatch overhead: if there are texturing messages, it still costs a couple hundred cycles to not-actually-read from the texture results. This patch finds the next instruction that could re-enable channels and sets JIP accordingly. Reviewed-by: Eric Anholt <eric@anholt.net>
2012-12-12 02:20:05 -08:00
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
case BRW_OPCODE_HALT:
/* From the Sandy Bridge PRM (volume 4, part 2, section 8.3.19):
*
* "In case of the halt instruction not inside any conditional
* code block, the value of <JIP> and <UIP> should be the
* same. In case of the halt instruction inside conditional code
* block, the <UIP> should be the end of the program, and the
* <JIP> should be end of the most inner conditional code block."
*
* The uip will have already been set by whoever set up the
* instruction.
*/
i965: Rename next_ip() -> next_offset(). That we were comparing its return value with offsets should have been a clue. :) Make it take a void *store in preparation for making the function useful elsewhere. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-17 12:53:56 -07:00
if (block_end_offset == 0) {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, brw_inst_uip(devinfo, insn));
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
} else {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
brw_inst_set_jip(devinfo, insn, (block_end_offset - offset) / scale);
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
}
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
assert(brw_inst_uip(devinfo, insn) != 0);
assert(brw_inst_jip(devinfo, insn) != 0);
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 10:15:08 -08:00
break;
intel/eu: Encode and decode native instruction opcodes from/to IR opcodes. Change brw_inst_set_opcode() and brw_inst_opcode() to call brw_opcode_encode/decode() transparently in order to translate between hardware and IR opcodes, and update the EU compaction code in order to do the same as needed, so we can eventually drop the one-to-one correspondence between hardware and IR opcodes. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-01-23 19:23:20 -08:00
default:
break;
i965: Add support for gen6 BREAK ISA emit. There are now two targets: the hop-to-end-of-block target, and the target for where to resume execution for active channels.
2010-12-01 11:46:46 -08:00
}
}
}
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_ff_sync(struct brw_codegen *p,
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_reg_nr,
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
struct brw_reg src0,
intel: Convert from GLboolean to 'bool' from stdbool.h. I initially produced the patch using this bash command: for file in {intel,i915,i965}/*.{c,cpp,h}; do [ ! -h $file ] && sed -i 's/GLboolean/bool/g' $file && sed -i 's/GL_TRUE/true/g' $file && sed -i 's/GL_FALSE/false/g' $file; done Then I manually added #include <stdbool.h> to fix compilation errors, and converted a few functions back to GLboolean that were used in core Mesa's function pointer table to avoid "incompatible pointer" warnings. Finally, I cleaned up some whitespace issues introduced by the change. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Chad Versace <chad@chad-versace.us> Acked-by: Paul Berry <stereotype441@gmail.com>
2011-10-07 12:26:50 -07:00
bool allocate,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned response_length,
intel: Convert from GLboolean to 'bool' from stdbool.h. I initially produced the patch using this bash command: for file in {intel,i915,i965}/*.{c,cpp,h}; do [ ! -h $file ] && sed -i 's/GLboolean/bool/g' $file && sed -i 's/GL_TRUE/true/g' $file && sed -i 's/GL_FALSE/false/g' $file; done Then I manually added #include <stdbool.h> to fix compilation errors, and converted a few functions back to GLboolean that were used in core Mesa's function pointer table to avoid "incompatible pointer" warnings. Finally, I cleaned up some whitespace issues introduced by the change. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Chad Versace <chad@chad-versace.us> Acked-by: Paul Berry <stereotype441@gmail.com>
2011-10-07 12:26:50 -07:00
bool eot)
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
i965: Refactor Sandybridge implied move handling. This was open-coded in three different places, and more are necessary. Extract this into a function so it can be reused. Unfortunately, not all variations were the same: in particular, one set compression control and checked that the source register was not ARF_NULL. This seemed like a good idea, so all cases now do so.
2011-03-16 14:09:17 -07:00
gen6_resolve_implied_move(p, &src0, msg_reg_nr);
i965: ff sync message change for sandybridge
2010-09-17 14:23:48 +08:00
insn = next_insn(p, BRW_OPCODE_SEND);
i965: Add support for the instruction compression bits on gen6. Since the 8-wide first-quarter and 16-wide first-half have the same bit encoding, we now need to track "do you want instruction compression" in the compile state.
2010-12-03 11:49:29 -08:00
brw_set_dest(p, insn, dest);
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_src0(p, insn, src0);
brw_set_src1(p, insn, brw_imm_d(0));
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-14 18:00:06 -07:00
if (devinfo->gen < 6)
brw_inst_set_base_mrf(devinfo, insn, msg_reg_nr);
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
i965: Pass brw_compile pointer to brw_set_src[01]. This makes it symmetric with brw_set_dest, which is convenient, and will also allow for assertions to be made based off of intel->gen. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-05-10 16:51:12 -07:00
brw_set_ff_sync_message(p,
i965: Remove constant or ignored-by-hw args from FF sync message setup.
2010-05-13 22:15:34 -07:00
insn,
allocate,
response_length,
eot);
i965: add support for new chipsets 1. new PCI ids 2. fix some 3D commands on new chipset 3. fix send instruction on new chipset 4. new VUE vertex header 5. ff_sync message (added by Zou Nan Hai <nanhai.zou@intel.com>) 6. the offset in JMPI is in unit of 64bits on new chipset 7. new cube map layout
2009-07-13 10:48:43 +08:00
}
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
/**
* Emit the SEND instruction necessary to generate stream output data on Gen6
* (for transform feedback).
*
* If send_commit_msg is true, this is the last piece of stream output data
* from this thread, so send the data as a committed write. According to the
* Sandy Bridge PRM (volume 2 part 1, section 4.5.1):
*
* "Prior to End of Thread with a URB_WRITE, the kernel must ensure all
* writes are complete by sending the final write as a committed write."
*/
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_svb_write(struct brw_codegen *p,
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
struct brw_reg dest,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_reg_nr,
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
struct brw_reg src0,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned binding_table_index,
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
bool send_commit_msg)
{
i965: Let the caller of brw_set_dp_write/read_message control the target cache. brw_set_dp_read_message already had a target_cache argument, but its interpretation was rather convoluted (on Gen6 the render cache was used if the caller asked for it, otherwise it was ignored using the sampler cache instead), and the constant cache wasn't representable at all. brw_set_dp_write_message used the data cache on Gen7+ except for RENDER_TARGET_WRITE messages, in which case it would use the render cache. On Gen6 the render cache was always used. Instead of the above, provide the shared unit SFID that the caller expects will be used. Makes no functional changes. v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:36:16 +03:00
const struct gen_device_info *devinfo = p->devinfo;
const unsigned target_cache =
(devinfo->gen >= 7 ? GEN7_SFID_DATAPORT_DATA_CACHE :
devinfo->gen >= 6 ? GEN6_SFID_DATAPORT_RENDER_CACHE :
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
BRW_SFID_DATAPORT_WRITE);
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *insn;
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
gen6_resolve_implied_move(p, &src0, msg_reg_nr);
insn = next_insn(p, BRW_OPCODE_SEND);
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
brw_inst_set_sfid(devinfo, insn, target_cache);
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
brw_set_dest(p, insn, dest);
brw_set_src0(p, insn, src0);
intel/eu: Use descriptor constructors for dataport write messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:12:59 -07:00
brw_set_desc(p, insn,
brw_message_desc(devinfo, 1, send_commit_msg, true) |
brw_dp_write_desc(devinfo, binding_table_index,
0, /* msg_control: ignored */
GEN6_DATAPORT_WRITE_MESSAGE_STREAMED_VB_WRITE,
0, /* last_render_target: ignored */
send_commit_msg)); /* send_commit_msg */
i965 gen6: Initial implementation of transform feedback. This patch adds basic transform feedback capability for Gen6 hardware. This consists of several related pieces of functionality: (1) In gen6_sol.c, we set up binding table entries for use by transform feedback. We use one binding table entry per transform feedback varying (this allows us to avoid doing pointer arithmetic in the shader, since we can set up the binding table entries with the appropriate offsets and surface pitches to place each varying at the correct address). (2) In brw_context.c, we advertise the hardware capabilities, which are as follows: MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS 64 MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS 4 MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS 16 OpenGL 3.0 requires these values to be at least 64, 4, and 4, respectively. The reason we advertise a larger value than required for MAX_TRANSFORM_FEEDBACK_SEPARATE_COMPONENTS is that we have already set aside 64 binding table entries, so we might as well make them all available in both separate attribs and interleaved modes. (3) We set aside a single SVBI ("streamed vertex buffer index") for use by transform feedback. The hardware supports four independent SVBI's, but we only need one, since vertices are added to all transform feedback buffers at the same rate. Note: at the moment this index is reset to 0 only when the driver is initialized. It needs to be reset to 0 whenever BeginTransformFeedback() is called, and otherwise preserved. (4) In brw_gs_emit.c and brw_gs.c, we modify the geometry shader program to output transform feedback data as a side effect. (5) In gen6_gs_state.c, we configure the geometry shader stage to handle the SVBI pointer correctly. Note: ordering of vertices is not yet correct for triangle strips (alternate triangles are improperly oriented). This will be addressed in a future patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-11-28 06:55:01 -08:00
}
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
i965: Pass number of components explicitly to brw_untyped_atomic and _surface_read. And calculate the message response size based on the number of components rather than the other way around. This simplifies their interface somewhat and allows the caller to request a writeback message with more than one vector component in SIMD4x2 mode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 12:56:19 +02:00
static unsigned
intel/compiler: Silence unused parameter warning in brw_surface_payload_size src/intel/compiler/brw_eu_emit.c: In function ‘brw_surface_payload_size’: src/intel/compiler/brw_eu_emit.c:3070:46: warning: unused parameter ‘p’ [-Wunused-parameter] 3070 | brw_surface_payload_size(struct brw_codegen *p, | ~~~~~~~~~~~~~~~~~~~~^ Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6826>
2020-09-22 14:10:56 -07:00
brw_surface_payload_size(unsigned num_channels,
intel/fs: Take an explicit exec size in brw_surface_payload_size() Instead of magically falling back to SIMD8 for atomics and typed messages on Ivy Bridge, explicitly figure out the exec size and pass that into brw_surface_payload_size. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 13:40:31 -05:00
unsigned exec_size /**< 0 for SIMD4x2 */)
i965: Pass number of components explicitly to brw_untyped_atomic and _surface_read. And calculate the message response size based on the number of components rather than the other way around. This simplifies their interface somewhat and allows the caller to request a writeback message with more than one vector component in SIMD4x2 mode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 12:56:19 +02:00
{
intel/fs: Take an explicit exec size in brw_surface_payload_size() Instead of magically falling back to SIMD8 for atomics and typed messages on Ivy Bridge, explicitly figure out the exec size and pass that into brw_surface_payload_size. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 13:40:31 -05:00
if (exec_size == 0)
return 1; /* SIMD4x2 */
else if (exec_size <= 8)
i965: Pass number of components explicitly to brw_untyped_atomic and _surface_read. And calculate the message response size based on the number of components rather than the other way around. This simplifies their interface somewhat and allows the caller to request a writeback message with more than one vector component in SIMD4x2 mode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 12:56:19 +02:00
return num_channels;
intel/fs: Take an explicit exec size in brw_surface_payload_size() Instead of magically falling back to SIMD8 for atomics and typed messages on Ivy Bridge, explicitly figure out the exec size and pass that into brw_surface_payload_size. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 13:40:31 -05:00
else
return 2 * num_channels;
i965: Pass number of components explicitly to brw_untyped_atomic and _surface_read. And calculate the message response size based on the number of components rather than the other way around. This simplifies their interface somewhat and allows the caller to request a writeback message with more than one vector component in SIMD4x2 mode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 12:56:19 +02:00
}
i965/gen7: Implement code generation for untyped atomic instructions. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-09-11 14:01:50 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_untyped_atomic(struct brw_codegen *p,
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
struct brw_reg dst,
i965/fs: Use the GRF for UNTYPED_ATOMIC instructions Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-11 16:13:15 -07:00
struct brw_reg payload,
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
struct brw_reg surface,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned atomic_op,
unsigned msg_length,
intel/eu: Plumb header present bit to codegen helpers for HDC messages. This makes sure that the header-present bit of the message descriptor is in sync with the IR instruction fields, which gives the optimizer more control to avoid the overhead of setting up a message header when it's possible to do so. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-12-12 12:05:03 -08:00
bool response_expected,
bool header_present)
i965: Pass number of components explicitly to brw_untyped_atomic and _surface_read. And calculate the message response size based on the number of components rather than the other way around. This simplifies their interface somewhat and allows the caller to request a writeback message with more than one vector component in SIMD4x2 mode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 12:56:19 +02:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
const unsigned sfid = (devinfo->gen >= 8 || devinfo->is_haswell ?
HSW_SFID_DATAPORT_DATA_CACHE_1 :
GEN7_SFID_DATAPORT_DATA_CACHE);
intel/fs: Take an explicit exec size in brw_surface_payload_size() Instead of magically falling back to SIMD8 for atomics and typed messages on Ivy Bridge, explicitly figure out the exec size and pass that into brw_surface_payload_size. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 13:40:31 -05:00
const bool align1 = brw_get_default_access_mode(p) == BRW_ALIGN_1;
/* SIMD4x2 untyped atomic instructions only exist on HSW+ */
const bool has_simd4x2 = devinfo->gen >= 8 || devinfo->is_haswell;
const unsigned exec_size = align1 ? 1 << brw_get_default_exec_size(p) :
has_simd4x2 ? 0 : 8;
const unsigned response_length =
intel/compiler: Silence unused parameter warning in brw_surface_payload_size src/intel/compiler/brw_eu_emit.c: In function ‘brw_surface_payload_size’: src/intel/compiler/brw_eu_emit.c:3070:46: warning: unused parameter ‘p’ [-Wunused-parameter] 3070 | brw_surface_payload_size(struct brw_codegen *p, | ~~~~~~~~~~~~~~~~~~~~^ Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6826>
2020-09-22 14:10:56 -07:00
brw_surface_payload_size(response_expected, exec_size);
intel/eu: Provide single descriptor argument to brw_send_indirect_surface_message(). Instead of the current message_len, response_len and header_present arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:19:49 -07:00
const unsigned desc =
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
brw_message_desc(devinfo, msg_length, response_length, header_present) |
intel/eu: Rework surface descriptor helpers This commit pulls the surface descriptor helpers out into brw_eu.h and makes them no longer depend on the codegen infrastructure. This should allow us to use them directly from the IR code instead of the generator. This change is unfortunately less mechanical than perhaps one would like but it should be fairly straightforward. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 16:09:30 -05:00
brw_dp_untyped_atomic_desc(devinfo, exec_size, atomic_op,
response_expected);
i965: Mask out unused Align16 components in brw_untyped_atomic. This is currently not a problem because the vec4 visitor happens to mask out unused components from the destination, but it might become an issue when we start using atomics without writeback message. In any case it seems sensible to set it again here because the consequences of setting the wrong writemask (random graphics memory corruption) are difficult to debug and can easily go unnoticed. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 13:58:21 +02:00
/* Mask out unused components -- This is especially important in Align16
* mode on generations that don't have native support for SIMD4x2 atomics,
* because unused but enabled components will cause the dataport to perform
* additional atomic operations on the addresses that happen to be in the
* uninitialized Y, Z and W coordinates of the payload.
*/
const unsigned mask = align1 ? WRITEMASK_XYZW : WRITEMASK_X;
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
brw_send_indirect_surface_message(p, sfid, brw_writemask(dst, mask),
payload, surface, desc);
i965/gen7: Implement code generation for untyped atomic instructions. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-09-11 14:01:50 -07:00
}
i965/gen7: Implement code generation for untyped surface read instructions.
2013-09-11 14:03:13 -07:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_untyped_surface_read(struct brw_codegen *p,
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
struct brw_reg dst,
struct brw_reg payload,
struct brw_reg surface,
i965: Don't use GL types in files shared with intel-gpu-tools. sed -i -e 's/GLuint/unsigned/g' -e 's/GLint/int/g' \ -e 's/GLfloat/float/g' -e 's/GLubyte/uint8_t/g' \ -e 's/GLshort/int16_t/g' \ brw_eu* brw_disasm.c brw_structs.h Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-25 15:51:24 -08:00
unsigned msg_length,
i965: Pass number of components explicitly to brw_untyped_atomic and _surface_read. And calculate the message response size based on the number of components rather than the other way around. This simplifies their interface somewhat and allows the caller to request a writeback message with more than one vector component in SIMD4x2 mode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 12:56:19 +02:00
unsigned num_channels)
i965/gen7: Implement code generation for untyped surface read instructions.
2013-09-11 14:03:13 -07:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
const unsigned sfid = (devinfo->gen >= 8 || devinfo->is_haswell ?
HSW_SFID_DATAPORT_DATA_CACHE_1 :
GEN7_SFID_DATAPORT_DATA_CACHE);
intel/fs: Take an explicit exec size in brw_surface_payload_size() Instead of magically falling back to SIMD8 for atomics and typed messages on Ivy Bridge, explicitly figure out the exec size and pass that into brw_surface_payload_size. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 13:40:31 -05:00
const bool align1 = brw_get_default_access_mode(p) == BRW_ALIGN_1;
const unsigned exec_size = align1 ? 1 << brw_get_default_exec_size(p) : 0;
intel/eu: Provide single descriptor argument to brw_send_indirect_surface_message(). Instead of the current message_len, response_len and header_present arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:19:49 -07:00
const unsigned response_length =
intel/compiler: Silence unused parameter warning in brw_surface_payload_size src/intel/compiler/brw_eu_emit.c: In function ‘brw_surface_payload_size’: src/intel/compiler/brw_eu_emit.c:3070:46: warning: unused parameter ‘p’ [-Wunused-parameter] 3070 | brw_surface_payload_size(struct brw_codegen *p, | ~~~~~~~~~~~~~~~~~~~~^ Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6826>
2020-09-22 14:10:56 -07:00
brw_surface_payload_size(num_channels, exec_size);
intel/eu: Provide single descriptor argument to brw_send_indirect_surface_message(). Instead of the current message_len, response_len and header_present arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:19:49 -07:00
const unsigned desc =
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
brw_message_desc(devinfo, msg_length, response_length, false) |
intel/eu: Rework surface descriptor helpers This commit pulls the surface descriptor helpers out into brw_eu.h and makes them no longer depend on the codegen infrastructure. This should allow us to use them directly from the IR code instead of the generator. This change is unfortunately less mechanical than perhaps one would like but it should be fairly straightforward. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 16:09:30 -05:00
brw_dp_untyped_surface_rw_desc(devinfo, exec_size, num_channels, false);
i965/gen7: Implement code generation for untyped surface read instructions.
2013-09-11 14:03:13 -07:00
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
brw_send_indirect_surface_message(p, sfid, dst, payload, surface, desc);
i965: Add untyped surface write opcode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:24:14 +03:00
}
void
brw_untyped_surface_write(struct brw_codegen *p,
struct brw_reg payload,
struct brw_reg surface,
unsigned msg_length,
intel/eu: Plumb header present bit to codegen helpers for HDC messages. This makes sure that the header-present bit of the message descriptor is in sync with the IR instruction fields, which gives the optimizer more control to avoid the overhead of setting up a message header when it's possible to do so. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-12-12 12:05:03 -08:00
unsigned num_channels,
bool header_present)
i965: Add untyped surface write opcode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:24:14 +03:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add untyped surface write opcode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:24:14 +03:00
const unsigned sfid = (devinfo->gen >= 8 || devinfo->is_haswell ?
HSW_SFID_DATAPORT_DATA_CACHE_1 :
GEN7_SFID_DATAPORT_DATA_CACHE);
intel/eu: Add has_simd4x2 bools to surface_write functions Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 14:15:58 -05:00
const bool align1 = brw_get_default_access_mode(p) == BRW_ALIGN_1;
/* SIMD4x2 untyped surface write instructions only exist on HSW+ */
const bool has_simd4x2 = devinfo->gen >= 8 || devinfo->is_haswell;
intel/eu: Rework surface descriptor helpers This commit pulls the surface descriptor helpers out into brw_eu.h and makes them no longer depend on the codegen infrastructure. This should allow us to use them directly from the IR code instead of the generator. This change is unfortunately less mechanical than perhaps one would like but it should be fairly straightforward. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 16:09:30 -05:00
const unsigned exec_size = align1 ? 1 << brw_get_default_exec_size(p) :
has_simd4x2 ? 0 : 8;
intel/eu: Provide single descriptor argument to brw_send_indirect_surface_message(). Instead of the current message_len, response_len and header_present arguments. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:19:49 -07:00
const unsigned desc =
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
brw_message_desc(devinfo, msg_length, 0, header_present) |
intel/eu: Rework surface descriptor helpers This commit pulls the surface descriptor helpers out into brw_eu.h and makes them no longer depend on the codegen infrastructure. This should allow us to use them directly from the IR code instead of the generator. This change is unfortunately less mechanical than perhaps one would like but it should be fairly straightforward. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 16:09:30 -05:00
brw_dp_untyped_surface_rw_desc(devinfo, exec_size, num_channels, true);
i965: Add untyped surface write opcode. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-23 14:24:14 +03:00
/* Mask out unused components -- See comment in brw_untyped_atomic(). */
intel/eu: Add has_simd4x2 bools to surface_write functions Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-11-01 14:15:58 -05:00
const unsigned mask = !has_simd4x2 && !align1 ? WRITEMASK_X : WRITEMASK_XYZW;
i965: Add typed surface access opcodes. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:28:25 +03:00
intel/eu: Use descriptor constructors for dataport typed surface messages. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:27:06 -07:00
brw_send_indirect_surface_message(p, sfid, brw_writemask(brw_null_reg(), mask),
payload, surface, desc);
i965/gen7: Implement code generation for untyped surface read instructions.
2013-09-11 14:03:13 -07:00
}
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
static void
brw_set_memory_fence_message(struct brw_codegen *p,
struct brw_inst *insn,
enum brw_message_target sfid,
intel/fs: Add support for SLM fence in Gen11 Gen11 SLM is not on L3 anymore, so now the hardware has two separate fences. Add a way to control which fence types to use. At this time, we don't have enough information in NIR to control the visibility of the memory being fenced, so for now be conservative and assume that fences will need a stall. With more information later we'll be able to reduce those. Fixes Vulkan CTS tests in ICL: dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_nonlocal.workgroup.guard_local.buffer.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_local.buffer.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_local.image.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.buffer.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.image.guard_nonlocal.workgroup.comp The whole set of supported tests in dEQP-VK.memory_model.* group should be passing in ICL now. v2: Pass BTI around instead of having an enum. (Jason) Emit two SHADER_OPCODE_MEMORY_FENCE instead of one that gets transformed into two. (Jason) List tests fixed. (Lionel) v3: For clarity, split the decision of which fences to emit from the emission code. (Jason) Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
2019-07-10 12:02:23 -07:00
bool commit_enable,
unsigned bti)
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_set_desc(p, insn, brw_message_desc(
devinfo, 1, (commit_enable ? 1 : 0), true));
brw_inst_set_sfid(devinfo, insn, sfid);
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
switch (sfid) {
case GEN6_SFID_DATAPORT_RENDER_CACHE:
brw_inst_set_dp_msg_type(devinfo, insn, GEN7_DATAPORT_RC_MEMORY_FENCE);
break;
case GEN7_SFID_DATAPORT_DATA_CACHE:
brw_inst_set_dp_msg_type(devinfo, insn, GEN7_DATAPORT_DC_MEMORY_FENCE);
break;
default:
unreachable("Not reached");
}
if (commit_enable)
brw_inst_set_dp_msg_control(devinfo, insn, 1 << 5);
intel/fs: Add support for SLM fence in Gen11 Gen11 SLM is not on L3 anymore, so now the hardware has two separate fences. Add a way to control which fence types to use. At this time, we don't have enough information in NIR to control the visibility of the memory being fenced, so for now be conservative and assume that fences will need a stall. With more information later we'll be able to reduce those. Fixes Vulkan CTS tests in ICL: dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_nonlocal.workgroup.guard_local.buffer.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_local.buffer.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_local.image.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.buffer.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.image.guard_nonlocal.workgroup.comp The whole set of supported tests in dEQP-VK.memory_model.* group should be passing in ICL now. v2: Pass BTI around instead of having an enum. (Jason) Emit two SHADER_OPCODE_MEMORY_FENCE instead of one that gets transformed into two. (Jason) List tests fixed. (Lionel) v3: For clarity, split the decision of which fences to emit from the emission code. (Jason) Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
2019-07-10 12:02:23 -07:00
assert(devinfo->gen >= 11 || bti == 0);
brw_inst_set_binding_table_index(devinfo, insn, bti);
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
}
intel/fs,vec4: Pull stall logic for memory fences up into the IR Instead of emitting the stall MOV "inside" the SHADER_OPCODE_MEMORY_FENCE generation, use the scheduling fences when creating the IR. For IvyBridge, every (data cache) fence is accompained by a render cache fence, that now is explicit in the IR, two SHADER_OPCODE_MEMORY_FENCEs are emitted (with different SFIDs). Because Begin and End interlock intrinsics are effectively memory barriers, move its handling alongside the other memory barrier intrinsics. The SHADER_OPCODE_INTERLOCK is still used to distinguish if we are going to use a SENDC (for Begin) or regular SEND (for End). This change is a preparation to allow emitting both SENDs in Gen11+ before we can stall on them. Shader-db results for IVB (i965): total instructions in shared programs: 11971190 -> 11971200 (<.01%) instructions in affected programs: 11482 -> 11492 (0.09%) helped: 0 HURT: 8 HURT stats (abs) min: 1 max: 3 x̄: 1.25 x̃: 1 HURT stats (rel) min: 0.03% max: 0.50% x̄: 0.14% x̃: 0.10% 95% mean confidence interval for instructions value: 0.66 1.84 95% mean confidence interval for instructions %-change: 0.01% 0.27% Instructions are HURT. Unlike the previous code, that used the `mov g1 g2` trick to force both `g1` and `g2` to stall, the scheduling fence will generate `mov null g1` and `mov null g2`. During review it was decided it was not worth keeping the special codepath for the small effect will have. Shader-db results for HSW (i965), BDW and SKL don't have a change on instruction count, but do report changes in cycles count, showing SKL results below total cycles in shared programs: 341738444 -> 341710570 (<.01%) cycles in affected programs: 7240002 -> 7212128 (-0.38%) helped: 46 HURT: 5 helped stats (abs) min: 14 max: 1940 x̄: 676.22 x̃: 154 helped stats (rel) min: <.01% max: 2.62% x̄: 1.28% x̃: 0.95% HURT stats (abs) min: 2 max: 1768 x̄: 646.40 x̃: 362 HURT stats (rel) min: <.01% max: 0.83% x̄: 0.28% x̃: 0.08% 95% mean confidence interval for cycles value: -777.71 -315.38 95% mean confidence interval for cycles %-change: -1.42% -0.83% Cycles are helped. This seems to be the effect of allocating two registers separatedly instead of a single one with size 2, which causes different register allocation, affecting the cycle estimates. while ICL also has not change on instruction count but report changes negative changes in cycles total cycles in shared programs: 352665369 -> 352707484 (0.01%) cycles in affected programs: 9608288 -> 9650403 (0.44%) helped: 4 HURT: 104 helped stats (abs) min: 24 max: 128 x̄: 88.50 x̃: 101 helped stats (rel) min: <.01% max: 0.85% x̄: 0.46% x̃: 0.49% HURT stats (abs) min: 2 max: 2016 x̄: 408.36 x̃: 48 HURT stats (rel) min: <.01% max: 3.31% x̄: 0.88% x̃: 0.45% 95% mean confidence interval for cycles value: 256.67 523.24 95% mean confidence interval for cycles %-change: 0.63% 1.03% Cycles are HURT. AFAICT this is the result of the case above. Shader-db results for TGL have similar cycles result as ICL, but also affect instructions total instructions in shared programs: 17690586 -> 17690597 (<.01%) instructions in affected programs: 64617 -> 64628 (0.02%) helped: 55 HURT: 32 helped stats (abs) min: 1 max: 16 x̄: 4.13 x̃: 3 helped stats (rel) min: 0.05% max: 2.78% x̄: 0.86% x̃: 0.74% HURT stats (abs) min: 1 max: 65 x̄: 7.44 x̃: 2 HURT stats (rel) min: 0.05% max: 4.58% x̄: 1.13% x̃: 0.69% 95% mean confidence interval for instructions value: -2.03 2.28 95% mean confidence interval for instructions %-change: -0.41% 0.15% Inconclusive result (value mean confidence interval includes 0). Now that more is done in the IR, more dependencies are visible and more SWSB annotations are emitted. Mixed with different register allocation decisions like above, some shaders will see more `sync nops` while others able to avoid them. Most of the new `sync nops` are also redundant and could be dropped, which will be fixed in a separate change. Reviewed-by: Francisco Jerez <currojerez@riseup.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3278>
2020-01-17 15:07:44 -08:00
void
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
brw_memory_fence(struct brw_codegen *p,
i965: Add ARB_fragment_shader_interlock support. Adds suppport for ARB_fragment_shader_interlock. We achieve the interlock and fragment ordering by issuing a memory fence via sendc. Signed-off-by: Plamena Manolova <plamena.manolova@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2018-04-27 15:06:56 +01:00
struct brw_reg dst,
intel/fs,vec4: Use g0 as the header for MFENCE We set header_present but then pass it some random garbage. Give it g0 instead. I'm not actually sure this does anything but g0 is the usual header data and this is what the windows driver does so it seems like a good idea. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-05-22 12:20:01 -05:00
struct brw_reg src,
intel/fs: Do a stalling MFENCE in endInvocationInterlock() Fixes: 939312702e "i965: Add ARB_fragment_shader_interlock support" Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-05-22 12:36:17 -05:00
enum opcode send_op,
intel/fs,vec4: Pull stall logic for memory fences up into the IR Instead of emitting the stall MOV "inside" the SHADER_OPCODE_MEMORY_FENCE generation, use the scheduling fences when creating the IR. For IvyBridge, every (data cache) fence is accompained by a render cache fence, that now is explicit in the IR, two SHADER_OPCODE_MEMORY_FENCEs are emitted (with different SFIDs). Because Begin and End interlock intrinsics are effectively memory barriers, move its handling alongside the other memory barrier intrinsics. The SHADER_OPCODE_INTERLOCK is still used to distinguish if we are going to use a SENDC (for Begin) or regular SEND (for End). This change is a preparation to allow emitting both SENDs in Gen11+ before we can stall on them. Shader-db results for IVB (i965): total instructions in shared programs: 11971190 -> 11971200 (<.01%) instructions in affected programs: 11482 -> 11492 (0.09%) helped: 0 HURT: 8 HURT stats (abs) min: 1 max: 3 x̄: 1.25 x̃: 1 HURT stats (rel) min: 0.03% max: 0.50% x̄: 0.14% x̃: 0.10% 95% mean confidence interval for instructions value: 0.66 1.84 95% mean confidence interval for instructions %-change: 0.01% 0.27% Instructions are HURT. Unlike the previous code, that used the `mov g1 g2` trick to force both `g1` and `g2` to stall, the scheduling fence will generate `mov null g1` and `mov null g2`. During review it was decided it was not worth keeping the special codepath for the small effect will have. Shader-db results for HSW (i965), BDW and SKL don't have a change on instruction count, but do report changes in cycles count, showing SKL results below total cycles in shared programs: 341738444 -> 341710570 (<.01%) cycles in affected programs: 7240002 -> 7212128 (-0.38%) helped: 46 HURT: 5 helped stats (abs) min: 14 max: 1940 x̄: 676.22 x̃: 154 helped stats (rel) min: <.01% max: 2.62% x̄: 1.28% x̃: 0.95% HURT stats (abs) min: 2 max: 1768 x̄: 646.40 x̃: 362 HURT stats (rel) min: <.01% max: 0.83% x̄: 0.28% x̃: 0.08% 95% mean confidence interval for cycles value: -777.71 -315.38 95% mean confidence interval for cycles %-change: -1.42% -0.83% Cycles are helped. This seems to be the effect of allocating two registers separatedly instead of a single one with size 2, which causes different register allocation, affecting the cycle estimates. while ICL also has not change on instruction count but report changes negative changes in cycles total cycles in shared programs: 352665369 -> 352707484 (0.01%) cycles in affected programs: 9608288 -> 9650403 (0.44%) helped: 4 HURT: 104 helped stats (abs) min: 24 max: 128 x̄: 88.50 x̃: 101 helped stats (rel) min: <.01% max: 0.85% x̄: 0.46% x̃: 0.49% HURT stats (abs) min: 2 max: 2016 x̄: 408.36 x̃: 48 HURT stats (rel) min: <.01% max: 3.31% x̄: 0.88% x̃: 0.45% 95% mean confidence interval for cycles value: 256.67 523.24 95% mean confidence interval for cycles %-change: 0.63% 1.03% Cycles are HURT. AFAICT this is the result of the case above. Shader-db results for TGL have similar cycles result as ICL, but also affect instructions total instructions in shared programs: 17690586 -> 17690597 (<.01%) instructions in affected programs: 64617 -> 64628 (0.02%) helped: 55 HURT: 32 helped stats (abs) min: 1 max: 16 x̄: 4.13 x̃: 3 helped stats (rel) min: 0.05% max: 2.78% x̄: 0.86% x̃: 0.74% HURT stats (abs) min: 1 max: 65 x̄: 7.44 x̃: 2 HURT stats (rel) min: 0.05% max: 4.58% x̄: 1.13% x̃: 0.69% 95% mean confidence interval for instructions value: -2.03 2.28 95% mean confidence interval for instructions %-change: -0.41% 0.15% Inconclusive result (value mean confidence interval includes 0). Now that more is done in the IR, more dependencies are visible and more SWSB annotations are emitted. Mixed with different register allocation decisions like above, some shaders will see more `sync nops` while others able to avoid them. Most of the new `sync nops` are also redundant and could be dropped, which will be fixed in a separate change. Reviewed-by: Francisco Jerez <currojerez@riseup.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3278>
2020-01-17 15:07:44 -08:00
enum brw_message_target sfid,
bool commit_enable,
intel/fs: Add support for SLM fence in Gen11 Gen11 SLM is not on L3 anymore, so now the hardware has two separate fences. Add a way to control which fence types to use. At this time, we don't have enough information in NIR to control the visibility of the memory being fenced, so for now be conservative and assume that fences will need a stall. With more information later we'll be able to reduce those. Fixes Vulkan CTS tests in ICL: dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_nonlocal.workgroup.guard_local.buffer.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_local.buffer.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.device.payload_local.image.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.buffer.guard_nonlocal.workgroup.comp dEQP-VK.memory_model.message_passing.core11.u32.coherent.fence_fence.atomicwrite.workgroup.payload_local.image.guard_nonlocal.workgroup.comp The whole set of supported tests in dEQP-VK.memory_model.* group should be passing in ICL now. v2: Pass BTI around instead of having an enum. (Jason) Emit two SHADER_OPCODE_MEMORY_FENCE instead of one that gets transformed into two. (Jason) List tests fixed. (Lionel) v3: For clarity, split the decision of which fences to emit from the emission code. (Jason) Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
2019-07-10 12:02:23 -07:00
unsigned bti)
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
intel/fs,vec4: Use g0 as the header for MFENCE We set header_present but then pass it some random garbage. Give it g0 instead. I'm not actually sure this does anything but g0 is the usual header data and this is what the windows driver does so it seems like a good idea. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-05-22 12:20:01 -05:00
dst = retype(vec1(dst), BRW_REGISTER_TYPE_UW);
src = retype(vec1(src), BRW_REGISTER_TYPE_UD);
i965/eu: Set execution size explicitly for memory fence send message. We don't want to emit a 32-wide send message in 32-wide programs. The memory fence message should have the same effect regardless of the execution size (as long as it's valid) so just set it to one. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-04-25 19:20:12 -07:00
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
/* Set dst as destination for dependency tracking, the MEMORY_FENCE
* message doesn't write anything back.
*/
intel/fs,vec4: Pull stall logic for memory fences up into the IR Instead of emitting the stall MOV "inside" the SHADER_OPCODE_MEMORY_FENCE generation, use the scheduling fences when creating the IR. For IvyBridge, every (data cache) fence is accompained by a render cache fence, that now is explicit in the IR, two SHADER_OPCODE_MEMORY_FENCEs are emitted (with different SFIDs). Because Begin and End interlock intrinsics are effectively memory barriers, move its handling alongside the other memory barrier intrinsics. The SHADER_OPCODE_INTERLOCK is still used to distinguish if we are going to use a SENDC (for Begin) or regular SEND (for End). This change is a preparation to allow emitting both SENDs in Gen11+ before we can stall on them. Shader-db results for IVB (i965): total instructions in shared programs: 11971190 -> 11971200 (<.01%) instructions in affected programs: 11482 -> 11492 (0.09%) helped: 0 HURT: 8 HURT stats (abs) min: 1 max: 3 x̄: 1.25 x̃: 1 HURT stats (rel) min: 0.03% max: 0.50% x̄: 0.14% x̃: 0.10% 95% mean confidence interval for instructions value: 0.66 1.84 95% mean confidence interval for instructions %-change: 0.01% 0.27% Instructions are HURT. Unlike the previous code, that used the `mov g1 g2` trick to force both `g1` and `g2` to stall, the scheduling fence will generate `mov null g1` and `mov null g2`. During review it was decided it was not worth keeping the special codepath for the small effect will have. Shader-db results for HSW (i965), BDW and SKL don't have a change on instruction count, but do report changes in cycles count, showing SKL results below total cycles in shared programs: 341738444 -> 341710570 (<.01%) cycles in affected programs: 7240002 -> 7212128 (-0.38%) helped: 46 HURT: 5 helped stats (abs) min: 14 max: 1940 x̄: 676.22 x̃: 154 helped stats (rel) min: <.01% max: 2.62% x̄: 1.28% x̃: 0.95% HURT stats (abs) min: 2 max: 1768 x̄: 646.40 x̃: 362 HURT stats (rel) min: <.01% max: 0.83% x̄: 0.28% x̃: 0.08% 95% mean confidence interval for cycles value: -777.71 -315.38 95% mean confidence interval for cycles %-change: -1.42% -0.83% Cycles are helped. This seems to be the effect of allocating two registers separatedly instead of a single one with size 2, which causes different register allocation, affecting the cycle estimates. while ICL also has not change on instruction count but report changes negative changes in cycles total cycles in shared programs: 352665369 -> 352707484 (0.01%) cycles in affected programs: 9608288 -> 9650403 (0.44%) helped: 4 HURT: 104 helped stats (abs) min: 24 max: 128 x̄: 88.50 x̃: 101 helped stats (rel) min: <.01% max: 0.85% x̄: 0.46% x̃: 0.49% HURT stats (abs) min: 2 max: 2016 x̄: 408.36 x̃: 48 HURT stats (rel) min: <.01% max: 3.31% x̄: 0.88% x̃: 0.45% 95% mean confidence interval for cycles value: 256.67 523.24 95% mean confidence interval for cycles %-change: 0.63% 1.03% Cycles are HURT. AFAICT this is the result of the case above. Shader-db results for TGL have similar cycles result as ICL, but also affect instructions total instructions in shared programs: 17690586 -> 17690597 (<.01%) instructions in affected programs: 64617 -> 64628 (0.02%) helped: 55 HURT: 32 helped stats (abs) min: 1 max: 16 x̄: 4.13 x̃: 3 helped stats (rel) min: 0.05% max: 2.78% x̄: 0.86% x̃: 0.74% HURT stats (abs) min: 1 max: 65 x̄: 7.44 x̃: 2 HURT stats (rel) min: 0.05% max: 4.58% x̄: 1.13% x̃: 0.69% 95% mean confidence interval for instructions value: -2.03 2.28 95% mean confidence interval for instructions %-change: -0.41% 0.15% Inconclusive result (value mean confidence interval includes 0). Now that more is done in the IR, more dependencies are visible and more SWSB annotations are emitted. Mixed with different register allocation decisions like above, some shaders will see more `sync nops` while others able to avoid them. Most of the new `sync nops` are also redundant and could be dropped, which will be fixed in a separate change. Reviewed-by: Francisco Jerez <currojerez@riseup.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3278>
2020-01-17 15:07:44 -08:00
struct brw_inst *insn = next_insn(p, send_op);
brw_inst_set_mask_control(devinfo, insn, BRW_MASK_DISABLE);
brw_inst_set_exec_size(devinfo, insn, BRW_EXECUTE_1);
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
brw_set_dest(p, insn, dst);
intel/fs,vec4: Use g0 as the header for MFENCE We set header_present but then pass it some random garbage. Give it g0 instead. I'm not actually sure this does anything but g0 is the usual header data and this is what the windows driver does so it seems like a good idea. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-05-22 12:20:01 -05:00
brw_set_src0(p, insn, src);
intel/fs,vec4: Pull stall logic for memory fences up into the IR Instead of emitting the stall MOV "inside" the SHADER_OPCODE_MEMORY_FENCE generation, use the scheduling fences when creating the IR. For IvyBridge, every (data cache) fence is accompained by a render cache fence, that now is explicit in the IR, two SHADER_OPCODE_MEMORY_FENCEs are emitted (with different SFIDs). Because Begin and End interlock intrinsics are effectively memory barriers, move its handling alongside the other memory barrier intrinsics. The SHADER_OPCODE_INTERLOCK is still used to distinguish if we are going to use a SENDC (for Begin) or regular SEND (for End). This change is a preparation to allow emitting both SENDs in Gen11+ before we can stall on them. Shader-db results for IVB (i965): total instructions in shared programs: 11971190 -> 11971200 (<.01%) instructions in affected programs: 11482 -> 11492 (0.09%) helped: 0 HURT: 8 HURT stats (abs) min: 1 max: 3 x̄: 1.25 x̃: 1 HURT stats (rel) min: 0.03% max: 0.50% x̄: 0.14% x̃: 0.10% 95% mean confidence interval for instructions value: 0.66 1.84 95% mean confidence interval for instructions %-change: 0.01% 0.27% Instructions are HURT. Unlike the previous code, that used the `mov g1 g2` trick to force both `g1` and `g2` to stall, the scheduling fence will generate `mov null g1` and `mov null g2`. During review it was decided it was not worth keeping the special codepath for the small effect will have. Shader-db results for HSW (i965), BDW and SKL don't have a change on instruction count, but do report changes in cycles count, showing SKL results below total cycles in shared programs: 341738444 -> 341710570 (<.01%) cycles in affected programs: 7240002 -> 7212128 (-0.38%) helped: 46 HURT: 5 helped stats (abs) min: 14 max: 1940 x̄: 676.22 x̃: 154 helped stats (rel) min: <.01% max: 2.62% x̄: 1.28% x̃: 0.95% HURT stats (abs) min: 2 max: 1768 x̄: 646.40 x̃: 362 HURT stats (rel) min: <.01% max: 0.83% x̄: 0.28% x̃: 0.08% 95% mean confidence interval for cycles value: -777.71 -315.38 95% mean confidence interval for cycles %-change: -1.42% -0.83% Cycles are helped. This seems to be the effect of allocating two registers separatedly instead of a single one with size 2, which causes different register allocation, affecting the cycle estimates. while ICL also has not change on instruction count but report changes negative changes in cycles total cycles in shared programs: 352665369 -> 352707484 (0.01%) cycles in affected programs: 9608288 -> 9650403 (0.44%) helped: 4 HURT: 104 helped stats (abs) min: 24 max: 128 x̄: 88.50 x̃: 101 helped stats (rel) min: <.01% max: 0.85% x̄: 0.46% x̃: 0.49% HURT stats (abs) min: 2 max: 2016 x̄: 408.36 x̃: 48 HURT stats (rel) min: <.01% max: 3.31% x̄: 0.88% x̃: 0.45% 95% mean confidence interval for cycles value: 256.67 523.24 95% mean confidence interval for cycles %-change: 0.63% 1.03% Cycles are HURT. AFAICT this is the result of the case above. Shader-db results for TGL have similar cycles result as ICL, but also affect instructions total instructions in shared programs: 17690586 -> 17690597 (<.01%) instructions in affected programs: 64617 -> 64628 (0.02%) helped: 55 HURT: 32 helped stats (abs) min: 1 max: 16 x̄: 4.13 x̃: 3 helped stats (rel) min: 0.05% max: 2.78% x̄: 0.86% x̃: 0.74% HURT stats (abs) min: 1 max: 65 x̄: 7.44 x̃: 2 HURT stats (rel) min: 0.05% max: 4.58% x̄: 1.13% x̃: 0.69% 95% mean confidence interval for instructions value: -2.03 2.28 95% mean confidence interval for instructions %-change: -0.41% 0.15% Inconclusive result (value mean confidence interval includes 0). Now that more is done in the IR, more dependencies are visible and more SWSB annotations are emitted. Mixed with different register allocation decisions like above, some shaders will see more `sync nops` while others able to avoid them. Most of the new `sync nops` are also redundant and could be dropped, which will be fixed in a separate change. Reviewed-by: Francisco Jerez <currojerez@riseup.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3278>
2020-01-17 15:07:44 -08:00
brw_set_memory_fence_message(p, insn, sfid, commit_enable, bti);
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:30:28 +03:00
}
i965: add low-level support for send to pixel interpolator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-17 21:47:22 +13:00
void
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
brw_pixel_interpolator_query(struct brw_codegen *p,
i965: add low-level support for send to pixel interpolator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-17 21:47:22 +13:00
struct brw_reg dest,
struct brw_reg mrf,
bool noperspective,
unsigned mode,
i965/fs: Handle non-const sample number in interpolateAtSample If a non-const sample number is given to interpolateAtSample it will now generate an indirect send message with the sample ID similar to how non-const sampler array indexing works. Previously non-const values were ignored and instead it ended up using a constant 0 value. The generator will try to determine if the sample ID is dynamically uniform via nir_src_is_dynamically_uniform. If not it will query the pixel interpolator in a loop, once for each different live sample number. The next live sample number is found using emit_uniformize. If multiple live channels have the same sample number then they will be handled in a single iteration of the loop. The loop is necessary because the indirect send message doesn't seem to have a way to specify a different value for each fragment. This fixes the following two Piglit tests: arb_gpu_shader5-interpolateAtSample-nonconst arb_gpu_shader5-interpolateAtSample-dynamically-nonuniform v2: Handle dynamically non-uniform sample ids. v3: Remove the BREAK instruction and predicate the WHILE directly. Make the tokens arrays const. (Matt Turner) v4: Iterate over the live channels instead of each possible sample number. v5: Don't special case immediate values in brw_pixel_interpolator_query. Make a better wrapper for the function to set up the PI send instruction. Ensure that the SHL instructions are scalar. (Francisco Jerez). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-07-17 14:40:03 +01:00
struct brw_reg data,
i965: add low-level support for send to pixel interpolator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-17 21:47:22 +13:00
unsigned msg_length,
unsigned response_length)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
const uint16_t exec_size = brw_get_default_exec_size(p);
intel/eu: Use descriptor constructors for pixel interpolator messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:16:16 -07:00
const unsigned slot_group = brw_get_default_group(p) / 16;
const unsigned simd_mode = (exec_size == BRW_EXECUTE_16);
const unsigned desc =
brw_message_desc(devinfo, msg_length, response_length, false) |
brw_pixel_interp_desc(devinfo, mode, noperspective, simd_mode,
slot_group);
i965: add low-level support for send to pixel interpolator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-17 21:47:22 +13:00
i965/fs: Handle non-const sample number in interpolateAtSample If a non-const sample number is given to interpolateAtSample it will now generate an indirect send message with the sample ID similar to how non-const sampler array indexing works. Previously non-const values were ignored and instead it ended up using a constant 0 value. The generator will try to determine if the sample ID is dynamically uniform via nir_src_is_dynamically_uniform. If not it will query the pixel interpolator in a loop, once for each different live sample number. The next live sample number is found using emit_uniformize. If multiple live channels have the same sample number then they will be handled in a single iteration of the loop. The loop is necessary because the indirect send message doesn't seem to have a way to specify a different value for each fragment. This fixes the following two Piglit tests: arb_gpu_shader5-interpolateAtSample-nonconst arb_gpu_shader5-interpolateAtSample-dynamically-nonuniform v2: Handle dynamically non-uniform sample ids. v3: Remove the BREAK instruction and predicate the WHILE directly. Make the tokens arrays const. (Matt Turner) v4: Iterate over the live channels instead of each possible sample number. v5: Don't special case immediate values in brw_pixel_interpolator_query. Make a better wrapper for the function to set up the PI send instruction. Ensure that the SHL instructions are scalar. (Francisco Jerez). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-07-17 14:40:03 +01:00
/* brw_send_indirect_message will automatically use a direct send message
* if data is actually immediate.
*/
intel/eu: Use descriptor constructors for pixel interpolator messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-07-09 16:16:16 -07:00
brw_send_indirect_message(p,
GEN7_SFID_PIXEL_INTERPOLATOR,
dest,
mrf,
vec1(data),
intel/eu: Add an EOT parameter to send_indirect_[split]_message For split indirect sends we have to put the EOT parameter in the extended descriptor as well as the instruction itself so just calling brw_inst_set_eot is insufficient. Moving the EOT handling handling into the send_indirect_[split]_message helper lets us handle it properly.
2019-02-07 17:45:51 -06:00
desc,
false);
i965: add low-level support for send to pixel interpolator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-17 21:47:22 +13:00
}
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
void
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
brw_find_live_channel(struct brw_codegen *p, struct brw_reg dst,
struct brw_reg mask)
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
const unsigned exec_size = 1 << brw_get_default_exec_size(p);
const unsigned qtr_control = brw_get_default_group(p) / 8;
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
brw_inst *inst;
assert(devinfo->gen >= 7);
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
assert(mask.type == BRW_REGISTER_TYPE_UD);
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
brw_push_insn_state(p);
intel/compiler: Reset default flag register in brw_find_live_channel() emit_uniformize() emits SHADER_OPCODE_FIND_LIVE_CHANNEL with its flag_subreg set, so that the IR knows which flag is accessed. However the flag is only used on Gen7 in Align1 mode. To avoid setting unnecessary bits in the instruction words, get the information we need and reset the default flag register. This allows round-tripping through the assembler/disassembler. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2019-01-22 11:33:11 -08:00
/* The flag register is only used on Gen7 in align1 mode, so avoid setting
* unnecessary bits in the instruction words, get the information we need
* and reset the default flag register. This allows more instructions to be
* compacted.
*/
const unsigned flag_subreg = p->current->flag_subreg;
brw_set_default_flag_reg(p, 0, 0);
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
if (brw_get_default_access_mode(p) == BRW_ALIGN_1) {
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
if (devinfo->gen >= 8) {
/* Getting the first active channel index is easy on Gen8: Just find
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
* the first bit set in the execution mask. The register exists on
* HSW already but it reads back as all ones when the current
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
* instruction has execution masking disabled, so it's kind of
* useless.
*/
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
struct brw_reg exec_mask =
retype(brw_mask_reg(0), BRW_REGISTER_TYPE_UD);
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
if (mask.file != BRW_IMMEDIATE_VALUE || mask.ud != 0xffffffff) {
/* Unfortunately, ce0 does not take into account the thread
* dispatch mask, which may be a problem in cases where it's not
* tightly packed (i.e. it doesn't have the form '2^n - 1' for
* some n). Combine ce0 with the given dispatch (or vector) mask
* to mask off those channels which were never dispatched by the
* hardware.
*/
brw_SHR(p, vec1(dst), mask, brw_imm_ud(qtr_control * 8));
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_regdist(1));
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
brw_AND(p, vec1(dst), exec_mask, vec1(dst));
exec_mask = vec1(dst);
}
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
/* Quarter control has the effect of magically shifting the value of
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
* ce0 so you'll get the first active channel relative to the
* specified quarter control as result.
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
*/
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
inst = brw_FBL(p, vec1(dst), exec_mask);
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
} else {
intel/compiler: Reset default flag register in brw_find_live_channel() emit_uniformize() emits SHADER_OPCODE_FIND_LIVE_CHANNEL with its flag_subreg set, so that the IR knows which flag is accessed. However the flag is only used on Gen7 in Align1 mode. To avoid setting unnecessary bits in the instruction words, get the information we need and reset the default flag register. This allows round-tripping through the assembler/disassembler. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2019-01-22 11:33:11 -08:00
const struct brw_reg flag = brw_flag_subreg(flag_subreg);
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965/fs: Allow specifying arbitrary quarter control to FIND_LIVE_CHANNEL. This makes FIND_LIVE_CHANNEL behave like a normal instruction for non-zero quarter control. On Gen8+ we just leave the quarter control field of the emitted FBL instruction set to the default value so the hardware applies the expected shift to the execution mask signals. On Gen7 we apply the offset manually by specifying a non-zero subregister offset in the source region of the FBL instruction. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 16:25:42 -07:00
brw_MOV(p, retype(flag, BRW_REGISTER_TYPE_UD), brw_imm_ud(0));
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
i965/fs: Allow specifying arbitrary execution sizes up to 32 to FIND_LIVE_CHANNEL. Due to a Gen7-specific hardware bug native 32-wide instructions get the lower 16 bits of the execution mask applied incorrectly to both halves of the instruction, so the MOV trick we currently use wouldn't work. Instead emit multiple 16-wide MOV instructions in 32-wide mode in order to cover the whole execution mask. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 17:34:14 -07:00
/* Run enough instructions returning zero with execution masking and
* a conditional modifier enabled in order to get the full execution
* mask in f1.0. We could use a single 32-wide move here if it
* weren't because of the hardware bug that causes channel enables to
* be applied incorrectly to the second half of 32-wide instructions
* on Gen7.
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
*/
i965/fs: Allow specifying arbitrary execution sizes up to 32 to FIND_LIVE_CHANNEL. Due to a Gen7-specific hardware bug native 32-wide instructions get the lower 16 bits of the execution mask applied incorrectly to both halves of the instruction, so the MOV trick we currently use wouldn't work. Instead emit multiple 16-wide MOV instructions in 32-wide mode in order to cover the whole execution mask. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 17:34:14 -07:00
const unsigned lower_size = MIN2(16, exec_size);
for (unsigned i = 0; i < exec_size / lower_size; i++) {
inst = brw_MOV(p, retype(brw_null_reg(), BRW_REGISTER_TYPE_UW),
brw_imm_uw(0));
brw_inst_set_mask_control(devinfo, inst, BRW_MASK_ENABLE);
i965/fs: Allow specifying arbitrary quarter control to FIND_LIVE_CHANNEL. This makes FIND_LIVE_CHANNEL behave like a normal instruction for non-zero quarter control. On Gen8+ we just leave the quarter control field of the emitted FBL instruction set to the default value so the hardware applies the expected shift to the execution mask signals. On Gen7 we apply the offset manually by specifying a non-zero subregister offset in the source region of the FBL instruction. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 16:25:42 -07:00
brw_inst_set_group(devinfo, inst, lower_size * i + 8 * qtr_control);
i965/fs: Allow specifying arbitrary execution sizes up to 32 to FIND_LIVE_CHANNEL. Due to a Gen7-specific hardware bug native 32-wide instructions get the lower 16 bits of the execution mask applied incorrectly to both halves of the instruction, so the MOV trick we currently use wouldn't work. Instead emit multiple 16-wide MOV instructions in 32-wide mode in order to cover the whole execution mask. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 17:34:14 -07:00
brw_inst_set_cond_modifier(devinfo, inst, BRW_CONDITIONAL_Z);
brw_inst_set_exec_size(devinfo, inst, cvt(lower_size) - 1);
intel/compiler: Reset default flag register in brw_find_live_channel() emit_uniformize() emits SHADER_OPCODE_FIND_LIVE_CHANNEL with its flag_subreg set, so that the IR knows which flag is accessed. However the flag is only used on Gen7 in Align1 mode. To avoid setting unnecessary bits in the instruction words, get the information we need and reset the default flag register. This allows round-tripping through the assembler/disassembler. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2019-01-22 11:33:11 -08:00
brw_inst_set_flag_reg_nr(devinfo, inst, flag_subreg / 2);
brw_inst_set_flag_subreg_nr(devinfo, inst, flag_subreg % 2);
i965/fs: Allow specifying arbitrary execution sizes up to 32 to FIND_LIVE_CHANNEL. Due to a Gen7-specific hardware bug native 32-wide instructions get the lower 16 bits of the execution mask applied incorrectly to both halves of the instruction, so the MOV trick we currently use wouldn't work. Instead emit multiple 16-wide MOV instructions in 32-wide mode in order to cover the whole execution mask. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 17:34:14 -07:00
}
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
i965/fs: Allow specifying arbitrary quarter control to FIND_LIVE_CHANNEL. This makes FIND_LIVE_CHANNEL behave like a normal instruction for non-zero quarter control. On Gen8+ we just leave the quarter control field of the emitted FBL instruction set to the default value so the hardware applies the expected shift to the execution mask signals. On Gen7 we apply the offset manually by specifying a non-zero subregister offset in the source region of the FBL instruction. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 16:25:42 -07:00
/* Find the first bit set in the exec_size-wide portion of the flag
* register that was updated by the last sequence of MOV
* instructions.
*/
const enum brw_reg_type type = brw_int_type(exec_size / 8, false);
intel/eu: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 13:36:58 -07:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965/fs: Allow specifying arbitrary quarter control to FIND_LIVE_CHANNEL. This makes FIND_LIVE_CHANNEL behave like a normal instruction for non-zero quarter control. On Gen8+ we just leave the quarter control field of the emitted FBL instruction set to the default value so the hardware applies the expected shift to the execution mask signals. On Gen7 we apply the offset manually by specifying a non-zero subregister offset in the source region of the FBL instruction. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 16:25:42 -07:00
brw_FBL(p, vec1(dst), byte_offset(retype(flag, type), qtr_control));
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
}
} else {
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
if (devinfo->gen >= 8 &&
mask.file == BRW_IMMEDIATE_VALUE && mask.ud == 0xffffffff) {
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
/* In SIMD4x2 mode the first active channel index is just the
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 15:09:33 -07:00
* negation of the first bit of the mask register. Note that ce0
* doesn't take into account the dispatch mask, so the Gen7 path
* should be used instead unless you have the guarantee that the
* dispatch mask is tightly packed (i.e. it has the form '2^n - 1'
* for some n).
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
*/
inst = brw_AND(p, brw_writemask(dst, WRITEMASK_X),
negate(retype(brw_mask_reg(0), BRW_REGISTER_TYPE_UD)),
brw_imm_ud(1));
} else {
/* Overwrite the destination without and with execution masking to
* find out which of the channels is active.
*/
i965: set correct execsize for MOVS with a width of 4 in brw_find_live_channel Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-12-03 11:11:14 +01:00
brw_push_insn_state(p);
brw_set_default_exec_size(p, BRW_EXECUTE_4);
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
brw_MOV(p, brw_writemask(vec4(dst), WRITEMASK_X),
brw_imm_ud(1));
inst = brw_MOV(p, brw_writemask(vec4(dst), WRITEMASK_X),
brw_imm_ud(0));
i965: set correct execsize for MOVS with a width of 4 in brw_find_live_channel Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-12-03 11:11:14 +01:00
brw_pop_insn_state(p);
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 14:42:53 +03:00
brw_inst_set_mask_control(devinfo, inst, BRW_MASK_ENABLE);
}
}
brw_pop_insn_state(p);
}
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
void
brw_broadcast(struct brw_codegen *p,
struct brw_reg dst,
struct brw_reg src,
struct brw_reg idx)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 14:50:46 -07:00
const bool align1 = brw_get_default_access_mode(p) == BRW_ALIGN_1;
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
brw_inst *inst;
i965/ir: Make BROADCAST emit an unmasked single-channel move. Alternatively we could have extended the current semantics to 32-wide mode by changing brw_broadcast() to emit multiple indexed MOV instructions in the generator copying the selected value to all destination registers, but it seemed rather silly to waste EU cycles unnecessarily copying the exact same value 32 times in the GRF. The vstride change in the Align16 path is required to avoid assertions in validate_reg() since the change causes the execution size of the MOV and SEL instructions to be equal to the source region width. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 00:10:03 -07:00
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_exec_size(p, align1 ? BRW_EXECUTE_1 : BRW_EXECUTE_4);
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
assert(src.file == BRW_GENERAL_REGISTER_FILE &&
src.address_mode == BRW_ADDRESS_DIRECT);
intel/compiler: Add some restrictions to MOV_INDIRECT and BROADCAST These restrictions effectively already existed due to the way we use indirect sources but weren't being directly enforced. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 11:57:48 -07:00
assert(!src.abs && !src.negate);
assert(src.type == dst.type);
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
if ((src.vstride == 0 && (src.hstride == 0 || !align1)) ||
idx.file == BRW_IMMEDIATE_VALUE) {
/* Trivial, the source is already uniform or the index is a constant.
* We will typically not get here if the optimizer is doing its job, but
* asserting would be mean.
*/
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-22 19:41:30 -07:00
const unsigned i = idx.file == BRW_IMMEDIATE_VALUE ? idx.ud : 0;
intel/fs: Fix MOV_INDIRECT and BROADCAST of Q types on Gen11+ The immediate case is pretty uncommon to see but it can happen, in theory. BROADCAST is typically used to uniformize values and those are usually 32-bit. However, it does come up in some subgroup ops. Fixes: 49c21802cbca "intel/compiler: Split has_64bit_types into float/int" Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6211>
2020-07-17 16:22:11 -05:00
src = align1 ? stride(suboffset(src, i), 0, 1, 0) :
stride(suboffset(src, 4 * i), 0, 4, 1);
if (type_sz(src.type) > 4 && !devinfo->has_64bit_float) {
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 0),
subscript(src, BRW_REGISTER_TYPE_D, 0));
brw_set_default_swsb(p, tgl_swsb_null());
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 1),
subscript(src, BRW_REGISTER_TYPE_D, 1));
} else {
brw_MOV(p, dst, src);
}
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
} else {
intel/compiler: Add some restrictions to MOV_INDIRECT and BROADCAST These restrictions effectively already existed due to the way we use indirect sources but weren't being directly enforced. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 11:57:48 -07:00
/* From the Haswell PRM section "Register Region Restrictions":
*
* "The lower bits of the AddressImmediate must not overflow to
* change the register address. The lower 5 bits of Address
* Immediate when added to lower 5 bits of address register gives
* the sub-register offset. The upper bits of Address Immediate
* when added to upper bits of address register gives the register
* address. Any overflow from sub-register offset is dropped."
*
* Fortunately, for broadcast, we never have a sub-register offset so
* this isn't an issue.
*/
assert(src.subnr == 0);
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
if (align1) {
const struct brw_reg addr =
retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD);
intel/eu: Just modify the offset in brw_broadcast This means we have to drop const from a variable but it also means that 100% of the code which deals with the offset limit is in one place. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 14:16:31 -07:00
unsigned offset = src.nr * REG_SIZE + src.subnr;
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
/* Limit in bytes of the signed indirect addressing immediate. */
const unsigned limit = 512;
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
/* Take into account the component size and horizontal stride. */
assert(src.vstride == src.hstride + src.width);
brw_SHL(p, addr, vec1(idx),
replace _mesa_logbase2 with util_logbase2 Reviewed-by: Marek Olšák <marek.olsak@amd.com> Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com> Reviewed-by: Matt Turner <mattst88@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3024>
2019-12-06 09:20:09 -08:00
brw_imm_ud(util_logbase2(type_sz(src.type)) +
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
src.hstride - 1));
/* We can only address up to limit bytes using the indirect
* addressing immediate, account for the difference if the source
* register is above this limit.
*/
intel/eu: Just modify the offset in brw_broadcast This means we have to drop const from a variable but it also means that 100% of the code which deals with the offset limit is in one place. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 14:16:31 -07:00
if (offset >= limit) {
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_regdist(1));
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
brw_ADD(p, addr, addr, brw_imm_ud(offset - offset % limit));
intel/eu: Just modify the offset in brw_broadcast This means we have to drop const from a variable but it also means that 100% of the code which deals with the offset limit is in one place. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 14:16:31 -07:00
offset = offset % limit;
}
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
brw_pop_insn_state(p);
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_regdist(1));
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
/* Use indirect addressing to fetch the specified component. */
intel/eu: Fix broadcast instruction for 64-bit values on little-core We're not using broadcast for any 32-bit types right now since we mostly use it for emit_uniformize on 32-bit buffer indices. However, SPIR-V subgroups are going to need it for 64-bit so let's make it work. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 14:45:12 -07:00
if (type_sz(src.type) > 4 &&
intel/fs: Fix MOV_INDIRECT and BROADCAST of Q types on Gen11+ The immediate case is pretty uncommon to see but it can happen, in theory. BROADCAST is typically used to uniformize values and those are usually 32-bit. However, it does come up in some subgroup ops. Fixes: 49c21802cbca "intel/compiler: Split has_64bit_types into float/int" Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6211>
2020-07-17 16:22:11 -05:00
(devinfo->is_cherryview || gen_device_info_is_9lp(devinfo) ||
!devinfo->has_64bit_float)) {
intel/eu: Fix broadcast instruction for 64-bit values on little-core We're not using broadcast for any 32-bit types right now since we mostly use it for emit_uniformize on 32-bit buffer indices. However, SPIR-V subgroups are going to need it for 64-bit so let's make it work. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 14:45:12 -07:00
/* From the Cherryview PRM Vol 7. "Register Region Restrictions":
*
* "When source or destination datatype is 64b or operation is
* integer DWord multiply, indirect addressing must not be
* used."
*
* To work around both of this issue, we do two integer MOVs
* insead of one 64-bit MOV. Because no double value should ever
* cross a register boundary, it's safe to use the immediate
* offset in the indirect here to handle adding 4 bytes to the
* offset and avoid the extra ADD to the register file.
*/
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 0),
retype(brw_vec1_indirect(addr.subnr, offset),
BRW_REGISTER_TYPE_D));
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_null());
intel/eu: Fix broadcast instruction for 64-bit values on little-core We're not using broadcast for any 32-bit types right now since we mostly use it for emit_uniformize on 32-bit buffer indices. However, SPIR-V subgroups are going to need it for 64-bit so let's make it work. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 14:45:12 -07:00
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 1),
retype(brw_vec1_indirect(addr.subnr, offset + 4),
BRW_REGISTER_TYPE_D));
} else {
brw_MOV(p, dst,
retype(brw_vec1_indirect(addr.subnr, offset), src.type));
}
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
} else {
/* In SIMD4x2 mode the index can be either zero or one, replicate it
* to all bits of a flag register,
*/
inst = brw_MOV(p,
brw_null_reg(),
i965/ir: Make BROADCAST emit an unmasked single-channel move. Alternatively we could have extended the current semantics to 32-wide mode by changing brw_broadcast() to emit multiple indexed MOV instructions in the generator copying the selected value to all destination registers, but it seemed rather silly to waste EU cycles unnecessarily copying the exact same value 32 times in the GRF. The vstride change in the Align16 path is required to avoid assertions in validate_reg() since the change causes the execution size of the MOV and SEL instructions to be equal to the source region width. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 00:10:03 -07:00
stride(brw_swizzle(idx, BRW_SWIZZLE_XXXX), 4, 4, 1));
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
brw_inst_set_pred_control(devinfo, inst, BRW_PREDICATE_NONE);
brw_inst_set_cond_modifier(devinfo, inst, BRW_CONDITIONAL_NZ);
brw_inst_set_flag_reg_nr(devinfo, inst, 1);
/* and use predicated SEL to pick the right channel. */
inst = brw_SEL(p, dst,
i965/ir: Make BROADCAST emit an unmasked single-channel move. Alternatively we could have extended the current semantics to 32-wide mode by changing brw_broadcast() to emit multiple indexed MOV instructions in the generator copying the selected value to all destination registers, but it seemed rather silly to waste EU cycles unnecessarily copying the exact same value 32 times in the GRF. The vstride change in the Align16 path is required to avoid assertions in validate_reg() since the change causes the execution size of the MOV and SEL instructions to be equal to the source region width. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 00:10:03 -07:00
stride(suboffset(src, 4), 4, 4, 1),
stride(src, 4, 4, 1));
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
brw_inst_set_pred_control(devinfo, inst, BRW_PREDICATE_NORMAL);
brw_inst_set_flag_reg_nr(devinfo, inst, 1);
}
}
i965/ir: Make BROADCAST emit an unmasked single-channel move. Alternatively we could have extended the current semantics to 32-wide mode by changing brw_broadcast() to emit multiple indexed MOV instructions in the generator copying the selected value to all destination registers, but it seemed rather silly to waste EU cycles unnecessarily copying the exact same value 32 times in the GRF. The vstride change in the Align16 path is required to avoid assertions in validate_reg() since the change causes the execution size of the MOV and SEL instructions to be equal to the source region width. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 00:10:03 -07:00
brw_pop_insn_state(p);
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 20:14:24 +02:00
}
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
/**
* This instruction is generated as a single-channel align1 instruction by
* both the VS and FS stages when using INTEL_DEBUG=shader_time.
*
* We can't use the typed atomic op in the FS because that has the execution
* mask ANDed with the pixel mask, but we just want to write the one dword for
* all the pixels.
*
* We don't use the SIMD4x2 atomic ops in the VS because want to just write
* one u32. So we use the same untyped atomic write message as the pixel
* shader.
*
* The untyped atomic operation requires a BUFFER surface type with RAW
* format, and is only accessible through the legacy DATA_CACHE dataport
* messages.
*/
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 11:06:57 -07:00
void brw_shader_time_add(struct brw_codegen *p,
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 15:28:11 -07:00
struct brw_reg payload,
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
uint32_t surf_index)
{
TRIVIAL: intel/eu: Use a local devinfo variable in brw_shader_time_add(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 14:59:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
const unsigned sfid = (devinfo->gen >= 8 || devinfo->is_haswell ?
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
HSW_SFID_DATAPORT_DATA_CACHE_1 :
GEN7_SFID_DATAPORT_DATA_CACHE);
TRIVIAL: intel/eu: Use a local devinfo variable in brw_shader_time_add(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 14:59:08 -07:00
assert(devinfo->gen >= 7);
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
brw_push_insn_state(p);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 16:57:02 -07:00
brw_set_default_access_mode(p, BRW_ALIGN_1);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 14:29:25 -07:00
brw_inst *send = brw_next_insn(p, BRW_OPCODE_SEND);
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
/* We use brw_vec1_reg and unmasked because we want to increment the given
* offset only once.
*/
brw_set_dest(p, send, brw_vec1_reg(BRW_ARCHITECTURE_REGISTER_FILE,
BRW_ARF_NULL, 0));
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 15:28:11 -07:00
brw_set_src0(p, send, brw_vec1_reg(payload.file,
payload.nr, 0));
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
brw_set_desc(p, send, (brw_message_desc(devinfo, 2, 0, false) |
intel/eu: Rework surface descriptor helpers This commit pulls the surface descriptor helpers out into brw_eu.h and makes them no longer depend on the codegen infrastructure. This should allow us to use them directly from the IR code instead of the generator. This change is unfortunately less mechanical than perhaps one would like but it should be fairly straightforward. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 16:09:30 -05:00
brw_dp_untyped_atomic_desc(devinfo, 1, BRW_AOP_ADD,
false)));
intel/eu: Use descriptor constructors for dataport untyped surface messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 15:22:58 -07:00
TRIVIAL: intel/eu: Use a local devinfo variable in brw_shader_time_add(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 14:59:08 -07:00
brw_inst_set_sfid(devinfo, send, sfid);
brw_inst_set_binding_table_index(devinfo, send, surf_index);
i965: Fix the untyped surface opcodes to deal with indirect surface access. Change brw_untyped_atomic() and brw_untyped_surface_read() to take the surface index as a register instead of a constant and to use brw_send_indirect_message() to emit the indirect variant of send with a dynamically calculated message descriptor. This will be required to support variable indexing of image arrays for ARB_shader_image_load_store. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 14:21:31 +03:00
brw_pop_insn_state(p);
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 14:10:52 -08:00
}
i965: Add brw_WAIT to emit wait instruction This will be used to implement the barrier function. v2: * Rename to brw_WAIT (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-04 18:05:04 -08:00
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
/**
* Emit the SEND message for a barrier
*/
void
brw_barrier(struct brw_codegen *p, struct brw_reg src)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
struct brw_inst *inst;
assert(devinfo->gen >= 7);
i965: Use align1 mode for barrier messages. In commit 7428e6f86ab5 we switched the barrier SEND message's destination type to UW to avoid problems in SIMD16 compute shaders. Tessellation control shaders also use barriers, and in vec4 mode, we were emitting them in align16 mode. The simulator warns that only UD, D, F, and DF are valid destination types - UW is technically illegal. So, switch to align1 mode. Either mode should work fine. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2017-01-15 00:58:20 -08:00
brw_push_insn_state(p);
brw_set_default_access_mode(p, BRW_ALIGN_1);
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
inst = next_insn(p, BRW_OPCODE_SEND);
i965: Set dest type to UW for several send messages Without this, on SIMD 16 the send instruction destination will appear to write more than one destination register, causing the simulator to report an error. Of course, the send instruction can actually write more than one destination register regardless of the type set for the destination, so this is a bit strange. Suggested-by: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-01-31 18:28:42 -08:00
brw_set_dest(p, inst, retype(brw_null_reg(), BRW_REGISTER_TYPE_UW));
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
brw_set_src0(p, inst, src);
brw_set_src1(p, inst, brw_null_reg());
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_set_desc(p, inst, brw_message_desc(devinfo, 1, 0, false));
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
intel/eu: Use brw_set_desc() along with a helper to set common descriptor controls. This replaces brw_set_message_descriptor() with the composition of brw_set_desc() and a new inline helper function that packs the common message descriptor controls into an integer. The goal is to represent all message descriptors as a 32-bit integer which is written at once into the instruction, which is more flexible (SENDS anyone?), robust (see d2eecf0b0b24d203d0f171807681dffd830d54de fixing an issue ultimately caused by some bits of the extended message descriptor being left undefined) and future-proof than the current approach of specifying the individual descriptor fields directly into the instruction. This approach also seems more self-documenting, since it will allow removing calls to functions with way too many arguments like brw_set_*_message() and brw_send_indirect_message(), and instead provide a single descriptor argument constructed from an appropriate combination of brw_*_desc() helpers. Note that because brw_set_message_descriptor() was (conditionally?) overriding fields of the instruction which strictly speaking weren't part of the message descriptor, this involves calling brw_inst_set_sfid() and brw_inst_set_eot() in some cases in addition to brw_set_desc(). v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-11 10:49:39 -07:00
brw_inst_set_sfid(devinfo, inst, BRW_SFID_MESSAGE_GATEWAY);
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
brw_inst_set_gateway_subfuncid(devinfo, inst,
BRW_MESSAGE_GATEWAY_SFID_BARRIER_MSG);
brw_inst_set_mask_control(devinfo, inst, BRW_MASK_DISABLE);
i965: Use align1 mode for barrier messages. In commit 7428e6f86ab5 we switched the barrier SEND message's destination type to UW to avoid problems in SIMD16 compute shaders. Tessellation control shaders also use barriers, and in vec4 mode, we were emitting them in align16 mode. The simulator warns that only UD, D, F, and DF are valid destination types - UW is technically illegal. So, switch to align1 mode. Either mode should work fine. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2017-01-15 00:58:20 -08:00
brw_pop_insn_state(p);
i965: Add brw_barrier to emit a Gateway Barrier SEND This will be used to implement the Gateway Barrier SEND needed to implement the barrier function. v2: * notify => gateway_notify (Ken) * combine short lines of brw_barrier proto/decl (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-11-04 18:11:37 -08:00
}
i965: Add brw_WAIT to emit wait instruction This will be used to implement the barrier function. v2: * Rename to brw_WAIT (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-04 18:05:04 -08:00
/**
* Emit the wait instruction for a barrier
*/
void
brw_WAIT(struct brw_codegen *p)
{
intel: s/brw_device_info/gen_device_info/ Generated by: sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.c sed -i -e 's/brw_device_info/gen_device_info/g' src/intel/**/*.h sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.c sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.cpp sed -i -e 's/brw_device_info/gen_device_info/g' **/i965/*.h Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-08-22 15:01:08 -07:00
const struct gen_device_info *devinfo = p->devinfo;
i965: Add brw_WAIT to emit wait instruction This will be used to implement the barrier function. v2: * Rename to brw_WAIT (mattst88) Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-04 18:05:04 -08:00
struct brw_inst *insn;
struct brw_reg src = brw_notification_reg();
insn = next_insn(p, BRW_OPCODE_WAIT);
brw_set_dest(p, insn, src);
brw_set_src0(p, insn, src);
brw_set_src1(p, insn, brw_null_reg());
brw_inst_set_exec_size(devinfo, insn, BRW_EXECUTE_1);
brw_inst_set_mask_control(devinfo, insn, BRW_MASK_DISABLE);
}
i965/fs: Define new shader opcode to set rounding modes Although it is possible to emit them directly as AND/OR on brw_fs_nir, having a specific opcode makes it easier to remove duplicate settings later. v2: (Curro) - Set thread control to 'switch' when using the control register - Use a single SHADER_OPCODE_RND_MODE opcode taking an immediate with the rounding mode. - Avoid magic numbers setting rounding mode field at control register. v3: (Curro) - Remove redundant and add missing whitespace lines. - Match printing instruction to IR opcode "rnd_mode" v4: (Topi Pohjolainen) - Fix code style. Signed-off-by: Alejandro Piñeiro <apinheiro@igalia.com> Signed-off-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-07-01 08:12:59 +02:00
void
i965/fs/generator: refactor rounding mode helper in preparation for float controls v2: - Fix bug in defining BRW_CR0_FP_MODE_MASK. v3: - Update comment (Caio). v4: - Split the patch into the helper (this one) and the new opcode (Caio). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-13 01:34:35 +03:00
brw_float_controls_mode(struct brw_codegen *p,
unsigned mode, unsigned mask)
{
/* From the Skylake PRM, Volume 7, page 760:
* "Implementation Restriction on Register Access: When the control
* register is used as an explicit source and/or destination, hardware
* does not ensure execution pipeline coherency. Software must set the
* thread control field to ‘switch’ for an instruction that uses
* control register as an explicit operand."
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
*
* On Gen12+ this is implemented in terms of SWSB annotations instead.
i965/fs/generator: refactor rounding mode helper in preparation for float controls v2: - Fix bug in defining BRW_CR0_FP_MODE_MASK. v3: - Update comment (Caio). v4: - Split the patch into the helper (this one) and the new opcode (Caio). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-13 01:34:35 +03:00
*/
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
brw_set_default_swsb(p, tgl_swsb_regdist(1));
brw_inst *inst = brw_AND(p, brw_cr0_reg(0), brw_cr0_reg(0),
brw_imm_ud(~mask));
brw_inst_set_exec_size(p->devinfo, inst, BRW_EXECUTE_1);
intel/eu/gen12: Don't set thread control, it's gone. An effect similar to the one formerly provided by setting thread control to "switch" can be achieved now by setting a RegDist of 1 on the SWSB field. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-09-26 23:36:58 -07:00
if (p->devinfo->gen < 12)
brw_inst_set_thread_control(p->devinfo, inst, BRW_THREAD_SWITCH);
i965/fs/generator: refactor rounding mode helper in preparation for float controls v2: - Fix bug in defining BRW_CR0_FP_MODE_MASK. v3: - Update comment (Caio). v4: - Split the patch into the helper (this one) and the new opcode (Caio). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-13 01:34:35 +03:00
if (mode) {
brw_inst *inst_or = brw_OR(p, brw_cr0_reg(0), brw_cr0_reg(0),
brw_imm_ud(mode));
brw_inst_set_exec_size(p->devinfo, inst_or, BRW_EXECUTE_1);
intel/eu/gen12: Don't set thread control, it's gone. An effect similar to the one formerly provided by setting thread control to "switch" can be achieved now by setting a RegDist of 1 on the SWSB field. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2019-09-26 23:36:58 -07:00
if (p->devinfo->gen < 12)
brw_inst_set_thread_control(p->devinfo, inst_or, BRW_THREAD_SWITCH);
i965/fs: Define new shader opcode to set rounding modes Although it is possible to emit them directly as AND/OR on brw_fs_nir, having a specific opcode makes it easier to remove duplicate settings later. v2: (Curro) - Set thread control to 'switch' when using the control register - Use a single SHADER_OPCODE_RND_MODE opcode taking an immediate with the rounding mode. - Avoid magic numbers setting rounding mode field at control register. v3: (Curro) - Remove redundant and add missing whitespace lines. - Match printing instruction to IR opcode "rnd_mode" v4: (Topi Pohjolainen) - Fix code style. Signed-off-by: Alejandro Piñeiro <apinheiro@igalia.com> Signed-off-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-07-01 08:12:59 +02:00
}
intel/eu/gen12: Set SWSB annotations in hand-crafted assembly. Reviewers are encouraged to audit the code generation pass independently for the case I missed some potential data hazard or new code has been added in the meantime. v2: Add SYNC instruction to cr0 workaround in brw_float_controls_mode(). v3: Drop likely redundant (and potentially harmful) RegDist SWSB annotation from ce0 read in brw_find_live_channel() (Caio). Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-09-26 23:38:24 -07:00
if (p->devinfo->gen >= 12)
brw_SYNC(p, TGL_SYNC_NOP);
i965/fs: Define new shader opcode to set rounding modes Although it is possible to emit them directly as AND/OR on brw_fs_nir, having a specific opcode makes it easier to remove duplicate settings later. v2: (Curro) - Set thread control to 'switch' when using the control register - Use a single SHADER_OPCODE_RND_MODE opcode taking an immediate with the rounding mode. - Avoid magic numbers setting rounding mode field at control register. v3: (Curro) - Remove redundant and add missing whitespace lines. - Match printing instruction to IR opcode "rnd_mode" v4: (Topi Pohjolainen) - Fix code style. Signed-off-by: Alejandro Piñeiro <apinheiro@igalia.com> Signed-off-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-07-01 08:12:59 +02:00
}
intel/eu: Add a mechanism for emitting relocatable constant MOVs Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-08-08 12:55:29 -05:00
void
brw_update_reloc_imm(const struct gen_device_info *devinfo,
brw_inst *inst,
uint32_t value)
{
/* Sanity check that the instruction is a MOV of an immediate */
assert(brw_inst_opcode(devinfo, inst) == BRW_OPCODE_MOV);
assert(brw_inst_src0_reg_file(devinfo, inst) == BRW_IMMEDIATE_VALUE);
/* If it was compacted, we can't safely rewrite */
assert(brw_inst_cmpt_control(devinfo, inst) == 0);
brw_inst_set_imm_ud(devinfo, inst, value);
}
/* A default value for constants that will be patched at run-time.
* We pick an arbitrary value that prevents instruction compaction.
*/
#define DEFAULT_PATCH_IMM 0x4a7cc037
void
brw_MOV_reloc_imm(struct brw_codegen *p,
struct brw_reg dst,
enum brw_reg_type src_type,
uint32_t id)
{
assert(type_sz(src_type) == 4);
assert(type_sz(dst.type) == 4);
if (p->num_relocs + 1 > p->reloc_array_size) {
p->reloc_array_size = MAX2(16, p->reloc_array_size * 2);
p->relocs = reralloc(p->mem_ctx, p->relocs,
struct brw_shader_reloc, p->reloc_array_size);
}
p->relocs[p->num_relocs++] = (struct brw_shader_reloc) {
.id = id,
.offset = p->next_insn_offset,
};
brw_MOV(p, dst, retype(brw_imm_ud(DEFAULT_PATCH_IMM), src_type));
}
Reference in a new issue Copy permalink