gallivm: fix incorrect type for undefined texcoords

Texcoords may be 1-5 components in length.  We initialize the
unused components with an LLVMGetUndef().  But we were using
an int vec type rather than a float vec type.

This eventually led to a failed assertion in lp_build_clamp()
where 'a' was a vec of int[8] but 'min' and 'max' were float[8]
in a trace of the game Tom Clancy's Splinter Cell: Blacklist.

The game seems to have a bug where a texture sampler mistakenly has
shadow comparison turned on, but the shader's tex sample instructions
are sampling a 2D R8G8B8A8_UNORM texture.  The instruction has a
2-component texcoord so when we do the sampler comparison operation
we're using the undefined 5th coordinate component.

Signed-off-by: Brian Paul <brianp@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20233>
This commit is contained in:
Brian Paul 2022-12-07 14:49:11 -07:00 committed by Marge Bot
parent 5233551e19
commit bc048f5c51

View file

@ -2344,7 +2344,7 @@ visit_tex(struct lp_build_nir_context *bld_base, nir_tex_instr *instr)
LLVMValueRef texture_unit_offset = NULL;
LLVMValueRef texel[NIR_MAX_VEC_COMPONENTS];
unsigned lod_src = 0;
LLVMValueRef coord_undef = LLVMGetUndef(bld_base->base.int_vec_type);
LLVMValueRef coord_undef = LLVMGetUndef(bld_base->base.vec_type);
unsigned coord_vals = is_aos(bld_base) ? 1 : instr->coord_components;
memset(&params, 0, sizeof(params));
enum lp_sampler_lod_property lod_property = LP_SAMPLER_LOD_SCALAR;