Skip to content

resolve mesh face_count discrepancy with server (#853 #1901)#5413

Open
AndrewMeadows wants to merge 7 commits intodevelopfrom
leviathan/ghi-1901
Open

resolve mesh face_count discrepancy with server (#853 #1901)#5413
AndrewMeadows wants to merge 7 commits intodevelopfrom
leviathan/ghi-1901

Conversation

@AndrewMeadows
Copy link
Collaborator

This provides fixes necessary for some bugs (I think #853 and #1901) where texture changes to some mesh faces would be ignored by the server because the particular face being changed was "unknown" server-side. This client-side change doesn't fix everything, but allows the client to coordinate with similar server-side changes.

}

for (S32 i=0;i<(S32)mFaces.size();i++)
for (U8 i=0;i<mFaces.size();i++)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm changing almost all of the "face count" variables to use U8 instead of a mix of types (S32, U32, size_t). There are a few places I didn't change because in context the logic was expecting invalid values (-1).

// copies LLTextureEntry 'te'
// returns TEM_CHANGE_TEXTURE if successful, otherwise TEM_CHANGE_NONE
S32 LLPrimTextureList::copyTexture(const U8 index, const LLTextureEntry& te)
S32 LLPrimTextureList::copyTexture(const U8 index, const LLTextureEntry* te)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change was necessary because I was doing the development work on Linux and g++-11.0.4 considered it an error to cast a pointer to const reference and then pass it to a function that expected a real const reference.

After making it work on Linux I ported the changes to a branch off of develop.

{
const LLVolumeFace& vol_face = volume->getVolumeFace(face->getTEOffset());
LLVertexBuffer::drawElements(LLRender::TRIANGLES, vol_face.mPositions, NULL, vol_face.mNumIndices, vol_face.mIndices);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change because during my development I started crashing here for indexing out of bounds.

mMediaImplList.resize(num_tes) ;

LLViewerObject::setNumTEs(num_tes) ;
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I'm doing here changing if (a && b) else if (c) to be if (a) { if (b) else if (c) } . The reason being the old_num_tes == num_tes case was falling into the final else case but now it falls into the non-existent else case of the embedded if/else_if and is no longer handled.

//===============================================================
LLPrimitive::LLPrimitive()
: mTextureList(),
mNumTEs(0),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data member wasn't being used.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a long-standing issue where the client and server disagreed on the number of faces for mesh objects, causing PBR materials and other texture changes to be silently dropped. The fix involves both refactoring the face count tracking mechanism and implementing a workaround protocol to communicate face counts between client and server.

Changes:

  • Removed the mNumTEs member variable and replaced it with direct queries to mTextureList.size() to maintain a single source of truth
  • Changed face count types from S32/U32 to U8 throughout the codebase, consistent with MAX_TES = 45
  • Refactored TextureEntry packing/unpacking to encode face count information in the wire protocol for mesh objects
  • Added bounds checking in renderFace() to prevent crashes when accessing volume faces

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
indra/newview/llvovolume.cpp Reorganized setNumTEs logic for clarity without changing behavior
indra/newview/llface.cpp Added bounds check to prevent out-of-bounds access to volume faces
indra/llprimitive/tests/llprimitive_test.cpp Updated test stubs to match new U8 signatures
indra/llprimitive/llprimtexturelist.h Changed size() and setSize() signatures from S32 to U8, uncommented copyTexture overload
indra/llprimitive/llprimtexturelist.cpp Implemented U8-based iteration, added copyTexture overload, fixed typo in comment
indra/llprimitive/llprimitive.h Removed mNumTEs field, changed MAX_TES to U8, added helper methods for TE packing/unpacking
indra/llprimitive/llprimitive.cpp Major refactoring: extracted packing logic, implemented face count workaround for mesh, updated validTE to use mTextureList.size()
indra/llmath/llvolume.h Changed getNumFaces() and related constants to return/use U8
indra/llmath/llvolume.cpp Updated getNumFaces() implementation and loop variables to use U8

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +86 to +87
U8 this_size = static_cast<U8>(mEntryList.size());
U8 other_size = static_cast<U8>(other_list.mEntryList.size());
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential data loss when mEntryList.size() exceeds 255. The vector size is being cast to U8, which can only hold values 0-255, but there's no check to ensure the size doesn't exceed this limit before the cast. If mEntryList.size() is greater than 255, the cast will silently truncate the value, leading to incorrect behavior.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The good news is: for other reasons not evident here we don't expect mEntryList.size() to ever report more than 45.

mEntryList[index] = other_list.getTexture(index)->newCopy();
}

// add new entires if needed
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in comment: 'entires' should be 'entries'.

Suggested change
// add new entires if needed
// add new entries if needed

Copilot uses AI. Check for mistakes.
Comment on lines +1218 to +1226
if (flags && ! highest_flagged_index)
{
// initialize highest_flagged_index
while (flags > 1)
{
highest_flagged_index++;
flags >>= 1;
}
}
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug in highest_flagged_index calculation for multi-byte bitfields. When the first non-zero flags byte is not the first byte of the bitfield, the initialization logic (lines 1218-1226) finds the highest bit within that byte but doesn't account for the bit offset from previous bytes. For example, if only index 20 is flagged (requiring 3 bytes where bytes 1-2 are 0x00 and byte 3 is non-zero), the code would initialize highest_flagged_index to the bit position within byte 3 (e.g., 6) but fail to add the 14-bit offset from the previous two bytes. The increment at line 1210 only happens when highest_flagged_index is already non-zero, so it won't correct this. The initialization should account for how many bytes have already been processed.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately Copilot does not realize: the variable-length bitfield is only as long as necessary to pack the non-zero bits. Zero-value bytes at the front of the bitfield are never packed. This probably means this code at least deserves a comment about that fact make the code a little bit more understandable.

Comment on lines 1026 to 1027
// Decreases in the LLVolume's notion of "num_faces" happens for LOD reasons
// which do not reflect actual change of faces of the object, so we ignore them.
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment appears to be incorrect or misleading. The comment states "Decreases in the LLVolume's notion of 'num_faces' happens for LOD reasons which do not reflect actual change of faces of the object, so we ignore them." However, the actual check (getNumTEs() < mVolumep->getNumFaces()) handles the case when the volume has MORE faces than the texture entries, not fewer. If the intent is to ignore decreases, the condition should be inverted, or if the condition is correct, the comment should explain that we're updating texture entries when the volume has more faces.

Suggested change
// Decreases in the LLVolume's notion of "num_faces" happens for LOD reasons
// which do not reflect actual change of faces of the object, so we ignore them.
// If the LLVolume's notion of "num_faces" increases, update the number of
// texture entries to match. Decreases in "num_faces" happen for LOD reasons
// which do not reflect an actual change in the object's faces, so we ignore them.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the effort Copilot but you misunderstand how the comment relates to the logic flow. The "ignore" statement is trying to explain why there is no "else" case. Nevertheless, I will modify the comment to be less confusing to friendly AI assistants.

U8 LLVolume::getNumFaces() const
{
return mIsMeshAssetLoaded ? getNumVolumeFaces() : (S32)mProfilep->mFaces.size();
return mIsMeshAssetLoaded ? getNumVolumeFaces() : static_cast<U8>(mProfilep->mFaces.size());
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential data loss when mFaces.size() exceeds 255. The vector size is being cast to U8, which can only hold values 0-255. If mFaces.size() is greater than 255, the cast will silently truncate the value, leading to incorrect face count.

Suggested change
return mIsMeshAssetLoaded ? getNumVolumeFaces() : static_cast<U8>(mProfilep->mFaces.size());
// Clamp face count to maximum value representable by U8 to avoid truncation.
U32 num_faces = mIsMeshAssetLoaded
? static_cast<U32>(getNumVolumeFaces())
: static_cast<U32>(mProfilep->mFaces.size());
if (num_faces > 255u)
{
num_faces = 255u;
}
return static_cast<U8>(num_faces);

Copilot uses AI. Check for mistakes.
else if (new_size < current_size)
{
for (size_t index = current_size-1; index >= new_size; --index)
for (U8 index = current_size-1; index >= new_size; --index)
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Infinite loop bug when new_size is 0. The loop variable 'index' is U8, so when it reaches 0 and decrements (--index), it will underflow to 255 instead of going negative, causing the condition 'index >= new_size' to remain true indefinitely. The loop should either use a signed type for the index, iterate in the opposite direction, or explicitly check for the underflow condition.

Suggested change
for (U8 index = current_size-1; index >= new_size; --index)
for (S32 index = current_size - 1; index >= new_size; --index)

Copilot uses AI. Check for mistakes.
Comment on lines 1567 to 1574
if (mVolumep && mVolumep->getParams().isMeshSculpt())
{
cur_ptr += pack_TE_field(cur_ptr, (U8 *)colors, 4 ,last_face_index, MVT_U8);
}
else
{
cur_ptr += pack_TE_field(cur_ptr, (U8 *)colors, 4 ,last_face_index, MVT_U8);
}
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead code: The if and else branches contain identical code, making the conditional check unnecessary. This appears to be leftover scaffolding that should be removed. The mesh sculpt-specific workaround for face count is correctly implemented later at lines 1587-1591 for the bump field.

Suggested change
if (mVolumep && mVolumep->getParams().isMeshSculpt())
{
cur_ptr += pack_TE_field(cur_ptr, (U8 *)colors, 4 ,last_face_index, MVT_U8);
}
else
{
cur_ptr += pack_TE_field(cur_ptr, (U8 *)colors, 4 ,last_face_index, MVT_U8);
}
cur_ptr += pack_TE_field(cur_ptr, (U8 *)colors, 4 ,last_face_index, MVT_U8);

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot is correct! Oh I see why this happened... during debugging I had a temporary method called pack_TE_field_verbose() which I was using to see what was happening for mesh objects only. When I cleaned up my debugging code I did not evaluate this code carefully enough.

Comment on lines +92 to 95
for (U8 index = this_size; index > other_size; --index)
{
delete mEntryList[index-1];
}
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential infinite loop bug when other_size is 0. The loop variable 'index' is U8, so when it reaches 0 and decrements (--index), it will underflow to 255 instead of going negative, causing the condition 'index > other_size' to remain true indefinitely. The loop should either use a signed type for the index or explicitly check for the underflow condition.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLPrimTextureList::copy() is only happening in the context of initializing this object as a copy of other object, which has already been initialized and should never have zero texture entries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants