Skip to content

Commit

Permalink
Defer dataAddr eval & use mainline store for nullSource OffHeap x86 A…
Browse files Browse the repository at this point in the history
…rrayStoreCHK

ArrayStoreCHK in x86 uses null check and OOL store path for nullSource
stores, evaluating the destination address early to be used in either
the mainline or OOL store paths. For OffHeap evaluating the destination
address includes loading the dataAddr pointer, where having it alive
across GC point (OOL ArrayTypeCheck) will cause a crash [1].

To defer the evaluation and loading the dataAddr into a register, this
consolidate the store paths for null and non-null stores and defers the
destination address evaluation till before the store happens for OffHeap.

Side-effect of this is null stores performing a wrtbar/card-marking even
thought it's not necessary but opted for instead of a second null check.

[1]
Reason of crashing is evaluating the dataAddr pointer into a collectable
register. A long-term solution is fixing that without the impact of
setting every dataAddr as an internal-pointer.
  • Loading branch information
rmnattas committed Feb 13, 2025
1 parent b6b5bb6 commit ff967f7
Showing 1 changed file with 26 additions and 5 deletions.
31 changes: 26 additions & 5 deletions runtime/compiler/x/codegen/J9TreeEvaluator.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2957,11 +2957,25 @@ TR::Register *J9::X86::TreeEvaluator::ArrayStoreCHKEvaluator(TR::Node *node, TR:
!fej9->classHasBeenExtended(node->getArrayStoreClassInNode())
) ? true : false;

// OffHeap runs defer destination evaluation after GC point.
static char *disableDeferDestinationEvaluation = feGetEnv("TR_DisableDeferDestinationEvaluation");
bool deferDestinationEvaluation = TR::Compiler->om.isOffHeapAllocationEnabled() && !disableDeferDestinationEvaluation;

doneLabel = generateLabelSymbol(cg);
doneLabel->setEndInternalControlFlow();

doNullStoreLabel = generateWriteBarrier ? generateLabelSymbol(cg) : doneLabel;
startOfWrtbarLabel = generateWriteBarrier ? generateLabelSymbol(cg) : doNullStoreLabel;
if(generateWriteBarrier)
{
startOfWrtbarLabel = generateLabelSymbol(cg);
// For OffHeap we use mainline store for null stores to consolidate store paths and defer
// destination evaluation. OffHeap will perform redundant wrtbar on null stores.
doNullStoreLabel = deferDestinationEvaluation ? startOfWrtbarLabel : generateLabelSymbol(cg);
}
else
{
startOfWrtbarLabel = doneLabel;
doNullStoreLabel = doneLabel;
}

bool usingCompressedPointers = false;
bool usingLowMemHeap = false;
Expand Down Expand Up @@ -2993,7 +3007,7 @@ TR::Register *J9::X86::TreeEvaluator::ArrayStoreCHKEvaluator(TR::Node *node, TR:

TR::MemoryReference *tempMR = NULL;

if (generateWriteBarrier)
if (generateWriteBarrier && !deferDestinationEvaluation)
{
tempMR = generateX86MemoryReference(firstChild, cg);
}
Expand Down Expand Up @@ -3123,6 +3137,12 @@ TR::Register *J9::X86::TreeEvaluator::ArrayStoreCHKEvaluator(TR::Node *node, TR:
sourceChild->setIsNonNull(true);
}

if (deferDestinationEvaluation)
{
// Perform deferred destination evaluation
tempMR = generateX86MemoryReference(firstChild, cg);
}

TR::TreeEvaluator::VMwrtbarWithStoreEvaluator(
node,
tempMR,
Expand Down Expand Up @@ -3153,7 +3173,8 @@ TR::Register *J9::X86::TreeEvaluator::ArrayStoreCHKEvaluator(TR::Node *node, TR:

if (!isRealTimeGC)
{
if (generateWriteBarrier)
// OffHeap uses the already generated mainline VMwrtbarWithStoreEvaluator for null stores
if (generateWriteBarrier && !deferDestinationEvaluation)
{
assert(isNonRTWriteBarrierRequired);
assert(tempMR);
Expand All @@ -3176,7 +3197,7 @@ TR::Register *J9::X86::TreeEvaluator::ArrayStoreCHKEvaluator(TR::Node *node, TR:
generateLabelInstruction(TR::InstOpCode::JMP4, node, doneLabel, cg);
og.endOutlinedInstructionSequence();
}
else
else if (!generateWriteBarrier)
{
// No write barrier emitted. Evaluate the store here.
//
Expand Down

0 comments on commit ff967f7

Please sign in to comment.