DOUBLE_FAULT exception under win7 x86 with KB4088878 #54
Comments
|
Thank you for detailed report and attaching files. That helps. I was able to repro the same issue and plan to work for a fix this weekend. |
//in VmmpHandleCrAccess
if (UtilIsX86Pae()) {
UtilLoadPdptes(UtilVmRead(VmcsField::kGuestCr3));
}the guest cr3 is an _KPROCESS::UserDirectoryTableBase // Returns a kernel CR3 value of the current process;
/*_Use_decl_annotations_*/ static ULONG_PTR VmmpGetKernelCr3() {
auto guest_cr3 = UtilVmRead(VmcsField::kGuestCr3);
// Assume it is an user-mode CR3 when the lowest bit is set. If so, get CR3
// from _KPROCESS::DirectoryTableBase.
if (guest_cr3 & 1) {
static const long kDirectoryTableBaseOffsetX64 = 0x28;
static const long kDirectoryTableBaseOffsetX86 = 0x18;
auto process = reinterpret_cast<PUCHAR>(PsGetCurrentProcess());
if (IsX64())
{
auto OriginalGsBase = UtilReadMsr(Msr::kIa32GsBase);
UtilWriteMsr(Msr::kIa32GsBase, UtilVmRead(VmcsField::kGuestGsBase));
guest_cr3 =
+ *reinterpret_cast<PULONG_PTR>(process + kDirectoryTableBaseOffsetX64);
UtilWriteMsr(Msr::kIa32GsBase, OriginalGsBase);
}
else
{
auto OriginalBase = UtilReadMsr(Msr::kIa32FsBase);
UtilWriteMsr(Msr::kIa32FsBase, UtilVmRead(VmcsField::kGuestFsBase));
guest_cr3 =
+ *reinterpret_cast<PULONG_PTR>(process + kDirectoryTableBaseOffsetX86);
UtilWriteMsr(Msr::kIa32FsBase, OriginalBase);
}
}
return guest_cr3;
} |
|
Does this resolve the issue? I was able to repro but could not find a solution last weekend. If it does, feel free to make a PR or send me a patch so I can give you a credit if you like. |
|
I'm sorry but the issue has not been resolved. The DOUBLE_FAULT exception still exists in the VMExit for the specific CR3 access after I attempted to modify the code. The guest CR3 value (such as
|
|
BTW the target to judge should be the source operand of the |
|
@tandasat sorry I misread the issue. The best way to solve #52 , #46, and this issue is to stop reading/writing guest virtual memory by switching host cr3 in any vmexit handler. A emulated GuestVA accessing & emulated linear-address translation could do the same without any problem. I did a small test that replace all CR3Switching job with my ReadWriteGuestMemory code referenced from Intel/haxm, it do work fine for now. Idtr gdtr;
gdtr.base = (ULONG_PTR)UtilVmRead(VmcsField::kGuestGdtrBase);
gdtr.limit = (USHORT)UtilVmRead(VmcsField::kGuestGdtrLimit);
VmmpWriteGuestVirtual((ULONG_PTR)operation_address,
sizeof(gdtr), &gdtr, sizeof(gdtr), 0);SIZE_T VmmpWriteGuestVirtual(ULONG_PTR addr, SIZE_T dst_buflen, const void *src, SIZE_T size, ULONG flag)
{
// TODO: use guest CPL for access checks
const char *srcp = (const char *)src;
SIZE_T offset = 0;
void *hva, *hva_base;
while (offset < size) {
ULONG64 gpa;
ULONG64 len = size - offset;
int r = VmmpTranslateVirtualAddress(addr + offset, TF_WRITE, &gpa, &len,
flag != 2);
if (r != 0) {
if (flag != 0)
return offset; // Number of bytes successfully written
if (r & TF_GP2HP) {
HYPERPLATFORM_LOG_ERROR_SAFE("VmmpWriteGuestVirtual(%llx, %x) failed\n", addr, size);
}
HYPERPLATFORM_LOG_ERROR_SAFE("VmmpWriteGuestVirtual(%llx, %x) injecting #PF\n", addr, size);
VmmpInjectPageFault(r & 0x1f, (ULONG_PTR)addr + offset);
return false;
}
hva_base = VmmpMapGuestPA(gpa, len);
if (hva_base) {
hva = (UCHAR *)hva_base + (gpa & 0xfff);
RtlCopyMemory(hva, (void *)(srcp + offset), min(len, dst_buflen - offset ));
}
else {
HYPERPLATFORM_LOG_ERROR_SAFE("VmmpWriteGuestVirtual Failed to VmmpMapGuestPA\n");
}
VmmpUnmapGuestPA(hva_base, len);
offset += (SIZE_T)len;
}
return flag != 0 ? size : true;
}
update1: After a 2 hours test, I got a GP fault inside PatchGuard code, dont know if the VmmpRead/WriteGuestVirtual is the I uploaded my ntoskrnl for further analysis.ntoskrnl.zip update2: It seems the missing update3: After adding PowerCallbackInitialization it still went BSOD with #GP fault, but no BSOD when descriptor_table_exiting = false, I will take a look at KVM to see if I have missed something. |
|
@hzqst Thank you for looking into a correct solution and referring to the sample implementation. I agree that avoiding CR3 is the right approach in general. Let me know if you find anything new. I will continue to look into both quick possible fix specific to this issue and your suggestion too. |
|
I am hitting a similar issue, but via kInvalidGuestState (0x21) vmexit reason instead of access to/from CR3. The exit is inside one of the new subroutines introduced by Microsoft related to Spectre/Meltdown fixes and involves a move to CR3: nt!KxIsrLinkageShadow:
fffff803`3cb6ba40 f644241001 test byte ptr [rsp+10h], 1
fffff803`3cb6ba45 7440 je nt!KxIsrLinkageShadow+0x47 (fffff803`3cb6ba87)
fffff803`3cb6ba47 56 push rsi
fffff803`3cb6ba48 0f01f8 swapgs
fffff803`3cb6ba4b 65488b342500700000 mov rsi, qword ptr gs:[7000h]
fffff803`3cb6ba54 650fba24251870000001 bt dword ptr gs:[7018h], 1
fffff803`3cb6ba5e 7203 jb nt!KxIsrLinkageShadow+0x23 (fffff803`3cb6ba63)
fffff803`3cb6ba60 0f22de mov cr3, rsi
fffff803`3cb6ba63 488d742438 ---guest IP ----> lea rsi, [rsp+38h]
fffff803`3cb6ba68 65488b242508700000 mov rsp, qword ptr gs:[7008h]
fffff803`3cb6ba71 ff76f8 push qword ptr [rsi-8]
fffff803`3cb6ba74 ff76f0 push qword ptr [rsi-10h]
fffff803`3cb6ba77 ff76e8 push qword ptr [rsi-18h]
fffff803`3cb6ba7a ff76e0 push qword ptr [rsi-20h]
fffff803`3cb6ba7d ff76d8 push qword ptr [rsi-28h]
fffff803`3cb6ba80 ff76d0 push qword ptr [rsi-30h]
fffff803`3cb6ba83 488b76c8 mov rsi, qword ptr [rsi-38h]
fffff803`3cb6ba87 e94405ecff jmp nt!KxIsrLinkage (fffff803`3ca2bfd0)
fffff803`3cb6ba8c c3 ret Here is the state of VM at time of exit:
The platform is Windows 10, x64. Here's a great write up that explains the reason for this CR3 move: |
|
@mgreshis did you try with the latest commit of HyperPlatform? This is likely a separate issue and the similar issue was fixed already. If it still reproduces with the latest commit, please file an another issue. |
|
@tandasat apologies, the issue has already been fixed by you. I am on MemoryMon branch and had to backport. Thanks again. |
Description
Under Windows 7 x86 SP1 with KB4088878 (the 2018-03 security update: http://download.windowsupdate.com/c/msdownload/update/software/secu/2018/03/windows6.1-kb4088878-x86_7512ab54d6a6df9d7e3d511d84a387aaeaeef111.msu), everytime loading and running HyperPlatform driver, a DOUBLE_FAULT exception would be raised during handling cr3 access in vmexit.
Expected behavior
Executing normally.
Actual behavior
The DOUBLE_FAULT exception was triggered by instruction
mov cr3,ecxin functionUtilLoadPdptes.Steps to reproduce the problem
Install the 2018-03 security update for Windows 7 x86 SP1: http://download.windowsupdate.com/c/msdownload/update/software/secu/2018/03/windows6.1-kb4088878-x86_7512ab54d6a6df9d7e3d511d84a387aaeaeef111.msu
Load and run the latest version of HyperPlatform (without any extra modification) in the Windows 7 environment.
The exception would be triggered immediately.
Specifications
Commit: 97d6ccc (the latest commit so far)
OS version: Windows 7 SP1 7601 with KB4088878
Architecture: x86
Hardware: VMware Workstation (any version)
Details:
The DOUBLE_FAULT exception was triggered by instruction
mov cr3,ecxin functionUtilLoadPdptes:The current VMExit is handling a cr3 access from system function
KiKernelSysretExit:Detail files: https://1drv.ms/u/s!ApQpgQkWR0QOi8ZOVXUBX83WEQIE8Q
The text was updated successfully, but these errors were encountered: