Skip to contentSkip to conversation

File corruption when running VMware.

DISCUSSION Petr Vandrovec 3 participants 8 messages Apr 26, 2002 – May 3, 2002

Subsystems (2)
lore
Share & tools
lore
#0 lore raw
Quoted 9 lines · Rik van Riel
Quoted 3 lines · Hong-Gunn Chew

I have a repeatable problem when running VMware workstation 3.00 and 3.01. The cause is still unknown, and could be VMware itself, the hardware or the kernel.

If you can reproduce it without VMware or with only the open source part of VMware (ie without any of the binary only parts) we might have a chance of debugging it.

Hi again,
one of 2.4.x kernel images available in SuSE's 8.0 has patched&enabled
support for page tables in high memory, and this quickly revealed incompatibility between VMware's vmmon page table handling and ptes above directly mapped range.

So if you have >890MB of RAM and your kernel is compiled with support
for pte in high memory, please stop using VMware, or reconfigure your kernel to not use pte in high memory (4GB config without pte-in-highmem is OK). Using pte-in-highmem with vmmon will cause kernel oopses and/or memory corruption :-(

If you do not have >890MB of memory, then reason for your memory corruption
is still unknown to me.
Best regards,
Petr Vandrovec [email protected]

#1 lore raw

Hi Petr,

Quoted 7 lines

Hi again,
one of 2.4.x kernel images available in SuSE's 8.0 has
patched&enabled
support for page tables in high memory, and this quickly revealed incompatibility between VMware's vmmon page table handling and ptes above directly mapped range.

Quoted 7 lines

So if you have >890MB of RAM and your kernel is compiled
with support for pte in high memory, please stop using VMware, or reconfigure your
kernel to not use pte in high memory (4GB config without pte-in-highmem is OK). Using pte-in-highmem with vmmon will cause kernel oopses and/or
memory corruption :-(

I do have 1GB of memory. I will try to reconfigure my kernel and see if there's still a problem.
Thanks for the info!

Cheers,
Hong-Gunn

#2 lore raw
In reply to Petr Vandrovec
Quoted 17 lines · Petr Vandrovec
Quoted 9 lines · Rik van Riel
Quoted 3 lines · Hong-Gunn Chew

I have a repeatable problem when running VMware workstation 3.00 and 3.01. The cause is still unknown, and could be VMware itself, the hardware or the kernel.

If you can reproduce it without VMware or with only the open source part of VMware (ie without any of the binary only parts) we might have a chance of debugging it.

Hi again,
one of 2.4.x kernel images available in SuSE's 8.0 has patched&enabled
support for page tables in high memory, and this quickly revealed incompatibility between VMware's vmmon page table handling and ptes above directly mapped range.

Quoted 4 lines

So if you have >890MB of RAM and your kernel is compiled with support
for pte in high memory, please stop using VMware, or reconfigure your kernel to not use pte in high memory (4GB config without pte-in-highmem is OK). Using pte-in-highmem with vmmon will cause kernel oopses and/or

passing to the kernel mem=850M in lilo at boot will be enough.

Quoted 8 lines

memory corruption :-(

If you do not have >890MB of memory, then reason for your memory corruption
is still unknown to me.
Best regards,
Petr Vandrovec [email protected]

Quoted 5 lines

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Andrea

#3 lore raw
In reply to Petr Vandrovec
Quoted 18 lines · Andrea Arcangeli
Quoted 17 lines · Petr Vandrovec
Quoted 9 lines · Rik van Riel
Quoted 3 lines · Hong-Gunn Chew

I have a repeatable problem when running VMware workstation 3.00 and 3.01. The cause is still unknown, and could be VMware itself, the hardware or the kernel.

If you can reproduce it without VMware or with only the open source part of VMware (ie without any of the binary only parts) we might have a chance of debugging it.

Hi again,
one of 2.4.x kernel images available in SuSE's 8.0 has patched&enabled
support for page tables in high memory, and this quickly revealed incompatibility between VMware's vmmon page table handling and ptes above directly mapped range.

Quoted 6 lines
Quoted 4 lines

So if you have >890MB of RAM and your kernel is compiled with support
for pte in high memory, please stop using VMware, or reconfigure your kernel to not use pte in high memory (4GB config without pte-in-highmem is OK). Using pte-in-highmem with vmmon will cause kernel oopses and/or

passing to the kernel mem=850M in lilo at boot will be enough.

I downloaded your latest driver from your site (vmware-ws-any-update16 package) and I adjusted it this way:

vmmon-only/linux/hostif.c renamed from vmmon-only/linux/hostif.c.~1~ +6 -4
@@ -176,7 +176,7 @@
 	unsigned long pagenr;
 	pgd_t *pgd;
 	pmd_t *pmd;
-	pte_t *pte;
+	pte_t *ptep, pte;
 	
 	pgd = pgd_offset(current->mm, addr);
 	if (pgd_none(*pgd))
@@ -184,10 +184,12 @@
 	pmd = pmd_offset(pgd, addr);
 	if (pmd_none(*pmd))
 		return 0;
-	pte = pte_offset(pmd, addr);
-	if (!pte_present(*pte))
+	ptep = pte_offset_atomic(pmd, addr);
+	pte = *ptep;
+	pte_kunmap(ptep);
+	if (!pte_present(pte))
 		return 0;
-	pagenr = pte_pagenr(*pte);
+	pagenr = pte_pagenr(pte);
 	return pagenr;
 #else
    int pdoffset = PFN_2_PDOFF(ppn);
vmnet-only/vmnetInt.h renamed from vmnet-only/vmnetInt.h.~1~ +2 -4
@@ -96,10 +96,8 @@
 #endif
 
 
-#ifndef KERNEL_2_5_5
-#   define pte_offset_map(_dir, _address) pte_offset(_dir, _address)
-#   define pte_unmap(_pte)
-#endif
+#   define pte_offset_map(_dir, _address) pte_offset_atomic(_dir, _address)
+#   define pte_unmap(_pte) pte_kunmap(_pte);
 
 
 #ifndef KERNEL_2_4_8

I'm running the patched driver right now with vmware 3.0 workstataion on my main desktop with 1G using 2.4.19-pre7 as kernel (pte-highmem enabled of course). If I'll find any instability of the host OS I'll let you know, so far it looks solid.

Hope this helps,

Andrea

#5 lore raw
In reply to Petr Vandrovec
Quoted 2 lines · Andrea Arcangeli

enabled of course). If I'll find any instability of the host OS I'll let you know, so far it looks solid.

The instability appears only during the poweron/resume, I left it running for a long time and it was solid, but only after I now restarted/stopped it a few times it showed stability problems still. If the poweron doesn't reboot the machine then it is solid (that's why I couldn't notice it yesterday). Also correcting the #if 0 in the patch or adapting the lower part doesn't help. The big question is: are them the only two places touching the pagetables? I also wonder why you're using cr3 instead of using the pointer in current->mm, I assume they're different and that you swap the cr3 internally to the vmware module during ctx switches of tasks?

thanks,

Andrea

#6 lore raw

RE: Memory corruption when running VMware. (was File curruption when running VMware)

In reply to Petr Vandrovec

Hi Petr, Andrea,

Quoted 7 lines · Petr

So if you have >890MB of RAM and your kernel is compiled
with support for pte in high memory, please stop using VMware, or reconfigure your
kernel to not use pte in high memory (4GB config without pte-in-highmem is OK). Using pte-in-highmem with vmmon will cause kernel oopses and/or
memory corruption :-(

I have been trying different kernel configurations to overcome this problem. I found that turning on APIC seem to work properly. However turning on IO-APIC causes it to hang just after X is started, which seems to be during the startup of gdm.
I do have a silly question to ask though. How do I compile the kernel NOT to use pte in high memory?

Quoted 1 line · Andrea

passing to the kernel mem=850M in lilo at boot will be enough.

This did not work as it causes vmware to seg fault with a kernel oops:

	Unable to handle kernel paging request at virtual address
ffffe350
	 printing eip:
	f61ff30a
	*pde = 00001063
	*pte = 00000000
	Oops: 0000
	CPU:    0
	EIP:
0010:[nls_iso8859-1:__insmod_nls_iso8859-1_S.data_L2336+139562/11435686]
Tainted: PF
	EIP:    0010:[<f61ff30a>]    Tainted: PF
	EFLAGS: 00013286
	eax: ffffe350   ebx: e7652600   ecx: ffffe350   edx: 000000d0
	esi: e7652600   edi: 00000069   ebp: e7625de8   esp: e7625de8
	ds: 0018   es: 0018   ss: 0018
	Process vmware (pid: 2595, stackpage=e7625000)
	Stack: e7625e08 f61ff34d ffffe350 c02f8ee0 e7625e2c f5095a00
000000d0 e6d21000  
	       e7625eb8 f61ff41f e7652600 e7625e8d e7625e8e e7625e8f
e7625e40 00003286  
	       00003286 efe78d80 000001f0 00000000 e7589a80 c01e330f
000000bc 000001f0
	Call Trace:

[nls_iso8859-1:__insmod_nls_iso8859-1_S.data_L2336+139629/11435619] [nls_iso8859-1:__insmod_nls_iso8859-1_S.data_L2336+139839/11435409] [sys_sendmsg+303/480] [wait_for_buffers+86/144] [nls_iso8859-1:__insmod_nls_iso8859-1_S.data_L2336+137406/11437842]

	Call Trace: [<f61ff34d>] [<f61ff41f>] [<c01e330f>] [<c0134a36>]

[<f61fea9e>]

[nls_iso8859-1:__insmod_nls_iso8859-1_S.data_L2336+132081/11443167] [generic_file_readahead+288/304] [bounce_end_io_read+164/288] [isapnp_set_mem+23/272] [vfs_link+23/256] [system_call+51/56]

	   [<f61fd5d1>] [<c01272b0>] [<c0131e94>] [<c01e0527>]

[<c013e487>] [<c0106f0b>]

	Code: 8b 11 89 d0 25 00 07 01 00 3d 00 04 00 00 75 12 81 ca 00

00

Cheers,
Hong-Gunn

#7 lore raw

Re: Memory corruption when running VMware. (was File curruption when running VMware)

In reply to Petr Vandrovec
Quoted 11 lines · Hong-Gunn Chew

Hi Petr, Andrea,

Quoted 7 lines · Petr

So if you have >890MB of RAM and your kernel is compiled
with support for pte in high memory, please stop using VMware, or reconfigure your
kernel to not use pte in high memory (4GB config without pte-in-highmem is OK). Using pte-in-highmem with vmmon will cause kernel oopses and/or
memory corruption :-(

Quoted 6 lines

I have been trying different kernel configurations to overcome this problem. I found that turning on APIC seem to work properly. However turning on IO-APIC causes it to hang just after X is started, which seems to be during the startup of gdm.
I do have a silly question to ask though. How do I compile the kernel NOT to use pte in high memory?

the simplest patch to avoid pte in highmem is this. It's against 19pre7aa3 but it will apply cleanly to the kernel you're using.

mm/memory.c renamed from mm/memory.c.~1~ +1 -1
@@ -1493,7 +1493,7 @@
 {
 	struct page * page;
 
-	page = alloc_page(GFP_KERNEL | __GFP_HIGHMEM);
+	page = alloc_page(GFP_KERNEL);
 	if (page)
 		clear_pagetable(page);
 	return page;

However if I would be in you I'd simply add mem=850M to the append line in /etc/lilo.conf until the vmmon/vmnet update is released, that should be the most confortable approch from your part.

Quoted 4 lines
Quoted 1 line · Andrea

passing to the kernel mem=850M in lilo at boot will be enough.

This did not work as it causes vmware to seg fault with a kernel oops:

Hmm looking at the bounce_end_io_read it looks like the kernel didn't recognized you specified mem=850M via lilo, not idea why. You should check with free that your kernel is using only 850M of ram before starting vmware. Can you show the contents of /proc/meminfo right after boot?

Quoted 10 lines

[generic_file_readahead+288/304] [bounce_end_io_read+164/288] [isapnp_set_mem+23/272] [vfs_link+23/256] [system_call+51/56]

	   [<f61fd5d1>] [<c01272b0>] [<c0131e94>] [<c01e0527>]

[<c013e487>] [<c0106f0b>]

	Code: 8b 11 89 d0 25 00 07 01 00 3d 00 04 00 00 75 12 81 ca 00

00

Cheers,
Hong-Gunn

Andrea