From: Chris Lalancette <clalance@redhat.com> Date: Mon, 9 Jun 2008 11:45:36 +0200 Subject: [xen] HV: memory corruption with large number of cpus Message-id: 484CFBC0.9070404@redhat.com O-Subject: [RHEL5.3 PATCH]: Fix random memory corruption in the HV Bugzilla: 449945 RH-Acked-by: Bill Burns <bburns@redhat.com> RH-Acked-by: Markus Armbruster <armbru@redhat.com> RH-Acked-by: Don Dutile <ddutile@redhat.com> All, During 5.1 we got sporadic reports of hypervisor crashes during context switch with machines with lots of cores (16 or more, bz 294551). During 5.2 these crashes seemed to go away, but we had a sneaking suspicion that the bug was hiding and not actually fixed. I recently went back to try and find the root cause of this bug, and found out that it is, indeed, just hiding in 5.2. Basically what's happening is that we are writing data beyond the end of an array, which was causing the memory corruption. The upstream xen-3.1-testing repository has fixed this in c/s 15601 by just removing the array and associated code in question, since it was never used. The attached patch is a backport of that code. This resolves BZ 449945 for 5.3, and I also believe we should put this into the 5.2.z stream. Tested by me on an x86_64 machine with 16 cores. With a stock 5.2 HV, the bug is kind of hard to trigger, but to induce failure, I rebuilt the 5.2 HV with max_phys_cpus=32 (the old default). Before the patch, I was able to get the HV to crash within a few minutes of a fully virtualized guest install. After the patch, I was able to start hundreds of guest installs without crashing. Please ACK. Chris Lalancette diff --git a/arch/x86/smpboot.c b/arch/x86/smpboot.c index 2556314..667f533 100644 --- a/arch/x86/smpboot.c +++ b/arch/x86/smpboot.c @@ -543,40 +543,6 @@ extern struct { unsigned short ss; } stack_start; -#ifdef CONFIG_NUMA - -/* which logical CPUs are on which nodes */ -cpumask_t node_2_cpu_mask[MAX_NUMNODES] __read_mostly = - { [0 ... MAX_NUMNODES-1] = CPU_MASK_NONE }; -/* which node each logical CPU is on */ -int cpu_2_node[NR_CPUS] __read_mostly = { [0 ... NR_CPUS-1] = 0 }; -EXPORT_SYMBOL(cpu_2_node); - -/* set up a mapping between cpu and node. */ -static inline void map_cpu_to_node(int cpu, int node) -{ - printk("Mapping cpu %d to node %d\n", cpu, node); - cpu_set(cpu, node_2_cpu_mask[node]); - cpu_2_node[cpu] = node; -} - -/* undo a mapping between cpu and node. */ -static inline void unmap_cpu_to_node(int cpu) -{ - int node; - - printk("Unmapping cpu %d from all nodes\n", cpu); - for (node = 0; node < MAX_NUMNODES; node ++) - cpu_clear(cpu, node_2_cpu_mask[node]); - cpu_2_node[cpu] = 0; -} -#else /* !CONFIG_NUMA */ - -#define map_cpu_to_node(cpu, node) ({}) -#define unmap_cpu_to_node(cpu) ({}) - -#endif /* CONFIG_NUMA */ - u8 cpu_2_logical_apicid[NR_CPUS] __read_mostly = { [0 ... NR_CPUS-1] = BAD_APICID }; static void map_cpu_to_logical_apicid(void) @@ -585,13 +551,11 @@ static void map_cpu_to_logical_apicid(void) int apicid = hard_smp_processor_id(); cpu_2_logical_apicid[cpu] = apicid; - map_cpu_to_node(cpu, apicid_to_node(apicid)); } static void unmap_cpu_to_logical_apicid(int cpu) { cpu_2_logical_apicid[cpu] = BAD_APICID; - unmap_cpu_to_node(cpu); } #if APIC_DEBUG