From: Larry Woodman <lwoodman@redhat.com> Date: Tue, 15 Sep 2009 16:24:32 -0400 Subject: [x86_64] PCI space below 4GB forces mem remap above 1TB Message-id: 1253046272.25569.105.camel@dhcp-100-19-198.bos.redhat.com O-Subject: [RHEL5-U5 patch] Prevent boot-time panic when system has 1T RAM and the PCI space below 4GB forces memory remapping above 1TB Bugzilla: 523522 RH-Acked-by: Rik van Riel <riel@redhat.com> RH-Acked-by: Anton Arapov <anton@redhat.com> RH-Acked-by: Stefan Assmann <sassmann@redhat.com> RH-Acked-by: Prarit Bhargava <prarit@redhat.com> When holes in RAM push end of memory above 1TB system panics on boot freeing the bootmem because end of RAM is higher than 2^40(MAX_PHYSMEM_BITS). This e820 map has 1TB but since PCI re-mapped end_pfn is above 1TB: ---------------------------------------------------------------- BIOS-provided physical RAM map: BIOS-e820: 0000000000010000 - 0000000000095c00 (usable) BIOS-e820: 0000000000095c00 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 000000007f750000 (usable) BIOS-e820: 000000007f75e000 - 000000007f760000 type 9 BIOS-e820: 000000007f760000 - 000000007f76e000 (ACPI data) BIOS-e820: 000000007f76e000 - 000000007f7d0000 (ACPI NVS) BIOS-e820: 000000007f7d0000 - 000000007f7e0000 (reserved) BIOS-e820: 000000007f7ec000 - 0000000090000000 (reserved) BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved) BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved) BIOS-e820: 0000000100000000 - 0000010080000000 (usable) ----------------------------------------------------------------- Unable to handle kernel paging request at 000000038000001c RIP: [<ffffffff8041050c>] __free_pages_bootmem+0x72/0xc0 PGD 0 Oops: 0002 [1] SMP last sysfs file: CPU 0 Modules linked in: Pid: 0, comm: swapper Not tainted 2.6.18-164.el5 #1 RIP: 0010:[<ffffffff8041050c>] [<ffffffff8041050c>] __free_pages_bootmem+0x72/0xc0 RSP: 0018:ffffffff803f1ee0 EFLAGS: 00010206 RAX: 0000000380000054 RBX: ffff81e081c369c8 RCX: 0000000000000001 RDX: 000000038000001c RSI: 0000000000000006 RDI: 000000038000001c RBP: ffffffffffffffff R08: 0000000000000000 R09: ffffffff803f1ea8 R10: ffffffff803f1ea8 R11: 0000000000001000 R12: 0000000010000000 R13: 0000000000000000 R14: 0000000001f80000 R15: ffffffff80467a40 FS: 0000000000000000(0000) GS:ffffffff803c0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: 000000038000001c CR3: 0000000000201000 CR4: 00000000000006a0 Process swapper (pid: 0, threadinfo ffffffff803f0000, task ffffffff802ffae0) Stack: ffffffff8040fb88 ffff81e080006000 0000000001f1179a 0000000002000000 0000000100000000 0000000380000000 0000000000000007 000000000dc88a04 00000000000000d0 0000000000000000 0000000000000000 ffff810000000000 Call Trace: [<ffffffff8040fb88>] free_all_bootmem_core+0xf9/0x221 [<ffffffff8040c920>] numa_free_all_bootmem+0x3b/0x74 [<ffffffff8027cd02>] _etext+0x0/0x17e2fe [<ffffffff8040ba4a>] mem_init+0x78/0x19b [<ffffffff804113c4>] inode_init_early+0x53/0x71 [<ffffffff803fb75d>] start_kernel+0x180/0x225 [<ffffffff803fb22f>] _sinittext+0x22f/0x236 ------------------------------------------------------------------------------ The attached patch fixes this problem by limiting the end_pfn of physical memory to 1TB. Fixes BZ 523522 diff --git a/arch/x86_64/kernel/e820.c b/arch/x86_64/kernel/e820.c index f6702b2..6505e3b 100644 --- a/arch/x86_64/kernel/e820.c +++ b/arch/x86_64/kernel/e820.c @@ -194,7 +194,7 @@ void __init e820_bootmem_free(pg_data_t *pgdat, unsigned long start,unsigned lon free_bootmem_node(pgdat, addr, last-addr); } } - +#define TB_PFN (1UL << (40 - PAGE_SHIFT)) /* * Find the highest page frame number we have available */ @@ -219,7 +219,9 @@ unsigned long __init e820_end_of_ram(void) end_pfn_map = end>>PAGE_SHIFT; } } - + /* cap last pfn at 1TB */ + if (end_pfn > TB_PFN) + end_pfn = TB_PFN; if (end_pfn > end_pfn_map) end_pfn_map = end_pfn; if (end_pfn_map > MAXMEM>>PAGE_SHIFT)