From: Vitaly Mayatskikh <vmayatsk@redhat.com> Date: Tue, 24 Jun 2008 17:05:17 +0200 Subject: [x86] brk: fix RLIMIT_DATA check Message-id: m363ryeuvm.fsf@gravicappa.englab.brq.redhat.com O-Subject: Re: [RHEL-5.3 PATCH] BZ315681 data segment limits cause libc init to fail on static binaries. Bugzilla: 315681 RH-Acked-by: Jerome Marchand <jmarchan@redhat.com> RH-Acked-by: Rik van Riel <riel@redhat.com> https://bugzilla.redhat.com/show_bug.cgi?id=315681 Description: ============ Randomization of heap location was introduced in RHEL-5, but RLIMIT_DATA check in sys_brk() was incorrect. This cause random segmentation faults when data segment limit is set. Upstream status: ================ commit 4cc6028d4040f95cdb590a87db478b42b8be0508 commit c1d171a002942ea2d93b4fbd0c9583c56fce0772 Test status of the patch: ========================= Tested by me on x86_64 diff --git a/arch/i386/kernel/process.c b/arch/i386/kernel/process.c index da00938..253ed50 100644 --- a/arch/i386/kernel/process.c +++ b/arch/i386/kernel/process.c @@ -957,6 +957,6 @@ void randomize_brk(unsigned long old_brk) range_end = range_start + 0x02000000; new_brk = randomize_range(range_start, range_end, 0); if (new_brk) - current->mm->brk = new_brk; + current->mm->brk = current->mm->start_brk = new_brk; } diff --git a/arch/x86_64/kernel/process.c b/arch/x86_64/kernel/process.c index cce5327..9c60cb2 100644 --- a/arch/x86_64/kernel/process.c +++ b/arch/x86_64/kernel/process.c @@ -868,6 +868,6 @@ void randomize_brk(unsigned long old_brk) new_brk = randomize_range(range_start, range_end, 0); if (new_brk) - current->mm->brk = new_brk; + current->mm->brk = current->mm->start_brk = new_brk; } diff --git a/mm/mmap.c b/mm/mmap.c index d89ab1f..264fb13 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -242,7 +242,7 @@ asmlinkage unsigned long sys_brk(unsigned long brk) down_write(&mm->mmap_sem); - if (brk < mm->end_code) + if (brk < mm->start_brk) goto out; /* @@ -252,7 +252,8 @@ asmlinkage unsigned long sys_brk(unsigned long brk) * not page aligned -Ram Gupta */ rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur; - if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim) + if (rlim < RLIM_INFINITY && (brk - mm->start_brk) + + (mm->end_data - mm->start_data) > rlim) goto out; newbrk = PAGE_ALIGN(brk);