Subject: bootmem and mem_map allocation problem

From: Nicolas Pitre <[email-masked]>
Date: Sun, 5 Dec 1999 23:18:19 -0500 (EST)

I'm currently re-integrating the StrongARM SA1100 support into 2.3.29 but
here is the problem:

Physical RAM banks are located at 0xc0000000, 0xc8000000, 0xd0000000 and
0xd8000000.  The system I have contains 4 megs for each bank, but the
SA1100 can support many other configurations as well.  

The bootmem allocator keep track of all physical ram pages linearly, which
is still OK since the holes between all banks are marked reserved.
However this represents a total of 
((0xd8000000 + 4MB - 0xc0000000) >> PAGE_SHIFT) = 99328 pages total.

When it is time to allocate the mem_map array which is also indexing
physical pages, we have this:

	/*
	 * Some architectures (with lots of mem and discontinous memory
	 * maps) have to search for a good mem_map area:
	 */
	map_size = totalpages*sizeof(struct page);
	mem_map = (struct page *) alloc_bootmem(map_size);

map_size is then equal to 6754304 which is larger than a single ram
bank.  Then, alloc_bootmem() fails since it can't find enough
contiguous memory.  

Even if the mem_map "holes" can be freed later, the valuable
entries may not fit with the available memory layout, especially if we
want to keep the virt_to_phys()/phys_to_virt() macros simple.

Maybe the mem_map array could be indexed with virtual page location since
the virtual memory space is being compressed a bit, thus reducing the
number of entries required.  However this would require modifications to
the free_all_bootmem() function which isn't architecture specific.

What should be the correct fix for this?

Also I needed this patch to make things work on a machine with a non
null physical memory offset:

--- ../2.3.29/linux/mm/bootmem.c   Thu Dec  2 22:08:08 1999
+++ linux/mm/bootmem.c  Sat Dec  4 14:17:24 1999
@@ -38,7 +38,7 @@
        unsigned long mapsize = (pages+7)/8;

        mem_start = mstart;
-       bootmem_map = phys_to_virt(start << PAGE_SHIFT);
+       bootmem_map = phys_to_virt((start << PAGE_SHIFT) + mem_start);
        max_low_pfn = pages;

        /*
@@ -217,7 +217,7 @@
                        count++;
                        ClearPageReserved(page);
                        set_page_count(page, 1);
-                       if (i >= (virt_to_phys((char *)MAX_DMA_ADDRESS) >>
PAGE_SHIFT))
+                       if (i >= MAP_NR(MAX_DMA_ADDRESS) )
                                clear_bit(PG_DMA, &page->flags);
                        __free_page(page);
                }

Thanks.

Nicolas

unsubscribe: body of `unsubscribe linux-arm-kernel' to [email-masked]

---

From: Roman Zippel <[email-masked]>
Date: Mon, 6 Dec 1999 10:44:28 +0100 (MET)

Hi,

> want to keep the virt_to_phys()/phys_to_virt() macros simple.

I used the patch below (for 2.3.28, but should still apply, I had time yet
to update yet), but it removes the requirement for an 1:1 mapping of
physical memory to the memmap. Instead the whole logic is in MAP_NR(),
where an abitrary logic to convert a virtual address into a map nr can be
implemented.

bye, Roman

--- linux-2.3.28.org/mm/bootmem.c	Mon Nov 15 15:03:35 1999
+++ linux-2.3.28/mm/bootmem.c	Mon Nov 15 19:07:35 1999
@@ -203,18 +203,19 @@
 unsigned long __init free_all_bootmem (void)
 {
 	struct page * page;
-	unsigned long i, count, total = 0;
+	unsigned long i, nr, count, total = 0;
 
 	if (!bootmem_map) BUG();
 
-	page = mem_map;
 	count = 0;
-	for (i = 0; i < max_low_pfn; i++, page++) {
+	for (i = 0; i < max_low_pfn; i++) {
 		if (!test_bit(i, bootmem_map)) {
+			nr = MAP_NR(phys_to_virt(i << PAGE_SHIFT));
+			page = mem_map + nr;
 			count++;
 			ClearPageReserved(page);
 			set_page_count(page, 1);
-			if (i >= (virt_to_phys((char *)MAX_DMA_ADDRESS) >> PAGE_SHIFT))
+			if (nr >= (virt_to_phys((char *)MAX_DMA_ADDRESS) >> PAGE_SHIFT))
 				clear_bit(PG_DMA, &page->flags);
 			__free_page(page);
 		}

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Mon, 6 Dec 1999 19:14:02 +0000 (GMT)

> I'm currently re-integrating the StrongARM SA1100 support into 2.3.29 but

As I said quite prominently in the release notes for 2.3.29 for ARM, this is
expected at the moment:

> The 2.3.29 kernel patch is now on the FTP site.  I can say as a certainty
> that it will NOT work on any architecture which has a non-zero physical
> memory offset.  Depending on which patches get into Linus' kernel will
> depend on when this becomes available again.
> ...
> Again, please note that this kernel will not work on some of the ARM
> architectures (RiscPC for example).  There are fixes in the pipeline
> already.

Ingo has a fix for this, and I am currently waiting for him to integrate it
into the kernel; there is no point fixing something twice when it has already
been fixed.  Ingo's fix works well, and have tried to encourage him to forward
them to Linus, but have not received any response to date.

Unfortunately, Ingo's patch affects a number of files, which make it difficult
to manage when his patch goes into the mainstream.  If you're interested, I
could redistribute the patch separately.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |  http://www.arm.linux.org.uk/~rmk/armlinux.html    /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |