Subject: Zones and Nodes

From: Alexander Schulz <[email-masked]>
Date: Tue, 23 May 2000 19:30:58 +0200

Hello,

is there any documentation about the current memory management in Linux?
In particular: what are the nodes? Where will I have to start if I want
to change the zone some part of memory is in?

Thanks
  Alexander

unsubscribe: body of `unsubscribe linux-arm-kernel' to [email-masked]

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Tue, 23 May 2000 23:07:10 +0100 (BST)

> to change the zone some part of memory is in?

There is only the stuff in Documentation/, and some stuff pointed to by links
from there.  However, in answer to your question:

There are 3 zones in Linux:

 - one which all DMA is sourced from
 - one which all usual memory is sourced from
 - one which so called indirectly mapped "highpages" are sourced from

Allocations requested further down the list can fall up the list if the
relevent zone is empty.  The idea is to prevent situations where you may
have 256MB of memory in a PC, and all your tasks are sitting in the first
16MB of that memory, which means you can't reliably obtain DMA memory.

"Nodes" are to do with non-unified memory architecture (NUMA) stuff, which
does have a separate web site.  The support in Linux for this provides us
with the following:

 - we can allocate memory from a "node" which is fast to access
   (eg, one "node" may be local, another may be across a PCI bus, and
   is therefore slower)
 - reduced size of the mem_map array which contains information about
   every single page of memory in a system, when the memory is not
   contiguous.

Why do you want to change the zone some memory is in?
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: S A McConnell <[email-masked]>
Date: Wed, 24 May 2000 09:18:38 -0500

>  - one which so called indirectly mapped "highpages" are sourced from

What is the recomended approach for allocating memory for non-PCI lower
24-bit 8-word aligned  (0x00fffffe0) dedicated Memory-to-Device DMA?
(Used by many LCD controllers for moving video data.)

Users need a virtual address of the Video RAM the controller needs a
physical address.

Documentation/DMA-mapping text says the PCI interface does not support
such limited DMA addressing.

The only approach that we can see to this is to:
	- pre-allocate the memory
	- Remove it from the memory pool (with a fix up)
	- Provide a virtual to physical mapping.

We were wondering if there was a better method? Can you point out an
example?

Thanks,
SAM

Scott A McConnell

---

From: [email-masked] (Erik Mouw)
Date: Wed, 24 May 2000 19:28:26 +0200 (CEST)

> We were wondering if there was a better method? Can you point out an

Have a look at consistent_alloc() (and friends) in arch/arm/mm/consistent.c,
this function will exactly do what you want.

Erik

-- 
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031,  2600 GA Delft, The Netherlands
Phone: +31-15-2783635  Fax: +31-15-2781843  Email: [email-masked]
WWW: http://www-ict.its.tudelft.nl/~erik/

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Wed, 24 May 2000 21:16:00 +0100 (BST)

> We were wondering if there was a better method? Can you point out an

There is.

Under your architecture, you would have to allocate the usable DMA memory
to ZONE_DMA, and the rest to ZONE_NORMAL.  Whenever you ask any allocator
in the kernel for memory with a gfp mask value ored with GFP_DMA, then
you will end up getting memory only from ZONE_DMA, ie, in your case memory
which has the top 8 bits of the physical address clear.

Depending on the requirements of your peripheral, you may be able to
do with non-contiguous pages for DMA, in which case I'd strongly
recommend that you use the PCI consistency stuff with a NULL pci_device
pointer to get a region of uncached PCI space.  You can then mmap()
this space (but that's not trivial at the moment).

The alternative is to allocate a large region using __get_free_pages(),
but not that this can, and most likely will fail when you're getting low
on memory.

The former is the preferred method if at all possible.
   _____

---

From: Nicolas Pitre <[email-masked]>
Date: Wed, 24 May 2000 17:46:12 -0400 (EDT)

> this space (but that's not trivial at the moment).

Incidentally, I was looking at some code which looks like this:

	db->rawbuf = pci_alloc_consistent(s->dev, PAGE_SIZE << order,
					  &db->dmaaddr);
	[...]
	/* now mark the pages as reserved; otherwise remap_page_range
	 * doesn't do what we want */
	 */
	mapend = MAP_NR(db->rawbuf + (PAGE_SIZE << db->buforder) - 1);
	for (map = MAP_NR(db->rawbuf); map <= mapend; map++)
	        set_bit(PG_reserved, &mem_map[map].flags);

(excert from drivers/sound/es1371.c)

It looks to me that this will work on I386, but fail on ARM, Sparc and
probably other architectures as well since the provided virtual address is
generated through a vmalloc() which MAP_NR() isn't able to make sense of.

What's the correct way to handle mmap() onto such addresses?

This is maybe a LK question...

Nicolas

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Wed, 24 May 2000 22:48:13 +0100 (BST)

> What's the correct way to handle mmap() onto such addresses?

Yes, well, ahem.  That's the messy bit.  Although video-cyberpro.c doesn't
use pci_alloc_consistent, it does remap_page_range() on each page
individually.

However, I'm planning to investigate the kiobuf stuff, which should clean
some (and I mean only some) of it up.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Nicolas Pitre <[email-masked]>
Date: Wed, 24 May 2000 19:49:04 -0400 (EDT)

> some (and I mean only some) of it up.

Maybe a generic method is required so all those drivers would
be portable across all architectures, just like what happened with PCI
consistent memory.

Nicolas

---

From: S A McConnell <[email-masked]>
Date: Wed, 24 May 2000 21:52:17 -0500

Russell King - ARM Linux Admin wrote:

> to ZONE_DMA, and the rest to ZONE_NORMAL.

O.k. I will check to see what we are placing in ZONE_DMA.

We also have GPIO DMA and memory-to-memory. I have not dug into those
ideas.

>  Whenever you ask any allocator

Code taken from sa1100fb.c

I tried this and it returned an out of range address >24 bits.

       if ((allocated_region = 
             (char *)__get_free_pages(GFP_KERNEL | GFP_DMA, order)) ==
NULL)
           return -ENOMEM;

> this space (but that's not trivial at the moment).

I will have to think about this. The device expects a physical address
of the start of video RAM. You also program it with Pixels Per Line and
number of lines per screen. That tells me it wants contiguous physical
memory.

> but not that this can, and most likely will fail when you're getting low
Yep :(

Thanks for the info!

As for the mmap() you and Eric have really confused me. Expect some
follow up questions :-) after I do some more studying.

SAM

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Thu, 25 May 2000 09:05:56 +0100 (BST)

> this function will exactly do what you want.

Please, don't use consistent_alloc() and friends directly - go via the
pci_* versions for the moment and pass a NULL pci_device pointer.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |