Subject: memory mapping

From: Mark van Doesburg <[email-masked]>
Date: Tue, 1 Sep 1998 23:37:17 +0200

There is no segmentation of the memory space.  Hence, if you want the
	kernel to be located at 0x10000000 (the SDRAM physical address), all
	user binaries will be limited to 0x00008000 to 0x10000000, ie 250MB.

OK, understood. (btw the 21285 maps SDRAM to 0x0)

	Do you have a device that provides 2GB of memory at one time?  I'd think
	that this is unlikely, and that you probably have a set of cards.  The
	resulting PCI usage will generally be quite sparse.  Therefore, you
	should be able to get away with mapping the areas that contain useful
	information only.

Well, the prototype would probably have 256MB in the host and 8*64 on
the 8 sa co-processors. This would mean 768MB, however this is all
mapped in PCI space by the bios on the host, which means i have no
control over it.  Most PC bioses would map the RAM of the host on the
lower 256MB in PCI space and would map the PCI cards growing downward
from the 4GB top.

	ioremap is not currently implemented on ARM yet, but it is in my pipeline
	of things to do.  If you want to code it, please look at what the i386 
	one does and send the relevent (tested) patches to my patch system.

If i write patches that might be usefull i will mail them to your patch
system. (The 21285 serial driver is still untested and will be for some
time i'm afraid)

	> I've already registered another architecture to do all of this, since
	> the virt_to_bus and bus_to_virt would be different for my board.

	Are they significantly different?

The ebsa-285 assumes the SDRAM is mapped at a fixed location in PCI
space.  This  certainly won't be the case when the ebsa-285 is used as a
co-processor since it is mapped by the host. The bus_to_virt and
virt_to_bus functions would have to read the SDRAM base address
registers from the PCI configuration space. This would probably work
just as well on the current ebsa-285 arch.

Another reason for the new arch is that i want to have an easy way to
give the kernel parameters. I also want the reboot/halt code to restart
the ebsa-285 in a way so i can load a new kernel without rebooting the
host.

	> Finally, i would like to know how the user process sees the memory
	> space.  Does the kernel appear in a non accessable part of the memory,
	> and if so, where does it appear ?

	You've already mentioned it!  The kernel appears at 0xc0000000 in all
	current machines.

	I would prefer that the memory structure is identical across all machines,
	especially the user-space, to guarantee binary compatability.

I agree, i don't want this system to be different.

Loading the kernel at 0x60000000 would solve my memory problems. This
would give me access to all the memory/io the 21285 can have (including
the 4GB PCI memory space). It would still leave 1.5GB for user binaries.

Would this be compatible ?

Would this 1.5GB be per process or for all processes together ?  (Is the
MMU register reloaded on a task switch ?)

Thank for your explanation,

Mark.
unsubscribe: body of `unsubscribe linux-arm' to [email-masked]

---

From: Philip Blundell <[email-masked]>
Date: Wed, 02 Sep 1998 10:20:36 +0100

>space.  This  certainly won't be the case when the ebsa-285 is used as a

This needs to be fixed anyway.  Basically we need to read the bit from the 
21285 that tells it whether or not to be the PCI "central function", and if 
not avoid doing all the PCI config stuff.  Then __virt_to_bus() and so on need 
to be taught that the offset might not be constant.  I think the overhead is 
small enough we can ignore it but you could always make it a config option if 
people were that worried.

>Would this be compatible ?

It would be compatible for user processes.  It doesn't all that much matter 
where the kernel is loaded across different architectures so you could change 
it if you really wanted to.

No matter where you map the kernel you aren't going to be able to get at the 
whole of the 4GB PCI memory space.  I would have thought the best thing to do 
would be to leave the kernel where it is and ioremap() memory in as and when 
you want it rather than trying to always map the whole lot at once.

>MMU register reloaded on a task switch ?)

It's per process, and yes.  This is why context switches are so slow on the 
ARM; we need to flush the cache because the memory mappings have changed.

p.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Wed, 2 Sep 1998 22:16:53 +0100 (BST)

> the 4GB PCI memory space). It would still leave 1.5GB for user binaries.

The user space is not just a simple matter of one single binary image.
User space is built up of multiple binary images - the user program
and it's librarys, plus sysv ipc shm areas as well.

Your suggestion is that we redefine the location of all of these, which IMHO
is not a good idea.

I am assuming that you're thinking about doing some sort of multi-processor
Linux kernel on this machine, in which case have you considered all the issues
to do with cache coherency?

The StrongARM does not have any cache snooping like the Intel does, and this
means that shared memory space is fraught with problems.

Looking at the Intel code, it appears that they only map in the memory on an
as and needed basis.  I don't see that doing a similar thing will reduce
performance, if it's done properly.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |  http://www.arm.uk.linux.org/~rmk/armlinux.html    /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |
unsubscribe: body of `unsubscribe linux-arm' to [email-masked]

---

From: Philip Blundell <[email-masked]>
Date: Wed, 02 Sep 1998 23:26:37 +0100

>and it's librarys, plus sysv ipc shm areas as well.

The libraries are (assuming ELF) pretty much completely relocatable.  I'm sure 
SysV shared memory can be battered into line somehow.

>From what I've seen he seems to be planning most of a cluster-type 
architecture rather than actual multiprocessing, in which case cache coherency 
doesn't enter into it.  We've been through the reasons that real SMP is bad 
news on current ARM machines quite recently, but a Beowulf-type cluster that 
used PCI for message passing would be a neat toy.

p.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Wed, 2 Sep 1998 23:54:43 +0100 (BST)

> used PCI for message passing would be a neat toy.

Would this really require all memory to be accessible at all times by
everything?
   _____

---

From: "Adam 'WeirdArms' Wiggins" <[email-masked]>
Date: Thu, 3 Sep 1998 10:14:22 +1000 (EST)

> used PCI for message passing would be a neat toy.

	What about some mutliproccesing systems with loose cupling via
dual ported SRAMS for msg passing regions? It's not SMP but its
mutliprocessing of a kind.
	Cheers Adam

---

From: Matthew Wilcox <[email-masked]>
Date: Thu, 03 Sep 1998 14:25:58 +0200

> Would this really require all memory to be accessible at all times by

No, Beowulf is a NUMA (Non-Uniform Memory Architecture) system - processors
can't access remote memory directly.  They communicate by message passing. 
Conventional Beowulf systems communicate over 100MBit ethernet, so
communicating via PCI should be significantly faster.

-- 
Matthew Wilcox <[email-masked]>
"I decry the current tendency to seek patents on algorithms.  There are
better ways to earn a living than to prevent other people from making use of
one's contributions to computer science."  -- Donald E. Knuth, TAoCP vol 3
unsubscribe: body of `unsubscribe linux-arm' to [email-masked]

---

From: Philip Blundell <[email-masked]>
Date: Thu, 03 Sep 1998 13:37:43 +0100

>Would this really require all memory to be accessible at all times by

No, it's certainly not necessary but I can see why Mark would prefer it to be 
that way.  So long as he's aware of the tradeoffs I don't see any reason why 
he shouldn't change the mappings to do what he wants.  It sounds like this is 
always going to be a system of fairly specialist interest.

Another option incidentally might be to dedicate a region of say 1MB on each 
CPU for message passing and have only that area mapped into PCI space rather 
than the whole of RAM.

p.