Subject: Re: memory mapping

From: Mark van Doesburg <[email-masked]>
Date: Thu, 3 Sep 1998 03:40:31 +0200

The user space is not just a simple matter of one single binary
	image.  User space is built up of multiple binary images - the
	user program and it's librarys, plus sysv ipc shm areas as well.

Can you give me details of the current mapping ?

I assume it is not the intention to map kernel only memory below the
address where the kernel is mapped. Am i right ?

	Your suggestion is that we redefine the location of all of
	these, which IMHO is not a good idea.

Would this be much work ?

I am not suggesting you should remap the kernel to 0x60000000 on all
architectures, but what would be so bad about it ?

	I am assuming that you're thinking about doing some sort of
	multi-processor Linux kernel on this machine, in which case have
	you considered all the issues to do with cache coherency?

What i want to make is a multiprocessor render engine. It is a "normal"
computer (pentium/alpha/sparc) with one or more PCI cards in it. These
PCI cards have 4 or 8 sa-110/21285/64MB SDRAM sets on it. They are going
to communicate with the host and each other via TCP/IP links. These
links are made by having the processors copying the packets to each
others memories. The only time a strongarm would have to flush the cache
is when it receives a packet from another strongarm. These packets
would present only a fraction of the communication. Most packets will
come from/go to the host, and no cache flushes would be needed for those
packets.

After having this setup i would add shared memory segments. These would
be located on the host, which has cache coherency hardware.

	The StrongARM does not have any cache snooping like the Intel
	does, and this means that shared memory space is fraught with
	problems.

I want to map the PCI memory space uncached. That would not give a
preformance penalty for me. I have to calculate a lot of data en put the
results somewhere. This putting of the data would be relativly random
access. Sending it with TCP/IP would be too much overhead.

	Looking at the Intel code, it appears that they only map in the
	memory on an as and needed basis.  I don't see that doing a
	similar thing will reduce performance, if it's done properly.

It won't reduce preformance, but there simply is not enough room to map
it.  What i need right now is 768MB, this won't fit with the current
mapping.  There are also other plans, these future projects will require
much more, we will probably put over 3GB ram in the machine. I need to
map the entire 2GB PCI memory area of the 21285 to access all of this.

There are alternatives offcourse. One thing i could do is map the memory
of the host first. After that i would map as many strongarms as
possible.  All strongarms that can not be accessed by shared memory
would then be accessed with the DMA processor. Doing this has two major
problems The first is that i would have to flush the cache of both
strongarms when i send a packet from one sa to another. The second
reason is that i would have to put the packet messages in the memory of
the host. This would block all upstream and downstream PCI busses, and
it would also cause a big delay. (A third reason could be that this
solution is really messy)

I could also iomap/iounmap pages as needed, this would be both a
nightmare to code and a real preformance killer.

Remapping the kernel to 0x60000000 really seems to be the best thing to
do in my case. I would prefer binary compatibility, but considering that
this setup would only be interesting only to people with special
computing needs, it would not be a disaster if it isn't binary
compatible.

If it really is a bad idea to change the kernel mapping, i guess i'll
just have to use the DMA channel.

greetings,

Mark.
unsubscribe: body of `unsubscribe linux-arm' to [email-masked]

---

From: Philip Blundell <[email-masked]>
Date: Thu, 03 Sep 1998 11:41:56 +0100

>be located on the host, which has cache coherency hardware.

Be warned that posted-write FIFOs and the like can still screw you up even if 
you have cache coherency in the chipset.

>preformance penalty for me.

Are you sure?  The StrongARM never performs burst accesses to uncacheable 
regions and this can be quite a hit.

>just have to use the DMA channel.

Changing the kernel mapping for your particular application is probably OK, 
though you might consider using the 21285's DMA engine anyway as you might 
well get better performance that way.

p.