Subject: Re: switched to strongARM and 160 Mb and Debian 2.2 doesn't boot

From: [email-masked]
Date: Mon, 28 Aug 2000 17:43:50 -0400

Russell King wrote:
> is probably what you were meaning, and yes, it still fits in the 0xc0000000
> to 0xcfffffff space.  But what about the vmalloc area, when you've got all
> the banks used and full (ie 256MB of memory)?
>

Just for the record, According to the SA1110 Advance Developer Manual each of
the four SDRAM banks in an SA1110 can contain 128 MB of RAM (using 32M x 8, or
64M x 16, or 64M x 4 components) for a total of 512MB of memory.  I'm not saying
you'd want to do this, but that it is technically possible using commercially
available parts.

For example:
     Using 4 Samsung K4S560832A 32Mx8 component Yields 128MB bank w/32 bit bus
     Using 4 Samsung K4S560432A 64Mx4 component
Enjoy,
Erik

unsubscribe: body of `unsubscribe linux-arm-kernel' to [email-masked]

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Mon, 28 Aug 2000 22:48:34 +0100 (BST)

>      Using 4 Samsung K4S560432A 64Mx4 component

Yep, full size is 512MB, but with those macros modified as described, you
can't use all of that.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Erik Mouw <[email-masked]>
Date: Tue 29 Aug 2000 00:00:08 +0100 (MDT)

> Yep, full size is 512MB, but with those macros modified as described, you

Correct. But if we want to support the full 512MB, we need 0xc0000000 to
0xdfffffff. Vmalloc space should go on top of that at 0xe0000000, but
that's already occupied by the SA1100 I/O space (PCMCIA slots and SA1100 
registers). Can we just move the memory down to (say) 0xb0000000
0xcfffffff and put vmalloc space at 0xd0000000?

Erik
[oops, time to leave this building...]

-- 
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031,  2600 GA Delft, The Netherlands
Phone: +31-15-2783635  Fax: +31-15-2781843  Email: [email-masked]
WWW: http://www-ict.its.tudelft.nl/~erik/

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Mon, 28 Aug 2000 23:19:27 +0100 (BST)

> 0xcfffffff and put vmalloc space at 0xd0000000?

You'll need to change both TASK_SIZE and PAGE_OFFSET (TASK_SIZE is the
"top" of the 32-bit task space, PAGE_OFFSET is the virtual address of
the start of the kernel direct mapped memory).

However, I suspect that you don't actually need to keep everything
mapped.  Eg, On the assabet:

1. The flash should be mapped in as needed by the MTD drivers
2. The scr on the Assabet can be unmapped once its been read.
3. The BCR and SA1111 can be mapped at 0xde000000 and df000000
   respectively.

With those changes, you've just gained 224MB of extra space.  If you also
consider that PCMCIA IO normally doesn't need more than 1MB, then that's
another 126MB of space.  So now we're at 0xe6000000 being the lower end
of the mapped IO region.  So, we've just gained a total of 350MB, which
will cover the 256MB of extra ram space.

That leaves you 94MB for vmalloc/ioremap.  This may be enough without
ioremap, but once you start using it, you really want to free up some
more (by making the pcmcia drivers ioremap the minimal space required
on an as-needed basis).
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Nicolas Pitre <[email-masked]>
Date: Mon, 28 Aug 2000 21:46:31 -0400 (EDT)

[To John G. Dorsey:  If you're listening, I would like your coments on the
 PCMCIA stuff below.]

Woohoooo!

I got my inbox flodded...  ;)

I wanted to get home and have a look at the code before suggesting the
requested quick hack... but apparently Erik got it first.

> the start of the kernel direct mapped memory).

IMHO that's not a good idea.

> 1. The flash should be mapped in as needed by the MTD drivers

Indeed.

> 2. The scr on the Assabet can be unmapped once its been read.

The scr is in reality IO pins which aren't in the way... and not desirable
to unmap at all.

> 3. The BCR and SA1111 can be mapped at 0xde000000 and df000000

The cache invalidation area is already there... and this doesn't solve the
512MB possibility either.

Well... here's the plan:

virt_to_phys()/phys_to_virt() become:

#define __virt_to_phys(x)  (x)
#define __phys_to_virt(x)  (x)

That gcc knows how to optimize!  ;-)

Next the PCMCIA 0 and 1 (IO only) are mapped at 0xf6000000 and 0xf7000000.
This should cover all designs with hardwired devices i.e. straight IDE
interface, ehternet, etc. (16MB for each should be more than enough,
especially since this usually uses ISA component which are used to a
very tiny address space).

The integrated peripherals mapping from 0xf8000000 stays unchanged.

Cache invalidation area goes to 0xf5000000 and 0xf5800000.

And... all guizmos like flash, SA-1111, CPLDs, etc go between 0xf0000000
to 0xf4ffffff... and it could possibly be grown downwards if needed.

RAM is now from 0xc0000000 to 0xdfffffff, the same as physically.  This
leaves 0xe0000000 to 0xefffffff for vm areas.

That's it!

**HOWEVER** this requires the generic SA1100 PCMCIA driver to be modified
so it actually ioremap() its memory and attribute space (even IO could be
done that way too for more clarity).  This change must occur before all
the rest otherwise PCMCIA will be broken and people will step on my toes.  

John --> You have code that did just that in the past.  Could it be done
quickly?

In the mean time I'll include the 64MB change... but hopefully the real
solution will come sooner than later, otherwise someone will certainly
come by with a 128MB per bank design...

Any comments?

Nicolas

---

From: John G Dorsey <[email-masked]>
Date: Mon, 28 Aug 2000 22:22:39 -0400

> John --> You have code that did just that in the past.  Could it be done

I'm on it.

(Might as well just do all three per-slot spaces on the fly, as Card Services
is already issuing the setup calls for them. We just configure timings now and
no-op the actual allocation request.)

> come by with a 128MB per bank design...

Someone (me) will come by in late October with such a design. It passed final
engineering review last week. =)

-jd

---

From: Erik Mouw <[email-masked]>
Date: Tue 29 Aug 2000 23:22:22 +0100 (MDT)

On Tue, 29 Aug 2000 17:05:42 -0400, John G Dorsey wrote:
> I've tried it with pcnet_cs on Assabet and Neponset, and ide_cs on Assabet. I
> was surprised when I found:
> 
>   #define PCMCIA_IO_0_BASE 0xe0000000
>   #define PCMCIA_IO_1_BASE 0xe4000000
> 
> ...in asm/arch/ide.h. Owners of this driver for devices that don't go through
> Card Services should probably tell Nicolas what they want this code to look
> like now.

That's my stuff (well, I recently cleaned it up), although it is in the
kernel since linux-2.2.x (or maybe longer). The Empeg, LART, and Victor
have an IDE interface mapped into PCMCIA IO space 0. It is independent of
the card services because it doesn't use card services, the drive is just
always there. What's your suggestion how it should look like?

Erik

---

From: John G Dorsey <[email-masked]>
Date: Tue, 29 Aug 2000 18:50:05 -0400

Erik Mouw wrote:

> kernel since linux-2.2.x (or maybe longer).

I haven't tested ide_cs with any regularity since Erik Bunce first got it running
(2.3.99), but removing the hardwired base address (now in ide_init_hwif_ports())
was part of that conversion. The code as it is currently written should fail when
run against CS because of the address arithmetic being performed.

> The Empeg, LART, and Victor

Once the static mapping is removed from the official tree, this code simply won't
work. Constructing the mapping on demand, as the sa1100 socket driver now does,
seems appropriate. I'm not sure where I'd do this in the code, though. I know
ide_init_hwif_ports() is inline on other architectures, but it's becoming portly
on SA-1100. (Also, on SPARC, there's a warning in the code against using ioremap()
in this routine, as it can be called with degenerate parameters.)

I can also imagine a simple implementation of the low-level socket driver
operations that might allow these boards to just use Card Services. (Advantage:
consolidated resource management code. Disadvantage: needs ramdisk to load ide_cs
at boot.)

-jd

---

From: John G Dorsey <[email-masked]>
Date: Tue, 29 Aug 2000 17:05:42 -0400

> the rest otherwise PCMCIA will be broken and people will step on my toes.

Done.

See linux-2.4.0-test6-rmk5-np2-jd1.patch.gz on
http://www.cs.cmu.edu/~wearable/software/assabet.html (under "Bleeding Edge").

-jd

---

From: Mike Touloumtzis <[email-masked]>
Date: Tue, 29 Aug 2000 16:23:53 -0700

> That gcc knows how to optimize!  ;-)

Are you planning on using multiple NUMA nodes, or one node with multiple,
discontiguous banks?  From looking at the code, gaps in mem_map get
returned to the free page pool, so I'm not sure what the advantage of
having multiple NUMA nodes is; can anyone explain?

miket

---

From: Nicolas Pitre <[email-masked]>
Date: Tue, 29 Aug 2000 19:24:02 -0400 (EDT)

> always there. What's your suggestion how it should look like?

It should stay there, but only the address will change.

Those are used for designs that actually have an hardwired IDE interface
on the PCMCIA space, just like if it was an ISA bus.  It shouldn't
conflict with real PCMCIA implementations.

Nicolas

---

From: Nicolas Pitre <[email-masked]>
Date: Tue, 29 Aug 2000 19:29:21 -0400 (EDT)

On Tue, 29 Aug 2000, John G Dorsey wrote:

> work.

I wanted to leave a small mapped area for those.  This might be considered
like a restricted ISA bus and some boards, like the empeg, use it to stick
more than one device.

Nicolas

---

From: Mike Touloumtzis <[email-masked]>
Date: Tue, 29 Aug 2000 17:05:58 -0700

> having multiple NUMA nodes is; can anyone explain?

Apologies for responding to my own post; after further thought,
I realized that the SA1100 has much larger gaps between physical
memory banks than the hardware I'm dealing with (two banks with
only an 8MB gap between them); a mem_map array able to address
all of this space as one node would be impractically large.

I'm currently in the process of retooling my EP7211 port (whose
memory handling was based on SA1100) with physical == virtual
and two NUMA nodes, each with a local mem_map; I'll see if I
can get that configuration working...

miket

---

From: Erik Mouw <[email-masked]>
Date: Wed 30 Aug 2000 09:45:30 +0100 (MDT)

> run against CS because of the address arithmetic being performed.

The base address is no longer in ide_init_hwif_ports(). I send a patch to
this list (or was it sa1100-linux?) that fixes it. I assume Nico will put
it in his next patch.

> in this routine, as it can be called with degenerate parameters.)

Yes, some old stuff in the IDE code simply calls this function with "all
the world is a PC" in mind.

> consolidated resource management code. Disadvantage: needs ramdisk to load
> ide_cs at boot.)

That's not a clean solution, IMHO. The LART doesn't need Card Services, so
why should the IDE driver depend on it? The idea is that it emulates ISA
I/O space which is not larger than 64k. A small static mapping for this
kind of devices would be nicer. 

Erik

---

From: Erik Mouw <[email-masked]>
Date: Wed 30 Aug 2000 10:09:11 +0100 (MDT)

> all of this space as one node would be impractically large.

Oh, but the current implementation would still do. The LART also has an
8MB gap between two 8MB regions in a single bank, and the current
implementation works well. The trick is to mark the 8MB gap as reserved.
Have a look at arch/arm/kernel/arch.c and search for LART.

Erik