Subject: char array, int pointer and cross boundary access

From: zhu qun ying <[email-masked]>
Date: Thu, 04 May 2000 16:42:56 +0800

Hi,

While trying to figure out how StrongARM behave on cross boundary access, I
encounter the following error ( I have the kernel-mode alignment trap handler on
and off with the same result). For comparisom, I have the same program compile
at x86 platform.

The arm result is very strange. The behavior is realy unacceptable. It should
either return run time error or some software emulation should return the
correct result?  I am using kernel 2.3.99-pre6-rmk1-np1. The kernel-mode
alignment trap handler suppose to be able to catch this and return the correct
result, but it has no effect for this case.

A kernel problem? A GCC problem? Any hints?

==== x86
kk's starting address: 0x0xbffff86c
i's address for kk + 0: 0x0xbffff86c 3 5 6 t
i's address for kk + 1: 0x0xbffff86d 5 6 t g
i's address for kk + 2: 0x0xbffff86e 6 t g h
i's address for kk + 3: 0x0xbffff86f t g h n
i's address for kk + 4: 0x0xbffff870 g h n o
i's address for kk + 5: 0x0xbffff871 h n o r
i's address for kk + 6: 0x0xbffff872 n o r q
i's address for kk + 7: 0x0xbffff873 o r q i
i's address for kk + 8: 0x0xbffff874 r q i p
=== SA1100
kk's starting address: 0x0xbffffd8c
i's address for kk + 0: 0x0xbffffd8c 3 5 6 t
i's address for kk + 1: 0x0xbffffd8d 5 6 t 3
i's address for kk + 2: 0x0xbffffd8e 6 t 3 5
i's address for kk + 3: 0x0xbffffd8f t 3 5 6
i's address for kk + 4: 0x0xbffffd90 g h n o
i's address for kk + 5: 0x0xbffffd91 h n o g
i's address for kk + 6: 0x0xbffffd92 n o g h
i's address for kk + 7: 0x0xbffffd93 o g h n
i's address for kk + 8: 0x0xbffffd94 r q i p

The source code for this program:
========================
#include <stdio.h>

int
main(void)
{
    char kk[12] = {'3',  '5', '6', 't',
                   'g',  'h', 'n', 'o',
                   'r', 'q', 'i', 'p'};
    int *i;
    int y, j;
   
    printf("kk's starting address: 0x%p\n", kk);
    for (j = 0; j < 9; ++j) {
    i = (int *)(kk + j);
    printf("i's address for kk + %d: 0x%p %c %c %c %c\n", j, i,
           (*i) & 0xFF, ((*i) >> 8) & 0xff,
           ((*i)>>16)&0xff, ((*i)>>24) & 0xff);
    }
    return 0;
}

-- 
 (~._.~)  Öì Èº Ó¢  (Qun Ying)         (65) 874-6743
  ( O )         Kent Ridge Digital Labs (KRDL)
 ()~*~()  21 Heng Mui Keng Terrace, Singapore 119613
 (_)-(_)    [email-masked]  *  [email-masked]

unsubscribe: body of `unsubscribe linux-arm-kernel' to [email-masked]

---

From: Philip Blundell <[email-masked]>
Date: Thu, 04 May 2000 18:05:58 +0100

In message <[email-masked]>, zhu qun ying writes:
>The arm result is very strange. The behavior is realy unacceptable. 
>It should either return run time error or some software emulation should 
>return the correct result?

No, your program is in error.  You can't cast a pointer of arbitrary 
alignment to some other type and expect it to work.  If you play these sorts 
of games you need to know the limitations of the machine.

>The kernel-mode alignment trap handler suppose to be able to catch this 
>and return the correct result,

Only for code inside the kernel.  If your program is in user context the trap 
handler will have no effect.

p.

---

From: Jim Studt <[email-masked]>
Date: Thu, 4 May 2000 13:09:49 -0500 (CDT)

> A kernel problem? A GCC problem? Any hints?

This is just the way ARMs work.  Consult the documentation of the LDR
instruction on page 36 (paper) 40 (PDF) of
  http://www.arm.com/Documentation/UserMans/PDF/ARM7vC.pdf

It has a pretty diagram on the next couple of pages.  In short you
aren't allowed to read misaligned words, but if you do it has some
useful quirks that might be helpful for unscrambling data.

-- 
                                     Jim Studt, President
                                     The Federated Software Group, Inc.

---

From: zhu qun ying <[email-masked]>
Date: Fri, 05 May 2000 08:17:04 +0800

I know the limit of ARM, and I know that I can't cast arbitrary alignment to
some other type. I want to know why the kernel allow this operation instead of
issuing run time error, like illeagl operation/access, or emulating the
behaviour using software. I used to port some program over to WINCE, and that is
a horrible experience, that kind of operation causes runtime error. Any
advantage that allow this kind of operation?

---

From: Nicolas Pitre <[email-masked]>
Date: Thu, 4 May 2000 21:10:05 -0400 (EDT)

> advantage that allow this kind of operation?

If you look at linux/arch/arm/mm/fault-armv.c, line 134, you'll see the
do_alignment() function.  Few lines further you'll see:

        if (user_mode(regs)) {
                set_cr(cr_no_alignment);
                ai_user += 1;
                return 0;
        }

This disables the alignment trap for user space code.  See the
documentation for the expected results...

If you want the alignment trap handler to work on user space code as well
as kernel code, just modify the above so it looks like:

        if (user_mode(regs)) {
                ai_user += 1;
        }

Alternately, if you want to fix broken user space programs and kill them
whenever a misaligned access occur, just use:

        if (user_mode(regs)) {
                ai_user += 1;
                return 1;
        }

The above is based on linux-2.3.99-pre6-rmk1.

Nicolas

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Fri, 5 May 2000 08:28:41 +0100 (BST)

> advantage that allow this kind of operation?

For the past 13 years that the ARM architecture has been around, the behaviour
of unaligned loads is something that was 100% defined to happen, and that the
compiler can rely on to optimise its code.

If you start causing user-mode runtime errors for these cases, then you will
effectively prevent any program which has not been compiled with the
appropriate flags to fail in perhaps undetectable ways (eg, if you compile
up a program with a compiler that does not generate half-word instructions).

There are a fair number of machines out there that use the StrongARM, but
are not able to use the half-word instructions, or don't use the StrongARM
at all that would be adversely affected by this.

Basically to maintain consistent behaviour across all current machines, we
ignore user-mode unaligned accesses.

It's just not worth the hastle.  Make sure your programs follow the
appropriate C standards, and you won't have a problem.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Fri, 5 May 2000 08:30:53 +0100 (BST)

> as kernel code, just modify the above so it looks like:

Either case is not good.  (see previous mails).  It's best left as-is and
the program must follow the C alignment rules to have expected behaviour.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Kira Brown <[email-masked]>
Date: Fri, 5 May 2000 11:07:29 +0100 (BST)

> at all that would be adversely affected by this.

The only ones I know of are the Medusa architecture machines; is it really
worth worrying about them?  They're obsolete and becoming increasingly
rare.

kira.

---

From: David Bath <[email-masked]>
Date: Fri, 05 May 2000 20:06:02 +1000

> the program must follow the C alignment rules to have expected behaviour.

While I agree with your "if it hurts don't do it" policy, life is all too
often about porting existing old code rather than writing new code, and old
code from x86 and 68K worlds has definitely been known to be sloppy in this
area.

It seems there are two classes of programs, those that have been optimised by
the ARM compiler to take advantage of this well defined behaviour and don't
want exceptions or fixups, and those sloppy programs being ported from other
platforms that expect more "traditional" behaviour, and desperately need
exceptions or fixups.    I would imagine debugging the latter on an ARM/Linux
platform that produces no exception would be very unpleasant.

How feasible would it be to make this run-time configurable, on a per-process
basis (perhaps by use of the /proc filesystem) ?

dB.

---

From: Nicolas Pitre <[email-masked]>
Date: Fri, 5 May 2000 09:44:31 -0400 (EDT)

> the program must follow the C alignment rules to have expected behaviour.

The idea is to debug misbehaving code.  Generating a SIGBUS is really nice
to quickly spot where are the alignment abuses.  This is not intended for
mainstream kernels.

Nicolas

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Fri, 5 May 2000 13:10:10 +0100 (BST)

> worth worrying about them?  They're obsolete and becoming increasingly

Try any ARM based machine which does not support V4 architecture - ie, ARM6
cores and some, if not most ARM7 cores.  It's far wider than just Medusa.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Fri, 5 May 2000 13:25:03 +0100 (BST)

> code from x86 and 68K worlds has definitely been known to be sloppy in this

I wasn't talking about an "if it hurts".  It doesn't hurt - it breaks.

> platform that produces no exception would be very unpleasant.

This is an old topic of discussion - please read the mailing list archives
for the results of the previous discussions.

The problem here is:

1. without the user-mode exception handler

   Sloppily written programs aren't flagged with an obvious warning message.
   Correctly written, binary compatible programs across the whole range of
   ARM processors will run as expected.

2. with the user-mode exception handler

   Sloppily written programs are flagged with a warning message.
   Correctly written, binary compatible programs will fail, maybe in
   unpredicatable ways.

Ok, so you choose option 2, and suddenly your currently running processes
die.  That is extremely unpleasant.

So, you make it per-process selectable.  So long as you get the right compiler
options, all well and good - you enable the user-mode alignment trap.  But if
you get the options wrong?  You either end up with either case (1) or (2) for
that particular program.

Most packages are built using a standard set of options, and therefore these
problems may not be detected.  It depends really on the setup of the binutils,
and whether binutils can be teached to "do the right thing" automagically.
Really, it can't do because you've got this thing called the C library which
will be an unknown quantity.

No, there are just too many variables here - binutils, glibc, compiler options,
hardware platform.  All have to be exactly right, or else all hell can break
loose.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Fri, 5 May 2000 15:39:06 +0100 (BST)

> to quickly spot where are the alignment abuses.  This is not intended for

In that case, what about hiding it under the kernel hacking options, so that
it can only be enabled when EXPERIMENTAL is also turned on?  Of course, it
will have to display a message so when people complain that they're having
problems its obvious why...
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Scott Bambrough <[email-masked]>
Date: Fri, 05 May 2000 10:55:43 -0400

-- 
Scott Bambrough - Software Engineer
REBEL.COM    http://www.rebel.com
NetWinder    http://www.netwinder.org

---

From: Scott Bambrough <[email-masked]>
Date: Fri, 05 May 2000 10:56:48 -0400

I would only recommend this procedure for debugging/testing of programs, not a
production kernel.   Don't write such non-portable code in the first place.

Scott

---

From: Kira Brown <[email-masked]>
Date: Fri, 5 May 2000 16:26:15 +0100 (BST)

> cores and some, if not most ARM7 cores.  It's far wider than just Medusa.

Yes, I know, but we were talking about StrongARM.  The ability to compile
kernels specific to your processor is one of the advantages of
Linux...  can't we have this as an optional feature in StrongARM kernels?

kira.

---

From: Scott Bambrough <[email-masked]>
Date: Fri, 05 May 2000 11:24:41 -0400

> problems its obvious why...

I like this idea myself.

Scott

---

From: David Bath <[email-masked]>
Date: Sat, 06 May 2000 07:58:18 +1000

> problems its obvious why...

So, if we  now all accept that there is a time when experienced users (namely
those attempting to port poorly written x86 Linux apps to ARM) will want to
turn user level exceptions on, it seems only a small step (conceptually, if not
technically) to allow them to turn it on at runtime.

It seems a bit excessive making them rebuild their kernel just so they can
debug some poorly written user level code, and then reboot their original
kernel so they can continue to run their correctly written ARM code.

There's plenty of precedence with Linux for controlling experimental
/diagnostic stuff  at runtime.  For example...

echo 1 >/proc/sys/net/ipv4/tcp_syncookies               if I want SYN cookie
protection
echo "scsi log token [level]" > /proc/scsi/scsi            if I want scsi
logging enabled
echo "0 0 0 y y y - y - 10 " >/proc/scsi/tmscsim/0   if I want to try out
tagged queuing on a scsi device

If I've just copied a source tar file down from somewhere, typed "make", had a
clean build, and then had it crash or behave very strangely, it would be very
useful to be able to do something like...

echo "usr-fixup" /proc/.....<some fname here>    [to enable user level fixups]

run it again, and see if it works.   If it does work, I could then do a

echo "usr-sigbus" /proc/....<some fname here>   [to enable user level
exceptions]

run it again,  and gather all the info I need to work out how to fix it from
SIGBUS crash details.  Fix it,  rebuild, and then

echo "usr-none"  /proc/....<some fname here>    [to return to no user level
intervention]

run it again, notice it works, and declare victory.

If you believe that ARM-Linux will only ever be used in a single-user
environment then this is probably sufficient.  If you think one day a corporate
server with 100 users might one day be running on ARM-Linux then you might want
to make the above work on a per-process basis.  I might be trying to debug my
sloppy x86 code while 99 other users are trying to run their correctly written
ARM code.

dB.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Fri, 5 May 2000 23:04:26 +0100 (BST)

> technically) to allow them to turn it on at runtime.

And then its all too easy for it to be turned on permanently.  No, that is not
something we want, especially when we end up with users complaining.

> sloppy x86 code while 99 other users are trying to run their correctly written

But then a decent sysadmin of the corporate server would never let you near to
debug your program.  He'd want to lock it down as tightly as possible to prevent
misuse/hacking/etc, which, if you're talking about a distribution-only box will
probably mean no C compiler either.

(If you need the machine to stay up, you should only be running tried and tested
programs on it).
   _____

---

From: David Bath <[email-masked]>
Date: Sat, 06 May 2000 08:40:53 +1000

> something we want, especially when we end up with users complaining.

Hadn't we agreed we were going to let them turn it on at kernel config time?
Turning it on at config time seems to have more potential for permanence then
controlling it at runtime.  If it's a config time option you need to reboot your
machine again to turn it off.  If it's a runtime option, then it's much easier to
turn it off again.

> debug your program.

I think that depends on what business the corporation is in.  Do a 'top' on some big
corporate servers at s/w companies like Sun or Cisco and you'll see dozens of people
running cc on big thumping SMP servers.  Clearly you wouldn't want one of those
users to be able to influence user-level exception handling for other users, hence
the need to be able to control your own destiny, on  a per-process basis.

dB.

---

From: Philip Blundell <[email-masked]>
Date: Sun, 07 May 2000 01:24:41 +0100

>SIGBUS crash details.  Fix it,  rebuild, and then

Both of these solutions are pretty poor because they carry the risk of 
breaking already-running programs.  If your shell was compiled with 
-march=armv3, for example, enabling either fixups or exceptions for unaligned 
accesses might well leave you with an unusable machine.  (The default for GCC 
is -march=armv3.)

>you might want to make the above work on a per-process basis.

Yes.  I guess you could conceive of a bit in the program header that says 
"this program shouldn't be doing unaligned accesses" and caused them to be 
trapped.

p.

---

From: David Bath <[email-masked]>
Date: Sun, 07 May 2000 21:07:47 +1000

> "this program shouldn't be doing unaligned accesses" and caused them to be

hmmm... maybe we could just add a system call to enable it.   Then the first thing
you could do when porting some code that doesn't seem to work right, is enable the
exception handler  on the first line.  Then you could  use the exception to locate
and fix the real bug,  and then remove the fist line you added.

dB.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Sun, 7 May 2000 12:40:48 +0100 (BST)

> and fix the real bug,  and then remove the fist line you added.

No.  Phil's point is more or less the same as one of the points I've been trying to
make.

If all your binaries are compiled up so that they rely on unaligned accesses, and
you turn on the SIGBUS, suddenly every process on your system receives SIGBUS when
it next runs, leaving you with zero processes, and therefore an unusable system.

I did make the suggestion of using the compiler to indicate this, but then dismissed
this as a solution because it is not a "binary" thing.  It can only be turned on if
your binary, your dynamic linker and your libraries are compiled up so they don't
use unaligned accesses.

IMHO, this is a deep complicated issue which needs a lot of further discussion.
Until all the issues are resolved, I don't think we can even start to consider
putting this in for 2.4.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Philip Blundell <[email-masked]>
Date: Sun, 07 May 2000 13:47:41 +0100

Russell King wrote:
>David Bath writes:
>> hmmm... maybe we could just add a system call to enable it.  
>
>No.  Phil's point is more or less the same as one of the points I've been 
>trying to make.

Actually, I think you dismiss David's suggestion too lightly.  You could have 
a system call that turned trapping on for the calling process only.

p.

---

From: David Bath <[email-masked]>
Date: Sun, 07 May 2000 21:49:03 +1000

> it next runs, leaving you with zero processes, and therefore an unusable system.

Actually, I meant for the system call to only turn on exception handling for the
process that issued the system call, not for everybody.

>  It can only be turned on if

Yes, dynamic libraries to complicate things.

> Until all the issues are resolved, I don't think we can even start to consider

Yep, there do seem to be a lot of gotcha's.

dB.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Sun, 7 May 2000 12:55:49 +0100 (BST)

> a system call that turned trapping on for the calling process only.

But what about the dynamic linker and libraries?
   _____

---

From: Philip Blundell <[email-masked]>
Date: Sun, 07 May 2000 14:41:47 +0100

>But what about the dynamic linker and libraries?

Yes, there's still a pitfall there for the unwary.  But it's not nearly as 
bad - the worst that happens is that the process you're debugging dies 
unexpectedly (and I think it's fair to say that if you want to do this, you 
need to take care of how the linker and libraries were built).  Whereas with 
the global approach that some people were advocating, other users can be 
screwed over without them realising what's going on.

p.

---

From: David Bath <[email-masked]>
Date: Mon, 08 May 2000 10:41:06 +1000

Philip Blundell wrote:

> need to take care of how the linker and libraries were built).

It would be cool if the exception handler could inspect the code and/or address
that caused the misaligned access, and then determine whether it was an
intended misaligned access (as caused by -march=armv3 etc.) or an unintended
one, as caused by sloppy x86 code.  I haven't looked at the generated code, but
I would imagine they'd be indistinguishable :-(.

dB.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Mon, 8 May 2000 07:15:26 +0100 (BST)

> I would imagine they'd be indistinguishable :-(.

They could well be identical.  However, it may be worth looking at what
the sloppy x86 case is producing.  Could you create a simple test case
and post the instructions around the offending unaligned load?
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: David Bath <[email-masked]>
Date: Tue, 09 May 2000 10:37:23 +1000

> and post the instructions around the offending unaligned load?

I don't have an example handy, but if I come across one I will.  Perhaps the
original instigator of this thread can help out here.

Can you give us an example of some correct C code that inspires arm-gcc to generate
intentional misaligned accesses?

I suspect porting application code directly to ARM-Linux will remain a task for true
masochists.  Happily, most unix apps have long ago been ported to MIPS platforms,
and so have long ago had their misaligned accesses fixed.  The MIPS architecture is
far friendlier in this regard, since all misaligned accesses are bad; it doesn't
have the concept of good ones and bad ones.

Maybe ARM-Linux application porters should keep an old DECstation of SGI Indigo
handy.  Get the code running there first, and then get it working on ARM.

dB.

---

From: zhu qun ying <[email-masked]>
Date: Tue, 09 May 2000 09:14:26 +0800

> handy.  Get the code running there first, and then get it working on ARM.

I have the simple C source for this thread at my first post, but it is not
intentional. The original idea comes from re-construct a packed network packet
into its original structure form, one integer element is misalligned and I cast
the pointer to integer and cause some porblem.

Just curious to know, as I am working with StrongARM for only a few months, what
are the intentional misalignment asccess for? what are the advantage to use
misalignement access to have the roation, as StrongARM does have the rotation
instruction.

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Tue, 9 May 2000 07:26:22 +0100 (BST)

> misalignement access to have the roation, as StrongARM does have the rotation

Take the following code on a machine without half-word instructions (ie,
any V3 architecture or broken V4 architecture):

	struct {
		unsigned short a;
		unsigned short b;
	} *c;

	c = <4-byte aligned pointer>

	printf("%d %d\n", c->a, c->b);

The code will be:

	@ r4 = c
	adr	r0, string
	ldr	r1, [r4, #0]
	ldr	r2, [r4, #2]
	mov	r1, r1, asl #16
	mov	r2, r2, asl #16
	mov	r1, r1, lsr #16	@ a
	mov	r2, r2, lsr #16 @ b
	bl	printf
   _____