Created attachment 140 [details]
bpeg.s compiled by mac to an a.out object file

Created attachment 141 [details]
bpeg.s compiled by rmac to an a.out object file

Created attachment 142 [details]
Hex dump (xxd) of bpeg.mac.o for easier diffing

Created attachment 143 [details]
Hex dump (xxd) of bpeg.rmac.o for easier diffing

Linkovitch forced me to think about this a little harder, and I ended up finding that a work-around is possible.  Replacing this code in bpeg.s:
 

ZZQTLMatrix    .equ     *                      ; Luminance Quantization Matrix ($100 Bytes)
ZZQTCMatrix     .equ    *+$100                  ; Chrominance Quantization Matrix ($100 Bytes)
MCUBuffer       .equ    *+$200                  ; I/O MCU Buffer ($600 Bytes)
MCUTmpBuffer    .equ    *+$800                  ; Inv. DCT Temporary Buffer ($100 Bytes)

With this: 

ZZQTLMatrix:                       ; Luminance Quantization Matrix ($100 Bytes)
ZZQTCMatrix     .equ    ZZQTLMatrix+$100                  ; Chrominance Quantization Matrix ($100 Bytes)
MCUBuffer       .equ    ZZQTLMatrix+$200                  ; I/O MCU Buffer ($600 Bytes)
MCUTmpBuffer    .equ    ZZQTLMatrix+$800                  ; Inv. DCT Temporary Buffer ($100 Bytes)

Causes both assemblers to generate identical object files.

First of all, thanks a bunch for taking so much time to look into this and reporting it.

Now, I've been staring at this for most of the day in between other things and my brain is kind of foggy now, so I'll just state what's happening and the implications that follow the (so far) fix.

First of all, rmac did indeed assemble this file correctly. It was shipped with qUaKe and from that archive I saw that v1.1.1 used to assemble the file properly.

So after assembling the file using the old and new rmac I dumped both files' symbols using nm:

../../rmac.exe -lbpeg.lst -o bpeg.o a.S && m68k-atari-mint-nm.exe -p bpeg.o > bpeg_nm.txt
../../rmac.exe -lbpeg_ref.lst -o bpeg_ref.o a.S && m68k-atari-mint-nm.exe -p bpeg_ref.o > bpeg_nm_ref.txt

(a.s is just a file that includes jaguar.inc and bpeg.s)

This produced two files that were almost the same except for the following (left is current rmac, right is old rmac):

00f03608 t ZZQTLMatrix  | 00f03608 a ZZQTLMatrix 
00f03708 t ZZQTCMatrix  | 00f03708 a ZZQTCMatrix
00f03808 t MCUBuffer    | 00f03808 a MCUBuffer
00f03e08 t MCUTmpBuffer | 00f03e08 a MCUTmpBuffer 

According to https://sourceware.org/binutils/docs/binutils/nm.html "t" is a symbol belonging in text and "a" is an absolute symbol. So all that's written above is correct - rmac is now exporting symbols that are defined absolute as relative (inside text segment).

So yes, the problematic lines in bpeg.s are the following:

ZZQTLMatrix	.equ	*			; Luminance Quantization Matrix ($100 Bytes)

ZZQTCMatrix	.equ	*+$100			; Chrominance Quantization Matrix ($100 Bytes)

MCUBuffer	.equ	*+$200			; I/O MCU Buffer ($600 Bytes)

MCUTmpBuffer	.equ	*+$800			; Inv. DCT Temporary Buffer ($100 Bytes)


My fix so far was to modify AddBSDSymEntry in object.c from:

if (w1 & EQUATED)
...

switch (w1 & TDB)
...

to:

if (w1 & EQUATED)
...
else
switch (w1 & TDB)
...

so the exporter will stop when encountering an absolute symbol. This fixes the issue.

HOWEVER....

Remember bug #140? Yup, with this change the code in Removers' lib is breaking.

00000450 T _init_module
000006b0 T _play_module
00000660 T _clear_module
00000668 T _pause_module

With the "fix" the "T"s become "A"s, so that's no good!

I think the proper way to fix the issue would be to make AddBSDSymEntry realise that the symbols are defined and are assigned an absolute value, even though they are inside the TEXT segment. The thing is that this code is inside the .gpu code block, so that makes all symbols automatically fully defined as we know their addresses during assembly time and they're not going to change during link time.

So it's probably really easy to do but my brain just won't cooperate right now! So I'll go do something else for today and will think about this soon (tomorrow?)...

Thanks for taking a look.  That all make sense.  Somewhere in my rambling through the code, I found something like (Pseudo-code from memory, forgive me):

if (orgActive) {
   doPcIsAbsolute();
} else {
   doPcIsRelative();
}

I thought it was somewhere dealing with '*', but perhaps not.  I'll try to take another look as well after work & family time tonight.

I still haven't figured out how the expression evaluator converts the difference between two relative/non-absolute labels into an absolute value either though.  It does seem to work; I just don't know why, nor why it doesn't work for equates, as is the case here.  It seems like that should be fixed as well.

Created attachment 144 [details]
The patches!

There we go, I knew I should have stopped last night as I was getting utterly useless!

So, the fix is simple: when calculating an expression that contains "*" if ORG mode is active then simply don't add any section to the symbol's attributes. Then during export time we read that and we know that the symbol is a "free agent" so we mark it as absolute.

Patch attached, and I'll add two more regression tests for the suite. These are good tripwires in case we decide to tinker with coff in the future!

Created attachment 145 [details]
Additions to the regression suite

Thanks. Verified it works and produces an identical object file to mac. Looks like a fair amount of cruft removal made it in there as well!

Thanks for the patch!  :-)