[ia16-gcc] General ia16-gcc toolchain questions

[rafael2k]

With all variables set, ia16-gcc gives me error, in a file where the first line is "#include <signal.h>"

without -std (just a warning):

ia16-elf-gcc -Os -melks-libc -mtune=i8086 -mcmodel=small -mno-segment-relocation-stuff -fno-inline -fno-builtin-printf -fno-builtin-fprintf -Wall -Wextra -Wtype-limits -Wno-unused-parameter -Wno-sign-compare -Wfatal-errors -Wextra -Wshadow -Wundef -Wwrite-strings -Wredundant-decls -Wdisabled-optimization -Wdouble-promotion -Wmissing-declarations -Wconversion -Wstrict-overflow=2 -Wlogical-op -Wno-aggressive-loop-optimizations  -I/home/rafael2k/programs/devel/elks/include -I/home/rafael2k/programs/devel/elks/libc/include -I/home/rafael2k/programs/devel/elks/elks/include -D__ELKS__     -c -o test.o test.c
In file included from /home/rafael2k/programs/devel/elks/libc/include/signal.h:7:0,
                 from test.c:1:
/home/rafael2k/programs/devel/elks/elks/include/linuxmt/signal.h:230:5: warning: "UNUSED" is not defined [-Wundef]
 #if UNUSED
     ^~~~~~

with -stc=c99:

ia16-elf-gcc -Os -melks-libc -std=c99 -mtune=i8086 -mcmodel=small -mno-segment-relocation-stuff -fno-inline -fno-builtin-printf -fno-builtin-fprintf -Wall -Wextra -Wtype-limits -Wno-unused-parameter -Wno-sign-compare -Wfatal-errors -Wextra -Wshadow -Wundef -Wwrite-strings -Wredundant-decls -Wdisabled-optimization -Wdouble-promotion -Wmissing-declarations -Wconversion -Wstrict-overflow=2 -Wlogical-op -Wno-aggressive-loop-optimizations  -I/home/rafael2k/programs/devel/elks/include -I/home/rafael2k/programs/devel/elks/libc/include -I/home/rafael2k/programs/devel/elks/elks/include -D__ELKS__     -c -o test.o test.c
In file included from /home/rafael2k/programs/devel/elks/libc/include/signal.h:7:0,
                 from test.c:1:
/home/rafael2k/programs/devel/elks/elks/include/linuxmt/signal.h:186:23: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘void’
 typedef stdcall __far void (*__kern_sighandler_t)(int);
                       ^~~~
compilation terminated due to -Wfatal-errors.

Other time I get (when trying to compile Lua):

ia16-elf-gcc -Os -melks-libc -mtune=i8086 -mcmodel=small -mno-segment-relocation-stuff -fno-inline -fno-builtin-printf -fno-builtin-fprintf -Wall -Wextra -Wtype-limits -Wno-unused-parameter -Wno-sign-compare -Wfatal-errors -Wextra -Wshadow -Wundef -Wwrite-strings -Wredundant-decls -Wdisabled-optimization -Wdouble-promotion -Wmissing-declarations -Wconversion -Wstrict-overflow=2 -Wlogical-op -Wno-aggressive-loop-optimizations  -I/home/rafael2k/programs/devel/elks/include -I/home/rafael2k/programs/devel/elks/libc/include -I/home/rafael2k/programs/devel/elks/elks/include -D__ELKS__     -c -o lapi.o lapi.c
In file included from /home/rafael2k/programs/devel/elks/libc/include/sys/cdefs.h:6:0,
                 from /home/rafael2k/programs/devel/elks/libc/include/features.h:10,
                 from /home/rafael2k/programs/devel/elks/libc/include/sys/types.h:4,
                 from /home/rafael2k/programs/devel/elks/libc/include/string.h:4,
                 from lapi.c:14:
/home/rafael2k/programs/devel/elks/elks/include/arch/cdefs.h:16:25: error: expected ‘)’ before ‘__attribute__’
 #define noreturn        __attribute__((__noreturn__)) /* don't require <stdnoreturn.h> */
                         ^
llimits.h:194:38: note: in expansion of macro ‘noreturn’
 #define l_noret  void __attribute__((noreturn))
                                      ^~~~~~~~
lmem.h:79:11: note: in expansion of macro ‘l_noret’
 LUAI_FUNC l_noret luaM_toobig (lua_State *L);
           ^~~~~~~
compilation terminated due to -Wfatal-errors.

I always struggle with the correct order to add the includes, specially when porting others software. Is there any cookbook recipe or may be I just hit a bug or cornercase?

[ghaerr]

Each of your compilation issues are slightly different:

without -std (just a warning):

Given that signal.h is a kernel include, the options given to ia16-elf-gcc are including something that is causing failure, when the "normal" options found in elkscmd/Makefile-rules don't give this error. I will have to look into exactly what is the problem, given your use of lots of options.

with -stc=c99:

It appears that the C99 option doesn't allow __far which is required for the kernel signal handler mechanism. This could probably be specially worked around with some kernel include modifications but it is unclear exactly what support for C99 is included in ia16-elf-gcc in the first place - IIRC it is based on GCC 6.3, pretty old?

Other time I get (when trying to compile Lua):

"Reading the error message" (!) shows this is related to Lua source and ELKS source using the noreturn define in non-compatible ways. This would have to be changed in one or the other's source. We'll likely see more of this name collision as more outside programs are brought into ELKS.

[rafael2k]

Uff, sorry about not properly reading the error messages, I was in "automatic" mode trying to port stuff with the least effort. So for software fpu, I better stick with ia16-gcc, right? Lua needs some fpu operations.

[ghaerr]

Yes, definitely read the error messages!!

for software fpu, I better stick with ia16-gcc, right?

Probably yes, but not necessarily: while we have been successful getting programs like ELKS Basic to run using floating point compiled with OWC, the ELKS math library (include/math.h) has not been ported to OWC; so while you can use OWC to successfully produce floating point code using hardware FPU instructions, only the basic C operations (+,-,*,/) are supported. Much else requires the math library which isn't available. (Our OWC Basic runs well displaying floating point numbers except none of the Basic math functions are compiled in, for instance).

[ghaerr]

without -std (just a warning):

The "normal" options found in elkscmd/Makefile-rules don't give this error.

We know why the other two errors are occurring. For this one, you'll need to compare the options given to ia16-elf-gcc in elkscmd/Makefile-rules with your own. I haven't had time to do this, but would like to hear what you find out. I find this interesting because I have more strictly followed the options specified in Makefile-rules, but that doesn't necessarily mean we should not deviate from them. In particular, having a warning on an undefined symbol like #if UNUSED is a bit worrisome since there's a lot of code in our codebase that depends on such a feature (and we don't want a warning, either).

For the Lua noreturn redefinition, we can possibly fix that by adding an #ifndef noreturn around its definition in include/arch/cdefs.h, although I think actually that's a bug in the Lua codebase - they should be using __noreturn__.

[ghaerr]

In particular, having a warning on an undefined symbol like #if UNUSED is a bit worrisome since

The reason for this warning is -Wundef, which you provided to the compiler. That option specifically warns about undefined symbols - so output warning is exactly as you specified!

[ghaerr]

with -std=c99:

Looking further, the error occurs because of the forward declaration of the system call _signal, which uses a __far type, which is not in C99. We don't really need to have _signal in signal.h, unless we're compiling libc, which code can be otherwise protected using __LIBC__.

This would probably be worth making a change for, so that our header files can be used strictly as C99. Shall I make this change? Are you finding it useful to compile code as C99-compliant?

[rafael2k]

with -std=c99:

Looking further, the error occurs because of the forward declaration of the system call _signal, which uses a __far type, which is not in C99. We don't really need to have _signal in signal.h, unless we're compiling libc, which code can be otherwise protected using __LIBC__.

This would probably be worth making a change for, so that our header files can be used strictly as C99. Shall I make this change? Are you finding it useful to compile code as C99-compliant?

Hi @ghaerr. I like to use C99 when possible, so if the change is not intrusive, that would be welcome.

[rafael2k]

Yes, definitely read the error messages!!

for software fpu, I better stick with ia16-gcc, right?

Probably yes, but not necessarily: while we have been successful getting programs like ELKS Basic to run using floating point compiled with OWC, the ELKS math library (include/math.h) has not been ported to OWC; so while you can use OWC to successfully produce floating point code using hardware FPU instructions, only the basic C operations (+,-,*,/) are supported. Much else requires the math library which isn't available. (Our OWC Basic runs well displaying floating point numbers except none of the Basic math functions are compiled in, for instance).

Got it. This is why I got tons of undefined math library symbols when trying to compile Lua with OWC. Then I changed to ia16-gcc for this reason.

[rafael2k]

Thanks for the support. Just fixing the noreturn, and copied the content of Makefile-rules, I compiled all Lua!
Just some missing functions that should be easy to replace, but not the small model size limitation I think.

ia16-elf-ar rc liblua.a lapi.o lcode.o lctype.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o ltests.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o lutf8lib.o loadlib.o lcorolib.o linit.o
ia16-elf-ranlib liblua.a
ia16-elf-gcc -mcmodel=small -melks-libc -mtune=i8086 -Wall -Os -mno-segment-relocation-stuff -fno-inline -fno-builtin-printf -fno-builtin-fprintf -Wno-implicit-int -Wno-parentheses  -I/home/rafael2k/programs/devel/elks/include -I/home/rafael2k/programs/devel/elks/libc/include -I/home/rafael2k/programs/devel/elks/elks/include -Wextra -Wtype-limits -Wno-unused-parameter -Wno-sign-compare -Wno-empty-body -D__ELKS__ -DELKS_VERSION=\"0.9.0-dev\"   -c -o lua.o lua.c
ia16-elf-gcc -o lua   -mcmodel=small -melks-libc -mtune=i8086 -Wall -Os -mno-segment-relocation-stuff -fno-inline -fno-builtin-printf -fno-builtin-fprintf -Wl,-E lua.o liblua.a   
/home/rafael2k/programs/devel/elks/cross/lib/gcc/ia16-elf/6.3.0/../../../../ia16-elf/bin/ld.gold: error: address of section '.fartext' moves backward from 0x3f5d0 to 0x20000
/home/rafael2k/programs/devel/elks/cross/lib/gcc/ia16-elf/6.3.0/../../../../ia16-elf/bin/ld.gold: error: load segment overlap [0x10000 -> 0x3f5d0] and [0x30000 -> 0x34210]
liblua.a(ltable.o):function l_hashfloat: error: undefined reference to 'frexp'
liblua.a(lvm.o):function luaV_modf: error: undefined reference to 'fmod'
liblua.a(liolib.o):function io_tmpfile: error: undefined reference to 'tmpfile'
liblua.a(lmathlib.o):function math_ceil: error: undefined reference to 'ceil'
liblua.a(lmathlib.o):function math_modf: error: undefined reference to 'ceil'
liblua.a(lmathlib.o):function math_log: error: undefined reference to 'log2'
liblua.a(lmathlib.o):function math_fmod: error: undefined reference to 'fmod'
liblua.a(lmathlib.o):function math_atan: error: undefined reference to 'atan2'
liblua.a(loslib.o):function os_difftime: error: undefined reference to 'difftime'
liblua.a(loslib.o):function os_clock: error: undefined reference to 'clock'
liblua.a(lstrlib.o):function match_class: error: undefined reference to 'iscntrl'
liblua.a(lstrlib.o):function match_class: error: undefined reference to 'isgraph'
liblua.a(lstrlib.o):function addquoted: error: undefined reference to 'iscntrl'
collect2: error: ld returned 1 exit status
make: *** [makefile:149: lua] Error 1

I tried medium model, but then compilation fails fast, and I'll have some work to do.

ia16-elf-gcc -mcmodel=medium -melks-libc -mtune=i8086 -Wall -Os -mno-segment-relocation-stuff -fno-inline -fno-builtin-printf -fno-builtin-fprintf -Wno-implicit-int -Wno-parentheses  -I/home/rafael2k/programs/devel/elks/include -I/home/rafael2k/programs/devel/elks/libc/include -I/home/rafael2k/programs/devel/elks/elks/include -Wextra -Wtype-limits -Wno-unused-parameter -Wno-sign-compare -Wno-empty-body -D__ELKS__ -DELKS_VERSION=\"0.9.0-dev\"   -c -o lapi.o lapi.c
In file included from lapi.h:11:0,
                 from lapi.c:18:
lapi.c: In function ‘lua_topointer’:
llimits.h:126:23: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
 #define cast(t, exp) ((t)(exp))
                       ^
llimits.h:129:23: note: in expansion of macro ‘cast’
 #define cast_voidp(i) cast(void *, (i))
                       ^~~~
lapi.c:490:27: note: in expansion of macro ‘cast_voidp’
     case LUA_VLCF: return cast_voidp(cast_sizet(fvalue(o)));
                           ^~~~~~~~~~
llimits.h:138:23: note: in expansion of macro ‘cast’
 #define cast_sizet(i) cast(size_t, (i))
                       ^~~~
lapi.c:490:38: note: in expansion of macro ‘cast_sizet’
     case LUA_VLCF: return cast_voidp(cast_sizet(fvalue(o)));
                                      ^~~~~~~~~~
lapi.c: In function ‘index2value’:
lapi.c:59:16: error: cannot create ‘__far’ function or static storage variable
 static TValue *index2value (lua_State *L, int idx) {
                ^~~~~~~~~~~
lapi.c: In function ‘lua_pcallk’:
lapi.c:1092:12: error: cannot take address of far function or far static variable
     status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func);
     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[ghaerr]

I like to use C99 when possible, so if the change is not intrusive, that would be welcome.

I think my comment about being able to quickly "fix" signal.h for use with C99 was nearsighted - the fix would eliminate the use of _signal because of __far being used in the declaration. But there are many other routines in our C headers, for instance in string.h fmemcmp, fmemcpy, etc that also use __far declarations. We would have to eliminate all of them with further ifdefs.

The problem with removing these functions is that we're using them for various applications - so we're basically not C99 compliant - which I suppose is not surprising since we've got 16-bit ELKS extensions in order to get everything working.

I've left C99 header "support" on my list to think about, but don't think we should make header changes quickly at this point. BTW, we've get both OWC and C86 compilers set to C99 but they're not emitting warnings for __far. In the C86 case, we punted and defined __far to nothing which is not really correct, but there is no far support in C86. OWC doesn't complain about its own extensions when C99 is set.

[ghaerr]

I tried medium model, but then compilation fails fast, and I'll have some work to do.

Yes it appears that Lua wants to have sizeof(int) == sizeof(char *). Instead of using medium model, perhaps look further at configuring Lua for a very small version that will compile in small model. Otherwise, you may end up having to use OWC large model. Lua is very well designed from what I've heard, reading more of its documentation will hopefully how it should be configured for small platforms.

[rafael2k]

I friend advised me that both can be done - use owc and remove mathlib from the build, use large model, or with ia16-gcc, really strip lots of modules and make it fit in the desired size (small model).

I'll give another shot with owc and disabling mathlib.

[rafael2k]

I like to use C99 when possible, so if the change is not intrusive, that would be welcome.

I think my comment about being able to quickly "fix" signal.h for use with C99 was nearsighted - the fix would eliminate the use of _signal because of __far being used in the declaration. But there are many other routines in our C headers, for instance in string.h fmemcmp, fmemcpy, etc that also use __far declarations. We would have to eliminate all of them with further ifdefs.

The problem with removing these functions is that we're using them for various applications - so we're basically not C99 compliant - which I suppose is not surprising since we've got 16-bit ELKS extensions in order to get everything working.

I've left C99 header "support" on my list to think about, but don't think we should make header changes quickly at this point. BTW, we've get both OWC and C86 compilers set to C99 but they're not emitting warnings for __far. In the C86 case, we punted and defined __far to nothing which is not really correct, but there is no far support in C86. OWC doesn't complain about its own extensions when C99 is set.

Got it. Indeed, no need for c99 in ia16-gcc. Btw, what would need to be done for a math library for watcom? It is possible to also have soft-fpu? This is not really for Lua, as at least 5.1 and 5.2 can be built integer-only, but in case of scattered floating point usage, it would be nice to have software fpu (requiring a FPU seems not a good option).

[ghaerr]

I've left C99 header "support" on my list to think about

so we're basically not C99 compliant

Got it. Indeed, no need for c99 in ia16-gcc

I like to use C99 when possible

After more research, I have found that ia16-elf-gcc sets __STRICT_ANSI__ when -std=c99 is set, and some ELKS header files already have support for removing non-standard extensions when __STRICT_ANSI__ is set. Seeing this, I now think we should go ahead and support C99 (or other) C standards in our header files, through the use of __STRICT_ANSI__ being defined.

This change will also allow some #ifndef C86 to be replaced, and I'm thinking of having CPP86 automatically define __STRICT_ANSI__ itself, since C86 doesn't have any extensions (notwithstanding the asm keyword).

I'll go ahead and make the changes to the libc and kernel header files, so that -std=c99 can be used with ia16-elf-gcc. Note that OWC is already being passed -std=c99 (using ewcc) and that in this case, the OpenWatcom C compiler does not define __STRICT_ANSI__, which allows it to use the __far keyword based functions, which is just want we want!

[ghaerr]

what would need to be done for a math library for watcom? It is possible to also have soft-fpu?

You might want to experiment around with it a bit, but I have not yet figured out how to get OWC to emit direct calls to software floating point emulation, which is supposed to work using -msoft-float (see ewcc for more details, which uses -Wc,-fpc).

We're currently using the -Wc,fpi87 option which emits inline 8087 FPU instructions. There is another option -mhard-emu-float which uses -Wc,fpi but that option generates special linker magic which somehow catches the CPU trap when 8087 is nonexistent and then traps to the OWC software floating point library which I would assume is linked into the executable. It is all very complicated and tricky.

If you'd like to experiment with -msoft-float and then run wdis on the output .obj file to figure out what is going on, that would help, but ultimately even once that is figured out we would need to port the entire OWC floating point emulation ASM code as well as their own math library, a huge effort.

[rafael2k]

Cool! Thanks the explanation. When I go back to porting Lua I'll try all options and inspect the generated code, as I to have a real fully-featured Lua, I guess we'll need large memory model (it works fine on DOS, so it should also work fine on ELKS). Without mathlib in Lua, Lua retains float support, but just not exposing the math libraries (of course, if the compiler supports float).

Btw, at some day in the future we could also enable FPU in C86 and see what happens...
: )

[ghaerr]

we could also enable FPU in C86 and see what happens

I looked into that - C86 generates 8087 hardware FPU instructions only, no soft FP emulation. When I tried porting the ELKS math library to OWC I got a number of problems, but don't recall exactly what they were offhand. C86 will likely be the same.

Should we want to "require" hardware FPU support, that excludes systems with CPUs earlier than about 486DX and no extra FPU chip installed, right?

Does anyone have any thoughts about how many 386 systems had 8087/80387 chips installed? I think that was a premium option for a few years there up until 1989 or so. We would probably have to flag our FP programs so that they at least didn't crash on those earlier systems.

[rafael2k]

SoftFPU seems a sensible approach indeed, when needed. I'll take a look what owc does when "fpc" is using, which should "generate calls to floating-point library".

[ghaerr]

I just played with running ewcc -msoft-float and it is working, OWC is generating calls to FP emulation routines, which I have found the source for in /path/to/owc/bld/clib/cgsupp/a/. It may not be as hard as I was thinking to get these routines ported over to our libc/watcom/asm library. I'll take a further look....

[rafael2k]

Yay, I'm with the fingers crossed!

With soft-float, OWC will be complete for C language IMHO, able to compile big boss software like Doom, Lua and zlib, and also small model software, like elks image viewer, sound/midi players, any small tool. Most importantly, owc supports large model, which ia16-gcc does not.

[toncho11]

Doom uses only OWC. Would this soft FPU support help Doom with speed? Probably no. Maybe Doom already handles floating point operations using integer arithmetic.

[ghaerr]

In general, except for modern CPUs, games written for older CPUs don't use floating point at all, or only rarely. Integer operations are wildly faster than floating point. Software floating point is the slowest of all, but has the huge advantage that it'll run on any CPU.

Floating point concerns were big in the early '80s, when "fast" FP support was hard to come by. Intel charged a lot of money for the 8087+ series of FPU upgrades, but the time the 80486DX shipped, FP was on-chip. Nowadays, almost all but certain embedded processors have on-chip FP.

Maybe Doom already handles floating point operations using integer arithmetic.

I'm not aware of whether Doom uses any FP, but I doubt it. I suspect it uses "fixed point", which is faster as computations are much simpler, but has wider "gaps" between representable numbers. In fixed or floating point representation, mathematical "real" numbers are stored with an "integer" portion and a "fractional" portion. In fixed point the decimal point between the two is fixed, whereas in floating point the decimal point "floats" allowing more precision in certain cases, using the same number of bits.

The original OpenWatcom compiler (non-ELKS) had three separate upper-level options for FP: hardware FP, hardware FP w/software fallback, and software FP. ELKS OWC previously had the hardware FP support turned on by default, since no library code was required. We now have software-only FP, although even that is complicated, as the compiler itself doesn't emit FPU instructions but the library we're using for emulating FP may use hardware FP if present. I have that turned off until we figure out how more how the library works.

We still have no support for any math library (math.h) functions in OWC - that's a whole another story, as OWC documentation is saying that the emulated floating point library codes is not IEEE 754 compliant, and our existing math library requires IEEE 754 format.

At this point, we've made a good step forward by adding basic software FP support, which means one can use C floats/doubles, add, subtract, multiple and divide them, and display them.

[rafael2k]

I managed to compile Lua 5.1.5 (a version widely still used) with OWC for ELKS!

ps: source code in https://github.com/rafael2k/lua

[toncho11]

I managed to compile Lua 5.1.5 (a version widely still used) with OWC for ELKS!

ps: source code in https://github.com/rafael2k/lua

Nice!!! :) Some Lua implementations come with graphics support.
There is also this console based: https://github.com/asciimoo/drawille
And pixel based functions:
https://franko.github.io/graph-toolkit/graphics.html

[toncho11]

I have used https://love2d.org/ . We could reimplement the drawing api from Love: https://love2d.org/wiki/love.graphics for vga graphics.

Or we can simply port all graphics functions from Basic to Lua.

[rafael2k]

Indeed! I'm right now trying some simple games in Lua to check if it is ok. Love2D is good option. A friend writes this one: https://github.com/gamelly/gly-engine

But in any case, we'll need the low level vga graphics functions, as you mention.

[toncho11]

@rafael2k Can you enable "Issues" on your Lua fork repository?
What is the right way to add graphics functions in Lua? A Lua library is a .so file. But does OWC or gcc-ia16 support dynamic libraries?

[toncho11]

ChatGPT says no and no for both.

[toncho11]

Adding Functions:

• If you want to add functions outside of the math library, modify lbaselib.c (for global functions) instead.
• Use lua_pushcfunction(L, my_function); lua_setglobal(L, "my_function"); to register a function globally.

[rafael2k]

@rafael2k Can you enable "Issues" on your Lua fork repository?

Enabled.

What is the right way to add graphics functions in Lua? A Lua library is a .so file. But does OWC or gcc-ia16 support dynamic libraries?

I would suggest we embed the module direct to our fork, and link all together.