the memory model being used in elks?

Hello,

Finally I tried to test the "pres" utility on elks as opposed to elksemu.

Too bad, it reports lack of memory
and it looks like the maximum available memory is around 30K, the console shows:

sys_brk failed: len -32634 > end_seg 30864
sys_brk failed: len -32632 > end_seg 30864
sys_brk failed: len 31368 > end_seg 30864
...

I assumed somehow a data segment with around 60K available, is this not the case?

The same binary works just fine under elksemu.

Rl

Did you compile it with split I and D ? (-i)

Alan

Thanks for the suggestion.

I expected this to be the default, which corresponded to the report from the elks "file" command (all files and "pres" too were reported as split i/d). "File" might have been wrong though.

Now I also have rechecked the make logs and bcc documentation. There was no "-i" flag at the build and the linker command was "bcc".

The man page states that the "-i" flag makes bcc to avoid passing "-i" to the linker and would in that case lead to creation of an "impure executable".

IOW, as much as I can see, the binary is built correctly and it is the memory allocation by the kernel which looks surprizing.

I have tried building kernels both with and without the "advanced memory manager", the resulting memory limitation looks the same.

Rl

It is my understanding that bcc assumes DS=ES=SS and both stack and heap data use the same code segment. How did you determine the memory usage of the program to be 60K? Does that include all usage or only heap usage?

Yes, this is fine.

I counted the approximate size of the static and allocated data objects and also tested. This includes all the data ("data+bss"+heap+stack).

This worked fine with separate i/d on Venix/86 on IBM PC XT, where DS=SS and heap and stack grew from the opposite ends of the segment, heap from the bottom up and the stack from the top down (the hole in the middle if big enough was used for other processes' data, not wasted iirc).

Venix also had kernel support for ES as extra data of a process so f.i. access to the graphic memory was done via ES. The compiler was not involved in these tricks, it just did not use ES.

Rl

Compile with flags

-i -H E000

Flag H controls the size of BSS.

I see, a minix-like static allocation (I'm guessing this is needed because ELKS does not yet handle overlapping segments between different processes so it does not want to allocate a whole segment to each process per default?).

Yes, indeed, this helped, but I had to both set

-H0xff00
and
-DBLON=3584 (reduce the compression block size)

to get rid of "no memory".

The utility still performs better than ELKS compress but this scaling down should not be needed.

8KB compression block size works under elksemu (about 9KB extra for the buffers compared to the 3.5KB block), what causes this difference?

Rl

Classic Unix defines the memory map as

<--- Data ---><--- BSS ---> | <--- Space ---> | <--- Stack --- >

Where the upper divide is SP and the lower one is set by brk() or sbrk(). The space in the middle doesn't all really exist if you have an MMU with any kind of sane paging system.

If you have support built for ELKS pattern binaries then you can do resizing at runtime as they are laid out to allow the bss to extend

Alan

Even without MMU overlapping 8086 segments allow the use of this <-Space-> for stack or data+bss of other processes. Wonder if/how this is handled in elks (lacking the time to Use-The-Source...)

This is what I expected - but rebuilding the kernel with and without the corresponding option did not seem to change the result.

Indicating the desired data area size with -H improved the outcome but still the available space looks smaller than it should be. Sigh, I did not plan to do kernel hacking just to make the pres utility work as it used to. :)

/Rl

Only if you use far pointers which is incredibly expensive, otherwise taking the address of a stack based variable breaks.

Alan

Not at all. The process has its full 2^16 address space for data, the stack always at the top, the data+heap at the bottom, no far pointers, no problem to access stack based variables.

It is up to the kernel to keep track of the sizes of heaps and stacks of all processes (which it does in any case) and move processes' data segments in RAM when necessary so that the segments can and do overlap but not the data. Then the RAM area inside the unused address space in the middle of the segment is not wasted.

process1:      |data+heap>                            <stack|
process2:                     |--data+heap-->                    <--stack--|
               |<-------------------- 64K ----------------->|
                              |<-------------------- 64K ----------------->|

As many can overlap as it fits, with some extra margins for the stacks. The same works of course for combined code+data+heap too.

Rl

Very clever! You stil then need to know the stack sizes and automatically trap and move which means you need to call a helper on function entry or alloca, but that's not difficult.

It doesn't work in protected mode, which ELKS supports but for real mode that would work beautifully.

The ELKS format basically turns the traditional arrangement around with a predefined stack size rather than heap size, and stack at bottom, so the heap can then expand upwards without mess.

Alan

Venix/86 did not use such helpers. I think it was reserving some reasonable amount for the stack and also was checking the stack pointer of the running process regularly for entering the "red zone" or something like that. This worked quite good provided that nobody put large arrays on the stack - the latter did not work and could kill the system.

Indeed, why not.

Apparently it preallocates too much for the stack in my case. Is there any means to tune this per binary?

Rl