Lua can easily overflow an unsigned short counting nested calls.
(The limit to this value is the maximum stack size, LUAI_MAXSTACK,
which is currently 1e6.)
Since commit f407b3c4a, it was being used for two distinct (and
incompatible) meanings:
A: Function has TBC variables (now bit CIST_TBC)
B: Interpreter is closing TBC variables (original bit CIST_CLSRET)
B implies A, but A does not imply B.
That reduces the size of "CallInfo". Moreover, bit CIST_HOOKED from
call status is not needed. When in a hook, 'transferinfo' is always
valid, being zero when the hook is not call/return.
Therefore, fields ftransfer/ntransfer in lua_Debug must have type
'int'. (Maximum stack size must fit in an 'int'.) Also, this commit
adds check that maximum stack size respects size_t for size in bytes.
Several definitions that don't need to be "global" (that is, that
concerns only specific parts of the code) moved out of llimits.h,
to more appropriate places.
The generational mode also uses the parameters for the incremental
mode in its major collections, so it should be easy to change those
parameters without having to change the GC mode.
The flag CIST_FIN does not mark a finalizer, but the function that was
running when the finalizer was called. (So, the function did not call
the finalizer, but it looks that way in the stack.)
Instead of assuming that shrinking a block may be an emergency
collection, use an explicit field ('gcstopem') to stop emergency
collections while GC is working.
When, inside a coroutine, a C function with to-be-closed slots return,
the corresponding metamethods can yield. ('__close' metamethods called
through 'lua_closeslot' still cannot yield, as there is no continuation
to go when resuming.)
To-be-closed variables are linked in their own list, embedded into the
stack elements. (Due to alignment, this information does not change
the size of the stack elements in most architectures.) This new list
does not produce garbage and avoids memory errors when creating tbc
variables.
Completes commit b07fc10e91. '__close' metamethods can yield even
when they are being called due to an error. '__close' metamethods from
C functions are still not allowed to yield.
The stack size is derived from 'stack_last', when needed. Moreover,
the handling of stack sizes is more consistent, always excluding the
extra space except when allocating/deallocating the array.
The previous stackless implementations marked all 'luaV_execute'
invocations as fresh. However, re-entering 'luaV_execute' when
resuming a coroutine should not be a fresh invocation. (It works
because 'unroll' called 'luaV_execute' for each call entry, but
it was slower than letting 'luaV_execute' finish all non-fresh
invocations.)
A "with stack" implementation gains too little in performance to be
worth all the noise from C-stack overflows.
This commit is almost a sketch, to test performance. There are several
pending stuff:
- review control of C-stack overflow and error messages;
- what to do with setcstacklimit;
- review comments;
- review unroll of Lua calls.
