Instead of updating 'L->top' in every place that may call a
metamethod, the metamethod functions themselves (luaT_trybinTM and
luaT_callorderTM) correct the top. (When calling metamethods from
the C API, however, the callers must preserve 'L->top'.)
- OP_NEWTABLE can use 'ra + 1' to set top (instead of ci->top);
- OP_CLOSE doesn't need to set top ('Protect' already does that);
- OP_TFORCALL must use 'ProtectNT', to preserve the top already set.
(That was a small bug, because iterators could be called with
extra parameters besides the state and the control variable.)
- Comments and an extra test for the bug in previous item.
As an example, 'print(string.format("%.99f", 1e70))' may have a
lot of garbage after the number.
The old test to ensure that 'string.format("%.99f", n)' was not too
large, 'fabs(n) < 1e100', assumes that the number will fit in the 99
bytes; but the 99 is not the space for the number, it is the added
extra zeros. The option worked for smaller numbers because of the
extra space added to MAX_ITEM.
Open upvalues are kept alive together with their corresponding
stack. This change makes a simpler and safer fix to the issue in
commit 440a5ee78c, about upvalues in the list of open upvalues
being collected while others are being created. (That previous fix
may not be correct.)
The addresses of static variables may be different for different
instances of Lua, making these instances incompatible if they use
these addresses as unique keys in the registry (or other tables).
When creating an upvalue, an emergency collection can collect the
previous upvalue where the new one would be linked. The following
code can trigger the bug, using valgrind on Lua compiled with the
-DHARDMEMTESTS option:
local x; local y
(function () return y end)();
(function () return x end)()
- Several new comments in 'lmem.c'.
- Both 'luaM_growaux_' and 'luaM_shrinkvector_' use 'luaM_saferealloc_'
to check for errors. Moreover, the use of 'luaM_saferealloc_' makes
'luaM_shrinkvector_' try again if shrink fails (which can happen now).
- In 'checkSizes', save old debt only when needed.
- Macro 'checkliveness' (for debug) always uses 'L', to avoid warnings.
- Some old 'while' changed to 'for' in 'testes/gc.lua'.
- In 'testes/libs/makefile', do not make files depend on 'ltests.h',
which may not even exist.
This commit brings a new implementation for HARDMEMTESTS, which forces
an emergency GC whenever possible. It also fixes some issues detected
with this option:
- A small bug in lvm.c: a closure could be collected by an emergency
GC while being initialized.
- Some tests: a memory address can be immediatly reused after a GC;
for instance, two consecutive '{}' expressions can return exactly the
same address, if the first one is not anchored.
OP_RETURN must update trap before updating stack. (Bug detected with
-DHARDSTACKTESTS). Also, in 'luaF_close', do not create a variable
with 'uplevel(uv)', as the stack may change and invalidate this
value. (This is not a bug, but could become one if 'upl' was used
again.)
Constants directly assigned to other constants were not propagating:
For instance, in
local <const> k1 = 10
local <const> k2 = k1
'k2' were not treated as a compile-time constant.
String literal expressions have their own kind VKSTR, instead of the
generic VK. This allows strings to "cross" functions without entering
their constant tables (e.g., if they are used only by some nested
function).
An error in a closing method may be caused by a lack of resources,
such as memory or stack space, and the error may free enough resources
(by unwinding the stack) to allow the method to work if called again.
If the closing method is already running after some error (including
its own), it is not called again.
Opcodes OP_NEWTABLE and OP_SETLIST use the same representation to
store the size of the array part of a table. This new representation
can go up to 2^33 (8 + 25 bits).
OP_NEWTABLE is followed by an OP_EXTRAARG, so that it can keep
the exact size of the array part of the table to be created.
(Functions 'luaO_int2fb'/'luaO_fb2int' were removed.)
In the generic for loop, it is simpler for OP_TFORLOOP to use the
same 'ra' as OP_TFORCALL. Moreover, the internal names of the loop
temporaries "(for ...)" don't need to leak internal details (even
because the numerical for loop doesn't have a fixed role for each of
its temporaries).
This commit detaches the number of active variables from the
number of variables in the stack, during compilation. Soon,
compile-time constants will be propagated and therefore will
not exist during run time (in the stack).
VLOCAL expressions keep a reference to their corresponding 'Vardesc',
and 'Upvaldesc' (for upvalues) has a field 'ro' (read-only). So, it is
easier to check whether a variable is read-only. The decoupling in
VLOCAL between 'vidx' ('Vardesc' index) and 'sidx' (stack index)
should also help the forthcoming implementation of compile-time
constant propagation.
The syntax for local attributes ('const'/'toclose') was unified with
the regular syntax for local variables, so that we can have variables
with attributes in local definitions with multiple names; for instance:
local <toclose> f, <const> err = io.open(fname)
This new syntax does not implement constant propagation, yet.
This commit also has some small improvements to the manual.
In the 'io' library, changed the use of the metatable also as its
own "method table", so that metamethods cannot be accessed as if they
were methods. (For instance, 'io.stdin.__gc' does not result in
the finalizer metamethod anymore.)
- Better documentation in 'testes/cstack.lua' about using
'debug.setCstacklimit' to find a good limit.
- Constant LUAI_MAXCSTACK gets added CSTACKERR (extra stack for
error handling), so that it is compatible with the argument to
'debug.setCstacklimit'.
- Added a test for calling 'debug.traceback' after yields inside
hooks. (Lua 5.3 seems to have a bug there.)
- Removed test "repeat test with '__open' metamethod instead of a
function", as the previous test already uses the '__open' metamethod.
(It changed when functions were removed as possible to-be-closed
variables).
Removed the field 'name' from the structure 'Vardesc', as the name
of the local variable is already available in the prototype of the
function, through the index 'idx'.
Ensure that operation macros, such as 'luai_numdiv' and 'luai_numidiv',
operate only on variables, or at most at 's2v(ra)'. ('s2v' is a nop, a
cast from pointer to pointer.)
When there are multiple errors when closing objects, the error
reported by the protected call is the first one, for two reasons:
First, other errors may be caused by this one;
second, the first error is handled in the original execution context,
and therefore has the full traceback.
Instead of a 'tocont' flag, the function 'warn' in Lua now receives all
message pieces as multiple arguments in a single call. Besides being
simpler to use, this implementation ensures that Lua code cannot create
unfinished warnings.
- tests show progress in real time, so that we can see maximum
stack levels even if test crashes.
- new test for recursion continuing into message handler.
'luaL_traceback' changed to use an aux buffer instead of concats.
This should reduce the quantity of garbage it generates (in the
form of intermediate strings) while producing a trackback.
It also added information about the number of levels skipped when
skipping levels in a trace.
- new error message for "attempt to assign to const variable"
- note in the manual about compatibility options
- comments
- small changes in 'read_line' and 'pushstr'
- The preprocessor must work with at least 'long', and therefore must
do shifts of up to 31 bits correctly.
- Whenever possible, use unsigned types in shifts.
The flag for to-be-closed variables was changed from '*toclose'
to '<toclose>'. Several people found confusing the old syntax and
the new one has a clear terminator, making it more flexible for
future changes.
Back to how it was, a coroutine does not unwind its stack in case of
errors (and therefore do not close its to-be-closed variables). This
allows the stack to be examined after the error. The program can
use 'coroutine.kill' to close the variables.
The function created by 'coroutine.wrap', however, closes the
coroutine's variables in case of errors, as it is impossible to examine
the stack any way.
- 'L' added to the 'BuffFS' structure
- '%c' does not handle control characters (it is not its business.
This now is done by the lexer, who is the one in charge of that
kind of errors.)
- avoid the direct use of 'l_sprintf' in the Lua kernel
Both when setting a path and searching for a file ('searchpath'),
this commit reduces the number of intermediate strings created
in Lua.
(For setting a path the change is not relevant, because this is
done only twice when loading the module. Anyway, it is a nice example
of how to use auxlib buffers to manipulate strings in the C API.)
To-be-closed variables in C use 'ci.nresults' to code that there is
a variable to be closed in that function. The intialization of the
base C level (the one "running" when calling API functions outside
any Lua call) did not initialize 'ci.nresults', creating (correct)
warnings in valgrind.
Added a new function 'luaL_addgsub', similar to 'luaL_gsub' but that
adds its result directly to a preexisting buffer, avoiding the creation
of one extra intermediate string. Also added two simple macros,
'luaL_bufflen' and 'luaL_buffaddr', to query the current length
and the contents address of a buffer.
The function 'luaO_pushvfstring' now uses an internal buffer to
concatenate small strings, instead of pushing all pieces on the
stack. This avoids the creation of several small Lua strings for each
piece of the result. (For instance, a format like "n: '%d'" used to
create three intermediate strings: "n: '", the numeral, and "'".
Now it creates none.)
The result of "string.format("%.99f", -1e308) is 410 characters long,
but all other formats have much smaller limits (at most 99 plus a fex
extras). This commit avoids 'string.format' asking for a buffer
~400 chars large when ~100 will do.
Avoid creating extra strings when possible:
- avoid creating new resulting string when subject was not modified
(instead, return the subject itself);
- avoid creating strings representing the captured substrings when
handling replacements like '%1' (instead, add the substring directly
to the buffer).
This new field gets the length of 'source' in the same structure.
Unlike the other strings in that structure, 'source' can be
relatively large, and Lua already has its length readily available.
The 'GCSenteratomic' is just an auxiliary state for transitioning
to 'GCSatomic'. All GC traversals should be done either on the
'GCSpropagate' state or the 'GCSatomic' state.
Checks of the form '1 <= x && x <= M' were rewritten in the form
'(unsigned)x - 1 < (unsigned)M', which is usually more efficient.
(Other similar checks have similar translations.) Although
some compilers do these optimizations, that does not happen
for all compilers or all cases.
The limit LUAI_MAXCCALLS was renamed LUAI_MAXCSTACK, which better
represents its meaning. Moreover, its definition was moved to
'luaconf.h', given its importance now that Lua does not use
a "stackless" implementation.
- In 'readutf8esc' (llex.c), the overflow check must be done before
shifting the accumulator. It was working because tests were using
64-bit longs. Failed with 32-bit longs.
- In OP_FORPREP (lvm.c), avoid negating an unsigned value. Visual
Studio gives a warning for that operation, despite being well
defined in ISO C.
- In 'luaV_execute' (lvm.c), 'cond' can be defined only when needed,
like all other variables.
When calling metamethods for things like 'a < 3.0', which generates
the opcode OP_LTI, the C register tells that the operand was
converted to an integer, so that it can be corrected to float when
calling a metamethod.
This commit also includes some other stuff:
- file 'onelua.c' added to the project
- opcode OP_PREPVARARG renamed to OP_VARARGPREP
- comparison opcodes rewritten through macros
Changed some implementation details; in particular, it is back using
an internal variable to keep the index, with the control variable
being only a copy of that internal variable. (The direct use of
the control variable demands a check of its type for each access,
which offsets the gains from the use of a single variable.)
- LUAC_VERSION is equal to LUA_VERSION_NUM, and it is stored
as an int.
- 'sizeof(int)' and 'sizeof(size_t)' removed from the header, as
the binary format does not depend on these sizes. (It uses its
own serialization for unsigned integer values.)
The numerical 'for' loop over integers now uses a precomputed counter
to control its number of iteractions. This change eliminates several
weird cases caused by overflows (wrap-around) in the control variable.
(It also ensures that every integer loop halts.)
Also, the special opcodes for the usual case of step==1 were removed.
(The new code is already somewhat complex for the usual case,
but efficient.)
All UTF-8 encoding functionality (including the escape
sequence '\u') accepts all values from the original UTF-8
specification (with sequences of up to six bytes).
By default, the decoding functions in the UTF-8 library do not
accept invalid Unicode code points, such as surrogates. A new
parameter 'nonstrict' makes them accept all code points up to
(2^31)-1, as in the original UTF-8 specification.
- The warning functions get an extra parameter that tells whether
message is to be continued (instead of using end-of-lines as a signal).
- The user data for the warning function is a regular value, instead
of a writable slot inside the Lua state.
When called with no arguments, 'math.randomseed' uses time and ASLR
to generate a somewhat random seed. the initial seed when Lua starts
is generated this way.
Removed code to ensure that strings inside Lua (as returned by
'lua_tolstring') always start in fully aligned addresses.
Since version 5.3 the documentation does not ensure that.
Several small improvements (code style, warnings, comments, more tests),
in particular:
- 'lua_topointer' extended to handle strings
- raises an error in 'string.format("%10q")' ('%q' with modifiers)
- in the manual for 'string.format', the term "option" replaced by
"conversion specifier" (the term used by the C standard)
After a major bad collection (one that collects too few objects),
next collection will be major again. In that case, avoid switching
back to generational mode (as it will have to switch again to
incremental to do next major collection).
The function 'string.gmatch' now has an optional 'init' argument,
similar to 'string.find' and 'string.match'. Moreover, there was
some reorganization in the manipulation of indices in the string
library.
This commit also includes small janitorial work in the manual
and in comments in the interpreter loop.
To-be-closed variables must contain objects with '__toclose'
metamethods (or nil). Functions were removed for several reasons:
* Functions interact badly with sandboxes. If a sandbox raises
an error to interrupt a script, a to-be-closed function still
can hijack control and continue running arbitrary sandboxed code.
* Functions interact badly with coroutines. If a coroutine yields
and is never resumed again, its to-be-closed functions will never
run. To-be-closed objects, on the other hand, will still be closed,
provided they have appropriate finalizers.
* If you really need a function, it is easy to create a dummy
object to run that function in its '__toclose' metamethod.
This comit also adds closing of variables in case of panic.
