lush/issues/24-test-lua-compatibility.md

24 — Test against major Lua projects

Status: open

Problem

Lush modifies the Lua core (parser, lexer, runtime). We need to verify that these changes don't break compatibility with real-world Lua code.

Projects to test against

Language compliance (start here)

• Lua 5.4 test suite (testes/) — the gold standard; tests edge cases of the language spec directly
• LuaJIT test suite — good for catching subtle semantic differences; also runs the official Lua tests

Embedded / scripting (real-world stress tests)

• Neovim — uses Lua extensively for config and plugins; large test suite using busted
• OpenResty / nginx-lua (lua-nginx-module) — huge t/ test directory using Test::Nginx
• Kong — API gateway built on OpenResty; thousands of test cases, stresses coroutines, metatables, and C API

Game engines

• Minetest (Luanti) — open source voxel engine, massive Lua modding API with tests
• LÖVE (love2d) — 2D game framework; community test suites

Lua tooling / frameworks

• Luacheck — linter with a large Lua test suite
• busted — BDD test framework with good self-coverage
• lua-TestMore — port of Perl's Test::More to Lua

Recommended priority

1. testes/ from official Lua 5.4 source — most authoritative
2. Neovim's test suite — real-world, large scale
3. Kong's test suite — stresses coroutines, metatables, and C API
4. LuaJIT test suite — catches subtle semantic differences

Goal

Confirm that standard Lua programs run correctly on Lush without modification. Any failures should be triaged as either acceptable (due to intentional syntax changes) or bugs to fix.

Implementation

Add a new dir to the gitignore which will be used to clone projects. Clone/pull a set of projects that use lua and have a test suite. Make a shallow clone Each project will have different dependencies. We need to audit each one and investigate how feasible it is to integrate them into the script. Lush forks from lua, check which version we forked from and clone compatible tags/versions of each project. Run their tests using our language.

Many projects use busted/luacheck, we may be able to use a system installed version via luarocks or brew, we should investigate the compatiblility of this.