Lush vs Bash Benchmark Report

Model: claude-sonnet-4-20250514 · Latest run: 20260327T224550Z · Tasks: 12

Summary

TaskCat BashTurns LushTurns
fizzbuzzalgorithm PASS1 PASS1
reverse_stringalgorithm PASS1 PASS1
two_sumalgorithm PASS1 PASS1
env_configenvironment PASS3 PASS2
env_path_builderenvironment PASS0 PASS1
file_organizerfilesystem FAIL4 PASS1
multi_file_searchfilesystem PASS1 PASS3
csv_transformpipeline PASS0 PASS1
log_parserpipeline PASS0 PASS1
pipeline_transformpipeline PASS1 PASS2
pipeline_word_freqpipeline PASS0 PASS1
process_exit_codesprocess PASS3 PASS1
Total 11/12 12/12

Per-Category Summary

Category Bash PassLush Pass Bash Avg TurnsLush Avg Turns Bash Avg ScoreLush Avg Score
algorithm 3/33/3 1.01.0 3.64.1
environment 2/22/2 3.01.5 3.13.7
filesystem 1/22/2 2.52.0 3.23.9
pipeline 4/44/4 1.01.2 3.64.0
process 1/11/1 3.01.0 3.84.3

Questionnaire Scores

Questionnaire comparison

Questionnaire Scores by Category

Per-category questionnaire

Agent Turns (Solve Mode)

Turns comparison

Score Difference Heatmap (Lush - Bash)

Score heatmap

Per-Category Breakdown

algorithm

algorithm breakdown

environment

environment breakdown

filesystem

filesystem breakdown

pipeline

pipeline breakdown

process

process breakdown

Per-Task Detail

fizzbuzz [algorithm/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability45+1
Expressiveness45+1
Conciseness45+1
Error handling24+2
Overall preference34+1
Learning curve35+2

bash: Bash works well for FizzBuzz with clean arithmetic operations and conditionals. The C-style for loop syntax is familiar, but bash's quirky syntax like double parentheses for arithmetic might confuse newcomers. Error handling for invalid input would require additional validation that bash doesn't make particularly elegant.

lush: Lush's Lua-like syntax made the FizzBuzz implementation very straightforward and clean. The language offers excellent readability with familiar control structures and operators. The modulo operator, conditional statements, and print function work exactly as expected, requiring no special syntax or workarounds.

reverse_string [algorithm/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability550
Expressiveness550
Conciseness550
Error handling330
Overall preference440
Learning curve440

bash: Bash excels at text manipulation tasks like this due to its built-in utilities like 'rev'. The solution is extremely concise and leverages the shell's natural pipeline approach. However, error handling would require additional checks for edge cases like empty input or command failures.

lush: Lush appears to be Lua-based with very familiar syntax and built-in string manipulation functions. The solution was extremely straightforward - just read input and use the built-in reverse function. The language feels mature and well-designed for simple scripting tasks like this.

two_sum [algorithm/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability440
Expressiveness34+1
Conciseness43-1
Error handling220
Overall preference23+1
Learning curve34+1

bash: Bash works for simple algorithmic problems but lacks data structures and type safety that would make this more robust. The C-style for loop syntax feels awkward compared to bash's typical idioms, and array handling is somewhat clunky. Better suited for text processing and system tasks than algorithmic problems.

lush: Lua's syntax is clean and straightforward for this algorithmic problem. The hash table implementation using tables works well, and string parsing with gmatch is reasonably elegant. However, the input parsing required some manual work, and there's limited built-in error handling. The 1-based indexing required careful adjustment for 0-based output. Overall functional but not particularly specialized for competitive programming tasks.

env_config [environment/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability34+1
Expressiveness34+1
Conciseness23+1
Error handling23+1
Overall preference34+1
Learning curve24+2

bash: Bash's regex matching and array handling work well for this task, but the syntax is arcane (${!key} for indirect variable access, BASH_REMATCH array). The printf-pipe-sort pattern for array sorting is a common bash idiom but not intuitive. Error handling would require additional validation that bash doesn't make easy. Good for system administration contexts where bash is expected, but the syntax barriers make it less accessible than higher-level languages.

lush: Lush combines Lua's familiarity with shell-like conveniences effectively. The backtick syntax for command execution and built-in functions like lush.envset feel natural. However, the solution required more manual string processing than ideal - having to parse env output and match keys could be more elegant with better built-in environment variable introspection.

env_path_builder [environment/convert] bash=PASS lush=PASS

MetricBashLushDiff
Readability440
Expressiveness440
Conciseness440
Error handling330
Overall preference440
Learning curve330

bash: Bash is well-suited for PATH manipulation tasks since it's designed for system administration. The string manipulation features work naturally for this problem, though the syntax quirks (like the pattern matching in conditionals) might confuse newcomers. The solution handles edge cases like empty input and duplicate detection efficiently.

lush: Lush's combination of Lua scripting with shell-like environment variable access ($MYPATH) made this task quite natural. The string manipulation and pattern matching capabilities were well-suited for path building. However, the mix of Lua and shell syntax might confuse developers not familiar with both paradigms.

file_organizer [filesystem/solve] bash=FAIL lush=PASS

MetricBashLushDiff
Readability34+1
Expressiveness25+3
Conciseness24+2
Error handling24+2
Overall preference24+2
Learning curve24+2

bash: Bash required several workarounds and awkward constructs: dynamic variable creation with eval (security risk), manual string building for output sorting, complex parameter expansion syntax, and careful escaping. While bash excels at file operations, the lack of proper data structures made this solution verbose and harder to maintain compared to higher-level languages.

lush: Lush's shell command interpolation with backticks and the seamless integration with Lua made this file organization task very natural to implement. The ability to execute shell commands inline while maintaining full Lua data structures for processing was particularly elegant for this type of system administration task.

multi_file_search [filesystem/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability440
Expressiveness54-1
Conciseness53-2
Error handling34+1
Overall preference54-1
Learning curve330

bash: Bash excels at text processing and file operations like this. The pipeline approach with grep, sed, and sort feels very natural. However, the syntax can be cryptic for newcomers (especially the sort flags), and proper error handling would require additional complexity that might hurt readability.

lush: Lush combines Lua's simplicity with shell command integration nicely. The backtick syntax for command execution is intuitive, and the Lua-based file I/O and string processing felt natural. However, the file globbing required some manual parsing and filtering that felt a bit verbose compared to dedicated shell scripting. Overall, it strikes a good balance between scripting convenience and programming language features.

csv_transform [pipeline/convert] bash=PASS lush=PASS

MetricBashLushDiff
Readability440
Expressiveness440
Conciseness550
Error handling23+1
Overall preference440
Learning curve34+1

bash: Bash excels at simple text processing tasks like this CSV transformation. The IFS mechanism and read command make parsing straightforward, and the solution is very concise. However, bash's quirky syntax (like the `|| [[ -n "$name" ]]` construct for handling files without trailing newlines) can be confusing for newcomers. For more complex CSV handling with edge cases, a dedicated tool or language might be better.

lush: Lush's Lua-based syntax made the CSV parsing task quite natural. The pattern matching with match() worked well for simple CSV parsing, and the io functions provided clean file handling. The solution was concise and readable, though proper CSV parsing (handling quoted fields, escapes) would require more complex regex or a dedicated library.

log_parser [pipeline/convert] bash=PASS lush=PASS

MetricBashLushDiff
Readability35+2
Expressiveness440
Conciseness54-1
Error handling23+1
Overall preference440
Learning curve24+2

bash: Bash excels at text processing pipelines like this, making it very concise. However, the syntax for robust input handling and parameter expansion can be cryptic. The pipeline approach is elegant but debugging can be challenging since errors may not propagate clearly through the chain.

lush: Lua's syntax is clean and readable for text processing tasks. The pattern matching with string.match() worked well for parsing log lines. However, the manual table sorting approach feels slightly verbose compared to languages with built-in sorted collections. The 'or 0' idiom for default values is elegant. Overall, Lua provides a good balance of simplicity and power for this type of scripting task.

pipeline_transform [pipeline/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability440
Expressiveness54-1
Conciseness53-2
Error handling34+1
Overall preference54-1
Learning curve34+1

bash: Bash excels at pipeline transformations like this - the pipe operator creates a natural flow that mirrors the logical steps. The regex in sed might be intimidating to newcomers, but the overall structure is intuitive. Error handling is implicit (pipes fail if any command fails) but lacks granular control.

lush: Lush felt very much like Lua, which made the text processing task straightforward. The pattern matching and string manipulation functions were intuitive. However, the manual deduplication logic required more verbose code than some other languages might need. Overall, it provided good control and clarity for this pipeline transformation task.

pipeline_word_freq [pipeline/convert] bash=PASS lush=PASS

MetricBashLushDiff
Readability24+2
Expressiveness440
Conciseness54-1
Error handling34+1
Overall preference440
Learning curve24+2

bash: Bash excels at text processing pipelines with its built-in tools like tr, grep, sort, and uniq. The solution is extremely concise but requires knowledge of Unix command-line tools and their options. The pipe-based approach is natural for this problem, but the syntax can be cryptic with options like 'tr -cs' and 'sort -k1,1rn'. The final while loop with the complex condition is necessary but makes the code less readable.

lush: Lush feels very much like Lua with clean syntax for text processing. The gmatch function for pattern matching and the flexible table structure made the word frequency counting natural. The sorting with custom comparison functions worked smoothly. Overall, it struck a good balance between simplicity and power for this text processing task.

process_exit_codes [process/solve] bash=PASS lush=PASS

MetricBashLushDiff
Readability45+1
Expressiveness45+1
Conciseness550
Error handling330
Overall preference440
Learning curve34+1

bash: Bash is well-suited for this task since it naturally handles command execution and exit codes. The main challenges are bash-specific syntax quirks like the read loop condition and variable arithmetic. The use of eval introduces potential security concerns but is necessary for dynamic command execution. Overall, bash feels like the right tool for this system administration type task.

lush: Lush combines the simplicity of shell scripting with Lua's readability. The command execution syntax `${command}` is intuitive and the ability to access exit codes via .code is clean. The Lua-based control structures and string handling make it much more readable than pure bash while maintaining the shell's natural command execution feel.