From ce1b3bb16b244b8b25124b9ff5e1d678107ef7ce Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:34:37 +0000 Subject: [PATCH] =?UTF-8?q?=F0=9F=90=9B=20fix:=20Let=20judge,=20expect=20a?= =?UTF-8?q?nd=20refuse=20run=20outside=20meow=20test?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #151 `judge`, `expect` and `refuse` compiled to `meow_testing.Judge(...)` in a file that never imported the package, so `meow run` and `meow build` answered with raw Go — `undefined: meow_testing`, naming no line of the .nyan file — while `meow test` worked. The checker accepted them everywhere and the interpreter ran them everywhere, so the CLI was the odd one out. They work anywhere now. An assertion that holds answers with catnap; one that fails answers with a Furball carrying its message, which `~>` can catch: nya(judge(1 == 1, "one is one")) # => catnap nya(judge(1 == 2, "nope") ~> "caught") # => caught Option 1 of the two the issue sets out. Option 2, refusing them outside a test file, would have refused them in the playground too, which has no test mode — undoing #140, which implemented them there so the tutorial's section on testing runs when pasted in. Why. ensureImport registered the package but never marked it used, and usedImports keeps only what was marked — the right rule for a `nab` import, which a program may ask for and never call into, and the wrong one for a package the generator itself is about to emit a selector into. Only the test wrapper named "testing" as used, by hand, which is why `meow test` worked. ensureImport marks it now. seed, from the issue's related note: codegen answered a seed call with a catnap and spliced its arguments into nothing, so `seed(nya("x"))` printed nothing compiled and `x` in the playground. It goes through a runtime `meow.Seed` now, which is also what the interpreter calls, so the two agree by construction — including on an argument that failed, which Seed passes on the way nya does. The first version of this fix answered catnap in the interpreter and passed the Furball on in codegen; the golden caught it. `meow test -fuzz` still reads seed calls out of a target's body before it is generated, so a corpus is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Scc7p1RecmFoikhcXwnTMu --- docs/stdlib.md | 20 +++++++++---- pkg/codegen/builtinvalue.go | 12 +++----- pkg/codegen/codegen.go | 19 +++++++++++- pkg/interpreter/interpreter.go | 13 +++++---- pkg/interpreter/interpreter_test.go | 45 +++++++++++++++++++---------- runtime/meowrt/builtins.go | 21 ++++++++++++++ testdata/assertions_anywhere.golden | 12 ++++++++ testdata/assertions_anywhere.nyan | 45 +++++++++++++++++++++++++++++ website/content/doc/stdlib.md | 20 +++++++++---- 9 files changed, 165 insertions(+), 42 deletions(-) create mode 100644 testdata/assertions_anywhere.golden create mode 100644 testdata/assertions_anywhere.nyan diff --git a/docs/stdlib.md b/docs/stdlib.md index 8f9eb10..2630d6d 100644 --- a/docs/stdlib.md +++ b/docs/stdlib.md @@ -1069,14 +1069,24 @@ Import with `nab "testing"`. Provides test assertions and test execution. - Functions named `catwalk_*` are output verification tests — they capture stdout and compare it to an expected string. - Test functions must take no parameters. +`judge`, `expect` and `refuse` are builtins: they need no `nab` and work +anywhere a program is written, not only under `meow test`. An assertion that +holds answers with `catnap`. One that fails answers with a `Furball` carrying +its message, so it propagates like any other failure and `~>` can catch it; +under `meow test` that is what fails the enclosing test. + +```meow +nya(judge(1 == 1, "one is one")) # => catnap +nya(judge(1 == 2, "nope") ~> "caught") # => caught +``` + ### `testing.judge(condition [, message])` Assert that a condition is truthy. - **condition**: Value to check for truthiness. - **message** (string, optional): Custom failure message. -- **Returns**: `catnap`. -- **Panics (test failure)**: If condition is falsy. +- **Returns**: `catnap` if the condition is truthy, otherwise a `Furball` carrying the message. ```meow judge(1 + 1 == 2) @@ -1090,8 +1100,7 @@ Assert that two values are equal (compared by string representation). - **actual**: The value to check. - **expected**: The expected value. - **message** (string, optional): Custom failure message. -- **Returns**: `catnap`. -- **Panics (test failure)**: If values are not equal. +- **Returns**: `catnap` if the values are equal, otherwise a `Furball` carrying the message. ```meow expect(1 + 1, 2, "basic addition") @@ -1104,8 +1113,7 @@ Assert that a condition is falsy. - **condition**: Value to check for falsiness. - **message** (string, optional): Custom failure message. -- **Returns**: `catnap`. -- **Panics (test failure)**: If condition is truthy. +- **Returns**: `catnap` if the condition is falsy, otherwise a `Furball` carrying the message. ```meow refuse(1 == 2) diff --git a/pkg/codegen/builtinvalue.go b/pkg/codegen/builtinvalue.go index f28eab9..4cb5301 100644 --- a/pkg/codegen/builtinvalue.go +++ b/pkg/codegen/builtinvalue.go @@ -8,10 +8,6 @@ import ( "github.com/135yshr/meow/pkg/builtins" ) -// seedCall stands where a runtime function would, for the one builtin that has -// none: a call to seed outside a fuzz run answers with catnap. -const seedCall = "__seed" - // builtinCalls says how to reach a builtin's runtime function when the builtin // is named rather than called: the Go expression that invokes it. // @@ -30,8 +26,10 @@ var builtinCalls = map[string]string{ "hiss": "meow.Hiss", "scram": "meow.Scram", // seed marks a fuzz corpus entry and is read off the AST before a body is - // generated, so outside a fuzz run it is the no-op genCall also emits. - "seed": seedCall, + // generated, so outside a fuzz run it is the catnap genCall also answers + // with — a runtime function of its own now, so that naming seed as a value + // needs no special case here. + "seed": "meow.Seed", "gag": "meow.Gag", "is_furball": "meow.IsFurball", "len": "meow.Len", @@ -98,8 +96,6 @@ func (g *Generator) genBuiltinValue(name string) (string, bool) { var body string switch { - case call == seedCall: - body = "meow.NewNil()" case arity == builtins.Variadic: body = fmt.Sprintf("%s(__a...)", call) default: diff --git a/pkg/codegen/codegen.go b/pkg/codegen/codegen.go index bb47b90..9dcfa4a 100644 --- a/pkg/codegen/codegen.go +++ b/pkg/codegen/codegen.go @@ -503,6 +503,18 @@ func (g *Generator) emitTest() string { return b.String() } +// ensureImport records that the generator itself needs one of Meow's own +// packages, because it is about to emit a selector into it — `meow_testing` for +// an assertion, and nothing else so far. +// +// That is why the package is marked used here rather than where the selector is +// written. A `nab` import is the opposite case: the program asks for the +// package and may never call into it, so usedImports keeps it out of the import +// block until markPackageUsed says a selector was emitted. Registering without +// marking is what left `judge` compiling to `meow_testing.Judge(...)` in a file +// that imported nothing — raw Go, `undefined: meow_testing`, under `meow run` +// while `meow test` worked (#151). Only the test wrapper named the package as +// used, and it did so by hand. func (g *Generator) ensureImport(name string) { if g.imports == nil { g.imports = make(map[string]string) @@ -512,6 +524,7 @@ func (g *Generator) ensureImport(name string) { g.imports[name] = path } } + g.markPackageUsed(name) } func (g *Generator) needsMeowImport() bool { @@ -1955,8 +1968,12 @@ func (g *Generator) genCall(e *ast.CallExpr) string { case "refuse": g.ensureImport("testing") return fmt.Sprintf("meow_testing.Refuse(%s)", argStr) + // seed's arguments are arguments: meow.Seed answers with catnap and + // evaluating them is what calling it does. Answering with the catnap + // alone spliced them into nothing, so a side effect written in one + // never happened here while the interpreter ran it (#151). case "seed": - return "meow.NewNil()" + return fmt.Sprintf("meow.Seed(%s)", argStr) default: if ks, ok := g.kittyDefs[ident.Name]; ok { fieldNames := make([]string, len(ks.Fields)) diff --git a/pkg/interpreter/interpreter.go b/pkg/interpreter/interpreter.go index affc7bd..6304b13 100644 --- a/pkg/interpreter/interpreter.go +++ b/pkg/interpreter/interpreter.go @@ -693,12 +693,13 @@ var builtinTable = map[string]builtinFn{ // seed states one entry of a fuzz corpus. Only `meow test -fuzz` has // anything to do with one — it reads the calls out of the test's body - // before the body is generated — and everywhere else codegen compiles a - // seed call to catnap. The playground cannot fuzz, so everywhere else is - // all there is here. - "seed": countsItself("seed", func(_ *Interpreter, _ []meowrt.Value) meowrt.Value { - return meowrt.NewNil() - }), + // before the body is generated — and everywhere else it answers with + // catnap. It is the runtime's own function rather than a catnap written + // here, so that an argument which failed is passed on the way codegen + // passes it on: this answered catnap whatever it was handed, and + // `seed(judge(false, "x")) ~> "caught"` printed catnap here and "caught" + // from a compiled program (#151). + "seed": variadic("seed", meowrt.Seed), } func (interp *Interpreter) dispatchBuiltin(name string, args []meowrt.Value) (meowrt.Value, bool) { diff --git a/pkg/interpreter/interpreter_test.go b/pkg/interpreter/interpreter_test.go index 0b29236..b4ca13a 100644 --- a/pkg/interpreter/interpreter_test.go +++ b/pkg/interpreter/interpreter_test.go @@ -1867,20 +1867,35 @@ nya(g())`) } } -// The program that pins this rule for the compiled path prints the same thing -// here, byte for byte. #154 was the two backends agreeing with each other and -// disagreeing with the spec; the fix is worth nothing if they stop agreeing, -// and the golden file is what the compiled path is held to. -func TestTheShadowingGoldenPrintsTheSameUnderTheInterpreter(t *testing.T) { - src, err := os.ReadFile(filepath.Join("..", "..", "testdata", "binding_shadows_builtin.nyan")) - if err != nil { - t.Fatal(err) - } - want, err := os.ReadFile(filepath.Join("..", "..", "testdata", "binding_shadows_builtin.golden")) - if err != nil { - t.Fatal(err) - } - if got := runMeow(t, string(src)); got != string(want) { - t.Errorf("the interpreter prints\n%s\nthe compiled program prints\n%s", got, want) +// The golden programs written to pin a rule for the compiled path print the +// same thing here, byte for byte. Each of these rules was a place where the two +// backends disagreed; the fixes are worth nothing if they drift again, and the +// golden file is what the compiled path is held to. +// +// Only the goldens whose programs the interpreter can run belong here: most of +// the corpus reaches for `nab`, a file or a clock, and the playground has none +// of those. +func TestTheseGoldensPrintTheSameUnderTheInterpreter(t *testing.T) { + for _, name := range []string{ + // A binding takes a builtin's or a constructor's name wherever it is + // written (#154). + "binding_shadows_builtin", + // judge, expect and refuse work outside `meow test`, and seed's + // arguments are evaluated (#151). + "assertions_anywhere", + } { + t.Run(name, func(t *testing.T) { + src, err := os.ReadFile(filepath.Join("..", "..", "testdata", name+".nyan")) + if err != nil { + t.Fatal(err) + } + want, err := os.ReadFile(filepath.Join("..", "..", "testdata", name+".golden")) + if err != nil { + t.Fatal(err) + } + if got := runMeow(t, string(src)); got != string(want) { + t.Errorf("the interpreter prints\n%s\nthe compiled program prints\n%s", got, want) + } + }) } } diff --git a/runtime/meowrt/builtins.go b/runtime/meowrt/builtins.go index 7a522d6..c8101af 100644 --- a/runtime/meowrt/builtins.go +++ b/runtime/meowrt/builtins.go @@ -6,6 +6,27 @@ import ( "strings" ) +// Seed states one entry of a fuzz corpus, and answers with catnap. +// +// Only `meow test -fuzz` has anything to do with a seed call: it reads the +// calls out of a fuzz target's body before that body is generated. Everywhere +// else the call is this, which exists so that the arguments are arguments — +// evaluated where they are written, the way Meow evaluates every other call's. +// Codegen used to answer a seed call with the catnap alone and splice the +// argument expressions into nothing, so a side effect written in one never +// happened, while the playground's interpreter ran it (#151). +// +// An unhandled Furball among them is passed on rather than swallowed, as Nya +// does, so a seed whose argument failed says so. +func Seed(args ...Value) Value { + for _, a := range args { + if f, ok := a.(*Furball); ok && !f.Handled { + return f + } + } + return NewNil() +} + // Nya prints a value (the Meow print function). If any argument is an // unhandled Furball, Nya returns it without printing so the error // propagates instead of being stringified and silently swallowed. diff --git a/testdata/assertions_anywhere.golden b/testdata/assertions_anywhere.golden new file mode 100644 index 0000000..d48f562 --- /dev/null +++ b/testdata/assertions_anywhere.golden @@ -0,0 +1,12 @@ +catnap +catnap +catnap +caught the judge +caught the expect +caught the refuse +catnap +catnap +caught from a function +evaluated +catnap +caught the seed argument diff --git a/testdata/assertions_anywhere.nyan b/testdata/assertions_anywhere.nyan new file mode 100644 index 0000000..c823f23 --- /dev/null +++ b/testdata/assertions_anywhere.nyan @@ -0,0 +1,45 @@ +# judge, expect and refuse are ordinary builtins that answer with a Furball, +# and they work wherever a program is written — not only under `meow test`. +# +# They used to compile to `meow_testing.Judge(...)` in a file that never +# imported the package, so `meow run` answered with raw Go: `undefined: +# meow_testing`, naming no line of the .nyan file. The checker accepted them +# everywhere and the interpreter ran them everywhere, so the CLI was the odd one +# out (#151). What this file prints is what the interpreter prints for it, byte +# for byte. + +# An assertion that holds answers with catnap, as it does under `meow test`, +# where only the enclosing test's PASS line is printed. +nya(judge(1 == 1, "one is one")) +nya(expect(2 + 2, 4, "two and two")) +nya(refuse(1 == 2, "one is not two")) + +# One that fails is a Furball, so `~>` can catch it and the message is the one +# the assertion was given. +nya(judge(1 == 2, "one is not two") ~> "caught the judge") +nya(expect(2 + 2, 5, "two and two are not five") ~> "caught the expect") +nya(refuse(1 == 1, "one is one after all") ~> "caught the refuse") + +# Held as a value, an assertion is still the assertion — the wrapper #152 gave +# every builtin needs the same import. +nyan held = judge +nya(held(1 == 1, "still one")) + +# Inside a function, where the Furball propagates as any other does. +meow check(n int) furball { + bring judge(n > 0, "n must be positive") +} +nya(check(1)) +nya(check(-1) ~> "caught from a function") + +# seed states one entry of a fuzz corpus. Only `meow test -fuzz` has anything to +# do with one, and everywhere else it answers with catnap — but its arguments +# are arguments, and Meow evaluates those. Codegen used to splice them into +# nothing at all, so a side effect written in one never happened while the +# interpreter ran it. +nya(seed(nya("evaluated"))) + +# An argument that fails is passed on, the way nya passes one on, so a seed +# whose argument failed says so rather than answering catnap as if nothing had +# happened. Both backends call the same runtime function for this. +nya(seed(judge(1 == 2, "a seed argument failed")) ~> "caught the seed argument") diff --git a/website/content/doc/stdlib.md b/website/content/doc/stdlib.md index 11345cb..9daddde 100644 --- a/website/content/doc/stdlib.md +++ b/website/content/doc/stdlib.md @@ -1073,14 +1073,24 @@ Import with `nab "testing"`. Provides test assertions and test execution. - Functions named `catwalk_*` are output verification tests — they capture stdout and compare it to an expected string. - Test functions must take no parameters. +`judge`, `expect` and `refuse` are builtins: they need no `nab` and work +anywhere a program is written, not only under `meow test`. An assertion that +holds answers with `catnap`. One that fails answers with a `Furball` carrying +its message, so it propagates like any other failure and `~>` can catch it; +under `meow test` that is what fails the enclosing test. + +```meow +nya(judge(1 == 1, "one is one")) # => catnap +nya(judge(1 == 2, "nope") ~> "caught") # => caught +``` + ### `testing.judge(condition [, message])` Assert that a condition is truthy. - **condition**: Value to check for truthiness. - **message** (string, optional): Custom failure message. -- **Returns**: `catnap`. -- **Panics (test failure)**: If condition is falsy. +- **Returns**: `catnap` if the condition is truthy, otherwise a `Furball` carrying the message. ```meow judge(1 + 1 == 2) @@ -1094,8 +1104,7 @@ Assert that two values are equal (compared by string representation). - **actual**: The value to check. - **expected**: The expected value. - **message** (string, optional): Custom failure message. -- **Returns**: `catnap`. -- **Panics (test failure)**: If values are not equal. +- **Returns**: `catnap` if the values are equal, otherwise a `Furball` carrying the message. ```meow expect(1 + 1, 2, "basic addition") @@ -1108,8 +1117,7 @@ Assert that a condition is falsy. - **condition**: Value to check for falsiness. - **message** (string, optional): Custom failure message. -- **Returns**: `catnap`. -- **Panics (test failure)**: If condition is truthy. +- **Returns**: `catnap` if the condition is falsy, otherwise a `Furball` carrying the message. ```meow refuse(1 == 2)