nfnl client errors when evaluating a macro that contains a function

[jship]

When evaluating some Fennel code that uses a macro and that macro has a local function defined, Conjure errors with:

; --------------------------------------------------------------------------------
; eval (buf): <path-omitted>/src/app.fnl
; [Compile] .../.config/nvim/bundle/conjure/lua/conjure/nfnl/fennel.lua:1069: attempt to index field 'options' (a nil value)

That erroring line in fennel.lua has me suspect Conjure's initialization of fennel is mixed up if fennel's options haven't been defined yet, maybe through Conjure's (or nfnl's) use of autoload?

Here is a small repro - note this also depends on nfnl.module:

❯ tree -a
.
├── .nfnl.fnl
└── src
    ├── app.fnl
    ├── app.lua
    └── macros.fnlm

2 directories, 4 files

❯ cat .nfnl.fnl
{:source-file-patterns ["src/*.fnl"]}

❯ cat src/macros.fnlm
(fn foo []
  ;; Evaluating src/app.fnl in Conjure will succeed if this next line is removed
  (fn dummy [] nil)
  true)

{: foo}

❯ cat src/app.fnl
(import-macros m :src.macros)

(local {: define} (require :nfnl.module))

(local M (define :src.app))

(fn M.test []
  (m.foo))

M

If I comment out the dummy fn above, restart nvim, and try evaluating src/app.fnl, it works fine. At that point, I can then uncomment dummy, evaluate src/app.fnl and evaluation continues to succeed, but the updates to the macro do not seem to propagate. For example, if I change src/macros.fnlm to the following:

(fn foo []
  (fn dummy [] nil)
  (dummy))

{: foo}

Then re-evaluate src/app.fnl, evaluating (M.test) still returns true from the original version of src/macros.fnlm, rather than nil.

As this repro also uses nfnl, it's worth pointing out that nfnl compilation never errors, and the generated src/app.lua is always up-to-date with the actual macro contents whenever src/macros.fnlm is saved.

[jship]

I checked the above over using the stdio Fennel client and it worked great. On this bit specifically:

...but the updates to the macro do not seem to propagate.

Macro changes propagate if the macro module is reloaded via <localleader>eF in the stdio client. Since the nfnl client doesn't have an equivalent of the stdio client's reload command as far as I'm aware, I take it the lack of propagation is expected in the nfnl client. Wanted to make that distinction to avoid confusion. The core of the above issue is still as described, in that the nfnl client errors during the evaluation of code using a macro, when that macro has a local function.

[enderbeatt]

@Olical hi, i have found that adding :compilerEnv _G when calling fennel.repl fixes the problem, aniseed does it by default.

[Olical]

Interesting! So this is actually a bypass that I think is not recommended by the compiler, it turns off guards that help protect you against other subtle macro related bugs from what I remember which is why I didn't carry that mistake forward into nfnl.

I actually had a dig into this a week or so ago and pretty much came to the conclusion that this could be a bug in Fennel that may well be fixed in the repo but isn't released yet, so I was holding out for the next Fennel compiler version release. Although I can't remember the exact details.

I think for now if you're okay with disabling those guards go for it, but I plan on checking this again when Fennel releases a new version to see if there's anything else to be done.

Here's what the docs say on it:

Sandboxing

Inside macros or eval-compiler, by default there are only two ways that code can interact with "the outside world"; you can call print in order to debug, and you can call io.open in read mode on files inside the current directory or its subdirectories. The rest of the io table and the entire os table is not accessible.

You can loosen these restrictions by passing {:compiler-env _G} in the options table when using the compiler API or setting --no-compiler-sandbox on the command line to get full access.

Please note that the sandbox is not suitable to be used as a robust security mechanism. It has not been audited and should not be relied upon to protect you from running untrusted code.

Note that other internals of the compiler exposed in compiler scope but not listed above are subject to change.

[Olical]

;; .nfnl.fnl
{:compiler-options {:error-pinpoint false
                    :compiler-env _G}}

Just as an example config for this.