feat: implement no_std option

[Aire-One]

Following discussion from #947 (reply in thread), here is my attempt at implementing the ability to replace the stdlib.

This is working as I expect and allows me to run tl check --stdlib busted-env.d.tl some_spec.tl 🎉. Tests and documentation are missing. I open this PR as a draft to get first reviews and implementation validation (see in-code comments).

[github-actions]

Teal Playground URL: https://961--teal-playground-preview.netlify.app

[hishamhm]

Thanks for the initiative!

Given that --global-env-def exists and also loads a module into the global environment (see _init_env_modules), my first thought was that this mechanism could be used to load the alternative standard library, and then we'd just need a --no-std flag to skip the hardcoded stdlib.

But now I'm thinking about the block you have marked with TODO, where some special behaviors are defined for the standard library. I'm thinking that you might actually want those behaviors, so you get, for example, improved argument checks for string.format, compat support, etc. I think a good solution would be to apply those extra properties to the stdlib entries, if the entries are present in the stdlib.

I also think a few definitions (global types any, thread, userdata) should probably move out of the default stdlib string and turn into hardcoded globals loaded via code (like they used to be previously), so that they're always present regardless of the stdlib definition, since those are primitive types of the language used by the compiler. On second thought, that might be the case for the metatable record type as well... I'm wondering now if we should split the current stdlib into a prelude which always gets loaded (with the core language types), and a replaceable stdlib which maps to the Lua standard library globals. That sounds like the best way to deal with any, thread, userdata and metatable, so that users aren't forced to redefine them precisely at the risk of breaking the compiler.

[Aire-One]

Thank you for the quick review @hishamhm!

Given that --global-env-def exists and also loads a module into the global environment (see _init_env_modules), my first thought was that this mechanism could be used to load the alternative standard library, and then we'd just need a --no-std flag to skip the hardcoded stdlib.

Yup, I also had the feeling that --stdlib and --global-env-def were somehow redundant.

Just to be sure to be aligned with your ideas : for my busted example to work, where we need to override the assert definition with luassert, we should use both flags tl check --global-env-def busted-env.d.tl --no-std some_spec.tl (and busted-env.d.tl still needs to embed as much as possible from the stdlib).

[hishamhm]

we should use both flags tl check --global-env-def busted-env.d.tl --no-std some_spec.tl (and busted-env.d.tl still needs to embed as much as possible from the stdlib).

Yes. Perhaps we could also introduce --stdlib <file> in tl as an alias to --no-std --global-env-def <file>? That could be done entirely at the tl script level. At first I was thinking of something short like --std, but gcc and others use that to mean "the standard of the language in use", e.g. --std=c99. It's a bit unfortunate that there is a Lua library also called "stdlib" but I'm not sure if that would bring confusion; --replace-stdlib sounds a bit too verbose.

In any case, it would be important to be also able to do stdlib replacement via tlconfig.lua.

[hishamhm]

@Aire-One I just pushed a change to master where I split prelude and standard_library. This should make your task here easier!

[Aire-One]

I just pushed a change to master where I split prelude and standard_library. This should make your task here easier!

Nice! Thank you for that @hishamhm. It's indeed helpful to me.

I'll dismiss the --stdlib alias for now. Let's see first how much it's actually needed with some time. (I suspect most users that need to define a custom environment already use a tlconfig.lua and don't bother with cli parameters)

[hishamhm]

I think we have a tricky situation here.

These stdlib_globals customizations are being tested before the predefined_modules array below. So, if someone is using --global-env-def to load an alternative stdlib, that won't be picked up because it's too late.

But then, if you tried to switch the order around and do these customizations after the predefined modules load, then any other predefined module which does expect a fully configured standard library won't have one ready yet. I think the conclusion is that we do need a configuration for a custom standard library that is separate from (and loaded earlier than) any other predefined modules.

[Aire-One]

Sorry for the delay, I was on vacation and expected to find some time earlier.

These stdlib_globals customizations are being tested before the predefined_modules array below. So, if someone is using --global-env-def to load an alternative stdlib, that won't be picked up because it's too late.

You mean "to load an alternative stdlib" as in "redefining something from the stdlib"? In this situation, the --global-env-def option alone would yield an global was previously declared as <const>: ... error. You are required to also use the --no-std flag.

I agree this behavior is not optimal, but since the --no-std flag is used, it sounds correct to me 🤷

Alternatively, when the no_std option is trusty, we could load the first predefined_modules instead (and skip it to when loading the other predefined modules latter). This way, it could be documented as "when --no-std is used, the first predefined-module option can be an alternative stdlib".

I think the conclusion is that we do need a configuration for a custom standard library that is separate from (and loaded earlier than) any other predefined modules.

Do you suggest introducing (back) an --stdlib <file> CLI option that replaces the stdlib string?