Nushell Engine-q Design Overview

- lily-mara added [`into filesize`](https://github.com/nushell/nushell/pull/3987) - tw4452852 added [support to append when calling `save`](https://github.com/nushell/nushell/pull/3992) - Pantoshire improved [errors when bash-style alias was mistakingly used](https://github.com/nushell/nushell/pull/3995) - tranzystorek-io did a lot of [general code improvements](https://github.com/nushell/nushell/pull/3996) - kubouch did some fixes to p[aths and the `source` command](https://github.com/nushell/nushell/pull/3998) - elferherrera updated the [prompt environment variable to PROMPT_COMMAND](https://github.com/nushell/nushell/pull/4003) to show that it is nushell code that gets run ## Engine-q We've been hard at work on the upcoming set of engine updates for Nushell (codenamed 'engine-q'). These updates address some fundamental flaws in the current Nushell engine design and should help set us up for a strong engine in the future. In this section we'll talk a bit about the design of this engine and show some fun tricks it's able to do. ### Background The current Nushell code uses a single concept for the scope that's shared between the parser and the evaluation engine. This is how definitions could be added by the parser into the scope that the engine could find them. This scope used locks to maintain thread safety. This meant that each variable lookup had the additional cost of unlocking the lock around the scope. The way this was set up also had subtle bugs with how the scope was handled, including corner cases where variables would be visible in scopes where they shouldn't be. In addition, there wasn't an easy way to do a speculative parse of content like you might with syntax highlighting or completions. ### New design In the new design, both the parser and the engine have received a pretty thorough rework. The parser now uses an [interning system](https://en.wikipedia.org/wiki/String_interning) for its definitions, allowing the resulting parse tree to be simpler. It also has a change delta system where the parser can create a temporary working set to use, and this working set can optionally merge into the permanent state. We've also building in a lot of additional fun additions -- like typechecking! -- so you can get additional benefits from information the parser knows. Likewise, the engine has been rewritten to use proper scoping, a simpler value system, and more. As you can see, we're taking full advantage of the opportunity to fix long-standing issues we wished we could fix over the last couple of years working on Nu. Oh, there is one more thing.

The Nushell team is working on 'engine-q', a major update to address core engine flaws and strengthen the future. The current engine's design has issues like thread-safety locks causing overhead and scoping bugs. The new design features an interning system, change delta system, typechecking for the parser, proper scoping, and a simpler value system for the engine, aiming to fix long-standing issues.