Anyway, so this enables a lot of patterns -- in fact, it enables far too many patterns, such that a frontend developer has to make a lot of choices in how they map their high-level modules to Nickel expressions. But no matter what choices you make, Nickel is capable of supporting pretty much any module or package management system
So, in your case
It seems like your external packages are in some sense parameterized by their dependencies, in a way that lets their clients control and vet exactly what they have access to. In Nickel, you can literally model that by treating the external package as a function of its dependencies, and then passing implementations of those dependencies as arguments.
And in fact, I've been intending since the beginning for that to be the dominant means of representing dependencies in Nickel
So, for example, the absolute top level of a Nickel program is a function that takes in a giant tuple of everything the platform supports -- hundreds of fields worth of arithmetic operations, memory management primitives, core data structures, IO functions (all without breaking purity, of course! :) ), etc
And yes, our designs do seem well-aligned
And then the top level is responsible for passing those core primitives, or whatever subset of them it's willing to provide, down to its dependencies
Now, you can do cool things with this
The most obvious of which is audit the exact functionality that your dependencies use. So if you don't want to give them filesystem access, just don't pass the filesystem submodule of the standard library
But you can also, for example, mock out the stdlibs with a test harness
Or pass a version of the stdlibs where everything has been modified to detect, intercept, and forbid certain disallowed operations
Such as accessing a file not belonging to a particular filepath, or something
haha you beat me to it
or for example, only making http requests to
google.com/search/...
Honestly I thought the ability to do this was just kind of a weird side effect of the way I was representing modules -- it's really cool to see that it actually has a use!
Especially if we can address the worst UX aspects of having to inject everything
Anyway, in addition to all that, there's also all the safety that comes from just being a pure language. You can restrict side effects because they're represented in the type system and mediated through args and return values -- which doesn't really operate at the module / import level, but can still be very useful for this same sort of thing
Yeah! The most interesting challenges with this aren't theoretical or technical, I think, they're figuring out the UX so that it's simple and approachable
Which, I must warn you, Nickel certainly is not, nor is it intended to be, at least at this stage. I want to build simple and accessible languages, but first I'm building Nickel as an enabling technology for those larger plans
But anyway
So that's the dependency injection story
Now, as for performance
Nickel uses linear types -- the same trick as Rust and many academic languages before it -- to allow purity and memory safety to coexist with mutation and manual memory management. I don't know if you've used a language with linear types (technically, Rust has affine types), but the idea is beautifully simple: by preventing a value from being in more than one place at once, you can mutate it, or even deallocate it, without anyone noticing, and therefore can't break either functional purity or memory safety
Now, at the moment I have a bunch of sketches lying around for how different memory usage patterns can be represented safely in Nickel. It's not nearly as convenient or concise as Rust's system, but it's based on simpler theoretical primitives and it is in some ways more powerful. And it's all done at the library level, so you can extend it or replace it if you want to, as long as you're willing to write a small amount of unsafe native code (in C, raw WASM, or whatever) to power the whole thing
So far in Forest I've been taking the "write a small amount of unsafe native code" approach
currently have the world's simplest
malloc implementation, and no free
written by hand in Wasm
I won't go into all the implementation details right now, but some of the things I've figured out how to express in the type system (but not implement in Nickel -- they'd have to be backed by external unsafe code) are:
- Uniquely owned allocation, like
mallocornew-- thinkBoxin Rust - Reference counted heap allocation -- think
Rc - Dynamic stack allocation, like
allocain C - Arenas
- Contiguous array slicing and viewing
- Shared immutable references to a reclaimable borrowed object
- Heap-allocated LIFO allocators -- these are weird, and I'm especially excited about them :)
A heap-allocated LIFO allocator is basically a way to have your very own pet call stack, small and domesticated and entirely separate from the huge and inscrutable native call stack
And that's really interesting for implementing things like coroutines
Or futures / promises
Basically anything asynchronous (although not necessarily multi-threaded, although concurrency should be easy and safe for the same reasons it is in Rust)
And sort of a related point to all of this is that, as I said, you have a lot of choice in how you implement closures. Which means you don't need an expensive heap allocation every time you call a higher-order function. If you have a smart optimizer, or a language frontend like Rust's that makes these kind of issues explicit, you can very often stack-allocate all the data that a closure carries along with it, in a way that Nickel is fully capable of expressing and proving safe
And again, thanks for your interest and enthusiasm!