Jonathan Blow's libraries discussion
This post is just a dump of the things that came to mind while watching Jonathan Blow’s libraries discussion. I’m not an expert on any of this but I find it so fascinating that my mind can stop thinking and taking notes when watching Jon streams.
## Library as single-file/many-files with resources, binaries…
This is something that I would really recommend tackling since the very beginning. I’ve suffered in the past with iOS and its inability to easily make libraries with resources (Apple calls them frameworks) for incompatibilities between static and dynamic linking. I still remember the hacks that I had to do some years ago.
Having a system that works well with any of this combinations out of the box would be ideal for any type of project.
My experience with Objective-C and Swift and their package managers is precisely what Jon is talking about in the video.
The ideal scenario is that you can define a library with a specific version, that’s pulled into your workspace and checked in with your sources. From that point on is just part of your source code so no surprises can happen.
That’s how I use package managers, but you can also decide to not check in the sources. If you do that you still have some safety thanks to a lock file that describes the exact versions of the libraries, that helps by making sure that in other machines the exact same versions are gonna be installed… unless the source in the package manager repository has changed. This is necessary because usually package managers allow you to define loosely a version, like “>2.1” which allows you to update “easily” minor versions. I rarely use this feature, I prefer to fix the specific version and manually check the changes that a new version has done, even for a small bugfix bump, before deciding to update.
In any case, keep in mind that having the lock file won’t avoid the “npm situation” where the original source disappears. Check in your dependencies ;)
The idea of having the compiler integrated with “package manager” functinality is amazing. Being able to define dependencies just with an import is really awesome. There are some tools in Swift that allow you to do this, it becomes necessary if you want to distribute single file scripts with dependencies.
That said I do think that people would expect your package manager, or compiler if it’s integrated, to pull the dependencies for them. You can still do it only once and check in the code, and only update if a specific command line argument is passed, but I think the functionality should be there.
## Main program overwriting dependencies
The idea of the main program being able to overwrite the dependencies of its libraries feels crazy. It would fix some specific problems for sure, it won’t be the first time I’ve seen how importing two libraries that depend on different versions of the same library make your program imposible to run.
Import multiple times
Mounting libraries with specific names is something interesting that none of the languages I work on a day to day basis do. I don’t really know how useful it is in practice but it seems really cool to be able to import the same library multiple times with different names if each import receives it’s own memory space.
Library hooks
I’m not a huge fan of the idea of libraries and the main program sharing the global scope, looks really dangerous as Jon points out.
If you just want to provide a procedure that the libraries can call the best solution is for the library to provide hooks in form of variables where the main program can assign a procedure. With languages that have “first class functions” this is simple and seems a better approach than sharing scopes.
Parameters
Libraries with parameters seem… interesting at least? I’ve never used anything like that so I don’t really know how useful it can be. But if I think about the functionality that this powerful language already has it seems to me that passing parameters to an import could allow amazing things, like modifying the behaviour of the library and what is exposed even at compile time.
Now that I think about it, this seems a clean way of doing what sometimes has to be done with preprocessor macros in C based languages.
Retroactive modelling
The problem about how to tell Print (an imported library) how to print a struct from another library or the main program is something that reminds me to how Swift can do retroactive modelling by adding protocol conformance to types that you don’t own via extensions.
This is a fundamental part of Swift that makes the language really powerful. Obviously it has its drawbacks (see Rust restrictions), specially with some of the goals and requirements that Jai has put on itself. But nonetheless, I recommend you to take a look at how Swift does it, it may give some inspiration. I think that this requires runtime in other languages, but with Jai compilation model and introspection capabilities maybe this could be done in a static way.
During the stream somebody mentioned traits and Jon is like… what trait even means? 😂 I know the feeling.
Naming is important.
— Alejandro Martinez (@alexito4) 15 de septiembre de 2016
Bad day that one when 'mixin' and 'trait' definition was not clearly given 😿
It’s so frustrating. Like everytime I hear Java people say “interface”.
For me the best definition of trait is that it defines requirements and provides some default behaviour. That is exactly what I was referring to with Swift protocols and extensions.
protocol (requirements) > trait (+ behavior) > mixin (+ state)
— Alejandro Martinez (@alexito4) 15 de septiembre de 2016
All of that gives you the interface
🤗
It’s interesting to see how similar problems end-up being solved in different ways.
Q & A
At 32:00 the presentations ends and Q&A starts, so here I leave some comments about that last part.
The package manager needs to allow users to provide an array of severs from where to retrieve the libraries so it’s easy for people to have private servers with their private company code. Using a public git server by default is nice, but at some point any dependency manager needs some way to work with private repositories, and I assume that in the serious games industry that’s even a stronger requirement.
It’s really a neat idea that the package manager could warn you about new updates to your libraries.
As I mentioned before, I don’t think that the best is manually getting the library, put it in the folder and then import it. I feel that downloading should be part of the feature set together with updating. I agree that it should be done only once and per demand, obviously doing it every time “a la npm” it’s really bad, but a good first time installation experience should be part of the language toolchain.
About libraries with examples and docs: I found out that one useful thing is to have a tool to quickly download a library and create a playground that imports it with some basic code to start playing with it. It’s a great way to start exploring a new library and have a first impression before deciding to put more time in it.
Conclusion
And that’s it!
I think that Jon has great ideas on how to tackle this crucial aspect of a language toolchain. This is sometimes ignored by language designers but it’s one of the things that has the biggest impact to end users.
I’m grateful that he’s doing these streams in the open so we all can learn not only from the final decision but from the process itself.
Thanks!