Start a design document

[kubouch]

A WIP of a design document. Still has some parts to fill in

should close #1 by moving all the design into DESIGN.md 👍

👉 see the document here

[skelly37]

To summarize our meeting with @amtoine @kubouch @Yethal (I might have missed something, as I've written it down after meeting). Please add whatever I have forgotten to do so 😅

• We start out with implementing simple install & uninstall on Linux
• Everyone takes a look at their package managers and see what we would like to actually see here
• We default to the interactive way [Y/n] prompt, but allow some kind of --yes flag
• We need to add our desired API to the docs (something can be extracted from here): https://github.com/nupac/nupac/blob/main/docs/DESIGN.md
• install works by default only with non-installed packages. Reinstall, upgrade or specific revision needs explicit flag for that
• We use the virtual environment idea with the possibility to have global packages to handle both a programming language and a convenient shell
• We stick to some metadata.nuon, keys to consider: https://github.com/nupac/nupac/blob/main/docs/PACKAGE_METADATA.md. Also Nushell version should be addressed
• Dependencies should include versions
• We can use something like: ~/.nupm/packages/<package_name>/<version>/ to handle multiple versions of packages
• We should be able to support multiple package repositories
• Some kind of way to initialize a new package kinda template, like nupm new <package_name>
• Some other ideas can be extracted from Nupac's issues: https://github.com/nupac/nupac/issues

I'm off today, but I will surely try to add something to the design document on Monday

[amtoine]

thanks @skelly37, that looks very complete with what i remember from the meeting 👌

[skelly37]

Also, regarding configuration:

How about using profiles? We won't end up with a ridiculous amount of envs, the users won't be forced to edit json/nuon/whatever by themselves. And one could have multiple configs on their machine, they could be copy-pasted etc...

e.g. nupm profile set --default interactive_mode=false

For example conan uses that: https://docs.conan.io/1/reference/profiles.html

[amtoine]

i've made some tiny tweaks to the document => the API still is the biggest missing part 👌

[Yethal]

Regarding installation:
I don't think defaulting to git clone as main installation method is a good idea.

• It requires git which may or may not be installed on target system
• It requires packages to be stored in git repositories

IMO installing packages over HTTP is a better choice

• Requires no external dependencies, both download itself and file verification can be done with just nu commands
• Allow packages to be stored in places other than git repositories
• Can still be used to download files from git (the way we currently do it in nupac)

[amtoine]

❓ a question regarding this DESIGN.md document itself: do we add some bullet points to keep track of what has been done?

[Yethal]

I mentioned this during the call and then totally forgot about it. I'd like to include a set of helpers for with the module (maybe as a submodule that is not exported by default) to be used in build.nu scripts so that package maintainers do not have to reinvent the wheel and come up with 100 different ways to e.g download a file. Examples of such helpers that come to mind:

• Download a file and verify its checksum
• Assert required binary is available in PATH
• Extract an archive (by detecting and calling an OS-appropriate archive util)
• Attempt to install a dependency using a system package manager
• Clone a git repository in a temporary directory

[skelly37]

I thought I will post the metadata.md suggestion here, so it won't got lost on discord 😅

# `nupm`'s package metadata reference
## Required attributes
- `author`: Name of the developer/organization/etc.
- `files`: List of records of files being part of the package. Records reference:
  - `checksum`: SHA256 sum of the file
  - `name`: File path (relative to the package scope)
  - `raw-url`: `GET`table link to the file
  - `supported-os`: Exactly like in the top-level metadata
- `name`: Package name
- `nu-version`: Supported Nushell version. Just like `dependencies` might be either exact version or some matcher like "greater than 1.70"
- `short-description`: Short info about the package, displayed by default
- `supported-os`: Operating systems supported by the package, the most broad possibility: `{"arch": ["*"], "family": ["*"], "name": ["*"]}`. Matched by `$nu.os-info`
- `url`: Package website/GitHub repository. Basically a place where one can find some additional info about the package
- `version`: Version of the package. Semantic versioning is advised to enable users to have more generic requirements
## Optional attribtues
- `dependencies`: Packages needed by the package — versions have to be specified. e.g. `[nupm/0.7.0]`. Semantic versioning is also supported: `[nupm/~0.7]`
- `installer`: Name of a script (relative to the package scope) that will install the package instead (or in addition to) of default `nupm` logic
- `keywords`: List of keywords used by `nupm search` in addition to `name`
- `pre-install-hook`: Name of a script (relative to the package scope) to be executed before installation. Might be simple pre-install message or more complex logic
- `post-install-hook`: Name of a script (relative to the package scope) to be executed after installation. Might be simple post-install message or more complex logic

[skelly37]

Can't wait to actually start implementing it, great job everyone with this brainstorming 😋

[kubouch]

Some changes I did:

• renamed NUPM_HOME/lib to NUPM_HOME/modules because modules can be used in many different ways. Also, it's better to leave lib empty because if we add in the future the ability to install 3rd party non-Nushell tools, they could install their libraries there. Better keep Nushell modules separate.
• The overlays need a bit of thinking and experimentation, we can start without it. The main idea is that a project would create an overlay which you can install as a globally-accessible overlay (Python-style global virtual environment)

Two things I don't agree with:

• Commands should have only one description available via -h or help. Nushell's help system already contains short and long description which should be enough for our purposes.
• The required metadata fields should be absolutely minimal. Most of the currently required fields could be optional and only required for publishing.
  I'll address the above concerns in a separate commit

[kubouch]

Seems good, if everyone is OK with these, we can merge it.

[skelly37]

* Commands should have only one description available via `-h` or `help`. Nushell's help system already contains short and long description which should be enough for our purposes.

@kubouch I'm afraid you misunderstood the intent. It was about the description of packages, that will be displayed when doing some actions like search.

It's actually my bad that I missed it when preparing the metadata.md file.

But I won't argue if you wouldn't want that anyway. So, either make short-description a description or add long-description to the metadata reference 😅

[kubouch]

Ah, oops, I misread, I put the long description back

[amtoine]

📯 📯 📯 @kubouch @skelly37 @Yethal 📯 📯 📯

in order to start contributing, let's merge this 👌

Note
this document IS NOT frozen and can be tweaked as we go 😌