Add support for managing package environments with ghc-pkg
Needs RevisionPublic

Authored by fugyk on Aug 1 2015, 1:58 PM.

Details

Summary

This patch adds support for creation, deletion, listing of packages and modification of packages of package environments(https://phabricator.haskell.org/D558) to ghc-pkg. It is written such that it is compatible with earlier version of both cabal and package DB and also it does not changes any old commands of package management for users. It is needed for cabal changes (https://github.com/haskell/cabal/pull/2697)

Details about this patch are here: https://ghc.haskell.org/trac/ghc/wiki/Commentary/GSoC_Cabal_nix.

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped
fugyk retitled this revision from to View support in ghc-pkg.Aug 1 2015, 1:58 PM
fugyk updated this object.
fugyk edited the test plan for this revision. (Show Details)
fugyk added reviewers: ezyang, duncan.
ezyang requested changes to this revision.Aug 1 2015, 7:45 PM

You will at least need documentation for the new ghc-pkg flags in the manual.

This revision now requires changes to proceed.Aug 1 2015, 7:45 PM
ezyang added a comment.Aug 1 2015, 8:05 PM

I'm sorry, this is kind of late for a general comment like this, but why does ghc-pkg need to know about views? ghc-pkg's job is to intermediate between Cabal and bin-package-db, but views do not have a binary format for GHC. So could this just entirely live in Cabal?

utils/ghc-pkg/Main.hs
301

Good thing to mention here in the manual: how do view-names turn into view-files?

Also, does view delete view-file DELETE the file, or just delete the symlink?

309

Does that mean if I add a package which already is in the view, does that delete the old package? Does ghc-pkg enforce consistency of the view?

768

It would probably be better if this could report exactly WHAT went wrong. Also, it's a bit funny that error is reported by setting packagesInView to Nothing

fugyk added a comment.Aug 5 2015, 9:50 PM

Sorry for being so late.
Theoretically cabal can also manage it, but I think ghc-pkg is better as firstly it is changing ~/.ghc which cabal does not directly changes till now. Also it will allow a easy creation and modification of package environments which itself is a nice feature. Cabal cannot allow it without porting many things like packageArg. We can directly build a sandbox using it without cabal. Also we can add option to not unregister the packages which are in some view without force which can give free garbage collection and will allow better consistency of database.

utils/ghc-pkg/Main.hs
309

No, it will tell that package cannot be added.

768

Changes done by ghc-pkg cannot leave it in bad state. So I didn't thought about it much.

fugyk updated this revision to Diff 3780.Aug 5 2015, 9:51 PM

OK, fine that sounds like a good reason to me. Hopefully the other stakeholders can help bikeshed the exact command structure.

utils/ghc-pkg/Main.hs
309

OK SGTM

768

In this case, ghc-pkg should BLEAT LOUDLY if there's an inconsistency (like a GHC panic; something went wrong, indicates a bug in the code).

What if someone manually edits the view file? This (I assume) is a supported operation.

fugyk updated this revision to Diff 3782.Aug 6 2015, 12:23 AM
thomie added a subscriber: thomie.Aug 9 2015, 7:18 AM

From just reading the current User's guide entry, I still have no idea what Views are for, or why I should care. There are lots of typos also. Please add a section explaining the problem that Views solve, and how it solves it.

I guess it's all explained here, but that information should go in the User's guide then:
https://ghc.haskell.org/trac/ghc/wiki/Commentary/GSoC_Cabal_nix

fugyk retitled this revision from View support in ghc-pkg to Add support for managing package environments with ghc-pkg.Aug 22 2015, 8:13 AM
fugyk updated this object.
fugyk updated this revision to Diff 3905.Aug 22 2015, 8:15 AM

I think one of the major part of confusion is the using of new term. I was using that term before I found ghc already had a feature I was trying to implement. So now I have replaced 'View' with already used term 'package environment' globally.

A package environment is a collection of IPID that are exposed to ghc. Now there is no good way to manage it. We have to find IPID from a package manually and add it to a file. This patch makes allows managing package environment from ghc-pkg with PackageArg. Suppose if I want to use two packages foo and bar-0.2 for development environment, I can do

ghc-pkg pkgenv create myenv --user
ghc-pkg pkgenv-modify myenv add-package foo
ghc-pkg pkgenv-modify myenv add-package bar-0.2
export GHC_ENVIRONMENT=myenv

There are some more features like ghc-pkg can warn if I unregister the package in some package environment, and also it can create a symlink from a file(used for nix-like symlinking).

The second benefit is that cabal no longer has to use separate DB for sandboxes. It can just register the sandboxed package in the user DB and does not have to worry about being used in global ghc. It will also be used for garbage collection, so that every package which cannot be reached from any package environments can be garbage collected.

Also updated the docs.

fugyk added inline comments.Aug 22 2015, 9:14 AM
utils/ghc-pkg/Main.hs
768

OK, I will do better error reporting and some other things once the basic premise of this patch seems good to be merged.

austin requested changes to this revision.Aug 24 2015, 8:20 PM
austin added a reviewer: austin.
austin added a subscriber: austin.

This looks nice! Unfortunately...

Macro phoenixwright: objection Bikeshedding

But I think it's mostly simple. Here's what I propose, if we read this description of the commands as a kind of grammar:

ghc-pkg pkgenv { create | delete | list-packages } <env>
ghc-pkg pkgenv list-pkgenvs
ghc-pkg pkgenv-modify <env> { add-package | remove-package} <pkg>

Notice that:

  • pkgenv by itself is a bit redundant with the name being ghc-pkg.
  • Duplication: list-packages vs list-pkgenvs
  • Some inconsistency: pkgenv-modify is a special case simply for adding or removing packages, which isn't very interesting.
  • Wording: Compound phrases vs singular actions (list-pkgenvs vs create) describe one command.
  • Command placement: in pkgenv-modify the <env> comes first before other subcommands!
  • Lots of typing needed for the basic stuff! e.g. ghc-pkg pkgenv-modify foobar add-package text-1.1 could just be ghc-pkg env add foobar text-1.1

Here's an interface I just scribbled up, which I think is OK at a glance, shorter and a bit more consistent:

ghc-pkg env { create | delete } <env>
ghc-pkg env list { environments | packages <env>}
ghc-pkg env add <env> <pkg>
ghc-pkg env delete <env> <pkg>

In this one:

  • Subcommands always come before any arguments.
  • All subcommands are singular words and shorter in general.
  • Collapsed pkgenv-modify into the toplevel env command.

I think this is a lot nicer, although rewriting all the docs and whatnot may be annoying. Sorry :( But I do think the above is better, so in lieu of anyone else saying otherwise, I'm going to request changes on this...

This revision now requires changes to proceed.Aug 24 2015, 8:20 PM
ezyang requested changes to this revision.Aug 24 2015, 11:51 PM

I am in support of austin's proposed API changes.