A few things to note:

• This will likely not work for cross compilers in general, since plugins are loaded from haskell libraries rather than arbitrary shared libraries. Interface files are incompatible between build and target machines (building the plugin itself might also be tricky)
• It's still impossible to enable Template Haskell in a stage1 compiler with this, since the code paths leading up to the runMeta hook (and those for installing the plugin!) depend on GHCI being defined.

Hooks are rather low-level and can easily break the compiler, this extension to allow plugins to modify DynFlags early in the pipeline is useful by itself, but for Template Haskell, I think a different strategy should be used eventually.

I'd rename the installHooks field to something else by the way, since this is conceivably useful for anything that modifies DynFlags before the compiler pipeline is started, perhaps dependency injection, or adjusting flags for some specific purpose.

I would like to propose this change for 7.10 and a subsequent revision of the design for 7.12.

What do you mean by revision for 7.12? Is the API likel going to change in 7.12?

PS: Is there an associated Trac ticket and/or Wiki page? If not, this definitely deserves one. (We use Phabricator only as an extension to Trac for code-review, not as a replacement for Trac's ticket management and wiki)

Take this with a grain of salt since this is my first time looking at the Plugin facility, hooks and the new type checker plugin (TcPlugin).

Here are a few notes:

• The idea of threading plugin state done in the type checker TcPlugin is quite useful, why not use that for the whole Plugin?
• The Plugin API is quite useful and a natural API to extend GHC. What about making the Plugin interface the standard way of customizing GHC in both cases: GHC API and -fplugin=...? This way there is a single facility to extend GHC and extensions are all concentrated in a single API.
• There are a few more interesting hooks described here https://www.haskell.org/pipermail/ghc-devs/2013-August/002182.html that may be worth considering.
• I would like to use the Plugin facility to integrate Lambdachine's byte code generator instead of hardwiring it into a fork of GHC. This means more hooks.

Luite, maybe the Plugin facility should also allow loading from a shared library?

In any case, it seems that there are too many open points for discussion here and it feels a bit rushed to get this into 7.10.

luite is completely right here. The primary motivation for adding the DynFlags -> DynFlags plugin was the motivation to be able to install the hooks from plugins, so that I could sidestep recompiling ghc all the time during development. Hence the name. I also think changing the name to a better or more appropriate one would be good!

I am fully aware of the issues the out-of-proc-th approach has. But I don't think that this patch has anything to do directly with oopth. It just ended up being a useful byproduct.

On the oopth side I plan to continue with adding an OOPTH CPP define to enable the necessary code paths for oopth in stage1. This will be for non cross compiler for now. Integrating the plugin directly in ghc or even better using the pipes approach outlined by luite on GHC to reduce the complexity and dependency of ghc on the oopth driver will then have to be decided. But this is as mentioned above orthogonal to the patch discussed here.

This Patxh uses GHCI CPP due to the fact that the used function are in part only available with GHCI.

Can we get rid of the GHCI requirement for this type of plugins, even if it's just for when GHC has been linked with dynamic libs? If we can, it'd require just a few more changes to make runMeta reachable again in stage1 compilers, making this plugin thing usable in the short term for oopth for cross compilers.

In D535#14686, @hvr wrote:

I would like to propose this change for 7.10 and a subsequent revision of the design for 7.12.

What do you mean by revision for 7.12? Is the API likel going to change in 7.12?

PS: Is there an associated Trac ticket and/or Wiki page? If not, this definitely deserves one. (We use Phabricator only as an extension to Trac for code-review, not as a replacement for Trac's ticket management and wiki)

I don't think there is a trac or wiki page yet (for proposing an overhaul of the plugin system) as far as I know. @rodlogic, could you open the wiki and trac page?

I renamed the new Plugin field, but there is still a build failure that I still need to investigate. It seems related to referencing a plugin that needs to be compiled from source.

fyi, this change is a bit stuck since it breaks one of the Plugin test cases. From a limited inspection, it seems that GHC supports compiling a Plugin from source before using it in the same GHC invocation. This change set assume the Plugin is compiled already and resides in the package db. On top of that I won't have much time to work on this this week.

@rodlogic, do you think we could get this unstuck for 7.12?

I have been struggling to find the spare time for GHC the past 6 months or so, but I am hopeful that will change in the near future. Let me give it a shot in the next couple of months.

@bgamari The lack of time will not change in the near future so I won't be able to pursue this further at the moment. I think it is best we get rid of this altogether as I was not the main driver behind this change.

What is the ticket number for this? Click "Edit revision" to add it, thanks.

In D535#72117, @thomie wrote:

What is the ticket number for this? Click "Edit revision" to add it, thanks.

I don't think there is one yet. I'll certainly add it, once I'm back from vacation. However the scope of the
diff has changed considerably over time and I still expect it to evolve as needed.

The latest updates motivation has been primarily to have all the code publicly available to allow test-driving
https://github.com/angerman/data-bitcode-plugin.