configure on Windows now downloads a mingw distribution on the fly if needed.
AbandonedPublic

Authored by thomie on Oct 13 2014, 3:33 PM.

Details

Summary

Also, migrated away from the rubenvb mingw distribution to a more up-to-date
and semi-official mingw-builds. Updated the version of gcc to 4.8.3.

This change obviates the need for a ghc-tarballs repository.

Also removed use of an embedded perl (also from ghc-tarballs) since it is provided by msys2.

Diff Detail

gintas updated this revision to Diff 868.Oct 13 2014, 3:33 PM
gintas retitled this revision from to configure on Windows now downloads a mingw distribution on the fly if needed..
gintas updated this object.
gintas edited the test plan for this revision. (Show Details)
gintas updated the Trac tickets for this revision.
hvr added a subscriber: hvr.EditedOct 13 2014, 3:39 PM

Is there any precedent for having ./configure download something from the Internet?

(I've seen ./configure scripts telling you download something manually, and place it in a specific location, but never actually performing network I/O w/o the user's consent)

In D339#7797, @hvr wrote:

Is there any precedent for having ./configure download something from the Internet?

(I've seen ./configure scripts telling you download something manually, and place it in a specific location, but never actually performing network I/O w/o the user's consent)

Yeah, it's a bit unorthodox. I wanted to keep the number of installation steps down to a minimum, but maybe this is a bit much. Telling the user to download something manually is ugly though, because it's hard to automate a full checkout & build that way.

I could extract the download logic to a separate script, then ./configure could point to that script in an error message. Still a bit ugly though, because if we want to pin down a version in the configure script, both configure and the other script need to know the filename at least...

hvr added a comment.Oct 13 2014, 3:53 PM

alternatively: tell user to either, a) download file from given URL manually *or* b) re-run ./configure with some --enable-autodownload flag

That way we'd have at least an explicit consent for doing something nasty like downloading several MiBs w/o prior warning

In D339#7799, @hvr wrote:

alternatively: tell user to either, a) download file from given URL manually *or* b) re-run ./configure with some --enable-autodownload flag

That way we'd have at least an explicit consent for doing something nasty like downloading several MiBs w/o prior warning

That's a great idea! I'll update the patch in the evening.

gintas updated this revision to Diff 871.Oct 14 2014, 5:16 PM
gintas edited edge metadata.
  • Added --enable-mingw-autodownload to avoid unexpected network load during ./configure
In D339#7799, @hvr wrote:

alternatively: tell user to either, a) download file from given URL manually *or* b) re-run ./configure with some --enable-autodownload flag

How does this look?

austin accepted this revision.Oct 15 2014, 6:17 PM
austin edited edge metadata.

Nice. Much better. And yes, we definitely should never default to doing network I/O any time in the build system.

Every time you do this, kittens cry and distro package maintainers (like myself) are sad (because we like to enable security features like no-network for building, and writing patches to change this is bad.)

This revision is now accepted and ready to land.Oct 15 2014, 6:17 PM
In D339#7933, @austin wrote:

Nice. Much better.

Thanks.

Without push rights, I can't run "arc land" on this myself, right? Can someone help out there?

hvr added a comment.Oct 20 2014, 7:58 AM
In D339#8388, @gintas wrote:

Without push rights, I can't run "arc land" on this myself, right? Can someone help out there?

Yeah, I'm sure @austin will push this one soon... is there anything that needs to be taken into account for existing GHC developers w/ checked out ghc-tarballs.git repos?

is there anything that needs to be taken into account for existing GHC developers w/ checked out ghc-tarballs.git repos?

They will have to rebuild to get the new gcc, but there should be no weird failure modes. I'm not 100% sure if the Makefile is correct about declaring the dependency on gcc everywhere so in theory some old .o files could be retained (so it might be a good idea to clean the workspace beforehand), but that's the usual SNAFU with Make.

There are some more cleanups related to dropping all references to ghc-tarballs, I'd like to deal with those after this has landed cleanly.

gintas updated this revision to Diff 935.Oct 21 2014, 3:36 PM
gintas edited edge metadata.
  • Fixed quoting in p7zip installation note.
  • Fixed extraction directory for mingw32.
  • Whitespace fixes
  • Ensure that inplace/mingw is not created as a file if mv fails.

Hold on, there are some more issues for the 32-bit build. This might take a while...

gintas updated this revision to Diff 972.Oct 24 2014, 2:20 PM
  • Define EH_UNWINDIN1G in seh_excn.h.
  • Avoid exporting certain symbols in Linker.c under mingw-w64-i686.
gintas updated this revision to Diff 973.Oct 24 2014, 2:53 PM
  • Removed references to ghc-tarballs.
  • Removed remaining references to Perl that were tripping up ./validate
gintas updated this revision to Diff 1007.Oct 26 2014, 2:48 PM
  • Updated RTS imports for i686-mingw-w64.
  • Updated expected output in ghcilink tests.

OK, this should be in better shape now. Phew! The diff may not look large, but this took *days* of work. Especially the part about exporting symbols took quite a while to figure out, and I'm still not sure if this is a reasonable solution or not.

Before submitting, please make sure that the following other patches have been checked in:

https://phabricator.haskell.org/D372
https://phabricator.haskell.org/D373
https://github.com/gintas/win32/commit/8b7e4de5c6509a02a4f854e24a1da55e6b8ef3c7

awson added a subscriber: awson.Oct 27 2014, 3:54 AM

I propose slightly different patchset for rts/Linker.c here: https://ghc.haskell.org/trac/ghc/ticket/9218#comment:12.

It solves name decoration problem in a slightly more uniform way, (almost)unifies RTS import list between 32 and 64 bit cases, adds COMDAT support necessary for gcc-4.9.x and does some minor things here and there.

gintas updated this revision to Diff 1028.Oct 27 2014, 8:35 AM
  • Added --enable-mingw-autodownload
  • Fixed quoting in p7zip installation note.
  • Fixed extraction directory for mingw32.
  • Whitespace fixes
  • Ensure that inplace/mingw is not created as a file if mv fails.
  • Define EH_UNWINDIN1G in seh_excn.h.
  • Avoid exporting certain symbols in Linker.c under mingw-w64-i686.
  • Removed references to ghc-tarballs.
  • Removed remaining references to Perl that were tripping up ./validate
  • Updated RTS imports for i686-mingw-w64.
  • Updated expected output in ghcilink tests.
  • Merging in awson's patch.
  • Updated another test expecting noisy DLL compilation output on Windows.

I've run validate.sh on Windows with x86_64 and i686 toolchains, both are looking reasonable with kyra's patch. There are some failures, but most of them are also there with the baseline.

I think the only remaining blocker is the mirror location for mingw-w64 packages on download.haskell.org in case the files on sourceforge are removed.

ezyang removed a subscriber: ezyang.Oct 27 2014, 2:05 PM
gintas updated this revision to Diff 1121.Oct 30 2014, 9:14 AM
  • Fixed a few more tests expecting noisy DLL linker output.
gintas updated this revision to Diff 1299.Nov 6 2014, 8:43 AM
  • Ensure that inplace/mingw/bin is on PATH for the build.
hvr awarded a token.Nov 10 2014, 10:15 AM
austin requested changes to this revision.Nov 20 2014, 10:06 PM
austin edited edge metadata.

@gintas, this patch mostly worked great, but it doesn't have a perl binary we can use? We need this for ghc-split which is written in Perl. Really, we should get rid of all the perl utilities, but oh well. Am I missing something?

BTW, the downloads for these MinGW binaries are now online: https://downloads.haskell.org/~ghc/mingw

This revision now requires changes to proceed.Nov 20 2014, 10:06 PM

Hi,

I didn't realize ghc needed Perl at runtime. Ugh.

I'm on vacation, will take a closer look when I get back.

If absolutely necessary, we should be able to download and install Perl
using exactly the same routine as mingw. There should be a decent
redistributable somewhere...

Feel free to roll back in the meantime if this causes problems.

thomie requested changes to this revision.Jun 10 2015, 6:10 AM
thomie added a reviewer: thomie.

Size of the mingw directory in the ghc-7.10.1 windows binaries:

327M	ghc-7.10.1-x86_64-unknown-mingw32/mingw/
110M	ghc-7.10.1-i386-unknown-mingw32/mingw

Size of unpacked files from https://downloads.haskell.org/~ghc/mingw:

$ du -hs mingw*
570M	mingw32
618M	mingw64

As it stands, all of this would be included in the binary distributions (see install_mingw target in ghc.mk). We should probably do something about this before landing this patch. See Trac #9014.

thomie added inline comments.Jun 10 2015, 12:21 PM
configure.ac
401

The mingw-builds project offers a choice between threads-posix and threads-win32 downloads. Please add a comment why threads-posix is needed.

Note that rts/win32/OSThreads.c calls CreateThread, which is a Windows api function. The mingw wiki mentions there are problems with that function by the way, saying that _beginthreadex should be used instead.

Also sjlj vs dwarf.

405

Again posix vs win32, and sjlj vs seh.

thomie commandeered this revision.Aug 12 2015, 5:00 AM
thomie edited reviewers, added: gintas; removed: thomie.

Abandoning, because this is superseded by D1123.

thomie abandoned this revision.Aug 12 2015, 5:00 AM