Use ar for -staticlib
ClosedPublic

Authored by angerman on Jul 9 2017, 9:34 PM.

Details

Summary

Hopefully we can get rid of libtool, by using ar only

Depends on: D3579

Test Plan

validate

Diff Detail

Repository
rGHC Glasgow Haskell Compiler
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
angerman created this revision.Jul 9 2017, 9:34 PM
angerman updated this revision to Diff 13093.Jul 10 2017, 8:04 PM
  • cleanup
bgamari requested changes to this revision.Jul 11 2017, 10:34 AM

Can you fix the warnings?

Are you sure we want to do this? Afterall, libtool exists for a reason; creating static libraries can be quite tricky on some platforms.

This revision now requires changes to proceed.Jul 11 2017, 10:34 AM
angerman updated this revision to Diff 13127.Jul 11 2017, 8:18 PM
angerman edited edge metadata.
  • rebase & drop unused local bind verbose

Can you fix the warnings?

Are you sure we want to do this? Afterall, libtool exists for a reason; creating static libraries can be quite tricky on some platforms.

This is what libtool on macOS says

Apple Inc. version cctools-898
Libtool with -static is intended to replace ar(5) and ranlib.

and this is what libtool on my debian says:

libtool (GNU libtool) 2.4.2
Written by Gordon Matzigkeit <gord@gnu.ai.mit.edu>, 1996

Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
If OUTPUT-FILE ends in `.a' or `.lib', then a standard library is created using `ar' and `ranlib', or on Windows using `lib'.

While there are .la libtool archives, the -staticlib flag for GHC has always just produced an archive of all the object
files, ready to be linked with ld into an executable.

angerman updated this revision to Diff 13130.Jul 12 2017, 2:56 AM
  • rebase onto fixed master

Right, but do we know that this won't break, for instance, AIX or OpenBSD?

Right, but do we know that this won't break, for instance, AIX or OpenBSD?

-staticlib was available on iOS/macOS only so far. And the other patch requires a libtool that behaves sufficiently similar or a wrapper script (that uses ar). I just hope this allows us to stay out of the libtool for staticlibs mess. (And we already use ar to create archives.)

I still want to do some additional tests with this patch over the weekend.

Right, but do we know that this won't break, for instance, AIX or OpenBSD?

-staticlib was available on iOS/macOS only so far. And the other patch requires a libtool that behaves sufficiently similar or a wrapper script (that uses ar). I just hope this allows us to stay out of the libtool for staticlibs mess. (And we already use ar to create archives.)

Ahhh right, of course. I keep forgetting this was previously Darwin-only. Sounds fine to me.

bgamari requested changes to this revision.Jul 18 2017, 12:47 PM

@angerman is still testing this. Let me know when it's ready.

This revision now requires changes to proceed.Jul 18 2017, 12:47 PM
angerman planned changes to this revision.Jul 26 2017, 8:28 AM

This falls apart as soon as an archive contains multiple files of the same name.
Extracting files onto the file system thus inevitably results in conflicts.

angerman updated this revision to Diff 13357.Jul 28 2017, 2:45 AM
angerman edited edge metadata.
  • Do not rely on ar or libtool.
  • Teach GHC to read/write GNU and BSD .a Archives.
  • run ranlib over the produced archive.
angerman updated this revision to Diff 13362.Jul 28 2017, 3:07 AM
  • rebase onto D3579 to reduce merge conflicts.
angerman edited the summary of this revision. (Show Details)Jul 28 2017, 4:44 AM
angerman edited the test plan for this revision. (Show Details)
bgamari abandoned this revision.Jul 28 2017, 10:47 AM

If I'm not mistaken we decided against this approach.

angerman reclaimed this revision.Jul 28 2017, 2:01 PM

If I'm not mistaken we decided against this approach.

We did? When? With the Ar.hs this now specifically sidesteps the issues with libtool and ar.

If I'm not mistaken we decided against this approach.

We did? When? With the Ar.hs this now specifically sidesteps the issues with libtool and ar.

Wait a moment; I'm terribly confused in that case. I thought that using libtool handled the duplicate file case properly. Is this not so?

angerman added a comment.EditedJul 28 2017, 4:25 PM

If I'm not mistaken we decided against this approach.

We did? When? With the Ar.hs this now specifically sidesteps the issues with libtool and ar.

Wait a moment; I'm terribly confused in that case. I thought that using libtool handled the duplicate file case properly. Is this not so?

The earlier diff, which was landed a while ago uses libtool. However this requires the presence of libtool (or the libtool-lite script which relies on GNU ar with MRI librarian support). libtool is quite different on each platform (the one on macOS is nothing like GNU libtool).

libtool for static archives does essentially: concatinate the input archives, add the input objects, and create a symbol index.

As Archives are rather simple structurally, we can just build the archives with Haskell directly and use ranlib on the final result to get the symbol index. This is what this diff does and it seems simpler to me and superior to trying to deal with the differences/abailability of libtool across platforms.

bgamari requested changes to this revision.Aug 18 2017, 7:41 AM

Alright, I suppose this is reasonable if libtool really is that inconsistent (thanks, Apple!). However, let's document the reason why we are doing this. Can you add a small note to Ar.hs saying what you said above? Then I promise I'll accept it :)

This revision now requires changes to proceed.Aug 18 2017, 7:41 AM
angerman updated this revision to Diff 13737.Sep 5 2017, 2:18 AM
angerman edited edge metadata.
  • rebase
  • Add comment regarding Ar
angerman updated this revision to Diff 13754.Sep 5 2017, 7:13 AM
  • rebase
  • Add comment regarding Ar
angerman updated this revision to Diff 13762.Sep 6 2017, 3:52 AM
  • rebase
angerman updated this revision to Diff 13776.Sep 7 2017, 9:35 AM
  • help windows
angerman updated this revision to Diff 13784.Sep 8 2017, 7:34 AM
  • rebase
angerman updated this revision to Diff 13785.Sep 8 2017, 7:57 AM
  • Adds Semigroup
angerman updated this revision to Diff 13786.Sep 8 2017, 8:05 AM
  • Semigroup from GHC.Base
angerman updated this revision to Diff 13787.Sep 8 2017, 8:12 AM
  • From Data.Semigroup
angerman updated this revision to Diff 13788.Sep 8 2017, 8:29 AM
  • Yuck...
angerman updated this revision to Diff 13800.Sep 9 2017, 2:37 AM
  • reabse
bgamari accepted this revision.Sep 13 2017, 7:24 AM

Looks good to me. Thanks for your patience, @angerman!

This revision is now accepted and ready to land.Sep 13 2017, 7:24 AM
This revision was automatically updated to reflect the committed changes.