Generic and Generic1 instances for tuples
Needs RevisionPublic

Authored by int-index on Dec 26 2016, 2:12 PM.

Details

Summary

Add lacking Generic and Generic1 instances for tuples of size up to
62, which is the size limit for tuples in GHC.

Diff Detail

Repository
rGHC Glasgow Haskell Compiler
Branch
generic-tuples
Lint
Lint WarningsExcuse: line breaks make it look even worse
SeverityLocationCodeMessage
Warninglibraries/base/GHC/Generics.hs:1220TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1221TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1222TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1223TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1224TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1225TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1226TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1227TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1228TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1229TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1230TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1231TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1232TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1233TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1234TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1235TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1236TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1237TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1238TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1239TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1240TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1241TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1242TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1243TXT3Line Too Long
Warninglibraries/base/GHC/Generics.hs:1244TXT3Line Too Long
Unit
No Unit Test Coverage
Build Status
Buildable 12873
Build 17239: [GHC] Linux/amd64: Patch building
Build 17238: [GHC] OSX/amd64: Continuous Integration
Build 17237: arc lint + arc unit
int-index retitled this revision from to Generic and Generic1 instances for tuples.Dec 26 2016, 2:12 PM
int-index updated this object.

Hmm, it is quite surprising how much of an effect this has an haddock.base allocations.

I'm not sure how I feel about this. I don't know if we should be encouraging the use of obscenely large tuple types, even if they are technically possible to use in GHC. For comparison, I'm not aware of any instances for tuple types with arity greater than 15 (at least in base).

Encouraging/discouraging should be done in documentation and/or compiler warnings. The lack of those instances prevents my (correct) code from compiling and causes unnecessary frustration, for now I had to resort to orphan instances.

I'm all for adding -fwarn-large-tuples to GHC, but it's a separate concern.

Encouraging/discouraging should be done in documentation and/or compiler warnings. The lack of those instances prevents my (correct) code from compiling and causes unnecessary frustration, for now I had to resort to orphan instances.

I'm all for adding -fwarn-large-tuples to GHC, but it's a separate concern.

The problem is that adding these might increase compilation times for all users, regardless of whether their tuples are obscene or polite, due to the fact that GHC.Generics interface file now contains axioms with positively huge types. For instance, if I compile,

language=haskell
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE StandaloneDeriving #-}

module Test where
import GHC.Generics
deriving instance Generic ((,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,) a b c d e f g h i j k l m n o p q r s t u v w x y z a1 b1 c1 d1 e1 f1 g1 h1 i1 j1 k1 l1 m1 n1 o1 p1 q1 r1 s1 t1 u1 v1 w1 x1 y1 z1 a2 b2 c2 d2 e2 f2 g2 h2 i2 j2)

the output of --show-iface on the resulting interface file is over 400 lines long.

On the other hand, the interface file itself is only 10kB, which isn't huge. Moreover, the only performance issues which we are seeing in the testsuite are due to the haddock.base test, which is quite sensitive to any sort of addition, so perhaps this addition does affect program compilation as much as I suspected.

I have pushed a branch with this patch up to git.haskell.org (the wip/D2899 branch) in hopes that we can see what the Gipeda builder has to say about this. Hopefully we'll have more data in a few hours.

Perhaps we could make strategic uses of orphans here, placing in a

Hmm, indeed the gipeda results don't look promising.

I need to head out for the night but I'll try to return to this tomorrow.

Hello Ben,

thank you for looking into those performance issues. I didn't anticipate them.

I believe you didn't finish a sentence here:

Perhaps we could make strategic uses of orphans here, placing in a

but I'll reply anyway. Orphan instances are exactly what I'm trying to avoid: adding a bunch of deriving clauses isn't much of a problem for me, but if they're added in more than one library, those instances will conflict with each other. And if you propose to add orphans to base, wouldn't it be the first precedent? My impression was that base is orphan-free, but I might be wrong on this.

Hello Ben,

thank you for looking into those performance issues. I didn't anticipate them.

I believe you didn't finish a sentence here:

Perhaps we could make strategic uses of orphans here, placing in a

but I'll reply anyway. Orphan instances are exactly what I'm trying to avoid: adding a bunch of deriving clauses isn't much of a problem for me, but if they're added in more than one library, those instances will conflict with each other. And if you propose to add orphans to base, wouldn't it be the first precedent? My impression was that base is orphan-free, but I might be wrong on this.

Forget I mentioned this; the cut-off sentence was actually a bit of text that was supposed to be deleted.

I'll try to talk to @simonpj about possible directions here later this week.

dfeuer added a subscriber: dfeuer.Jan 4 2017, 11:15 AM

Is there a Trac ticket for this? If not, that might be a good idea; that way the problem will survive even if the solution doesn't.

I've spoken to @simonpj about this; the conclusion is that we would like to better understand the source of these allocations and how we might avoid them. In particular @niteria proposed that his recent work on improving the efficiency of the type family consistency check (D2833) might help. He checked this and unfortunately found that it did not. It would be great to see a differential profile of the compiler building, e.g., T5642 with and without this change.

It would also be interesting to know whether moving these instances to a new module (e.g. GHC.Generics.Tuples, which would be imported from GHC.Generics) would help the situation. The idea here is that few users will use these instances so we would like to ensure they aren't in the interface file for GHC.Generics, which is used far more often.

I second @dfeuer's call for a Trac ticket. Also, you (@int-index) mentioned in https://phabricator.haskell.org/D2899#84342 that you had some code that relies on this... I'd be curious to know what code requires these extraordinarily large tuples :)

@rwbarton pointed out that my thought of moving these instance to a separate module will be ineffective unless the corresponding tycons are also moved (due to GHC's treatment of orphans). However, given that most users only use small tuples anyways, this would likely be a good idea regardless. Naturally, the new GHC.BigTuples module would be exposed via the Prelude, so there will be no user-visible change other than faster builds. This proposal is being tracked by Trac #13072.

The substantial increase in the time taken to build GHC makes me reluctant to add these instances even if we sort out the issue about compile-time performance for user programs. Perhaps it can be reduced but the 7.3% overall increase shown by gipeda for this commit is clearly much too high.

It also doesn't seem feasible to avoid the user compile-time issue to the degree I had hoped. Even with some more non-trivial changes to GHC, I'm not sure how to avoid reading the interface file for these large tuples when compiling any user program that defines Generic instances.

I'm wondering about the need for these instances. Couldn't you use your own large tuple type?

rwbarton requested changes to this revision.Jan 11 2017, 8:13 AM
rwbarton added a reviewer: rwbarton.
This revision now requires changes to proceed.Jan 11 2017, 8:13 AM
int-index added a comment.EditedJan 11 2017, 11:17 AM

Also, you (@int-index) mentioned in https://phabricator.haskell.org/D2899#84342 that you had some code that relies on this... I'd be curious to know what code requires these extraordinarily large tuples :)

The code that relies on this uses a large tuple as a ReaderT environment, but all elements in the tuple have unique types and the getters distinguish them by type. So I have something like ask @Foo that looks up the value of type Foo in the large tuple.

I'm wondering about the need for these instances. Couldn't you use your own large tuple type?

Sure, I could define my own product type with no problem. But unlike built-in tuples, it wouldn't be blessed with the comma-based syntax. Also, if we don't support all features (instances) for large tuples and require end-users to define their own types, what's the point in having those tuples in base at all?

austin resigned from this revision.Nov 9 2017, 11:31 AM