Generate DOS short names for paths in response files if available.
AbandonedPublic

Authored by Phyx on Jan 4 2017, 9:00 PM.

Details

Reviewers
erikd
austin
bgamari
Trac Issues
#12971
Summary

There is a bug in libiberty where when reading the response file from disk
they assume the file is ASCII using a one byte per character encoding instead of UTF-8.

This means that unicode paths are broken. A proper fix will be to fix libiberty.

However this won't be done for GHC 8.2. Instead I propose this for 8.2 and a proper fix
for 8.4 since the issue is marked high priority.

This has several caveats though:

  1. The path must exist, otherwise the path won't be converted.
  2. DOS short names can be disabled (but really, almost no one does this.).
  3. We're doing a lot of extra processing of paths
Test Plan

./validate and make test TEST=T12971

Diff Detail

Repository
rGHC Glasgow Haskell Compiler
Branch
trac-12971-temporary-response-workaround
Lint
Lint ErrorsExcuse: unicode
SeverityLocationCodeMessage
Errortestsuite\tests\driver\Makefile:647TXT5Bad Charset
Errortestsuite\tests\driver\Makefile:648TXT5Bad Charset
Unit
No Unit Test Coverage
Build Status
Buildable 13052
Build 17876: [GHC] Linux/amd64: Patch building
Build 17875: [GHC] OSX/amd64: Continuous Integration
Build 17874: [GHC] Windows/amd64: Continuous Integration
Build 17873: arc lint + arc unit
Phyx updated this revision to Diff 10251.Jan 4 2017, 9:00 PM
Phyx retitled this revision from to Generate DOS short names for paths in response files..
Phyx updated this object.
Phyx edited the test plan for this revision. (Show Details)
Phyx updated the Trac tickets for this revision.
Phyx added a subscriber: Restricted Project.
Phyx updated this revision to Diff 10252.Jan 4 2017, 9:30 PM
Phyx edited edge metadata.
  • T12971: fix non windows.
awson added a subscriber: awson.Jan 5 2017, 1:15 AM

Your second premise is wrong. Many have short path generation switched off. Me is an example.

Please, be absolutely sure you code doesn't break things on non-unicode paths not having short counterparts.

I briefly glanced at the code and have got an impression this is not true ATM.

Phyx added a comment.EditedJan 5 2017, 1:52 AM

You're the first one I've met who has it off. Anyway, if the short path
fails it just returns the original path you tried shorten.

This happens often as we build ghc itself. Lots of paths don't exist yet
when gcc is called. These failed calls is what the handle and Win32.try
are for.

I'll also say I dislike having to call this API, but it's the only non-upstream fix I can think of while retaining response files.

Phyx updated this revision to Diff 10256.Jan 5 2017, 2:31 AM
Phyx edited edge metadata.
  • T12971: fix non windows imports.
bgamari requested changes to this revision.Jan 5 2017, 11:32 AM
bgamari edited edge metadata.

Oh dear. What a mess. However, it sounds like this is a pretty sensible solution in light of the limitations we face. Thanks for picking this up, @Phyx! I don't know what we would do without you.

However, since this issue is still rather unresolved in the case that the user has both non-ASCII filenames and short paths disabled, do you think you could add a quick note about this to docs/users_guide/bugs.rst?

This revision now requires changes to proceed.Jan 5 2017, 11:32 AM
Phyx added a comment.Jan 6 2017, 1:46 AM

Sure, I'll do it tonight.

Thanks

Phyx added a comment.Jan 6 2017, 1:47 AM

Sure, I'll do it tonight.

Thanks

Phyx updated this revision to Diff 10317.Jan 6 2017, 12:47 PM
Phyx edited edge metadata.
  • T12971: add bugs entry.
Phyx added a comment.Jan 6 2017, 2:40 PM

Thanks @bgamari, I'll add the newline at the end of of ResponseFile.hsc soon.

Phyx retitled this revision from Generate DOS short names for paths in response files. to Generate DOS short names for paths in response files if available..Jan 6 2017, 4:48 PM
Phyx edited edge metadata.
Phyx updated this revision to Diff 10341.Jan 8 2017, 12:21 AM
  • T12971: end file in newline.
Phyx abandoned this revision.Jan 11 2017, 11:30 AM

Abandoning this in favor of D2942, as this mostly only fixes GCC and only half.. only for paths that do exist.