Implement underscores in numeric literals (NumericUnderscores extension)
ClosedPublic

Authored by takenobu on Nov 25 2017, 7:26 AM.

Details

Summary

Implement the proposal of underscores in numeric literals.
Underscores in numeric literals are simply ignored.

The specification of the feature is available here:
https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0009-numeric-underscores.rst

For a discussion of the various choices:
https://github.com/ghc-proposals/ghc-proposals/pull/76

Implementation detail:

  • Added dynamic flag
    • NumericUnderscores extension flag is added for this feature.
  • Alex "Regular expression macros" in Lexer.x
    • Add @numspc (numeric spacer) macro to represent multiple underscores.
    • Modify @decimal, @decimal, @binary, @octal, @hexadecimal, @exponent, and @bin_exponent macros to include @numspc.
  • Alex "Rules" in Lexer.x
    • To be simpler, we have only the definitions with underscores. And then we have a separate function (tok_integral and tok_frac) that validates the literals.
  • Validation functions in Lexer.x
    • tok_integral and tok_frac functions validate whether contain underscores or not. If NumericUnderscores extensions are not enabled, check that there are no underscores.
    • tok_frac function is created by merging strtoken and init_strtoken.
    • init_strtoken is deleted. Because it is no longer used.
  • Remove underscores from target literal string
    • parseUnsignedInteger, readRational__, and readHexRational} use the customized span'` function to remove underscores.
  • Added Testcase
    • testcase for NumericUnderscores enabled. NumericUnderscores0.hs and NumericUnderscores1.hs
    • testcase for NumericUnderscores disabled. NoNumericUnderscores0.hs and NoNumericUnderscores1.hs
    • testcase to invalid pattern for NumericUnderscores enabled. NumericUnderscoresFail0.hs and NumericUnderscoresFail1.hs
Test Plan

validate including the above testcase

Diff Detail

Repository
rGHC Glasgow Haskell Compiler
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
takenobu created this revision.Nov 25 2017, 7:26 AM

Since master failed, validate is incomplete.
However, I confirmed that only the same test cases as master failed.
These test cases are about spaces/.

takenobu updated this revision to Diff 14876.Dec 3 2017, 12:07 AM

Improve testcases and users_guide

I am not in a hurry :)
Could anyone please review this after D4260?

takenobu updated this revision to Diff 15074.Jan 13 2018, 2:47 AM
  • Resolve conflict with current master
takenobu updated this revision to Diff 15075.Jan 13 2018, 7:46 AM
  • Rebase against current master (ghc8.5)
bgamari accepted this revision.Jan 21 2018, 10:53 AM

Looks good to me. Thanks @takenobu!

This revision is now accepted and ready to land.Jan 21 2018, 10:53 AM
This revision was automatically updated to reflect the committed changes.

Ben, thanks for reviewing, despite being busy for release.