Implement underscores in numeric literals (NumericUnderscores extension)

Authored by takenobu on Nov 25 2017, 7:26 AM.



Implement the proposal of underscores in numeric literals.
Underscores in numeric literals are simply ignored.

The specification of the feature is available here:

For a discussion of the various choices:

Implementation detail:

  • Added dynamic flag
    • NumericUnderscores extension flag is added for this feature.
  • Alex "Regular expression macros" in Lexer.x
    • Add @numspc (numeric spacer) macro to represent multiple underscores.
    • Modify @decimal, @decimal, @binary, @octal, @hexadecimal, @exponent, and @bin_exponent macros to include @numspc.
  • Alex "Rules" in Lexer.x
    • To be simpler, we have only the definitions with underscores. And then we have a separate function (tok_integral and tok_frac) that validates the literals.
  • Validation functions in Lexer.x
    • tok_integral and tok_frac functions validate whether contain underscores or not. If NumericUnderscores extensions are not enabled, check that there are no underscores.
    • tok_frac function is created by merging strtoken and init_strtoken.
    • init_strtoken is deleted. Because it is no longer used.
  • Remove underscores from target literal string
    • parseUnsignedInteger, readRational__, and readHexRational} use the customized span'` function to remove underscores.
  • Added Testcase
    • testcase for NumericUnderscores enabled. NumericUnderscores0.hs and NumericUnderscores1.hs
    • testcase for NumericUnderscores disabled. NoNumericUnderscores0.hs and NoNumericUnderscores1.hs
    • testcase to invalid pattern for NumericUnderscores enabled. NumericUnderscoresFail0.hs and NumericUnderscoresFail1.hs
Test Plan

validate including the above testcase

Diff Detail

rGHC Glasgow Haskell Compiler
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.
takenobu created this revision.Nov 25 2017, 7:26 AM

Since master failed, validate is incomplete.
However, I confirmed that only the same test cases as master failed.
These test cases are about spaces/.

takenobu updated this revision to Diff 14876.Dec 3 2017, 12:07 AM

Improve testcases and users_guide

I am not in a hurry :)
Could anyone please review this after D4260?

takenobu updated this revision to Diff 15074.Jan 13 2018, 2:47 AM
  • Resolve conflict with current master
takenobu updated this revision to Diff 15075.Jan 13 2018, 7:46 AM
  • Rebase against current master (ghc8.5)
bgamari accepted this revision.Jan 21 2018, 10:53 AM

Looks good to me. Thanks @takenobu!

This revision is now accepted and ready to land.Jan 21 2018, 10:53 AM
This revision was automatically updated to reflect the committed changes.

Ben, thanks for reviewing, despite being busy for release.