EastAsianWidth.txt Format Change after Unicode 15.1.0

[elfham]

The format of EastAsianWidth.txt seems to change after Unicode 15.1.0 (draft).

15.0.0: https://www.unicode.org/Public/15.0.0/ucd/EastAsianWidth.txt

# For legacy reasons, there are no spaces before or after the semicolon
# which separates the two fields. The comments following the number sign
# "#" list the General_Category property value or the L& alias of the
# derived value LC, the Unicode character name or names, and, in lines
# with ranges of code points, the code point count in square brackets.
#
# For more information, see UAX #11: East Asian Width,
# at https://www.unicode.org/reports/tr11/
#
# @missing: 0000..10FFFF; N
0000..001F;N     # Cc    [32] <control-0000>..<control-001F>
0020;Na          # Zs         SPACE
0021..0023;Na    # Po     [3] EXCLAMATION MARK..NUMBER SIGN
0024;Na          # Sc         DOLLAR SIGN

15.1.0 draft: https://www.unicode.org/Public/draft/UCD/ucd/EastAsianWidth.txt

# The comments following the number sign "#" list the General_Category
# property value or the L& alias of the derived value LC, the Unicode
# character name or names, and, in lines with ranges of code points,
# the code point count in square brackets.
#
# For more information, see UAX #11: East Asian Width,
# at https://www.unicode.org/reports/tr11/
#
# @missing: 0000..10FFFF; N
0000..001F     ; N  # Cc    [32] <control-0000>..<control-001F>
0020           ; Na # Zs         SPACE
0021..0023     ; Na # Po     [3] EXCLAMATION MARK..NUMBER SIGN
0024           ; Na # Sc         DOLLAR SIGN

Therefore, bin/generate_east_asian_width will fail.

% ruby bin/generate_east_asian_width EastAsianWidth-15.1.0-draft.txt
bin/generate_east_asian_width:15:in `block (2 levels) in <main>': undefined method `to_sym' for nil:NilClass (NoMethodError)

    type = type.to_sym
               ^^^^^^^
        from bin/generate_east_asian_width:10:in `each_line'
        from bin/generate_east_asian_width:10:in `block in <main>'
        from bin/generate_east_asian_width:8:in `open'
        from bin/generate_east_asian_width:8:in `<main>'
%

So, I fix this.

Test

Convert the current version 15.0.0.

% ruby bin/generate_east_asian_width EastAsianWidth-15.0.0.txt > eaw-15.0.0.rb
% diff lib/reline/unicode/east_asian_width.rb eaw-15.0.0.rb
3c3
<   # EastAsianWidth.txt
---
>   # EastAsianWidth-15.0.0.txt
%

The same as the current version is generated.

Convert the draft version 15.1.0.

% ruby bin/generate_east_asian_width EastAsianWidth-15.1.0-draft.txt > eaw-15.1.0.rb
% diff eaw-15.0.0.rb eaw-15.1.0.rb
3c3
<   # EastAsianWidth-15.0.0.txt
---
>   # EastAsianWidth-15.1.0-draft.txt
63c63
<     \u{2FF0}-\u{2FFB}
---
>     \u{2FF0}-\u{2FFF}
70c70
<     \u{31F0}-\u{321E}
---
>     \u{31EF}-\u{321E}
%

Probably OK.

By the way, this EastAsianWidth.txt is still a draft, so it does not seem to include any New CJK Ideographs, etc.

[tompng]

Thank you. The implementation looks good 👍
One thing, could it be m = a.match(b) and m[c] instead of Regexp.last_match(c) ?

Regexp.last_match is not used in this repository.
In some code in ruby/irb, Regexp.last_match is used in this form that Regexp#match can't be used.

when /regexp1/
  puts :a, Regexp.last_match(1)
when /regexp2/
  puts :b, Regexp.last_match(2)
end

[elfham]

Thank you for the suggestion.
I have corrected the code.

[tompng]

👍