Undocumented flag in DXF layer after dwg2dxf

[vagran]

Hello,
After converting my DWG file using dwg2dxf utility, I have flag 8 set in one of my layers flags (group 70). This value is not described in the DXF specification. Looking through the libredwg source code I have discovered this place:

libredwg/src/dwg.spec

Line 3788 in 56d392a

(FIELD_VALUE (is_xref_ref) << 3) |

I do not really aware of this macro-based syntax, but this place looks suspicious, and probably does not correspond to its preceding comment which cites flag values from DXF docs, also not mentioning value 8. Maybe it should be changed like this:

       (FIELD_VALUE (is_xref_ref) << 4) |
      ((FIELD_VALUE (is_xref_resolved) ? 1 : 0) << 5) |
      (FIELD_VALUE (is_xref_dep) << 6);

[michal-josef-spacek]

@vagran Do you have some example DWG file?

[vagran]

@vagran Do you have some example DWG file?

Unfortunately, I see this issue with our customer proprietary file, which I cannot share.

[michal-josef-spacek]

@vagran ok. AC1015, LAYER table, and something with xref.
It could be helpful to send dwgread -v3 __DWG_FILE__ for LAYER table only.
Like:

...
Next object: 13 Handleoff: 0x1 [UMC] Offset: -1019 [MC] @2665
==========================================
Object number: 13/D, Size: 31 [MS], Type: 51 [BS]
Add table record LAYER [13] Decode table record LAYER
bitsize: 170 [RL] @3.2
 Hdlsize: 0x4E,
 has_strings: 1
handle: 0.1.10 [H 5]

num_eed: 0
num_reactors: 0 [BL 0]
is_xdic_missing: 1 [B 0]
ownerhandle: (4.1.2) abs:2 [H 330]
name: "0" [TU 2]
is_xref_resolved: 0 [BS 0]
xref: (5.0.0) abs:0 [H 0]
flag0: 0x3f0 [BSx 0]
color.index: 256 [CMC.BS 62]
color.rgb: 0xc3000007 [CMC.BL 420]
color.method: 0xc3 Truecolor
color.flag: 0x0 [CMC.RC]
plotstyle: (5.1.F) abs:15 [H 0]
material: (5.1.98) abs:152 [H 0]
ltype: (5.1.16) abs:22 [H 6]
...

To see your version of the layer settings.

[michal-josef-spacek]

@vagran And do you export to AC1015 DXF?

[vagran]

@vagran ok. AC1015, LAYER table, and something with xref. It could be helpful to send dwgread -v3 __DWG_FILE__ for LAYER table only.

This file's version code is: AC1032 (r2018)
is_maint: 0x0 [RC 0]
zero_one_or_three: 0x3 [RC 0]
thumbnail_address: 1984 [RL 0]
dwg_version: 0x21 [RC 0]
maint_version: 0x0 [RC 0]
codepage: 30 [RS 0]
ANSI_1252
unknown_0: 0x0 [RC 0]
app_dwg_version: 0x0 [RC 0]
app_maint_version: 0x0 [RC 0]
security_type: 0 [RL 0]
rl_1c_address: 0 [RL 0]
summaryinfo_address: 288 [RL 0]
vbaproj_address: 0 [RL 0]
r2004_header_address: 128 [RL 0]

#### r2004 File Header ####
file_ID_string: "AcFssFcAJMB" [TFF 12 0]
41634673734663414A4D4200
header_address: 0x0 [RLx 0]
header_size: 108 [RL 0]
x04: 4 [RL 0]
root_tree_node_gap: 0 [RLd 0]
lowermost_left_tree_node_gap: 0 [RLd 0]
lowermost_right_tree_node_gap: 0 [RLd 0]
unknown_long: 1 [RL 0]
last_section_id: 643 [RL 0]
last_section_address: 0x6A11C0 [RLL 0]
secondheader_address: 0x69F0E4 [RLL 0]
numgaps: 0 [RL 0]
numsections: 641 [RL 0]
x20: 32 [RL 0]
x80: 128 [RL 0]
x40: 64 [RL 0]
section_map_id: 643 [RL 0]
section_map_address: 0x69E000 [RLL 0]
section_info_id: 642 [RLd 0]
section_array_size: 643 [RL 0]
gap_array_size: 0 [RL 0]
crc32: 0x9bfb676f [RLx 0]
padding: [TFFx 12 0]
4134F74DBAF3701C8FFA8EE8
crc32 => 0x9bfb676f (verified)

Object number: 52774/CE26, Size: 92 [MS], Hdlsize: 0x51 [UMC] , Type: 51 [BOT]
Add table record LAYER [52774] Decode table record LAYER

 has_strings: 1
handle: 0.3.52C32 [H 5]
EED[0] size: 46 [BS]
EED[0] handle: 5.2.AA9
EED[0] raw: 46
EED[0] code: 0 [RC], wstring: len=0 [RS] "(null)" [TU]
EED[1] code: 0 [RC], wstring: len=20 [RS] ""Annotation" V-ANNO " [TU]

EED[2] size: 5 [BS]
EED[2] handle: 5.2.8580
EED[2] raw: 5
EED[2] code: 71 [RC], long: 0 [RL]



num_eed: 3
num_reactors: 0 [BL 0]
is_xdic_missing: 1 [B 0]
has_ds_data: 0 [B 0]
ownerhandle: (4.1.2) abs:2 [H 330]
name: "BOX" [TU 2]
is_xref_resolved: 0 [BS 0]
xref: (5.0.0) abs:0 [H 0]
flag0: 0x60 [BSx 0]
color.index: 256 [CMC.BS 62]
color.rgb: 0xc3000006 [CMC.BL 420]
color.method: 0xc3 Truecolor
color.flag: 0x0 [CMC.RC]
plotstyle: (5.1.F) abs:15 [H 0]
material: (5.1.98) abs:152 [H 0]
ltype: (5.1.16) abs:22 [H 6]
visualstyle: (5.0.0) abs:0 [H 348]
crc: 5DB9 [RSx]
 check_CRC 95: 5DB9 == 5DB9

@vagran And do you export to AC1015 DXF?

I do not specify any options for dwg2dxf. Should I?

[vagran]

BTW isn't this just an obvious bug in the source code:

    // DXF: frozen (1), frozen by default in new viewports (2),
    //      locked (4), is_xref_ref (16), is_xref_resolved (32), is_xref_dep (64).
    FIELD_VALUE (flag) |= FIELD_VALUE (frozen) |
      (FIELD_VALUE (frozen_in_new) << 1) |
      (FIELD_VALUE (locked) << 2) |
      (FIELD_VALUE (is_xref_ref) << 3) |
      ((FIELD_VALUE (is_xref_resolved) ? 1 : 0) << 4) |
      (FIELD_VALUE (is_xref_dep) << 5);

Comment (which is taken from the specification) does not match the bits set for is_xref_ref, is_xref_resolved and is_xref_dep.

[michal-josef-spacek]

@vagran Ah ok.

You have AC1032 DWG file.
You are using dwg2dxf __DWG_FILE__ to convert. The result will be AC1032.
Ok, I see that layer 70 is bad from dwg2dxf.

[michal-josef-spacek]

BTW isn't this just an obvious bug in the source code:
Comment (which is taken from the specification) does not match the bits set for is_xref_ref, is_xref_resolved and is_xref_dep.

Yes, maybe. I need to check.

[michal-josef-spacek]

I created some fixes to LAYER table: #1078

[michal-josef-spacek]

@vagran Could you test fixes in master branch?

[vagran]

I tried to test it. It crashes with free(): invalid pointer. Fun fact - it seems to be a regression from the only bug I have reported some time ago - #986.

Core was generated by `./dwg2dxf -y ../test-data/epsg-2882.dwg -o ../test-data/workspace/converted.dxf'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140627744847680) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140627744847680) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140627744847680) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140627744847680, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007fe672cb4476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007fe672c9a7f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007fe672cfb676 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fe672e4db77 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#6  0x00007fe672d12cfc in malloc_printerr (str=str@entry=0x7fe672e4b744 "free(): invalid pointer") at ./malloc/malloc.c:5664
#7  0x00007fe672d14a44 in _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at ./malloc/malloc.c:4439
#8  0x00007fe672d17453 in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3391
#9  0x00007fe6736cce9e in dxf_fixup_string (dxf=301, opts=1, 
    str=0x55f0efddbfb0 "<?xml version=\"1.0\" encoding=\"UTF-16\" standalone=\"no\" ?>\n<Dictionary version=\"1.0\" xmlns=\"http://www.osgeo.org/mapguide/coordinatesystem\">\n\n  <ProjectedCoordinateSystem id=\"FLHP-WF\">\n    <Name>FLHP-WF"..., dat=0x7ffc27382fa0) at out_dxf.c:1246
#10 dwg_dxf_GEODATA_private (hdl_dat=0x7ffc27382fa0, str_dat=0x7ffc27382fa0, obj=0x7fe66ed4a5c0, dat=0x7ffc27382fa0) at /home/artyom/tmp/libredwg/src/dwg.spec:6388
#11 dwg_dxf_GEODATA (dat=dat@entry=0x7ffc27382fa0, obj=obj@entry=0x7fe66ed4a5c0) at /home/artyom/tmp/libredwg/src/dwg.spec:6332
#12 0x00007fe673739231 in dwg_dxf_variable_type (obj=0x7fe66ed4a5c0, dat=0x7ffc27382fa0, dwg=<optimized out>) at /home/artyom/tmp/libredwg/src/classes.inc:215
#13 dwg_dxf_object (dat=dat@entry=0x7ffc27382fa0, obj=obj@entry=0x7fe66ed4a5c0, i=i@entry=0x7ffc27382f24) at out_dxf.c:2949
#14 0x00007fe67374a96c in dxf_objects_write (dwg=0x7ffc273830a0, dat=0x7ffc27382fa0) at out_dxf.c:3742
#15 dwg_write_dxf (dat=0x7ffc27382fa0, dwg=<optimized out>) at out_dxf.c:3830
#16 0x000055f0ebc639aa in main (argc=5, argv=<optimized out>) at dwg2dxf.c:339
(gdb) f 9
#9  0x00007fe6736cce9e in dxf_fixup_string (dxf=301, opts=1, 
    str=0x55f0efddbfb0 "<?xml version=\"1.0\" encoding=\"UTF-16\" standalone=\"no\" ?>\n<Dictionary version=\"1.0\" xmlns=\"http://www.osgeo.org/mapguide/coordinatesystem\">\n\n  <ProjectedCoordinateSystem id=\"FLHP-WF\">\n    <Name>FLHP-WF"..., dat=0x7ffc27382fa0) at out_dxf.c:1246
1246      if (str && *str)
(gdb) f 9
#9  0x00007fe6736cce9e in dxf_fixup_string (dxf=301, opts=1, 
    str=0x55f0efddbfb0 "<?xml version=\"1.0\" encoding=\"UTF-16\" standalone=\"no\" ?>\n<Dictionary version=\"1.0\" xmlns=\"http://www.osgeo.org/mapguide/coordinatesystem\">\n\n  <ProjectedCoordinateSystem id=\"FLHP-WF\">\n    <Name>FLHP-WF"..., dat=0x7ffc27382fa0) at out_dxf.c:1246

The corresponding code looks quite bad:

libredwg/src/out_dxf.c

Line 1282 in 203edc0

cbuf += 250;

Besides the questionable decision to use static variables, it increments the pointer, which is freed two lines below.

Regarding the layer, it now looks like this:

  0
LAYER
  5
52C32
330
2
100
AcDbSymbolTableRecord
100
AcDbLayerTableRecord
  2
BOX
 70
     0
 62
     6
  6
Continuous
290
     0
370
    13
390
F
347
98
348
0
1001
AcAecLayerStandard
1000

1000
"Annotation" V-ANNO 
1001
AcCmTransparency
1071
        0

Group 70 is 0. Is it supposed to be zero now? If it was 8 before, should it become 16? Or the flag itself also was wrong?

[michal-josef-spacek]

Regarding the layer, it now looks like this:

  0
LAYER
  5
52C32
330
2
100
AcDbSymbolTableRecord
100
AcDbLayerTableRecord
  2
BOX
 70
     0
 62
     6
  6
Continuous
290
     0
370
    13
390
F
347
98
348
0
1001
AcAecLayerStandard
1000

1000
"Annotation" V-ANNO 
1001
AcCmTransparency
1071
        0

Group 70 is 0. Is it supposed to be zero now? If it was 8 before, should it become 16? Or the flag itself also was wrong?

I think yes.
You have:

is_xref_resolved: 0 [BS 0]
xref: (5.0.0) abs:0 [H 0]

This mean without xref.

[michal-josef-spacek]

@rurban I have output of dwgread in the way:

...
Free object LAYER [46]
       linewt: 0x1f
...

How to prevent printing these new flags by logging in Free section of output?

[rurban]

@rurban I have output of dwgread in the way:

...
Free object LAYER [46]
       linewt: 0x1f
...

How to prevent printing these new flags by logging in Free section of output?

#ifndef IS_FREE

[michal-josef-spacek]

@rurban #1079 ?