re-running detailed routing

[kareefardi]

Describe the bug

Once I triggered detailed router to resume routing after stopping at some iteration where there was still routing violations and it didn't completely finish. By resuming I mean a separate call to detailed_routing. I am trying to recreate this scenario but running into issues. I re-ran detailed routing multiple times with 1 iteration and the violations count is diverging. Tried to increase droute_end_iter to 1 but still couldn't get it to work.

set_thread_count 20
read_db ./spm.odb
read_liberty ./sky130_fd_sc_hd__tt_025C_1v80.lib
read_sdc ./spm.sdc
set min_layer met1
set max_layer met5
set i 0

detailed_route\
    -bottom_routing_layer $min_layer\
    -top_routing_layer $max_layer\
    -droute_end_iter 0\
    -or_seed 42\
    -save_guide_updates\
    -verbose 1

while {$i < 10} {
    puts "DEBUG iter $i"
    detailed_route\
        -bottom_routing_layer $min_layer\
        -top_routing_layer $max_layer\
        -no_pin_access\
        -disable_via_gen\
        -droute_end_iter 0\
        -or_seed 42\
        -save_guide_updates\
        -verbose 1
    set i [expr $i + 1]
}

Expected Behavior

Clarity on whether this is supported and how to correctly do it.

Environment

Git commit: 87af90f72f3f9be1fdfa1d886f0dd8d8b8f34694
kernel: Linux 6.11.10-1-liquorix-amd64
os: Ubuntu 22.04.5 LTS (Jammy Jellyfish)
cmake version 3.25.1
-- The CXX compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenROAD version: v2.0-17585-g87af90f72
-- System name: Linux
-- Compiler: GNU 11.4.0
-- Build type: RELEASE
-- Install prefix: /usr/local
-- C++ Standard: 17
-- C++ Standard Required: ON
-- C++ Extensions: OFF
-- The C compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test C_COMPILER_SUPPORTS__-Wall
-- Performing Test C_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Failed
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Failed
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive - Success
-- Performing Test C_COMPILER_SUPPORTS__-x
-- Performing Test C_COMPILER_SUPPORTS__-x - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-x
-- Performing Test CXX_COMPILER_SUPPORTS__-x - Failed
-- Performing Test C_COMPILER_SUPPORTS__c++
-- Performing Test C_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__c++
-- Performing Test CXX_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17 - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17 - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- TCL readline library: /usr/lib/x86_64-linux-gnu/libtclreadline.so
-- TCL readline header: /usr/include/x86_64-linux-gnu
-- Found SWIG: /usr/bin/swig4.0 (found suitable version "4.0.2", minimum required is "4.0")  
-- Found Boost: /usr/local/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found version "1.81.0")  
-- boost: 1.81.0
-- Found GTest: /home/karim/.local/lib/cmake/GTest/GTestConfig.cmake (found version "1.13.0")  
-- GTest: 1.13.0
-- Found Python3: /usr/include/python3.10 (found version "3.10.12") found components: Development Development.Module Development.Embed 
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
-- Found Threads: TRUE  
-- spdlog: 1.8.1
-- Found BISON: /usr/bin/bison (found version "3.8.2") 
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- STA version: 2.6.0
-- STA git sha: aa598a2f14c5c142e90391a69988523505e7db3d
-- System name: Linux
-- Compiler: GNU 11.4.0
-- Build type: RELEASE
-- Build CXX_FLAGS: -O3 -DNDEBUG
-- Install prefix: /usr/local
-- Found FLEX: /usr/bin/flex (found version "2.6.4") 
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- TCL readline library: /usr/lib/x86_64-linux-gnu/libtclreadline.so
-- TCL readline header: /usr/include/x86_64-linux-gnu/tclreadline.h
-- CUDD library: /usr/local/lib/libcudd.a
-- CUDD header: /usr/local/include/cudd.h
-- SSTA: 0
-- Found SWIG: /usr/bin/swig4.0 (found suitable version "4.0.2", minimum required is "3.0")  
-- STA executable: /home/karim/work/OpenROAD/src/sta/app/sta
-- Found re2: /home/karim/Downloads/or-tools_x86_64_Ubuntu-22.10_cpp_v9.5.2237/lib/cmake/re2/re2Config.cmake (found version "9.0.0") 
-- Found Clp: /home/karim/Downloads/or-tools_x86_64_Ubuntu-22.10_cpp_v9.5.2237/lib/cmake/Clp/ClpConfig.cmake (found version "1.17.7") 
-- Found Cbc: /home/karim/Downloads/or-tools_x86_64_Ubuntu-22.10_cpp_v9.5.2237/lib/cmake/Cbc/CbcConfig.cmake (found version "2.10.7") 
-- Found SCIP: /home/karim/Downloads/or-tools_x86_64_Ubuntu-22.10_cpp_v9.5.2237/lib/cmake/scip/scip-config.cmake (found version "8.0.1") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found OR-Tools: /home/karim/Downloads/or-tools_x86_64_Ubuntu-22.10_cpp_v9.5.2237/lib/cmake/ortools (version: 9.5.2237)
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found OpenMP: TRUE (found version "4.5")  
-- GUI is not enabled
-- Found Boost: /usr/local/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found version "1.81.0") found components: serialization 
-- Could NOT find VTune (missing: VTune_LIBRARIES VTune_INCLUDE_DIRS) 
-- Found Boost: /usr/local/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found suitable version "1.81.0", minimum required is "1.78")  
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- Found Boost: /usr/local/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found version "1.81.0") found components: serialization system thread 
-- Found Boost: /usr/local/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found version "1.81.0")  
-- Found Eigen3: /usr/share/eigen3/cmake/Eigen3Config.cmake (found version "3.4.0") 
-- TCL readline enabled
-- Tcl Extended disabled
-- Python3 enabled
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/tmp.3GXOpVUvbb

To Reproduce

test-case.tar.gz
Extract, cd into extracted directory and run openroad -exit run.tcl

Relevant log output

No response

Screenshots

No response

Additional Context

No response

[osamahammad21]

Unfortunately, this functionality is not currently supported in DRT. In fact, it should have displayed a warning and avoided performing any routing during the reruns (I am working on a PR to address this).

Currently, DRT supports incremental detailed routing, but not in the way you described. Incremental detailed routing focuses on routing only the unrouted nets, while rerouting nets with violations is not yet implemented.

If your goal is to capture snapshots of the design after each iteration, you can achieve this by adding the following line before calling the detailed_route command:

set_debug_level DRT snapshot 1

Additionally, include the following option in the detailed_route command:

-drc_report_iter_step 1

This approach will generate a database file and a DRC report for each iteration, allowing you to track the design's progress.

[kareefardi]

Unfortunately, this functionality is not currently supported in DRT. In fact, it should have displayed a warning and avoided performing any routing during the reruns (I am working on a PR to address this).

Currently, DRT supports incremental detailed routing, but not in the way you described. Incremental detailed routing focuses on routing only the unrouted nets, while rerouting nets with violations is not yet implemented.

If your goal is to capture snapshots of the design after each iteration, you can achieve this by adding the following line before calling the detailed_route command:

set_debug_level DRT snapshot 1

Additionally, include the following option in the detailed_route command:

-drc_report_iter_step 1

This approach will generate a database file and a DRC report for each iteration, allowing you to track the design's progress.

@osamahammad21 Thank you. Yes, although a primary goal is to save a snapshot, it is useful to be able to pause and continue the design. In fact, we used this previously in specific cases, but when I attempted to consistently do it, I ran into the issue above. Whenever such a feature is implemented please tag me and let me know.

[osamahammad21]

ok 👍