LiveHD is the compiler infrastructure that allows to work +with Hardware Description Languages (HDLs) like Verilog, CHISEL, and Pyrope.
+Pyrope is the new HDL built on top of LiveHD.
+Hardware Design provides an introduction to hardware design +for software designers.
+ + + + + + +LiveHD is an infrastructure designed for Live Hardware Development. By live, we +mean that small changes in the design should have the synthesis and simulation +results in a few seconds.
+As the goal of "seconds," we do not need to perform too fine grain incremental +work. Notice that this is a different goal from having an typical incremental +synthesis, where many edges are added and removed in the order of thousands of +nodes/edges.
+LiveHD: a fast and friendly hardware development flow that you can trust
+LiveHD is optimized for synthesis and simulation. The main components of LiveHD +includes LGraph, LNAST, integrated 3rd-party tools, code generation, and "live" +techniques.
+A compilation goes through the following steps:
+Source code goes through a lexer/parser to create a parse tree or language specific AST.
+The parse tree or language specific AST is translated to LNAST. LNAST is a +AST-like representation that it is independent of the specific language. +Pyrope, CHISEL, and Verilog translate to LNAST.
+The are several passes on LNAST to infer the correct type and bitsizes. The goal is to +expand tuples, macros at LNAST level, but this code is still not finished.
+The LNAST is a tree-like representation which is translated to Lgraph. In a +way, LNAST is a HIR (High-level IR) and Lgraph is a LIR (Low-level IR). For +each Lgraph node, there is an equivalent LNAST, but not the opposite.
+LGraph has a hierarchical graph representation designed for fast synthesis +and simulation. It interfaces other tools like Yosys, ABC, OpenTimer, and +Mockturtle.
+For code generation, it is possible to translate back to LNAST or to directly +output from Lgraph.
+
This is a high level description of how to build LiveHD.
+Although LiveHD should run on most common Linux distributions, it is heavily tested on both Arch and Kali (Debian based).
+The following programs are assumed to be present when building LiveHD:
+It is also assumed that bash is used to compile LiveHD.
+gcc and clang offers better warnings and execution speed dependent of the benchmark.
+If you're unsure if your copy of gcc or clang is new enough, you can check the version by typing
+g++ --version
+or
+clang++ --version
+Download LiveHD source
+git clone https://github.com/masc-ucsc/livehd
+Install Bazelisk
+Bazelisk is a wrapper around bazel that allows you to use a specific version.
+If you do not have system permissions, you can install a local bazelisk
+npm install @bazel/bazelisk
+alias bazel=$(pwd)/node_modules/\@bazel/bazelisk/bazelisk.js
+You can also install it directly if you have administrative permissions:
+macos: +
brew install bazelisk.
+Linux: +
npm install -g @bazel/bazelisk
+go install github.com/bazelbuild/bazelisk@latest
+export PATH=$PATH:$(go env GOPATH)/bin
+Arch linux: +
pacaur -S bazelisk # or yay or paru installers
+Build LiveHD
+LiveHD has several build options, detailed below. All three should result in a working executable, but may differ in speed or output.
+A binary will be created in livehd/bazel-bin/main/lgshell.
bazel build //main:all # fast build, no debug symbols, slow execution (default)
+bazel build -copt //main:all # fastest execution speed, no debug symbols, no assertions
+bazel build -cdbg //main:all # moderate execution speed, debug symbols
+If you have multiple gcc versions, you may need to specify the latest. E.g:
+CXX=g++-8 CC=gcc-8 bazel build //main:all -c opt # fast execution for benchmarking
+CXX=g++-8 CC=gcc-8 bazel build //main:all -c dbg # debugging/development
+If you want to run clang specific version:
+CXX=clang++-10 CC=clang-10 bazel build //main:all -c dbg # debugging/development
+Make sure that the openJDK installed is compatible with bazel and has the certificates to use tools. E.g in debian:
+dpkg-reconfigure openjdk-11-jdk
+/var/lib/dpkg/ca-certificates-java.postinst configure
+If you fail to build for the first time, you may need to clear the cache under your home directory before rebuilding:
+rm -rf ~/.cache/bazel
+Make sure to have enough memory (4+GB at least)
+To start using LiveHD, check out Usage. If you're interested in working on LiveHD, refer to Creating a pass.
+ + + + + + +This is a high level description of how to use LiveHD.
+Below are some sample usages of the LiveHD shell (lgshell). A bash prompt is
+indicated by $, an lgshell prompt is indicated by livehd>, and a Yosys
+prompt is indicated by a yosys>. Lgshell supports color output and
+autocompletion using the tab key.
Currently, LiveHD can interface Verilog, FIRRTL, and Pyrope HDLs through +different front-end commands. After the parsing step, the HDL source can +be be transformed into LiveHD's internal intermediate representations, LNAST and +LGraph, and perform mid-end optimizations based on the two IRs, and generates +the optimized (or synthesized) Verilog code at the back-end.
+$ ./bazel-bin/main/lgshell
+livehd> help
+ ...
+livehd> help pass.sample
+livehd> exit
+The following uses Verilog as the example to demonstrate the compilations
+commands. It imports a Verilog file with a specified the database path,
+translates to the LNAST IR, lowers the LNAST IR to LGraph IR, executes some
+compiler optimizations, and generates the optimized Verilog code to the tmp
+directory. By default, a database called lgdb will be set in the livehd
+directory to store the internal representations, but users can optionally
+specify a prefered path.
livehd> inou.liveparse path:/your/path/lgdb_foo files:/your/path/bar.v |> inou.verilog |> pass.lnast_tolg |> pass.cprop |> pass.bitwidth |> inou.cgen.verilog odir:tmp
+A command lgraph.match can also be used to specify a (sub)hierarchy to operate
+over, which can then be moved from pass to pass using the pipe (|>) operator.
When Verilog file(s) are compiled through a series of commands in lgshell, if a +problem occurs while compiling Verilog files (due to a syntax error, use of +un-synthesizable Verilog, or something else), the corresponding error will be +printed. Once a hierarchy has been created, other lgshell commands can read, +modify, or export this hierarchy freely.
+The Pyrope compilation flow is similar to the Verilog commands except the
+front-end Pyrope parser pass inou.pyrope
livehd> inou.pyrope path:/your/path/lgdb_foo files:/your/path/bar.prp |> pass.lnast_tolg |> pass.cprop |> pass.bitwidth |> inou.cgen.verilog odir:tmp
+Additionally, users can compile a Pyrope code with a mid-end command of
+pass.compiler which integrates standard compilation passes in LiveHD such as (1)
+pass.lnast_tolg for IR lowering, (2) pass.cprop for legacy compiler
+optimizations such as copy and constant propagation, peep-hole optimization, deadcode elimination, and high-level data-structure resolving (3) pass.bitwidth for circuit bitwidth optimization.
livehd> inou.pyrope path:/your/path/lgdb_foo files:/your/path/bar.prp |> pass.compiler |> inou.cgen.verilog odir:tmp
+LiveHD compiles FIRRTL code with the protocal buffer format. Users can reference +this doc +to generate the protocol buffers file from Chisel/FIRRTL compiler.
+The LiveHD FIRRTL compiler uses a integrated mid-end commands as explained in
+the previous Pyrope example to compiles the FIRRTL HDL. Set gviz option to
+true to automatically generate the visiual Graphviz for individual steps. Set
+hier option to true for hierarchical Chisel design in most cases. Specify the
+top module name with the top option.
livehd> inou.firrtl.tolnast path:/your/path/lgdb_foo files:/your/path/bar.pb |> pass.compiler gviz:false top:top_module_name hier:true |> inou.cgen.verilog odir:tmp
+To display the content of the LNAST IR after parse (Pyrope as the example)
+inou.pyrope files:foo.prp |> lnast.dump
To display the content of the LGraph IR (Pyrope as the example)
+inou.pyrope files:foo.prp |> pass.lnast_tolg |> lgraph.dump
To serialize the content of the LGraph IR (Pyrope as the example), default path is lgdb
+inou.pyrope files:foo.prp |> pass.lnast_tolg |> lgraph.save
To deserialize a stored LGraph, and pass it to some lgraph passes
+lgraph.open name:foo |> pass.cprop |> pass.bitwidth |> inou.cgen.verilog odir:tmp
To display the content of the LGraph IR (Pyrope after the cprop pass as the example)
+inou.pyrope files:foo.prp |> pass.lnast_tolg |> pass.cprop |> inou.graphviz.from
$ ./bazel-bin/main/lgshell
+livehd> inou.liveparse files:./inou/yosys/tests/trivial.v |> inou.verilog
+livehd> lgraph.match |> lgraph.stats
+livehd> lgraph.match |> lgraph.dump
+lgraph.match picks up any LGraphs matching the regex passed (or everything if no regex is provided) and treats every single one as the top of the hierarchy, whereas lgraph.open name:<root module> will just open the root module as the top of the hierarchy.$ ./bazel-bin/main/lgshell
+livehd> inou.pyrope files:./inou/pyrope/tests/if1.prp
+livehd> lgraph.match |> <pass name>
+$ bazel build -c dbg //pass/sample:pass_sample
+$ bazel build -c dbg //inou/yosys:all
+$ bazel build -c dbg //inou/yosys:all
+$./bazel-bin/inou/yosys/yosys2
+LiveHD support parallel compilation, it will set the thread number to the
+highest available resources in your system. To manually set the compilation thread
+number, for example, 1, set the following environment variable
+
export LIVEHD_THREADS=1
+This section explains the memory management or garbage collect principles used +and the relationship with concurrency models.
+Memory management in a multi-threaded environment main challenge arises from +deletes and additions that can trigger memory allocation/de-allocation. In +LiveHD, we address this problem the following way:
+Only one thread can allocate/deallocate memory in an object at a given time. + To allow updates, a RW access is required. Otherwise, a RD access is enough. + Both return a std::unique_ptr. The API guarantees that only one thread can do + RW-access for a given object. Notice that the calling thread can have + multiple RD access and WR access to the same object simulnateusly. The check + is only against "other threads". There are 2 APIs:
+ref_rd_snapshot(): gets a RD access, and an assertion checks that + there is no other thread has wr_snapshot during the lifetime of the + returned std::unique_ptr
+ref_wr_snapshot(): gets a RW access, and an assertion checks that + there is no other thread has rd_snapshot during the lifetime of the + returned std::unique_ptr.
+Note
+Updating an atomic counter inside an object does not require a RW-access, +but a rd_snapshot because it does not trigger memory +allocation/deallocation. Adding/deleting elements requires a wr_snapshot.
+Creating an object requires a ref_wr_snapshot.
+The only way to pass references across threads is with std::unique_ptr or + calling to the library which will create a std::unique_ptr.
+Note
+If a std::unique_ptr created from a snapshot is passed to another thread, +the creator thread is still the owner of the thread pointer. If this is not +the intention, it may be safer for the new thread to call the library to +access ownership.
+This approach is somewhat similar to a hazzard pointer. The snapshot API +indicates intention to modify (which can delete), but instead of failing with a +nullptr return, we trigger a compile failure because it should never be the +case. Each lgraph/lnast can be updated in parallel, but only if they are +independent. The assertion is to check that there is no bug.
+LiveHD uses 3 main techniques to perform memory management.
+When an object is created and there is a single user, the code should use RAII +or std::unique_ptr. RAII means that when the object is out of scope, the memory +is recycled. std::unique_ptr will automatically call the destructor when the +reference use is zero.
+For objects that can be shared across threads, the snapshot API must be used.
+std::shared_ptr is one of the "heavy" cost options for garbage collection. In +LiveHD, we do not use the atomic std::shared_ptr. The std::shared_ptr are NOT +allowed to be passed between threads. It is a memory management for data only +within a thread or compiler pass.
+To avoid reference counting overheads, when passed to methods, a const
+std::shared<XX> & should be used nearly in all the cases.
LiveHD is built on C++17, LGraph and LNAST are the two key data structures
+inside LiveHD to support new hardware design.
+* LGraph stands for Live Graph. It is graph or netlist data
+ structure at the core of LiveHD.
+* LNAST stands for Language Neutral AST. It is an Abstract
+ Syntax Tree (AST) designed to be simple but to allow the translation from
+ multiple languages like CHIRRTL, Verilog, and Pyrope.
While LNAST could be seen as a high level API with control flow information, +LGraph is a lower level graph API where many LNAST high level constructs are +simplified.
+There is a division of functionality between LNAST and LGraph:
+LNAST: Language Neutral AST, the high level tree based representation/API
+for and while loops LGraph: Live Graph, the low level graph/netlist level based representation/API
+Warning
+LiveHD is beta under active development and we keep improving the +API. Semantic versioning is a 0.+, significant API changes are expect.
+The LGraph is built directly through LNAST to LGraph translations. The LNAST +builds a gated-SSA which is translated to LGraph. Understanding the LGraph is +needed if you want to build a LiveHD pass.
+LGraph is a graph or netlist where each vertex is called a node, and it has a +cell type and a set of input/output pins.
+A single LGraph represents a single netlist module. LGraph is composed of
+nodes, node pins, edges, cell types, and tables of attributes. An LGraph node
+is affiliated with a cell node type and each type defines different amounts of input
+and output node pins. For example, a node can have 3 input ports and 2 output
+pins. Each of the input/output pins can have many edges to other graph nodes.
+Every node pin has an affiliated node pid. In the code, every node_pin has a
+Port_ID.
A pair of driver pin and sink pin constitutes an edge. The bitwidth of the +driver pin determines the edge bitwidth.
+new_node = lg->create_node()
+//note: type and/or bits still need to be assigned later
+new_node = lg->create_node(Node_type_Op)
+//note: recommended way if you know the target node type
+new_node = lg->create_node_const(value)
+//note: recommended way to create a const node
+driver_pin = new_node.setup_driver_pin();
+//note: every cell in LGraph has only one driver pin, pin0
+sink_pin = new_node.setup_sink_pin()
+//note: when you know the node type only has one input pin
+setup sink pin for pin_x of a node, for more information, please refer to the + Cell type section. For quick reference of the sink pin names of each cell + type, please see + cell.cpp +
sink_pin = new_node.setup_sink_pin("some_name")
+add an edge between driver_pin and sink_pin
+driver_pin.connect(sink_pin);
+driver_node = edge.driver.get_node()
+absl::flat_hash_map<Node::Compact, int> my_map;
+my_map[node1.get_compact()] = 77;
+my_map[node2.get_compact()] = 42;
+...
+absl::flat_hash_map<Node_pin::Compact, int> my_map;
+my_map[node_pin1.get_compact()] = 14;
+my_map[node_pin2.get_compact()] = 58;
+...
+Node_pin dpin(lg, some_dpin.get_compact())
+Node node(lg, some_node.get_compact())
+new_node_pin = lg->add_graph_input(std::string_view)
+node.debug_name()
+node_pin.debug_name()
+for (auto &out : node.out_edges()) {
+ auto dpin = out.driver;
+ auto dpin_pid = dpin.get_pid();
+ auto dnode_name = dpin.get_node().debug_name();
+ auto snode_name = out.sink.get_node().debug_name();
+ auto spin_pid = out.sink.get_pid();
+ auto dpin_name = dpin.has_name() ? dpin.get_name() : "";
+ auto dbits = dpin.get_bits();
+
+ fmt::print(" {}->{}[label=\"{}b :{} :{} :{}\"];\n"
+ , dnode_name, snode_name, dbits, dpin_pid, spin_pid, dpin_name);
+}
+LGraph allows forward and backward traversals in the nodes (bidirectional +graph). The reason is that some algorithms need a forward and some a backward +traversal, being bidirectional would help. Whenever possible, the fast iterator +should be used.
+for (const auto &node:lg->fast()) {...} // unordered but very fast traversal
+
+for (const auto &node:lg->forward()) {...} // propagates forward from each input/constant
+
+for (const auto &node:lg->backward()) {...} // propagates backward from each output
+The LGraph iterator such as for(auto node: g->forward()) do not visit graph
+input and outputs.
// simple way using lambda
+lg->each_graph_input([&](const Node_pin &pin){
+
+ //your operation with graph_input node_pin;
+
+});
+LGraph supports hierarchical traversal. Each sub-module of a hierarchical +design will be transformed into a new LGraph and represented as a sub-graph node +in the parent module. If the hierarchical traversal is used, every time the +iterator encounters a sub-graph node, it will load the sub-graph persistent +tables to the memory and traverse the subgraph recursively, ignoring the +sub-graph input/outputs. This cross-module traversal treats the hierarchical +netlist just like a flattened design. In this way, all integrated third-party +tools could automatically achieve global design optimization or analysis by +leveraging the LGraph hierarchical traversal feature.
+for (const auto &node:lg->forward(true)) {...}
+To iterate over the input edges of node, simply call:
+for (const auto &inp_edge : node.inp_edges()) {...}
+And for output edges:
+for (const auto &out_edge : node.out_edges()) {...}
+Design attribute stands for the characteristic given to a LGraph node or node +pin. For instance, the characteristic of a node name and node physical +placement. Despite a single LGraph stands for a particular module, it could be +instantiated multiple times. In this case, same module could have different +attribute at different hierarchy of the netlist. A good design of attribute +structure should be able to represent both non-hierarchical and hierarchical +characteristic.
+Non-hierarchical LGraph attributes include pin name, node name and line of +source code. Such properties should be the same across different LGraph +instantia- tions. Two instantiations of the same LGraph module will have the +exact same user-defined node name on every node. For example, instantiations of +a subgraph-2 in both top and subgraph-1 would maintain the same non-hierarchical +attribute table.
+node.set_name(std::string_view name);
+LGraph also support hierarchical attribute. It is achieved by using a tree data +structure to record the design hierarchy. In LGraph, every graph has a unique +id (lg_id), every instantiation of a graph would form some nodes in the tree and +every tree node is indexed by a unique hierarchical id (hid). We are able to +identify a unique instantiation of a graph and generate its own hierarchical +attribute table. An example of hierarchical attribute is wire-delay.
+node_pin.set_delay(float delay);
+For each LGraph node, there is a specific cell type. This section explains the +operation to perform for each node. It includes a precise way to compute the +maximum and minimum value for the output.
+In LGraph, the cell types operate like having unlimited precision with signed +numbers. Most HDL IRs have a type for signed inputs and another for unsigned. +LiveHD handles the superset (sign and unlimited precision) with a single node. +In LGraph, an unsigned value is signed value that is always positive. This +simplifies the mixing and conversions which simplifies the passes. The drawback +is that the export may have to convert back to signed/unsigned for some +languages like Verilog.
+Maybe even more important is that all the LGraph cell types generate the same +result if the input is sign-extended. This has implications, for example a +typical HDL IR type like "concat" does not exist because the result is +dependent on the inputs size. This has the advantage of simplifying the +decisions of when to drop bits in a value. It also makes it easier to guarantee +no loss of precision. Any drop of precision requires explicit handling with +operations like and-gate with masks or Shifts.
+The document also explains corner cases in relationship to Verilog and how to +convert to/from Verilog semantics. These are corner cases to deal with sign and +precision. Each HDL may have different semantics, the Verilog is to showcase +the specifics because it is a popular HDL.
+All the cell types are in core/cell.hpp and core/cell.cpp. The type
+enumerate is called Ntype. In general the nodes have a single output with the
+exception of complex nodes like subgraphs or memories. The inputs is a string in
+lower case or upper case. Upper case ('A') means that many edges (or output
+drivers) can connect to the same node input or sink pin, lower case ('a') means
+that only a driver can connect to the input or sink pin.
Each cell type can be called directly with Pyrope using a low level RTL syntax. +This is useful for debugging not for general use as it can result in less +efficient LNAST code.
+An example of a multi-driver sink pin is the Sum cell which can do Y=3+20+a0+a3
+where A_{0} = 3, A_{1} = 20, A_{2} = a0, and A_{3} = a3. Another way to
+represent in valid Pyrope RTL syntax is:
Y = __sum(A=(3,20,a0,a3))
+An example if single driver sink pin is the SRA cell which can do Y=20>>3.
+It is lower case because only one driver pin can connect to 'a' and 'b'. Another way
+to represent a valid Pyrope RTL syntax is:
Y = __sra(a=20,b=3)
+The section includes description on how to compute the maximum (max) and
+minimum (min) allowed result range. This is used by the bitwidth inference
+pass. To ease the explanation, a sign value means that the result may be
+negative (a.sign == a.min<0). known is true if the result sign is known
+(a.known == a.max<0 or a.min>=0), either positive or negative (neg ==
+a.max<0). The cells explanation also requires the to compute the bit mask
+(a.mask == (1<<a.bits)-1).
For any value (a), the number of bits required (bits) is a.bits = log2(absmax(a.max,a.min))+1.
Addition and substraction node is a single cell Ntype that performs +2-complement additions and substractions with unlimited precision.
+If the inputs do not have the same size, they are sign extended to all have the +same length.
+Forward Propagation
+%Y = A.reduce('+') - B.reduce('+')
+%max = 0
+%min = 0
+for a in A {
+ %max += A.max
+ %min += A.min
+}
+for b in B {
+ %max -= b.min
+ %min -= b.max
+}
+Backward Propagation
+Backward propagation is possible when all the inputs but ONE are known. The +algorithm can check and look for the inputs that have more precision than +needed and reduce the max/min backwards.
+For example, if and all the inputs but one A are known (max/min has the max/min +computed for all the inputs but the unknown one)
+A_{unknown}.max = Y.max - max
+A_{unknown}.min = Y.min - min
+If the unknow is in port B:
B_{unknown}.max = min - T.min
+B_{unknown}.min = max - Y.max
+Verilog Considerations
+In Verilog, the addition is unsigned if any of the inputs is unsigned. If any +input is unsigned. all the inputs will be "unsigned extended" to match the +largest value. This is different from Sum_Op semantics were each input is +signed or unsigned extended independent of the other inputs. To match the +semantics, when mixing signed and unsigned, all the potentially negative inputs +must be converted to unsign with the Ntype_op::Tposs.
+logic signed [3:0] a = -1
+logic signed [4:0] c;
+
+assign c = a + 1'b1;
+The previous Verilog example extends everything to 5 bits (c) UNSIGNED extended +because one of the inputs is unsigned (1b1 is unsigned in verilog, and 2sb1 is +signed +1). LGraph semantics are different, everything is signed.
+c = 5b01111 + 5b0001 // this is the Verilog semantics by matching size
+c == -16 (!!)
+The Verilog addition/substraction output can have more bits than the inputs.
+This is the same as in LGraph Sum. Nevertheless, Verilog requires to specify
+the bits for all the input/outputs. This means that whenever Verilog drops
+precision an AND gate must be added (or a SEXT for signed output). In the
+following examples only the 'g' and 'h' variables needed.
wire [7:0] a;
+ wire [7:0] b;
+ wire [6:0] c;
+ wire [8:0] f = a + b; // f = __sum(a,b) // a same size as b
+ wire [8:0] f = a + c; // f = __sum(a,__get_mask(c,-1))
+ wire [7:0] g = a + b; // g = __and(__sum(a,b),0x7F)
+ wire [6:0] h = a + b; // h = __and(__sum(a,b),0x3F)
+Peephole Optimizations
+Y = x-0+0+... becomes Y = x+...Y = x-x+... becomes Y = ...Y = x+x+... becomes Y = (x<<1)+...Y = (x<<n)+(y<<m) where m>n becomes Y = (x+y<<(m-n)<<nY = (~x)+1+... becomes Y = ...-xY = a + (b<<n) becomes Y = {(a>>n)+b, a&n.mask}Y = a - (b<<n) becomes Y = {(a>>n)-b, a&n.mask}Y=x+y+... becomes Y=((x>>1)+(y>>1)+..)<<1Multiply operator. There is no cell type that combines multiplication and
+division because unlike in Sum. The reason is that with integers the order of multiplication/division changes
+the result even with unlimited precision integers (a*(b/c) != (a*b)/c).
Forward Propagation
+Y = A.reduce('*')
+var tmax = 1
+vat tmin = 1
+var sign = 0
+for i in A {
+ tmax *= maxabs(A.max, A.min)
+ tmin *= minabs(A.max, A.min)
+ known = false when min<0 and max>0
+ sign += 1 when max<0
+}
+if know { // sign is know
+ if sign & 1 { // negative
+ %max = -tmin
+ %min = -tmax
+ }else{
+ %max = tmax
+ %min = tmin
+ }
+}else{
+ %max = tmax
+ %min = -tmax
+}
+Backward Propagation
+If only one input is missing, it is possible to infer the max/min from the
+output and the other inputs. Like in the sum case, if all the inputs but one
+and the output is known, it is possible to backward propagate to further
+constraint the unknown input.
A_{unknown}.max = Y.max / A.min
+A_{unknown}.min = Y.min / A.max
+Verilog Considerations
+Unlike the Sum, the Verilog 2 LiveHD translation does not need to extend the
+inputs to have matching sizes. Multiplying/dividing signed and unsigned numbers
+has the same result. The bit representation is the same if the result was
+signed or unsigned.
LiveHD mult node result (Y) number of bits can be more efficient than in
+Verilog. E.g: if the max value of A0 is 3 (2 bits) and A1 is 5 (3bits). If the
+result is unsigned, the maximum result is 15 (4 bits). In Verilog, the result
+will always be 5 bits. If the Verilog result was to an unsigned variable.
+Either all the inputs were unsigned, or there should pass to an get_mask to
+force the MSB as positive. This extra bit will be simplified but it will notify
+LGraph that the output is to be treated as unsigned.
Peephole Optimizations
+Y = a*1*... becomes Y=a*...Y = a*0*... becomes Y=0Y = power2a*... becomes Y=(...)<<log2(power2a)Y = (power2a+power2b)*... becomes tmp=... ; Y = (tmp+tmp<<power2b)<<(power2a-power2b) when power2a>power2bY = (power2a-power2b)*... becomes tmp=... ; Y = (tmp-tmp<<power2b)<<(power2a-power2b) when power2a>power2bDivision operator. The division operation is quite similar to the inverse of +the multiplication, but a key difference is that only one driver is allowed for +each input ('a' vs 'A').
+Forward Propagation
+Y = a/b
+%max = a.max/b.min
+%min = a.min/b.max
+
+for i in a.max,a.min {
+ for j in b.max,b.min {
+ next when j == 0
+ tmp = i / j
+ %max = tmp when tmp > max
+ %min = tmp when tmp < min
+ }
+}
+Backward Propagation
+The backward propagation from the division can extracted from the forward +propagation. It is a simpler case of multiplication backward propagation.
+Verilog Considerations
+The same considerations as in the multiplication should be applied.
+Peephole Optimizations
+Y = a/1 becomes Y=aY = 0/b becomes Y=0Y = a/power2b becomes Y=a>>log2(power2b) if Y.known and !Y.negY = a/power2b becomes Y=1+~(a>>log2(power2b)) if Y.known and Y.negY = (x*c)/a if c.bits>a.bits becomes Y = x * (c/a) which should be a smaller division.Y.known and !Y.neg. From the hackers delight, weY=(a*(((1<<(a.bits+2)))/b+1))>>(a.bits+2) If a sign is not known. Then `YThere is no mod cell (Ntype_op::Mod) in LGraph. The reason is that a modulo +different from a power of 2 is very rare in hardware. If the language supports +modulo operations, they must be translated to division/multiplication.
+y = a mod b
+It is the same as:
+y = a-b*(a/b)
+If b is a power of 2, the division optimization will transform the modulo operation to:
+y = a - (a>>n)<<n
+The add optimization should reduce it to:
+y = a & n.mask
+Bitwise Not operator
+Forward Propagation
+Y = ~a
+%max = max(~a.max,~a.min)
+%min = min(~a.max,~a.min)
+Backward Propagation
+a.max = max(~Y.max,~Y.min)
+a.min = min(~Y.max,~Y.min)
+Verilog Considerations*
+Same semantics as verilog
+Peephole Optimizations
+No optimizations by itself, it has a single input. Other operations like Sum_Op can optimize when combined with Not_Op.
+And is a typical AND gate with multiple inputs. All the inputs connect to pin
+'A' because input order does not matter. The result is always a signed number.
digraph And {
+ rankdir=LR;
+ size="1,0.5"
+
+ node [shape = circle]; And;
+ node [shape = point ]; q0
+ node [shape = point ]; q
+
+ q0 -> And [ label ="A" ];
+ And -> q [ label = "Y" ];
+}
+The And cell has a significant backpropagation impact. Even if some inputs had +more bits, after the And cell the upper bits are dropped. This allows the back +propagation to indicate that those bits are useless.
+LT, GT, EQ
+There are only 3 comparators. Other typically found like LE, GE, and NE can be
+created by simply negating one of the LGraph comparators. GT = ~LE, LT = ~GE, and NE = ~EQ.
Y = A LT B
Y = A0 LT B and A1 LT B
Y = A0 LT B0 and A1 LT B0 and A0 LT B1 and A1 LT B1
Verilog treats all the inputs as unsigned if any of them is unsigned. LGraph treats all the inputs as signed all the time.
+| size | +A | +B | +Operation | +
|---|---|---|---|
| a==b | +S | +S | +EQ(a,b) | +
| a==b | +S | +U | +EQ(a,b) | +
| a==b | +U | +S | +EQ(a,b) | +
| a==b | +U | +U | +EQ(a,b) | +
| a< b | +S | +S | +LT(a,b) | +
| a< b | +S | +U | +LT(a,Tposs(b)) | +
| a< b | +U | +S | +LT(Tposs(a),b) | +
| a< b | +U | +U | +LT(Tposs(a),Tposs(b)) | +
Shift Left performs the typical shift left when there is a single amount
+(a<<amt). The allow supports multiple left shift amounts. In this case the
+shift left is used to build one hot encoding mask. (1<<(1,2) == (1<<1)|(1<<2))
The result for when there are not amounts (a<<()) is -1. Notice that this
+is not ZERO but -1. The -1 means that all the bits are set. The reason is that
+when there are no offsets in the onehot encoding, the default functionality is
+to select all the bit masks, and hence -1.
Logical or sign extension shift right.
+Verilog has 2 types of shift >> and >>>. The first is unsigned right shift,
+the 2nd is arithmetic right shift. LGraph only has arithmetic right shift
+(ShiftRigt_op). The verilog translation should make the value unsigned
+(ShiftRigt(Join(0,a),b)) before calling the shift operation. Conversely, for
+a >>> if the input is Verilog unsigned (ShiftRigt(a,b))
reduce AND a =u= -1 // unsigned equal
reduce OR a != 0
reduce xor is a chain of XORs.
+Inputs - a, mask +Get_mask (a, mask) +Functionality - Output contains only those bits a[i], for which mask[i] = 1, other bits a[i] for which mask[i] = 0, are dropped. +a & mask are interpreted as signed numbers and sign extended to the size of the other, if required. +eg - Get_mask (0sb11000011, 0sb10101010) = 0sb1001 + Get_mask (0sb11110000, 0sb00001111) = 0sb0000 + Get_mask (0sb0011, 0sb10) = 0sb001 + Get_mask (0sb10, 0sb1010) = 0sb11
+Inputs - a, value, mask +Set_mask(a, mask, value) +Functionality - Replaces all bits a[i] for which mask[i] = 1, with value[i] +Retains all bits a[i] for which mask[i] = 0. +// Check - if a, value are signed, actually none of them should be extended and their signs should not matter, but a might need to retain it's sign +eg - Set_mask (0b101 01 010, 0sb000 11 000, 0b001 10 011) = 0sb 101 10 010
+Inputs - a, b +Sext (a, b) +Selects only bits a[b:0] dropping all remaining MSBs. +The selected a[b:0] is interpretded as a signed value, a's sign does not matter,b conyains the MSB index and hence is always unsigned/ positive +eg Sext (0b10101010, 4) = 0sb01010 = 0xA = +10 +Sext (0b10101010, 5) = 0sb101010 = 0x2A = -22
+Memory is the basic block to represent SRAM-like structures. Any large storage will benefit from using memory arrays instead of flops, which are slower to simulate. These memories are highly configurable.
+digraph Memory {
+ rankdir=LR;
+ size="2,1"
+
+ node [shape = circle]; Memory;
+ node [shape = point ]; q0
+ node [shape = point ]; q1
+ node [shape = point ]; q2
+ node [shape = point ]; q3
+ node [shape = point ]; q4
+ node [shape = point ]; q5
+ node [shape = point ]; q6
+ node [shape = point ]; q7
+ node [shape = point ]; q8
+ node [shape = point ]; q9
+ node [shape = point ]; q10
+ node [shape = point ]; q
+
+ q0 -> Memory [ label ="a (addr)" ];
+ q1 -> Memory [ label ="b (bits)" ];
+ q2 -> Memory [ label ="c (clock)" ];
+ q3 -> Memory [ label ="d (data in)" ];
+ q4 -> Memory [ label ="e (enable)" ];
+ q5 -> Memory [ label ="f (fwd)" ];
+ q6 -> Memory [ label ="l (latency)" ];
+ q7 -> Memory [ label ="m (wmask)" ];
+ q8 -> Memory [ label ="p (posedge)" ];
+ q9 -> Memory [ label ="s (size)" ];
+ q10 -> Memory [ label ="w (wmode)" ];
+ Memory -> q [ label ="Q (data out)" ];
+}
+s (size) is for the array size in number of entriesb (bits) is the number of bits per entryf (fwd) points to a 0/1 constant driver pin to indicate if writes forward value (0b0 for write-only ports). Effectively, it means zero cycles read latency when enabled. fwd is more than just setting latency=0. Even with latency zero, the write delay affects until the result is visible. With fwd enabled, the write latency does not matter to observe the results. This requires a costly forwarding logic.c,d,e,q... are the memory configuration, data, address portsPorts (a,c...p,w) are arrays/vectors to support multiported memories. If a single instance
+exists in a port, the same is used across all the ports. E.g: if clock (c) is populated:
mem1.c = clk1 // clk for all the memory ports
+
+mem2.c[0] = clk1 // clock for memory port 0
+mem2.c[1] = clk2 // clock for memory port 1
+mem2.c[2] = clk2 // clock for memory port 2
+Each memory port (rd, wr, or rd/wr) has the following ports:
+a (addr) points to the driver pin for the address. The address bits should match the array size (ceil(log2(s)))c (clock) points to the clock driver pind (data_in) points to the write data driver pin (read result is in q port).e (enable) points to the driver pin for read/write enable.l (latency) points to an integer constant driver pin (2 bits always). For writes latency from 1 to 3, for reads latency from 0 to 3w (wmask) Points to the write mask (1 == write, 0==no write). The mask bust be a big as the number of bits per entry (b). The wmask pin can be disconnected which means no write mask (a write will write all the bits).p (posedge) points to a 1/0 constant driver pinm (mode) points to the driver pin or switching between read (0) and write mode (1) (single bit)Q (data_out) is a driver pin with the data read from the memoryAll the entries but the wmask must be populated. If the wmask is not set, a
+full write size is expected. Read-only ports do not have data and wmask
+fields if the write use the low ports (0,1...). By placing the read-only ports
+to the high numbers, we can avoid populating the wmask (m) and data out (q)
+ports. If the read ports use low port numbers those fields must be populated to
+allow the correct matching between write port (a[n]) and write result
+(q[n]).
All the ports must be populated with the correct size. This is important
+because some modules access the field by bit position.
+If it is not used, it will point to a zero constant with the correct number of bits.
+The exception to this is wmask which, if b indicates 8 bits per entry,
+will be equivalent to 0xFF. Setting wmask to 0b1 will mean a 1 bit zero,
+and the memory will be incorrectly operated.
The memory usually has power of two sizes. If the size is not a power of 2, the +address is rounded up. Writes to the invalid addresses will generated random +memory updates. Reads should read random data.
+And_Op: bitwise AND with 2 outputs single bit reduction (RED) or bitwise +Y = VAL&..&VAL ; RED= &Y
+The sign can not be backward propagated because Pick_Op removes the sign no matter the input sign.
+Not all the nodes have the same complexity overhead. When performing peephole +optimization is possible to trade one set of nodes for others. In general, +we have this set of overheads:
+0 overhead: not, get_mask, set_mask, sext, and SHL/SRA with constant shift + amounts. The rational is that those are just "wiring" cells to connect or + extract wires across. The NOT gate is not really zero, but it could be easily + mixed with sorrounding cells.
+1 overhead: And, Or, Xor, LUT, Mux
+3 overhead: LT, GT, EQ, Ror
+4 overhead: Less than 4 bit output Sum, and SHL/SRA with non-compile time + shift amount. This can be costly an require hardware like barrel shifters.
+5 overhead: large Sum, SHL/SRA.
+6 Overhead: Mult/Div
+If a overhead level can be elininated having a small number of different cells +with a smaller overhead level,the translation makes sense. Notice the "small +number of cells", after all everything can be translated to nand gates. A 3x +factor is somewhat reasonable. This means that a 5-level overhead is fine to be +replaced for 3 4-level (or 3 3-level) but not for 4 4-level overhead. Zero +overhead cells are not included in the list of cells in the replacement.
+This is a heuristic. Once works, it is a nice target to use AI to decide +when/if a transformation is worth.
+ + + + + + +LNAST stands for Language-Neutral Abstract Syntax Tree, which is constituted of +Lnast_nodes and indexed by a tree structure.
+LiveHD has two main data structures: LNAST and LGraph. The LNAST is the higher +level representation with a tree structure. The LGraph is the lower level +representation with a graph structure. Each node in LGraph has a LNAST +equivalent node, but LNAST is more high level and several nodes in LNAST may +not have a one-to-one mapping to LGraph.
+Each Lnast_node should has a specific node type and contain the following information from source code tokens
+(a) line number
+(b) pos_start, pos_end
+(c) string_view (optional)
Every node construction method has four function overloadings.
+For example, to construct a Lnast_node with a type of reference,
+we could use one of the following functions:
// C++
+auto node_ref = Lnast_node::create_ref("foo");
+auto node_ref = Lnast_node::create_ref("foo", line_num);
+auto node_ref = Lnast_node::create_ref("foo", line_num, pos1, pos2);
+auto node_ref = Lnast_node::create_ref(token);
+In case (1), you only knows the variable name is "foo".
+In case (2), you know the variable name and the corresponding line number.
+In case (3), you know the variable name, the line number, and the charactrer position.
+In case (4), you are building LNAST from your HDL AST and you already have the Token.
+The toke should have line number, positions, and string_view information.
If you don't care the string_view to be stored in the lnast node, just leave it empty for set "foo" for it. +This is true for many of the operator node, for example, to build a node with type of assign.
+// C++
+auto node_assign = Lnast_node::create_assign();
+auto node_assign = Lnast_node::create_assign(line_num);
+auto node_assign = Lnast_node::create_assign(line_num, pos1, pos2);
+auto node_assign = Lnast_node::create_assign(token); // The token is not necessary to have a string_view
+| + | + | + | + | + |
|---|---|---|---|---|
top |
+stmts |
+if |
+uif |
+for |
+
func_call |
+func_def |
+assign |
+dp_assign |
+mut |
+
bit_and |
+bit_or |
+bit_not |
+bit_xor |
+reduce_or |
+
logical_and |
+logical_or |
+logical_not |
+plus |
+minus |
+
mult |
+div |
+mod |
+shl |
+sra |
+
sext |
+set_mask |
+get_mask |
+mask_and |
+mask_popcount |
+
mask_xor |
+is |
+ne |
+eq |
+lt |
+
le |
+gt |
+ge |
+ref |
+const |
+
range |
+tuple_concat |
+tuple_add |
+tuple_get |
+tuple_set |
+
attr_set |
+attr_get |
+err_flag |
+phi |
+hot_phi |
+
invalid |
++ | + | + | + |
top¶Every LNAST has a top node as the root. A top node has one or more child
+nodes, which can only be stmts.
<top> --| <stmts>
+ | <stmts>
+ | <stmts>
+ | ...
+stmts¶A stmts node represents a sequence of statements.
<stmts> --| <const> : scope name
+ | <assign>
+ | <plus>
+ | <func_def>
+ | ...
+if¶An if node represents a conditional branch, which can be a statement or an
+expression.
<if> --| <ref/const> : if condition variable
+ | <stmts> : if branch
+ | <ref/const> : elif condition variable \ N times
+ | <stmts> : elif branch /
+ | <stmts> : else branch
+uif¶Unique if. Similar to if, but add additional assertions to check if at most one condition
+is true.
<uif> --| <ref/const> : if condition variable
+ | <stmts> : if branch
+ | <ref/const> : elif condition variable \ N times
+ | <stmts> : elif branch /
+ | <stmts> : else branch
+for¶A for node represents a for-loop over a range or tuple. Note that the loop
+must be unrolled during compilation.
<for> --| <ref> : iterator variable
+ | <ref> : iterated variable (tuple or range)
+ | <stmts> : for-loop body
+func_def¶A func_def node represents a functional block with input/output arguments.
<func_def> --| <ref/const> : input arguments
+ | <ref/const> : output arguments
+ | <stmts> : function body
+func_call¶A func_call node represents an instantiation of a functional block.
<func_call> --| <ref/const> : Lvalue
+ | <ref> : function reference
+ | <ref/const> : input arguments
+assign¶An assign node represents a variable assignment. Note that the Rvalue can only
+be a const or ref.
<assign> --| <ref> : Lvalue
+ | <ref/const> : Rvalue
+dp_assign¶the "lhs := rhs" assignment (dp_assign) is like the "=" assignment but there is no check +for overflow. If the rhs has more bits than the lhs, the upper bits will be +dropped.
+<dp_assign> --| <ref> : Lvalue
+ | <ref/const> : Rvalue
+const¶Constant value.
+<const> "0x1234"
+ref¶Variable.
+<ref> "variable_name"
+range¶Range.
+<range> --| <ref> or <const> : from-value
+ | <ref> or <const> : to-value
+<op> --| <ref> : Lvalue
+ | <ref/const> : Rvalue
+bit_not¶Bitwise not. Flip all Rvalue bits.
+reduce_or¶Or all Lvalue bits.
+logical_not¶Logical Not. Flip Rvalue where Rvalue must be a boolean.
+<op> --| <ref> : Lvalue
+ | <ref/const> : R-1
+ | <ref/const> : R-2
+mod¶Modulo of R-1 over R-2.
+shl¶Left-shift R-1 by R-2.
+sra¶Right-shift R-1 by R-2.
+ne¶Not equal to.
+eq¶Equal to.
+lt¶Less than.
+le¶Less than or equal to.
+gt¶Greater than.
+ge¶Greater than or equal to.
+<op> --| <ref> : Lvalue
+ | <ref/const> : R-1 \
+ | <ref/const> : R-2 \
+ | <ref/const> : R-3 2 or more values
+ | ... /
+ | <ref/const> : R-N /
+bit_and¶Bitwise and.
+bit_or¶Bitwise or.
+bit_xor¶Bitwise xor.
+plus¶Summation of R-1 to R-N.
+minus¶R-1 minus summation of R-2 to R-N.
+mult¶Product of R-1 to R-N.
+div¶R-1 divided by product of R-2 to R-N
+tuple_concat¶<tuple_concat> --| <ref> : Lvalue
+ | <ref> : R-1 (tuple)
+ | <ref> : R-2 (tuple)
+ | ...
+ | <ref> : R-N (tuple)
+tuple_add¶<tuple_add> --| <ref> : Lvalue
+ | <ref/const>
+ | <assign> --| <ref> \ Field 0
+ | <ref/const> /
+ | <assign> --| <ref> \ Field 1
+ | <ref/const> /
+ | ...
+ | <assign> --| <ref> \ Field N
+ | <ref/const> /
+tuple_set¶<tuple_set> --| <ref> : Lvalue
+ | <ref/<const> : 1st-level selection \
+ | ... 1..N selections
+ | <ref/<const> : Nth-level selection /
+ | <ref/<const> : Rvalue
+tuple_get¶<tuple_get> --| <ref> : Lvalue
+ | <ref> : Rvalue (selected from this value)
+ | <ref/const> : 1st-level selection \
+ | ... 1..N selections
+ | <ref/const> : Nth-level selection /
+In LNAST, all input/output/register are defined in the node type reference +with differenct prefix of string_view, "$" stands for input, "%" stands for +output, and "#" stands for register.
+// Pyrope
+foo = $a
+// Verilog
+input a;
+// C++
+auto node_input = Lnast_node::create_ref("$a", line_num, pos1, pos2);
+// Pyrope
+%out
+// Verilog
+output out;
+// C++
+auto node_output = Lnast_node::create_ref("%out", line_num, pos1, pos2);
+// Pyrope
+reg_foo
+// Verilog
+reg reg_foo;
+// C++
+auto node_reg = Lnast_node::create_ref("reg_foo", line_num, pos1, pos2);
+Bazel is a relatively new build system open sourced by google. The main difference +with traditional Makefiles is that it checks to make sure that dependences are not +lost and the builds are reproducible and hermetic. This document explains how +to use Bazel in the LGraph project.
+Build targets are referred to using the syntax //<relative path to BUILD file>:<executable>, where
+// is the path of the root livehd directory.
To build the LiveHD shell and supporting libraries, the target would be //main:all.
+To build every target in LiveHD (helpful for checking if changes cause compilation failures), the target would be //:.... For more details on target syntax, see this page.
For debugging/development use -c dbg, for benchmarking and testing -c opt.
Fast build: no optimization, minimal debugging information (no local variable information), assertions turned on (default) +
$ bazel build <target>
+Debug build: some optimization, full debugging information, assertions turned on +
$ bazel build -c dbg <target>
+or use address sanitizer to detect memory leaks +
$ bazel build -c dbg --config asan //...
+or use thread sanitizer to detect data races +
$ bazel build -c dbg --config tsan //...
+Release build: most optimization, no debug symbols, assertions turned off +
$ bazel build -c opt <target>
+Benchmarking build: aggressive optimization for the current architecture (binary may not be portable!) +
$ bazel build --config=bench <target>
+The bazel '-s' option prints the command executed. The sandbox may still be deleted.
+$ bazel build -s //main:all
+Bazel runs process in a sandbox what it is deleted after each run. To preserve it for debugging a failing test.
+bazel test --sandbox_debug -c dbg //YOUR_TEST_HERE
+Check the failing log, it will show you the sandbox location. You can change directory to it, and debug as usual.
+Many times, we have new tests that make the regression fail. We use "fixme" if +the test is a new one and LGraph is still not patched. We want the test in the system, +but we do not want to make fail the regressions.
+Those tests are marked with tags "fixme" in the BUILD. E.g:
+sh_test(
+ name = "my_test.sh",
+ tags = ["fixme"], # This is a fixme test. It fails, but we should fix it
+ srcs = ["tests/pyrope_test.sh"],
+To run all the fixme tests +
$ bazel test --test_tag_filters "fixme" <target>
+$ bazel query 'attr(tags, fixme, tests(<target>))'
+$ bazel query <target>
+$ bazel query <target>
+$ bazel query "deps(<target>)"
+$ bazel query "rdeps(//pass/..., //core:all)" | grep pass_
+This command is useful for benchmarking build time, and when system parameters change (the compiler gets upgraded, for example) +
$ bazel clean --expunge
+In addition to the short tests, there are sets of long tests that are run frequently +but not before every push to main line. The reason is that those are multi-hour +tests. +
$ bazel test --test_tag_filters "long1" <target>
+To list the tests under each tag. E.g., to list all the tests with long1 tag. +
$ bazel query 'attr(tags, long1, tests(<target>))'
+First run the tests to see the failing one. Then run with debug options +the failing test. E.g: +
$ bazel run -c dbg //eprp:all
+$ LGRAPH_LOG=info bazel run -c dbg //eprp:all
+$ bazel build -c dbg //eprp:eprp_test
+$ gdb bazel-bin/eprp/eprp_test
+(gdb) b Eprp::run
+(gdb) r
+In the cc_binary of the relevant BUILD file, add linkopts = ['-static']
Notice that the lgshell still needs the directory inside
+bazel-bin/main/lgshell.runfiles when using inou.yosys.\*
This section is for more advanced users that want to build LiveHD with some external 3rd party tool.
+When integrating LiveHD with a 3rd party tool (nextpnr in this example), you can either bring the 3rd +party tool to LiveHD and hence build it with bazel, or you can export the LiveHD code/libraries +and integrate with the 3rd party project. This document covers the later case.
+Bazel pulls a specific set of library dependences, if you export, you must ensure that the 3rd party tool +uses the same library version. The 3 main source of likely conflict is "boost", "abseil", and "fmt". +The "fmt" library is unlikely to be a conflict because LiveHD uses it as "header" only to avoid conflicts +with other tools like slang.
+To check the boost and abseil version, the easiest way: +
bazel build -c dbg //main:all
+# boost version 1.71 in this case
+grep -i "define BOOST_VERSION " bazel-*/external/boost//boost/version.hpp
+#define BOOST_VERSION 107100
+
+# abseil version 20210324
+grep "define ABSL_OPTION_INLINE_NAMESPACE_NAME" bazel-*/external/com_google_absl/absl/base/options.h
+#define ABSL_OPTION_INLINE_NAMESPACE_NAME lts_20210324
+nextpnr uses boost, in the previous example, you need to compile it with boost 1.71, with the usual requirements:
+# nextpnr ice40 needs icestorm, so install it first
+git clone https://github.com/cliffordwolf/icestorm.git
+cd icestorm
+make
+sudo make install
+
+# compile nextpnr itself
+git clone https://github.com/YosysHQ/nextpnr.git
+cd nextpnr
+mkdir build
+cd build
+cmake -DARCH=ice40 ../
+make -j $(ncpus)
+The previous steps should compile before you attempt to integrate LiveHD to nextpnr.
+Then, you need to clone and compile LiveHD. If you clone and compile parallel to nextpnr
+git clone https://github.com/masc-ucsc/livehd.git
+cd livehd
+bazel build -c dbg //main:all # You could use -c opt for faster/optimized compilation
+cd ../nextpnr/build/
+ln -s ../../livehd/
+ln -s livehd/bazel-out
+ln -s livehd/bazel-livehd
+Then, we need to copy the bazel gcc build instructions and combine with the nextpnr build
+Copy this to a file called pp:
+
--- livehd.params 2021-09-25 17:47:36.656724997 -0700
++++ livehd.params 2021-09-25 17:40:24.365650808 -0700
+@@ -1,16 +1,17 @@
+--o
+-bazel-out/k8-dbg/bin/main/lgshell
++-std=c++17
++-Wno-unknown-pragmas
++-I livehd/eprp -I livehd/elab -I bazel-livehd/external/com_google_absl -I bazel-livehd/external/fmt/include/ -I bazel-livehd/external/iassert/src -I livehd/mmap_lib/include -I livehd/core -I livehd/task -I livehd/lemu -I ./bazel-livehd/external/rapidjson -I livehd/pass/common -I ./bazel-livehd/external/replxx/include
++./extra.cpp
+ -pie
+ -fuse-ld=gold
+ -Wl,-no-as-needed
+ -Wl,-z,relro,-z,now
+ -B/usr/bin
+ -pass-exit-codes
+ -lstdc++
+ -lm
+-bazel-out/k8-dbg/bin/main/_objs/lgshell/main.pic.o
+ -Wl,--start-lib
+ bazel-out/k8-dbg/bin/main/_objs/main/inou_lef_api.pic.o
+ bazel-out/k8-dbg/bin/main/_objs/main/main_api.pic.o
+ bazel-out/k8-dbg/bin/main/_objs/main/meta_api.pic.o
+ bazel-out/k8-dbg/bin/main/_objs/main/top_api.pic.o
+The patch adds a new c++ file to compile (extra.cpp). It will be nicer if the file is in the nextpnr directory structure, but this is as an example of how to integrate. extra.cpp has a call to LiveHD to open a database as example.
cp livehd/bazel-bin/main/lgshell-2.params livehd.params
+patch <pp
+This example uses extra.cpp as a sample LiveHD call inside nextpnr. The extra.cpp contents:
#include "lgraph.hpp"
+
+void some_func() {
+ Lgraph *lg = Lgraph::open("lgdb","top");
+
+ lg->dump();
+}
+Then you need to add the @livehd.params to the end of the nextpnr-ice40 link step. A way to get the command line
+is to use the VERBOSE=1 option.
rm -f nextpnr-ice40
+make VERBOSE=1 nextpnr-ice40
+Cut and paste the command, it will end with something like thon3.9.so @livehd.params to be something like:
+
/usr/bin/c++ -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -fPIC -O3 -g -pipe -flto -fno-fat-lto-objects CMakeFiles/nextpnr-ice40.dir/common/archcheck.cc.o CMakeFiles/nextpnr-ice40.dir/common/basectx.cc.o CMakeFiles/nextpnr-ice40.dir/common/bits.cc.o CMakeFiles/nextpnr-ice40.dir/common/command.cc.o CMakeFiles/nextpnr-ice40.dir/common/context.cc.o CMakeFiles/nextpnr-ice40.dir/common/design_utils.cc.o CMakeFiles/nextpnr-ice40.dir/common/embed.cc.o CMakeFiles/nextpnr-ice40.dir/common/handle_error.cc.o CMakeFiles/nextpnr-ice40.dir/common/idstring.cc.o CMakeFiles/nextpnr-ice40.dir/common/idstringlist.cc.o CMakeFiles/nextpnr-ice40.dir/common/log.cc.o CMakeFiles/nextpnr-ice40.dir/common/nextpnr.cc.o CMakeFiles/nextpnr-ice40.dir/common/nextpnr_assertions.cc.o CMakeFiles/nextpnr-ice40.dir/common/nextpnr_namespaces.cc.o CMakeFiles/nextpnr-ice40.dir/common/nextpnr_types.cc.o CMakeFiles/nextpnr-ice40.dir/common/place_common.cc.o CMakeFiles/nextpnr-ice40.dir/common/placer1.cc.o CMakeFiles/nextpnr-ice40.dir/common/placer_heap.cc.o CMakeFiles/nextpnr-ice40.dir/common/property.cc.o CMakeFiles/nextpnr-ice40.dir/common/pybindings.cc.o CMakeFiles/nextpnr-ice40.dir/common/report.cc.o CMakeFiles/nextpnr-ice40.dir/common/router1.cc.o CMakeFiles/nextpnr-ice40.dir/common/router2.cc.o CMakeFiles/nextpnr-ice40.dir/common/sdf.cc.o CMakeFiles/nextpnr-ice40.dir/common/str_ring_buffer.cc.o CMakeFiles/nextpnr-ice40.dir/common/svg.cc.o CMakeFiles/nextpnr-ice40.dir/common/timing.cc.o CMakeFiles/nextpnr-ice40.dir/common/timing_opt.cc.o CMakeFiles/nextpnr-ice40.dir/3rdparty/json11/json11.cpp.o CMakeFiles/nextpnr-ice40.dir/json/jsonwrite.cc.o CMakeFiles/nextpnr-ice40.dir/frontend/json_frontend.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/arch.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/arch_place.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/arch_pybindings.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/bitstream.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/cells.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/chains.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/delay.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/gfx.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/main.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/pack.cc.o CMakeFiles/nextpnr-ice40.dir/ice40/pcf.cc.o CMakeFiles/chipdb-ice40.dir/ice40/chipdb/chipdb-384.cc.o CMakeFiles/chipdb-ice40.dir/ice40/chipdb/chipdb-1k.cc.o CMakeFiles/chipdb-ice40.dir/ice40/chipdb/chipdb-5k.cc.o CMakeFiles/chipdb-ice40.dir/ice40/chipdb/chipdb-u4k.cc.o CMakeFiles/chipdb-ice40.dir/ice40/chipdb/chipdb-8k.cc.o -o nextpnr-ice40 -ltbb /usr/lib/x86_64-linux-gnu/libboost_filesystem.so /usr/lib/x86_64-linux-gnu/libboost_program_options.so /usr/lib/x86_64-linux-gnu/libboost_iostreams.so /usr/lib/x86_64-linux-gnu/libboost_system.so /usr/lib/x86_64-linux-gnu/libboost_thread.so -lpthread /usr/lib/x86_64-linux-gnu/libboost_regex.so /usr/lib/x86_64-linux-gnu/libboost_chrono.so /usr/lib/x86_64-linux-gnu/libboost_date_time.so /usr/lib/x86_64-linux-gnu/libboost_atomic.so -lpthread /usr/lib/x86_64-linux-gnu/libpython3.9.so @livehd.params
+You can check that the new binary includes liveHD with something like: +
nm nextpnr-ice40 | grep -i Lgraph
+This document provides some minimal suggestion on how to build a new LiveHD pass.
+Most LiveHD passes reside inside inou or pass. The only difference is that
+inou focuses on translation from some external tool to/from LiveHD while
+pass works on transformations from LiveHD to LiveHD.
Check the pass/sample directory for how to create a trivial pass.
The typical is to have these files:
+Finally, add the new pass to main/BUILD
One of the main goals is to have a uniform set of passes in lgshell. lgshell should use this common +variable names when possible
+ name:foo lgraph name
+ path:lgdb lgraph database path (lgdb)
+ files:foo,var comma separated list of files used for INPUT
+ odir:. output directory to generate files like verilog/pyrope...
+The regression system builds for both gcc and clang. To force a clang build, set the following environment variables before building:
+CXX=clang++ CC=clang bazel build -c dbg //...
+Use lgbench to gather statistics in your code block. It also allows to run perf record +for the code section (from lgbench construction to destruction). To enable perf record +set LGBENCH_PERF environment variable
+export LGBENCH_PERF=1
+For most tests, you can debug with
+gdb ./bazel-bin/main/lgshell
+or
+lldb ./bazel-bin/main/lgshell
+Note that breakpoint locations may not resolve until lgshell is started and the relevant LGraph libraries are loaded.
+LiveHD has the option to run it with address sanitizer to detect memory leaks.
+bazel build -c dbg --config asan //...
+To debug with concurrent data race.
+bazel build -c dbg --config tsan //...
+The travis/azure regressions run several docker images. To debug the issue, run the same as the failing +docker image. c++ OPT with archlinux-masc image
+mkdir $HOME/docker
+docker run --rm --cap-add SYS_ADMIN -it -e LOCAL_USER_ID=$(id -u $USER) -v ${HOME}/docker:/home/user mascucsc/archlinux-masc
+
+# Once inside docker image. Create local "user" at /home/user with your userid
+/usr/local/bin/entrypoint.sh
+git clone https://github.com/masc-ucsc/livehd.git
+CXX=g++ CC=gcc bazel build -c opt //...
+A docker distro that specially fails (address randomizing and muslc vs libc) is alpine. The command line to debug it:
+docker run --rm --cap-add SYS_ADMIN -it -e LOCAL_USER_ID=$(id -u $USER) -v $HOME:/home/user -v/local/scrap:/local/scrap mascucsc/alpine-masc
+LiveHD is the synthesis/emulation flow primarily maintained and developed by +the MASC lab at UC Santa Cruz. Since LiveHD is used for computer +architecture and VLSI research, the MASC lab has an internal private +repo for some still in progress works. This is done using a private repo so +that we can wait until the research is published before pushing changes to the +public repo hosted on GitHub.
+This guide explains how we use git at the MASC group, and how you could setup a +similar flow to contribute to the LiveHD project. Other groups may choose to +adapt this technique for their own use.
+LiveHD uses bazel as a build system, as a result, we no longer use submodules. +Instead we use the built-in bazel support to pull specific repositories.
+This section is for git first time users and to show the git configuration used +by the MASC group.
+Suggested options for git first time users
+# Rebase no merge by default
+git config --global pull.rebase true
+# Set your name and email
+git config --global user.email "perico@lospalotes.com"
+git config --global user.name "Perico LosPalotes"
+git config --global pull.rebase true
+git config --global rebase.autoStash true
+Rebase creates cleaner logs, but sometimes it gets difficult to fix conflicts +with rebase. For cases that you are struggling to merge a conflict, you could +do this:
+# undo the failed rebase merge
+git rebase --abort
+
+# make sure that your code changes were committed
+git commit -a -m "Your commit message"
+git pull --no-rebase
+
+# Fix the conflict without rebase (easier)
+git commit -a -m "your merge message"
+git pull --no-rebase
+git push
+Clean the directory from any file not in git (it will remove all the files not committed)
+git clean -fdx
+Save and restore un-committed changes to allow a new git pull. stash is like a +"push" and "pop" replays the changes in the current directory. This will happen +automatically if you have the autoStash configuration option.
+git stash
+git pull
+git stash pop
+See the differences against the server (still not pushed). Everything may be +committed, so git diff may be empty
+git diff @{u}
+hercules --languages C++ --burndown --burndown-people --pb https://github.com/masc-ucsc/livehd >hercules1.data
+labours -f pb -m overwrites-matrix -o hercules1a.pdf <hercules1.data
+labours -f pb -m ownership -o hercules1b.pdf <hercules1.data
+
+hercules --languages C++ --burndown --first-parent --pb https://github.com/masc-ucsc/livehd >hercules2.data
+labours -f pb -m burndown-project -o hercules2.pdf <hercules2.data
+
+hercules --languages C++ --devs --pb https://github.com/masc-ucsc/livehd >hercules3.data
+labours -f pb -m old-vs-new -o hercules3a.pdf <hercules3.data
+labours -f pb -m devs -o hercules3b.pdf <hercules3.data
+labours -f pb -m devs-efforts -o hercules3c.pdf <hercules3.data
+If you do not plan to do many changes, and just wants to try LiveHD or be a +LiveHD user, the easiest way is to just clone the repo:
+git clone https://github.com/masc-ucsc/livehd
+cd livehd
+From time to time, you should get the latest version to have the latest bug fixes/patches. +Just a typical git pull should suffice:
+git pull
+These are instructions for advanced users, more likely other university/company +institutions with a team working on this project. The larger team may want to +have some private repository with internal development and some pushes/pulls to +the main LiveHD repo. For single external users, I would suggest to just fork +the repository and do pull requests.
+If you work outside UCSC and/or you are an infrequent contributor, you have two +main options: fork or private clone. The fork approach requires you to have +your repository public, if you have publications or work-in-progress that you +do not want to share the best option is to have a private repo (livehd-private).
+The simplest way to contribute to LiveHD is to create a public repo or a public +fork, and a pull request. Most git guides use the origin/master (in fork or +private repo) to synchronize with upstream/master (upstream main LiveHD repo). +This means that your local changes should NOT go to your origin/master. +Instead, you should create a branch for your local work. This works like a +charm if you do pull requests, and it is reasonable if you have a long +development branch without intention to push upstream.
+Although it is possible to create a different setup, we recommend that you keep +the origin/master clean to synchronize with upstream/origin. You should create +a new branch for each feature that you may want to upstream (origin/feature-x), +and a local development branch (dev) for all your team members.
+Clone the repo:
+git clone https://github.com/masc-ucsc/livehd.git livehd
+cd livehd
+Create development branch (dev)
+git checkout -b dev
+Create a branch from origin/master to create a pull request to upstream/master
+git checkout -b pull_request_xxx
+Create a branch from dev for internal development if needed
+git checkout -b feature_xx_4dev dev
+Synchronize origin/master from main upstream/master
+Add remote upstream (if not added before)
+git remote -v
+If remote -v did not list upstream. Add them
+git remote add upstream https://github.com/masc-ucsc/livehd.git
+git fetch upstream
+Make sure that you are in origin/master
+git checkout master
+Bring the changes from the remote upstream/master to local master/origin
+git merge upstream/master
+Push to repo origin/master if everything was fine
+git push origin master
+To see the difference with upstream (it should be empty)
+git diff @{upstream}
+Synchronize the dev branch with the latest master sync
+git checkout dev
+git merge origin/master
+git push # same as "push origin dev" because dev is checkout
+In case that you did not, push to the corresponding branch to the server
+git push origin dev
+git push origin pull_request_xxx
+git push origin feature_xx_4dev
+Create new pull request to upstream
+Make sure that origin/master is in sync (step 5)
+git diff @{upstream} # should be empty
+Rebase/merge the feature request with latest origin master
+git checkout pull_request_xxx
+git rebase master
+git push upstream pull_request_xxx
+Now create a pull request through github, and the UCSC/MASC team will review it.
+If you just want to do some small contributions to LiveHD doing a public fork is the +easiest way to contribute. Just fork, commit to forked master, and click on the web link +after you push.
+If you are working on LiveHD at UC Santa Cruz, contact Jose +Renau to be added to the MASC organization +on GitHub so that you have write access to the repo. The setup is similar to +the infrequent contributor flow but you have +access to directly commit to the public repository. Even the upstream/master.
+ + + + + + +These are the coding style rules for LiveHD C++. Each rule can be broken, but it +should be VERY rare, and a small comment should be placed explaining why.
+Code should be the comments, try to keep comments concise. They should explain +the WHY not the HOW. The code is the HOW.
+Labels used in comments:
+// FIXME: Known bug/issue but no time to fix it at the moment
+
+// TODO: Code improvement that will improve perf/quality/??? but no time at the moment
+
+// WARNING: message for some "strange" "weird" code that if changes has effects
+// (bug). Usually, this is a "not nice" code that must be kept for some reason.
+
+// NOTE: Any comment that you want to remember something about (not critical)
+
+// STYLE: why you broke a style rule (pointers, iterator...)
+We use std::string and std::string_view. These are the rules:
+Arguments for functions are always std::string_view (no const std::string &)
+If the return argument is not allocated in the function, we return a std::string_view
+If the return argument can be a new string, the function returns std::string
+Use the absl::StrCat, absl::StrAppend, absl::StrSplit when possible
+To convert to/from integers use str_tools::to_i to_hex ...
+Use str_tools for sub-string operations like str_tools::ends_with
+foo_bar = Foo_bar(3);
+elem = entries[index];
+val = My_enum::Big;
+class Sweet_potato {
+Use the Pass::error or Pass:warn for error and likely error (warn). Internally, error generates +and exception capture by the main lgshell to move to the next task.
+Pass::error("inou_yaml: can only have a yaml_input or a graph_name, not both");
+Pass::warn("inou_yaml.to_lg: output:{} input:{} graph:{}", output, input, graph_name);
+Make sure to configure your editor to use 2 spaces
+You can configure your text editor to do this automatically
+First do C includes (try to avoid when possible), then an empty line with C++ +includes, then an empty line followed with lgraph related includes. E.g:
+#include <sys/types.h>
+#include <dirent.h>
+
+#include <iostream>
+#include <set>
+
+#include "graph_library.hpp"
+#include "lgedgeiter.hpp"
+You can configure your text editor to do this automatically
+You can configure your text editor to highlight them. + https://github.com/ntpeters/vim-better-whitespace
+for(auto idx:g->unordered()) {
+}
+Use structured returns when iterator is returned for cleaner code:
+for(const auto &[name, id]:name2id) {
+ // ...
+for(auto idx:g->unordered()) {
+ for(const auto &c:g->out_edges(idx)) {
+It may be too verbose to write const all the time. The coding style request to use +const (when possible) in iterators and pointers. The others are up to the programmer.
+#include "absl/container/flat_hash_map.h"
+#include "absl/container/flat_hash_set.h"
+
+absl::flat_hash_map<Index_ID, RTLIL::Wire *> my_example;
+Traverse the map/set, and as it traverses decide to erase some of the entries: +
for (auto it = m.begin(), end = m.end(); it != end;) {
+ if (condition_to_erase_it) {
+ m.erase(it++);
+ } else {
+ ++it;
+ }
+}
+To check if a key is present: +
if (set.contains(key_value)) {
+}
+absl::Span is the equivalent of string_view for a string but for vectors. Like
+string_view, it does not have ownership, and the size in the span can decrease
+(not increase) without changing the original vector with "subspan". Faster and
+more functional, no reason to return "const std::vector
#include "absl/types/span.h"
+
+absl::Span<Sub_node> get_sub_nodes() const {
+ I(sub_nodes.size()>=1);
+ return absl::MakeSpan(sub_nodes).subspan(1); // Skip first element from vector
+};
+void print(const Sub_node& g); //or
+
+void edit(Sub_node& g);
+Note that older code still uses pointers, this is no longer allowed.
+The idea is to RARELY directly allocate pointer allocation
+Use:
+foo = Sweet_potato(3, 7)
+instead of
+foo = new Sweet_potato(3, 7)
+Use: +
foo = std::make_unique<Sweet_potato>(3,7);
+instead of
+foo = new Sweet_potato(3, 7)
+fmt::print("This is a debug message, name = {}, id = {}\n",g->get_name(), idx);
+If a variable is const, it can be exposed directly without get/set accessors
+foo = x.const_var; // No need to have x.get_const_var()
+bitarray visited(g->max_size());
+This usually means use meaningful variable names and conditions that are easy to understand. +If the meaning is not clear from the assertion, use a comment in the same line. +This way, when the assertion is triggered it is easy to identify the problem.
+I(n_edges > 0); //at least one edge needed to perform this function
+We use the https://github.com/masc-ucsc/iassert package. Go to the iassert for more details on the advantages +and how to allow it to use GDB with assertions.
+Extra checks should be only in debug. Debug and release must execute the same, +only checks (not behavior change) allowed in debug mode.
+Benchmark in release. It is 2x-10x faster.
+Use clang-format as configured to catch style errors. LGraph clang-format is +based on google format, but it adds several alignment directives and wider +terminal.
+ cd XXXX
+ clang-format -i *pp
+std::vector<LGraph *> Inou_yaml::generate() {
+
+ if (opack.graph_name != "") {
+ // ...
+ } else {
+ // ..
+ }
+Attributes are parameters or information that an be per Node, Node_pin or Edge. In LGraph, attributes are +persistent. This means that they are kept across execution runs in the LGraph database (E.g: in lgdb).
+For persistent attributes, the structures to use are defined in core/annotate.hpp. Any new attribute +must be added to "annotate.hpp" to preserve persistence and to make sure that they are cleared when needed.
+Many times it is important to have information per node, but that it is not persistent across runs. For example, +when building a LGraph from Yosys, there is a need to remember pointers from yosys to LGraph. This by definition can not be persistent because pointers change across runs. For this case, there are several options.
+If the data structure needs to keep most of the node/pins in the Lgraph, use the compact_class notation: +
absl::flat_hash_map<SomeData, Node_pin::Compact_class> s2pin;
+absl::flat_hash_map<SomeData, Node::Compact_class> s2node;
+
+SomeData d1;
+Lgraph *lg; // LGraph owning the node
+s2pin[d1] = node.get_driver_pin().get_compact_class(); // Example of use getting a pint
+s2node[d1] = node.get_compact_class();
+auto name = s2pin[d1].get_node(lg).get_name(); // Pick previously set driver name
+Another example:
+absl::flat_hash_map<Node_pin::Compact, RTLIL::Wire *> input_map;
+
+input_map[pin.get_compact()] = wire;
+
+auto *wire = input_map[pin.get_compact()];
+
+for(const auto &[key, value]:input_map) {
+ Node_pin pin(lg, key); // Key is a ::Compact, not a Node_pin
+ auto name = pin.get_name();
+ // ... Some use here
+}
+If the data structure just holds a small subset of the graph, you can keep the +metadata, and use Node/Node_pin directly. E.g:
+absl::flat_hash_map<SomeData, Node_pin> s2pin;
+absl::flat_hash_map<SomeData, Node> s2node;
+
+SomeData d1;
+s2pin[d1] = node.get_driver_pin(); // Example of use getting a pint
+s2node[d1] = node;
+auto name = s2pin[d1].get_name(); // Pick previously set driver name
+In this case, it is fine to use the full Node, Node_pin, or Edge. This has some +pointers inside, but it is OK because it is not persistent.
+The rule is that if the same code appears in 3 places, it should be refactored
+Tool to detect duplication +
find . -name '*.?pp' | grep -v test >list.txt
+ duplo -ml 12 -pt 90 list.txt report.txt
+