You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Deep Learning models usually are stacked layers, from shallow layers to deep layers. With Transform models being prevalent, it is worth mentioning that Transformer Encoder Layers are homogeneous, i.e., different depths of layers are of the same class and the same initialization parameters, only different in which the positions that they are placed in the sequential connected ModuleList.
Therefore, when we visualize the model, it is ugly to see all the 12/24/40 and even more duplicated graph structures in the big svg generated. We should fold them.
Describe the solution you'd like
My solution may be, after we generated the graphviz graph, we can do a traversal on the graph and recognize whether there are such subgraphs a and b that satisfies:
a and b have the same input shape and output shape.
a and b are identical and isomorphic.
a's only output is b's only input
Then we should fold a and b to
a*2
Additional context
another visualization lib called bigmodelvis can implement this, we can refer to their algorithms.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Deep Learning models usually are stacked layers, from shallow layers to deep layers. With Transform models being prevalent, it is worth mentioning that Transformer Encoder Layers are homogeneous, i.e., different depths of layers are of the same class and the same initialization parameters, only different in which the positions that they are placed in the sequential connected ModuleList.
Therefore, when we visualize the model, it is ugly to see all the 12/24/40 and even more duplicated graph structures in the big svg generated. We should fold them.
Describe the solution you'd like
My solution may be, after we generated the graphviz graph, we can do a traversal on the graph and recognize whether there are such subgraphs a and b that satisfies:
Then we should fold a and b to
a*2
Additional context
another visualization lib called
bigmodelvis
can implement this, we can refer to their algorithms.The text was updated successfully, but these errors were encountered: