Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More matmul variants #153

Merged
merged 2 commits into from
Jul 26, 2024
Merged

Conversation

adam-smnk
Copy link
Collaborator

@adam-smnk adam-smnk commented Jul 26, 2024

Adds support for linalg.matmul and linalg.matmul_tranpose_a.

Tested with:

import torch
import torch.nn as nn
import openvino as ov


# Define a synthetic model
class MatmulVariants(nn.Module):
    def __init__(self, sizes_mnk, type=None):
        super(MatmulVariants, self).__init__()
        m = sizes_mnk[0]
        n = sizes_mnk[1]
        k = sizes_mnk[2]
        self.weight_b = torch.empty((k, n), dtype=type).data.normal_(0, 0.01)
        self.bias = torch.empty((m, n), dtype=type).data.fill_(0.01)
        self.relu = nn.ReLU()
    def forward(self, a, b):
        # matmul
        mm = torch.matmul(a, b)
        add1 = torch.add(mm, self.bias)
        out1 = self.relu(add1)
        # matmul_transpose_b
        mmt = torch.matmul(a, self.weight_b)
        add2 = torch.add(mmt, self.bias)
        out2 = self.relu(add2)
        return out1 - out2

# Create an instance of the model
input_size = 1024
output_size = 128
model = MatmulVariants([output_size, output_size, input_size])

input_data_A = torch.tensor(range(1, input_size*output_size+1)).to(torch.float32)\
    .view(output_size, input_size)
input_data_B = torch.tensor(range(1, input_size*output_size+1)).to(torch.float32)\
    .view(input_size, output_size)

with torch.no_grad():
    reference = model(input_data_A, input_data_B)
    print('Reference:\n', reference)

# Dynamic shapes
# ov_model = ov.convert_model(model, example_input=input_data)
# Static shapes
ov_model = ov.convert_model(model,
                            input=[(ov.PartialShape([output_size, input_size]), ov.Type.f32),
                                   (ov.PartialShape([input_size, output_size]), ov.Type.f32)])
# ov.save_model(ov_model, "simple_model.mm.xml")

# expect lots of debug output with MLIR fragments before and after lowering in this call:
ov_compiled = ov.compile_model(ov_model, 'CPU')

tested = ov_compiled([input_data_A, input_data_B])[0]
print('Tested:\n', tested)
diff = reference - tested
print('Reference - Tested:\n', diff)

@adam-smnk adam-smnk merged commit 7fba0a4 into slyalin:mlir Jul 26, 2024
13 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants