test: bigquery mock server

[PrathameshTugaonkar]

Because

• Essentially because we could mock the BQ server to test it seamlessly

This commit

• tried and researched various approaches and added final approach to mock BQ server

[chuang8511]

Hi @PrathameshTugaonkar
Thanks for your contribution.
The mock seems good.

Now, could you

1. Please pre-commit install, you can check the lint before commit.
2. Please follow the git guideline. It is in the 3. point of sending PR
3. Please refer to json operator test code to increase the test code coverage. At least, we need to test

• func (e *execution) Execute(ctx context.Context, jobs []*base.Job) error {
• func (c *component) GetDefinition(sysVars map[string]any, compConfig *base.ComponentConfig) (*pb.ComponentDefinition, error) {

Thanks for your contribution again!

[PrathameshTugaonkar]

Thanks @chuang8511 for reviewing and providing suggestions.

Is it good to go now? @chuang8511

Point 3 - Should we take that in another ticket?

[chuang8511]

Thanks for your question!

Point 3 - Should we take that in another ticket?

No, the issue in this ticket mainly lacks the test code for bigquery component.
And, the previous bottleneck is we don't find a good way to mock bigquery.
The end goal here is still adding test for bigquery component.

So, we will still need point 3 in this ticket!

[PrathameshTugaonkar]

Thanks for your question!

Point 3 - Should we take that in another ticket?

No, the issue in this ticket mainly lacks the test code for bigquery component. And, the previous bottleneck is we don't find a good way to mock bigquery. The end goal here is still adding test for bigquery component.

So, we will still need point 3 in this ticket!

Okay, Definitely will work towards it. Thanks @chuang8511 !

[PrathameshTugaonkar]

@chuang8511 Please review. Thanks!

[chuang8511]

Hi @PrathameshTugaonkar ,

Thanks for your contribution.

What I mean exactly is - we need to do the test like here.

In your code, you only test bigquery http fake server. But, you don't interact the bigquery component with the bigquery http fake server.

Recently, I happened to build the similar thing. You can also take a look on this as a reference.

[chuang8511]

Please check this comment.

[chuang8511]

You need to test the interaction between the component and fake bigquery server