[Torch][WeightCompression] Add Scale Estimation data-aware support

[kshpv]

Changes

Added data-aware support for the Torch backend for WeightCompression with Scale Estimation.
Introduced support for MeanVarianceReducer, MaxVarianceReducer, and MeanAbsMaxReducer.
Incorporated torch.inference_mode() context for WeightCompression.

Reason for changes

These changes enable the utilization of data-aware Scale Estimation for the Torch backend, specifically leveraging CUDA devices for improved performance.

Related tickets

Ticket ID: 158974

Tests

Added a template for WeightCompression tests for both Torch and OV backends, covering data-aware and Scale Estimation scenarios.
Extended the test scope to include tinyllama_data_aware and tinyllama_scale_estimation_per_channel for Torch.
Added a new test case tinyllama_scale_estimation_group_size_64 for both Torch and OV backends.

Performance Metrics

Note: All CUDA results are obtained locally on a single RTX 3090.

Model	Backend	Metric Name	Metric Value	Num int4	Num int8	Compression Time (from Performance Job)	RAM MiB (from Performance Job)
tinyllama_data_aware	OV	Similarity	0.8577	94	124	0:01:28	8545
tinyllama_data_aware	TORCH	Similarity	0.8577	94	124	0:02:15	1225
tinyllama_data_aware	TORCH (CUDA)	Similarity	0.8577	94	124	0:00:28	-
tinyllama_scale_estimation_per_channel	OV	Similarity	0.8139	188	124	0:02:57	8681
tinyllama_scale_estimation_per_channel	TORCH	Similarity	0.8139	188	124	0:03:25	5472
tinyllama_scale_estimation_per_channel	TORCH (CUDA)	Similarity	0.8139	188	124	0:00:35	-
tinyllama_scale_estimation_group_size_64	OV	Similarity	0.8566	94	124	0:04:17	8681
tinyllama_scale_estimation_group_size_64	TORCH	Similarity	0.8566	94	124	0:04:01	5575
tinyllama_scale_estimation_group_size_64	TORCH (CUDA)	Similarity	0.8566	94	124	0:00:36	-

[kshpv]

weight compression build - 291

[kshpv]

The proposed example can be added as a follow-up PR - will be excluded from this PR

[ljaljushkin]

Not for this PR.
Should this function really take the list?
Seems like it always contains one element. Is it only RawReducer that consumes the whole list and the rest works with one element?
Can we inherit all these "one-element" classes from a class that defines a method with a single element?
IMO, it would be more clear when one element expected and when not.
@daniil-lyakhov

[daniil-lyakhov]

Reducers was designed to receive several inputs and output several outputs as well. For now, there is no such reducers, but the possible use case - quantization error (fp32 input, int8 input) -> diff

We can create a class for that, it will make hierarchy tree more complicated, we than should introduce method like _reduce_out_of_place_one_input method, and I don't think this will make code more readable

[ljaljushkin]

Since all reducers use one input, it should be relatively easy, isn't?

1. keep in base class _reduce_out_of_place for many inputs
2. subclass an intermediate class with _reduce_out_of_place_one_input
3. rename inheritance from base class to intermediate class
4. rename _reduce_out_of_place to _reduce_out_of_place_one_input.

[ljaljushkin]

@kshpv Have you run the performance job after merging with develop? Could you share it?

[kshpv]

@kshpv Have you run the performance job after merging with develop? Could you share it?

Sure, performance build 41

[ljaljushkin]

If compare 40 and 41 performance build, one can notice that AWQ is sometimes slowly.
40 build doesn't include #2727

41 build includes #2727

Although time for mixed-precision, applying compession and total time are better with 41 build, I wonder why there is some slowness. Is it measurement error or expected numbers? @nikita-savelyevv

[nikita-savelyevv]

If compare 40 and 41 performance build, one can notice that AWQ is sometimes slowly. 40 build doesn't include #2727 41 build includes #2727

Although time for mixed-precision, applying compession and total time are better with 41 build, I wonder why there is some slowness. Is it measurement error or expected numbers? @nikita-savelyevv

It is possible that AWQ part could become a bit slower for a small model like tiny-llama because the added compiled functions are the most effective for compressing large tensors. When operating with small tensors, compilation overhead can overshadow the speedup from compiled computation.

That's why for example int8 tiny-llama compression is a bit slower after #2727

My assumption is that the same effect makes AWQ a bit slower in this case. It should not happen for larger models though. I will check this. Thanks for the observation.

[ljaljushkin]

t the same effect makes AWQ a bit slower in this case. It should not happen for larger models though. I will check this. Thanks for the observa

Thanks for the confirmation!