improve performance

[AvenSun]

hi @sdcb
sorry for that I didn't noticed they have variety size in last PR.

I thought they already have the same size after padding and resize.

I looked into your implementation of PaddleOcrRecognizer.

OpenVINO.NET/projects/PaddleOCR/Sdcb.OpenVINO.PaddleOCR/PaddleOcrRecognizer.cs

Lines 129 to 144 in 6b99f89

	int modelHeight = Model.Shape.Height;
	int maxWidth = StaticShapeWidth ?? (int)Math.Ceiling(srcs.Max(src =>
	{
	Size size = src.Size();
	double width = 1.0 * size.Width / size.Height * modelHeight;
	double padded = 32 * Math.Ceiling(1.0 * width / 32);
	return padded;
	}));
	using Mat final = PrepareAndStackImages(srcs, modelHeight, maxWidth);
	using InferRequest ir = _compiledModel.CreateInferRequest();
	using (Tensor input = final.StackedAsTensor(srcs.Length))
	{
	ir.Inputs.Primary = input;
	ir.Run();
	}

the input of model must have the same shape in one batch.

I think you have no doubt about this.

It should be like the image below after invoked the method PrepareAndStackImages.
four images stack

Am I right? Here's my question.

the implementation of PrepareAndStackImages invoked the method ResizePadding

why not the method of ResizePadding resize and pad image to maxWidth directly?

then we can invoke VConcat immediately.

OpenVINO.NET/projects/PaddleOCR/Sdcb.OpenVINO.PaddleOCR/PaddleOcrRecognizer.cs

Lines 208 to 239 in 6b99f89

	private static unsafe Mat PrepareAndStackImages(Mat[] srcs, int modelHeight, int maxWidth)
	{
	Mat[] normalizeds = null!;
	Mat final = new();
	try
	{
	normalizeds = srcs
	.Select(src =>
	{
	using Mat channel3 = src.Channels() switch
	{
	4 => src.CvtColor(ColorConversionCodes.RGBA2RGB),
	1 => src.CvtColor(ColorConversionCodes.GRAY2RGB),
	3 => src.FastClone(),
	var x => throw new Exception($"Unexpect src channel: {x}, allow: (1/3/4)")
	};
	return ResizePadding(channel3, modelHeight, maxWidth);
	})
	.ToArray();
	using Mat combined = normalizeds.StackingVertically(modelHeight, maxWidth);
	combined.ConvertTo(final, MatType.CV_32FC3, 2.0 / 255, -1.0);
	}
	finally
	{
	foreach (Mat normalized in normalizeds)
	{
	normalized.Dispose();
	}
	}
	return final;
	}

according to what you mentioned in last PR (#5) , I made this new PR ,

It fixs all problem, you can take a look.

I confirmed It keeps exactly the same OCR result compared with your commit (same input and same model).

meanwhile we also can get some improvement as shown below.

BenchmarkDotNet v0.13.10, Windows 10 (10.0.19044.3693/21H2/November2021Update)
  [Host]     : .NET 7.0.14 (7.0.1423.51910), X64 RyuJIT AVX2
  Job-PARHUX : .NET 7.0.14 (7.0.1423.51910), X64 RyuJIT AVX2
  Job-YLHAAW : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2

IterationCount=10  LaunchCount=1  WarmupCount=1  

Method	Runtime	modelHeight	maxWidth	Mean	Error	StdDev	Ratio	RatioSD	Allocated	Alloc Ratio
StackingVerticallyBySdcb	.NET 7.0	48	320	415.2 μs	15.14 μs	10.01 μs	1.00	0.00	1.93 KB	1.00
StackingVerticallyByAven	.NET 7.0	48	320	252.8 μs	8.97 μs	5.93 μs	0.61	0.02	1.79 KB	0.93
StackingVerticallyBySdcb	.NET 8.0	48	320	408.3 μs	11.43 μs	7.56 μs	1.00	0.00	1.93 KB	1.00
StackingVerticallyByAven	.NET 8.0	48	320	248.5 μs	12.37 μs	8.18 μs	0.61	0.02	1.79 KB	0.93
StackingVerticallyBySdcb	.NET 7.0	48	512	423.3 μs	18.53 μs	12.26 μs	1.00	0.00	1.54 KB	1.00
StackingVerticallyByAven	.NET 7.0	48	512	361.4 μs	14.72 μs	9.74 μs	0.85	0.04	1.62 KB	1.05
StackingVerticallyBySdcb	.NET 8.0	48	512	419.8 μs	14.95 μs	9.89 μs	1.00	0.00	1.54 KB	1.00
StackingVerticallyByAven	.NET 8.0	48	512	360.3 μs	14.88 μs	9.84 μs	0.86	0.03	1.62 KB	1.05

[sdcb]

Overall speaking, thanks for you PR again, it's really a promising performance improvement result even if you added padding manually, but I requested a few changes, can you check.

[sdcb]

I agreed that we can use the method of ResizePadding resize and pad image to maxWidth directly in previous code, but I believe the StackingVertically method should keeps the same behavior of not constraint the input srcs width, which allows input srcs able to be different width.

[AvenSun]

hi @sdcb
All suggestions you mentioned are adopted.
please review.

[sdcb]

@AvenSun Hi AvenSun, I created a branch and did some necessary work that I believe, can you merge my upstream https://github.com/sdcb/OpenVINO.NET/tree/mat-ext branch into your PR's branch? After that, I'll go ahead and merge this PR.

[AvenSun]

hi @sdcb
I just did what you said, you can have a try.

[sdcb]

Thanks, merged.