Make hash caches thread-safe, Netkan warning for uncompiled plugins

[HebaruSan]

Problem

The Inflator has been emitting Index was outside the bounds of the array. errors for a while now, which cannot be reproduced locally. After #4398, we captured the full stack trace:

System.IndexOutOfRangeException: Index was outside the bounds of the array.
 at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
 at CKAN.NetFileCache.GetFileHash(String filePath, String hashSuffix, Dictionary`2 cache, Func`1 getHashAlgo, IProgress`1 progress, Nullable`1 cancelToken)
 at CKAN.NetKAN.Transformers.DownloadAttributeTransformer.Transform(Metadata metadata, TransformOptions opts)+MoveNext()
 at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
 at System.Collections.Generic.SparseArrayBuilder`1.ReserveOrAdd(IEnumerable`1 items)
 at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.ToArray()
 at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
 at CKAN.NetKAN.Transformers.NetkanTransformer.Transform(Metadata metadata, TransformOptions opts)
 at CKAN.Extensions.EnumerableExtensions.<>c__DisplayClass9_1`2.<SelectManyTasks>b__2()
 at System.Threading.Tasks.Task`1.InnerInvoke()
 at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
 at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
 at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread t

The Dictionary.TryInsert call in question must be in NetFileCache.GetFileHash:

CKAN/Core/Net/NetFileCache.cs

Lines 649 to 682 in dd37ee2

	private string GetFileHash(string filePath,
	string hashSuffix,
	Dictionary<string, string> cache,
	Func<HashAlgorithm> getHashAlgo,
	IProgress<int>? progress,
	CancellationToken? cancelToken)
	{
	string hashFile = $"{filePath}.{hashSuffix}";
	if (cache.TryGetValue(filePath, out string? hash))
	{
	return hash;
	}
	else if (File.Exists(hashFile))
	{
	hash = File.ReadAllText(hashFile);
	cache.Add(filePath, hash);
	return hash;
	}
	else
	{
	using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
	using (BufferedStream bs = new BufferedStream(fs))
	using (HashAlgorithm hasher = getHashAlgo())
	{
	hash = BitConverter.ToString(hasher.ComputeHash(bs, progress, cancelToken)).Replace("-", "");
	cache.Add(filePath, hash);
	if (Path.GetDirectoryName(hashFile) == cachePath.FullName)
	{
	hash.WriteThroughTo(hashFile);
	}
	return hash;
	}
	}
	}

Later, that stack trace was replaced by one featuring AggregateException, which I have been trying to purge from our logs.

Causes

• Apparently Dictionary.Add is thread unsafe and can throw this exception when multiple threads use it concurrently.
  https://stackoverflow.com/questions/15095817/adding-to-a-generic-dictionary-causes-indexoutofrangeexception
• My handling for AggregateException only worked when there was a single InnerException; if it contained multiple in the InnerExceptions property, it wasn't caught.

Motivation

Twice in the past few weeks, we have had mods submitted by someone other than their primary C# coder, who didn't compile the plugin (see KSP-CKAN/NetKAN#10592 and KSP-CKAN/NetKAN#10600). If this is going to be a common pattern going forward, it would be good to detect it programmatically rather than making me dig through the ZIP file and repo to notice it.

Changes

• Now NetFileCache is updated to use ConcurrentDictionary for those caches, which will not throw IndexOutOfRangException when trying to add an element.
• Now GUI.Main is excluded from coverage because it is a Form which we will not instantiate in tests.
• Now CKAN.Extensions.ExceptionExtensions.RethrowInner is created to wrap the more complex logic needed to pick the first element from InnerExceptions and rethrow it, and called by all the places where we catch AggregateException.
• Now if Netkan spots *.cs, *.csproj, or *.sln files in a ZIP and doesn't see a *.dll, a warning is emitted, to make this problem easier to spot.

[coveralls]

Pull Request Test Coverage Report for Build 15888219272

Details

• 33 of 75 (44.0%) changed or added relevant lines in 9 files are covered.
• 6 unchanged lines in 3 files lost coverage.
• Overall coverage increased (+5.4%) to 47.631%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
Core/Net/NetAsyncModulesDownloader.cs	5	6	83.33%
Netkan/Validators/InstallsFilesValidator.cs	0	1	0.0%
Core/Extensions/EnumerableExtensions.cs	0	2	0.0%
Core/Net/NetFileCache.cs	20	22	90.91%
Core/Extensions/ExceptionExtensions.cs	4	7	57.14%
Core/Registry/InstalledModule.cs	0	3	0.0%
Netkan/Services/ModuleService.cs	1	4	25.0%
Core/Registry/Registry.cs	1	13	7.69%
Netkan/Validators/PluginsValidator.cs	2	17	11.76%

Files with Coverage Reduction	New Missed Lines	%
Core/Registry/InstalledModule.cs	1	61.43%
Netkan/Validators/PluginsValidator.cs	2	34.38%
Core/Registry/Registry.cs	3	69.15%

Totals
Change from base Build 15834236157:	5.4%
Covered Lines:	8334
Relevant Lines:	16913

---

💛 - Coveralls