You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Source files are ending up on the large object heap #4881
Looking through perf traces of IntelliSense, I can see that large source files are ending up on the large object heap because they are read as a single chunk:
This is going to put pressure on Gen2 collections, and I can see that in one trace opening a completion window in a large file causes a 77ms GC2 collection.
Is there a leaner way to get the char array from the stream?
In second location, in this case we don't keep the char array around at call site, we could save extra call to Array.copy (need to check other places we call LexBuffer<_>.FromChars, we might need a separate factory method skipping the copy).
This might be harder to fix than you think.... Or at least what you'll need to fix isn't what's described :)
Among other things we use the entire contents of source files to act as a key for some operations in the language service caches. That can be changed I'm sure (e.g. use the hash of the contents) but until we do that we need to store the whole file, breaking into chunks isn't solving the fundamental problem
@dsyme - the Roslyn workspace uses the editor's types to back all the files in the workspace, (sometimes privately for closed files, and sometimes publicly for open files), but we should be able to use editor version types or even instances to manage these caches. In fact, if we just threaded the Workspace/Solution/Document/Project types further through, we might be able to eliminate these caches.
#4948 by @auduchinok also reduces some copying. With that and @smoothdeveloper's changes in place it would be good to measure this again and see if it's had a positive impact.
We closed #5944 because we'll need to incorporate a hash code (or some integer ID) for an ISourceText in the language service cache keys, which should help us in reducing the size of these data structures that we keep in memory. But for the time being, things should be a lot better now to #6001
Activity
smoothdeveloper commentedon May 11, 2018
I wonder if we could avoid the copies occuring at those two places:
https://github.com/Microsoft/visualfsharp/blob/75e435e74b81bb47c4a0ced11be0d7560d2a66ea/src/fsharp/UnicodeLexing.fs#L68
https://github.com/Microsoft/visualfsharp/blob/44fa027b308681a1b78a089e44fa1ab35ff77b41/src/utils/prim-lexing.fs#L161
in first location, we create a
char arrayby calling https://msdn.microsoft.com/en-us/library/ezftk57x(v=vs.110).aspx, and then we callArray.copyin second location.Is there a leaner way to get the char array from the stream?
In second location, in this case we don't keep the char array around at call site, we could save extra call to
Array.copy(need to check other places we callLexBuffer<_>.FromChars, we might need a separate factory method skipping the copy).davkean commentedon May 11, 2018
Yes those too, there's a bunch of them, the following are all ending up on LOH:
smoothdeveloper commentedon May 11, 2018
I made a test PR #4882 that removes the call to
Array.copyin that code path.I still have issues to build the whole repository with VS tooling so I currently can't test the impact on intellisense.
dsyme commentedon May 13, 2018
This might be harder to fix than you think.... Or at least what you'll need to fix isn't what's described :)
Among other things we use the entire contents of source files to act as a key for some operations in the language service caches. That can be changed I'm sure (e.g. use the hash of the contents) but until we do that we need to store the whole file, breaking into chunks isn't solving the fundamental problem
Pilchie commentedon May 14, 2018
@dsyme - the Roslyn workspace uses the editor's types to back all the files in the workspace, (sometimes privately for closed files, and sometimes publicly for open files), but we should be able to use editor version types or even instances to manage these caches. In fact, if we just threaded the
Workspace/Solution/Document/Projecttypes further through, we might be able to eliminate these caches.cartermp commentedon Jul 21, 2018
#4948 by @auduchinok also reduces some copying. With that and @smoothdeveloper's changes in place it would be good to measure this again and see if it's had a positive impact.
cartermp commentedon Dec 18, 2018
This is likely lessened greatly by #6001
We closed #5944 because we'll need to incorporate a hash code (or some integer ID) for an
ISourceTextin the language service cache keys, which should help us in reducing the size of these data structures that we keep in memory. But for the time being, things should be a lot better now to #6001