Allow to disable GC compaction

[fabianoliver]

Summary

As far as I know, compaction is typically the largest (often only?) contributor to stop-the-world pause times in modern GCs, presumably .NET GC included. Would it be feasible to

1. Add a GC toggle that'd disable automatic compaction for all generations, and
2. Add an optional to manually trigger compaction (possibly during the next upcoming GC phase, similar to LOH compaction)

With the overall aim/hope this might enable users to effectively reduce GC pauses at the cost of various other metrics?

I'm curious if that would be arelatively low effort means to have a - probably not ideal, but still pauseless GC that might be net/net beneficial for certain kinds of applications (eg financial space/trading, gaming, etc.)

If this feature would be technically possible, I wonder what the typical trade-offs of enabling such a setting would be. I'd assume lower throughput and somewhat higher overall consumption - would any other metric likely be affected?

Motivation

• .NET performs compaction by default, currently without an opt-out; I'd assume that choice was made as benefits of doing so were seen as outweighting an increase/introduction of pause times
• On the flip side, some applications cannot tolerate pause times, and developers of these would often gravitate to other languages
• I could be horribly wrong there, but my impression is that compaction is often not done these other, non GC-ed languages. Taking Rust as an example, as far as I know, no stable allocator trait even exists, and only some private crates offer limited functionality there (eg arena allocations for a limited subset of collection types)
• That'd make it seem to me as though compaction and/or other more clever allocation strategies are not always, but at least often somewhat of an afterthought there, which in turn might mean some users moving away from C# towards non-GC'ed languages might value the benefits of pauselesness over other benefits compaction (or other clever strategies) bring to the table
• Or to put it simply: If non-compaction is a viable strategy for a huge amount of apps written in non-managed languages, would it not also be a viable optional strategy in the managed world? Is there anything about .NET that would make a fragmented heap more problematic than it would be in native apps?

I somehow expect this isn't possible, or my line of thinking here is wrong, but I'd be curious to hear thoughts on this!

[teo-tsirpanis]

Automatic compaction of the heap is the primary reason object allocations are just a pointer bump most of the time. Disabling it or not always doing it will likely affect performance in non-trivial ways.

[huoyaoyuan]

The objects need to be copied, and pointers on stack and heap need to be updated. The pause can be significant for Gen1 because it's not background.

[fabianoliver]

Interesting point. Hard to judge without more specific benchmarks, my gut feeling is that slower allocation paths would be tolerable for a range of applications.

Generational promotion isn't definitely a big conceptual issue though, have commented in that on the back of the comment by @neon-sunset

[neon-sunset]

Perhaps you meant disabling compaction for Gen 2 and LOH specifically? The collection process in generational GCs is naturally compacting as the surviving objects are relocated to an older generation. My understanding is that there is no way to not "compact" ephemeral heaps as surviving objects are placed sequentially in e.g. Gen1.

[fabianoliver]

I was referring to gen 0/1/2, but you make a great point - promotion with gen-specific heaps can't happen without relocation of course. With that in mind, the two options would appear to be:

Make this hypothetical setting effectively disable generations. Gen 0 and gen 1 are no more. Everything goes right into gen2 (or loh/poh). I'd be curious to see the effect, but would imagine performance impact would be so horrible that this is a nonstarter.

Option 2, Promotion still relocates and still causes stw pauses. However, the hypothetical new setting would still be able to disable gen2 compaction, thereby presumably getting rid of nonconcurrent gen2 collections?

[neon-sunset]

Make this setting effectively disable generations.

This would require a drastically different design. We do have an example of such design in the form of Go's specialized Garbage Collector. However, in order to make this work Go makes significant concessions that are likely to produce the results you may not be happy with:

• Go uses virtual dynamically-sized stack and heavily relies on escape analysis to place its struct boxes on this virtual stack. Go's GC assumes that objects placed on the heap will have longer than trivial lifetime and were it to be generational - the object survival rates would make generational design inefficient
• Go's can only ever be statically compiled and offers limited reflection in order to not violate escape analysis assumptions
• Go's GC allocation throughput straight up does not scale with cores. At some core counts it exhibits negative scaling. At all heap sizes it has poor throughput.
• When Go's GC cannot keep up with allocation rate, allocation throttling causes allocation-bound code to regress terribly. Companies with large scale and high-load deployments experiencing this go to great lengths to avoid this. Some of them have moved to Rust since then as Go offers limited options for hands-on memory management (unlike .NET languages)

While the above might seem like a workable combination, Go applications continue to significantly struggle in high-throughput scenarios and GoGC runtime does not have immediate solutions to get itself out of this predicament.

There are better and more interesting GC implementations in the land of Java: Shenandoah and ZGC. However, I do not know how well (if at all) they would work with byref pointers and object pinning (as a start). Both make .NET truly one of a kind platform with unique value proposition.

So, rather than attempting to change the GC implementation (and we could absolutely use low-pause or pauseless GC mode for consistent latency!), it would be great to know what kind of production issue you are trying to tackle.

If your allocation profile leads to almost all objects dying in Gen0, it is worth to tune the GC aggressiveness. I have measured that simply using SRV GC + DATAS can substantially impact P99 application latency.

Below you can see the dotMemory profile of an application and cumulative pause time highlighted by yellow segments. Note how the DATAS's heap scaling algorithm substantially increases pause frequency while reducing their cumulative time. While the total pause time remains (and likely slightly increases), the individual pause time decreases substantially. I feel like this could be a good start. From there, you can also tackle individual allocation sources that do not play nicely with .NET's GC (finalizable objects, dependent handles, LOH allocations, allocations with unnecessarily extended lifetime). This has a good chance of getting you to the desired SLA.

[fabianoliver]

Thanks for your insightful thoughts there, quite interesting to hear how something like that played out with go.

For context - I fortunately don't have any massively significant issues with pause times, but that's a bit of selection bias; projects where pause times were deemed unacceptable simply didn't choose c# to begin with (though some did choose java - back then thanks to azul c4, though as you say, there are now some new kids on the block for java as well). So my question is more a mix of academic curiosity & general interest for c# to gain a stronger foothold in "my field".