Garbage collection

The problem

Errare humanum est

• For each malloc(...) should be free(...)
• For each new ... should be del ...
• ...
• Application memory is a complex structure

The GC Solution

• Set developes free from manual memory management
• Find allocated memory not longer used by running program
• Reclaim that memory

Developer free from thinking too much... ☺

The GC Solution

• Dangling pointers
• Double free pointer
• Number of memory leaks

The GC problems

• Overall program performance can be slower
• Program response time can be very deviated (stalls)
  • Unacceptable in real-time systems?

GC in languages

• Java
• C# (.Net CLR)
• JS, Python, Ruby, Lua, ...
• C, C++, D

Object graph

• VM Heap — The memory from which VM allocates all class instances
• GC Root — object accessible from outside the heap
  1. GC traverses object graph
  2. Finds unreferenced
     from GC root subgraphs
  3. Removes unreferenced objects

Tracing GC algorithms

• Mark and sweep
• Tri-color marking
• ...

Mark and sweep

1. Traverse object graph from GC roots
2. Mark each each reachable object (set a flag)
3. Upon traversal remove all unmarked objects

Mark and sweep (MARK)

Mark and sweep (MARK)

Mark and sweep (MARK)

Mark and sweep (MARK)

Mark and sweep (MARK)

Mark and sweep (SWEEP)

Mark and sweep (SWEEP)

Simple mark and sweep drawbacks

• Full memory scan causes unnecessary OS pages IO
• Memory freezes during mark phase (No writes/reads).

Tri-color marking

Maintain 3 objects sets.

1. Initially all objects are white
2. Grey all GC roots
3. While gray set is not empty
   • If it has white child → color child grey
   • Otherwise color node black
   • Go to the step 3

Black nodes points only black or gray nodes.

If gray set is empty — all nodes in white set can be garbaged.

Tri-color marking

• Maintain W-G-B sets on the fly by intercepting:
  • Objects creation
  • Objects mutations

As result you do not have perform full memory scan.
Look at the gray set size and if needed free all objects from white set.

Tri-color marking + Mark-and-sweep

• Store in object bits (W-G-B) status

Mark-and-sweep-compact (MSC)

Mark-and-sweep-compact (MSC)

pros:

• Reduced memory fragmentation
• Releases unused memory to OS

cons:

• Memory moving.
• Pointers became not valid after object moving.

Concurrent mark-and-sweep (CMS)

TODO

Stop-and-copy GC

Two equal memory areas. Cheney's algorithm (1970).

1. Mark
2. Move live objects into free memory area
3. Prune old area and make it free
4. Now roles of Area1 ↔ Area2 flipped

Stop-and-copy GC explained

1. Managed memory is divided into two equal spaces. One is labeled "from-space", the other "to-space".
2. New objects are always allocated at one end of to-space.
3. An incremental collecter traverses from-space, starting with a "root set" of known-live objects, and copies each object encountered into the other end of to-space. This collector replaces the object in from-space with a "forwarder" to the to-space object.
4. An incremental "scanner" process scans the objects copied into to-space field by field, copying objects referenced by them from from-space to to-space. This scanner leaves behind a forwarder for each object that is copied. The scanner stops when it reaches the end of the objects that were copied into to-space by the incremental collector.

Stop-and-copy GC explained

1. Any access to a forwarder is redirected to the to-space object referenced by the forwarded, and the original reference is fixed to reference the to-space object.
2. When a reference to a from-space object is stored into a to-space object, the referenced object is copied to to-space and a forwarder left behind.
3. At any time after the scanner has finished, a "flip" takes place, where the to-space and from-space pointers are reversed, the (new) to-space is cleared, and the root set is copied.

Stop-and-copy GC

pros:

• Removes memory fragmentation
• Releases unused memory to OS

cons:

• Need to have two copies of the heap.
• Memory moving.
• Pointers became not valid after object moving.

Generational GC

Some empirical observations:
Most of resently created objects have a short lifetime

            Integer c = 0;
            for (int i = 0; i < ...; ++i) {
                c  = new Integer(i + c);  //replace old c with new one
                ...
                if ((c % 100) == 0) break;
            }
            //We have a bunch of young short-lived `c` objects
        

• Let us divide all heap objects for long-lived and short-lived
• Perform frequent GC on short-lived objects
• Perform less-frequent GC on long-lived objects
• Separate memory regions for different ages

Reference counting GC

• Each object holds number of references (RC) to it
• RC incremented on every new reference to object
• If RC is 0 an object is deleted

• CPython
• Mozilla XPCOM (Mozilla Firefox internals)
• C/C++ GC implementations

Problems:

• Very CPU consuming: RC changed on every assigment
• Atomic RC changes in multithreaded env
• How we can handle a cyclic references?

GC in JVM

• Serial GC
• Parallel GC
• Parallel Old GC
• Concurrent mark-and-sweep
• Garbage First (G1) GC

All GC in JVM uses generational GC model.

Review in next slides the JVM memory model used for generational GC.

Java memory model

1. All new objects created in Eden space
2. Eden objects survived in minor GC
   promoted into S1/S2 space.
3. Survived objects from S1/S2
   promoted into Tenures space.
4. From Java 8 Perm Generation has been replaced with Metaspace.

Three different types of GC

• Minor GC It’s also called as Scavenge GC. This is the GC which collects garbage from the Young Generation.
• Major GC This GC collects garbage from the Old Generation
• Full GC This GC collects garbage from all regions i.e. Young, Old, Perm, Metaspace.

When Major or Full GC run all application threads are paused. It’s called as stop-the-world events. In Minor GCs also stop-the-world event occurs but momentarily.

Scavenge GC

• For young object GC the copy-collection algorithm is used.
• On each minor GC iteration (when Eden space is full)
  1. All live objects are copied to another location (S2).
  2. Previous location is marked as clean.
  3. S1 ↔ S2 are swapped

Scavenge GC

How can we detect live Eden objects?

• Find all GC roots and perform mark-and-sweep.
  • GC roots from thread stack.
  • GC roots from old space.
• But searching GC roots from old space requires full scan on it.
  • To avoid full scans of refs from old space the duty pages list is used.

Full GC (Duty pages list)

Instead of scanning full old space for refs on Eden objects

1. Cleanup duty-page list after each minor GC
2. Track each refence change, e.g.: foo.bar = something;
3. And mark modified by reference segment of memory(page) as duty(red).
4. When minor GC started, inspect only duty pages for references into Eden.
5. Peform GC and goto step 1.

GC in JVM

• Serial GC
• Parallel GC
• Parallel Old GC
• Concurrent mark-and-sweep
• Garbage First (G1) GC

Serial GC

-XX:+UseSerialGC

Not used in production.

• GC executed by single thread
• Young collection is a copy collection. Requires STW (stop-the-world)
• Old space is collected using an implementation of MSC algorithm. Requires STW (stop-the-world)

Parallel GC

-XX:+UseParallelGC

• As the name stated multiple threads performed GC
• Reduced STW time
• Young space GC in parallel
• Old space GC by single thread

cons:

• Long pauses for old space GC

Parallel old GC

-XX:+UseParallelOldGC

Old is no the GC implementation age ☺

• GC of young space is the same as in "Parallel GC"
• GC of old generation performed by parallel MSC (mark-and-sweep compacting).

pros:

• Reduces STW by parallel processing old gen

cons:

• Old space GC mostly in STW

Concurrent mark and sweep GC (CMS)

-XX:+UseConcMarkSweepGC

• Young gen as previous
• Old gen GC by concurrent-mark-and-sweep.
• Old gen space is not compacted.

pros:

• Small STW pauses
• Old space GC mostly not in STW in parallel with application
• Do not relocate object in memory

cons:

• Memory fragmentation

Concurrent mark and sweep GC (CMS)

1. Initial mark. Short STW to find GC roots
2. Concurrent mark. Parallel tracing from GC roots (not in STW)
3. Remark. Fix garbage produced by app during previous phase. (performed in STW)
4. Concurent sweep (not in STW)

Concurrent mark and sweep GC (CMS)

• If memory fragmentation is high and CMS unable to allocate new objects it may fallback to serial mark-and-sweep-compact to defragment old space.
• Serial MSC STW pause is 100-500 times longer than CMS.

G1 garbage collector

-XX:+UseG1GC

• Incremental MSC
• Divide heap into small 1-32Mb chunks and perform partial collection.

pros:

• High throughput

cons:

• Overall time of pauses is higher than in CMS
• On large heaps not scales as well as CMS
• Weak on Java 7

Used as default GC in Java 9

GC tuning showcase

TestGCPause.java

• 1/2 of old heap filled
• 2 000 000 messages
• 50000 garbage objects
  every 0.2 sec produced

Measure the largest message delivering time ~ max GC pause

GC tuning showcase

GC tuning showcase

 - For small heap size ~64M  SerialGC is effective
 - STW time grows proportionally to the heap size.

GC tuning showcase

• Time of full GC generally proportional to heap size
• => Be very careful with -Xms JVM option
• Allow heap to be small

Turning On GC debug

            -XX:+PrintGC
        

...
[GC 139776K->135432K(506816K), 0,9174710 secs]
[GC 275208K->263794K(506816K), 1,0039820 secs]
[Full GC 403570K->384989K(506816K), 2,0204030 secs]
...
        

[GC <allocatedMemoryBefore>-><allocatedMemoryAfter>(<heap size>), <time> secs]
[Full GC <allocatedMemoryBefore>-><allocatedMemoryAfter>(<heap size>), <time> secs]
        

Turning On GC debug (get more info)

            -XX:+PrintGC -XX:+PrintGCDetails
        

[GC [DefNew: 139776K->17472K(157248K), 0,8694740 secs]
            139776K->135432K(506816K), 0,8696040 secs]
            [Times: user=0,76 sys=0,11, real=0,87 secs]

[Full GC [Tenured: 349567K->253689K(349568K), 1,6324180 secs]
            506815K->253689K(506816K),
            [Perm : 2769K->2769K(21248K)], 1,6325410 secs]
            [Times: user=1,64 sys=0,00, real=1,63 secs]
        

GC tuning showcase

 - Turning -XX+UseConcMarkSweepGC cause poor results :(
 - Let's go to under the hood with -XX:+PrintGC -XX:+PrintGCDetails ...
        

-XX:+UseConcMarkSweepGC

• ParNew — parallel young generation GC
• CMS-concurrent-abortable-preclean — the concurent part of remark phase
• concurent mode failure — concurrent GC of the tenured generation did not complete before the tenured generation filled up.

The problem

• We have the high rate of young objects promotion
• Promoted objects fill tenured heap before concurrent GC finished
• => Concurent GC forced to raise STW and perform serial heap GC - it is slow operation.

Possible solutions

• Increase the Tenured heap size: increase -Xmx
• Perform CMS more frequently
  • -XX:CMSInitiatingOccupancyFraction=68
  • -XX:+UseCMSInitiatingOccupancyOnly
• Force object to stay a bit longer in young space
  • Tune Eden/Survivor space ratio -XX:SurvivorRatio=8
  • The percentage of the survivor spaces which must be used before objects are promoted to the Tenured generation
    -XX:TargetSurvivorRatio=50
  • How long the objects in the Young generation may age.
    -XX:MaxTenuringThreshold=4
  • ...

GC tuning is an experiment with many trials/failures

1. Discover application specific characteristics which can affect on performance
2. Remember the Pareto principle: 80% of application bottlechecks usually caused by 20% of code

Examine TestGCPause.java

1. Very high ratio of short lived garbage objects produced by `GarbageProducer` thread.

GC tuning is an experiment with many trials/failures

1. Turn GC logging on
2. Change VM options (Use dichotomy here)
3. Change application workload parameters
4. Run application
5. Evaluate results
6. Inacceptable? Go to the steps 2 or 3

Our results TestGCPause.java

• 2x application speed improved
• >10x GC pauses reduced

Resources

• Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine
• Java HotSpot VM Options
• Understanding CMS GC Logs
• Diagnosing a Garbage Collection problem
• Сборщик мусора Concurrent Mark-Sweep (CMS)
• How to deal with concurrent mode failures in the Hotspot JVM - See more at: http://cybergav.in/tag/usecmsinitiatingoccupancyonly/#sthash.1bpMJuQJ.dpuf
• Understanding GC pauses in JVM, HotSpot's minor GC
• Garbage collection in HotSpot JVM

Resources

• Understanding Java Garbage Collection
• Using JConsole