Garbage Collection when your system does not have enough memory in your Python App

Garbage collection is an automatic memory management feature in many programming languages that reclaims memory occupied by objects that are no longer referenced by the program. Instead of the programmer explicitly deallocating memory, the garbage collector periodically identifies and deallocates unreachable objects, preventing memory leaks and simplifying memory management.
 

Python employs a garbage collection system primarily based on reference counting, supplemented by a cyclic garbage collector. Reference counting tracks how many references point to each object; when the count reaches zero, the object is immediately deallocated. The cyclic garbage collector detects and reclaims objects involved in reference cycles (where objects mutually refer to each other, preventing their reference counts from reaching zero), preventing memory leaks. This combination ensures automatic memory management, freeing developers from manual memory allocation and deallocation.
 

Below code is tested on CentOS 7 and verified working great when we process big data.

def CHECK_OS():
	import platform
	return platform.system()

def FREE_MEMORY():
    """
    Force the Python garbage collector to release unused memory back to the system.
    """
    import gc
    
    # Run garbage collection
    gc.collect()
    
    # If running on Linux/Unix, attempt to release memory back to the system
    if CHECK_OS().lower() in ['linux', 'darwin']:
        import ctypes
        libc = ctypes.CDLL('libc.so.6')
        # MALLOC_TRIM(0) releases memory back to the system if possible
        libc.malloc_trim(0)