K.Boopathi
BlogNotesSecond BrainProjectsHire MeYour reading file0 pts

Memory profiling

Jun 16, 20248 min read

A core dump is generated when a Node.js process encounters a critical error that it cannot recover from, such as segmentation faults, accessing invalid memory, or other fatal errors.

  • By default, Node.js does not generate core dumps automatically. Instead, you typically need to enable core dumps on the operating system level.
  • On Linux, for example, you can use ulimit -c unlimited to enable core dumps of unlimited size.
  • Once enabled, if a Node.js process crashes, a core dump file (core.<pid>) is generated in the current working directory or the directory specified by the core_pattern setting.
const memwatch = require('memwatch-next');
 
memwatch.on('leak', function(info) {
    console.error('Memory leak detected:', info);
    
    memwatch.gc();
});

Linux sudo gcore <pid> gdb /path/to/node /path/to/core.<pid>

strings heapdump-394083985.931455.heapsnapshot | grep "twilio" → will print all the char contain twilio

kill -USR2 PID → by default nodejs will listen and create a heapdump

Event loop monitor

import { createHook } from "node:async_hooks";
import { tracer } from "/blog/event-loop-lag/v3/tracer.server";
 
const THRESHOLD_NS = 1e8; // 100ms
 
const cache = new Map<number, { type: string; start?: [number, number] }>();
 
function init(
  asyncId: number,
  type: string,
  triggerAsyncId: number,
  resource: any
) {
  cache.set(asyncId, {
    type,
  });
}
 
function destroy(asyncId: number) {
  cache.delete(asyncId);
}
 
function before(asyncId: number) {
  const cached = cache.get(asyncId);
 
  if (!cached) {
    return;
  }
 
  cache.set(asyncId, {
    ...cached,
    start: process.hrtime(),
  });
}
 
function after(asyncId: number) {
  const cached = cache.get(asyncId);
 
  if (!cached) {
    return;
  }
 
  cache.delete(asyncId);
 
  if (!cached.start) {
    return;
  }
 
  const diff = process.hrtime(cached.start);
  const diffNs = diff[0] * 1e9 + diff[1];
  if (diffNs > THRESHOLD_NS) {
    const time = diffNs / 1e6; // in ms
 
    const newSpan = tracer.startSpan("event-loop-blocked", {
      startTime: new Date(new Date().getTime() - time),
      attributes: {
        asyncType: cached.type,
        label: "EventLoopMonitor",
      },
    });
 
    newSpan.end();
  }
}
 
export const eventLoopMonitor = singleton("eventLoopMonitor", () => {
  const hook = createHook({ init, before, after, destroy });
 
  return {
    enable: () => {
      console.log("🥸  Initializing event loop monitor");
 
      hook.enable();
    },
    disable: () => {
      console.log("🥸  Disabling event loop monitor");
 
      hook.disable();
    },
  };
});

I used the basic “collectDefaultMetrics” of the prom-client package. We use Prometheus to gather metrics from instances, and Grafana to display Graphs.

About the ELU, I’m not aware of any “standard” prometheus ways to compute and report it. We do it manually via a small snippet of code like this


let lastELU = performance.eventLoopUtilization();

this._intervalRef = setInterval(() => {

// Store the current ELU so it can be assigned later.

const tmpELU = performance.eventLoopUtilization();

// Calculate the diff between the current and last before sending.

const report = performance.eventLoopUtilization(tmpELU, lastELU);

this._idleGauge.set(report.idle);

this._activeGauge.set(report.active);

this._utilizationGauge.set(report.utilization);

// Assign over the last value to report the next interval.

lastELU = tmpELU;

}, this._interval);

https://www.npmjs.com/package/prom-client

https://prometheus.io/

https://grafana.com/

https://easyperf.net/blog/2024/02/12/Memory-Profiling-Part1?utm_source=tldrwebdev

https://blog.jiayihu.net/comprenhensive-guide-chrome-performance/

https://github.com/alibaba/JS-Memory-Analysor

A deepagent to witnessing slowdowns in your test runs.

https://github.com/christian-bromann/ZeitZeuge

node trace-sync-io server

trace-sync-io detects calls to the Node sync APIs, like fs.readFileSync. So these warnings are what you are trying to catch

—kernel-tracing —expreimental-report → provide report

node —trace_gc app.js print when ever the garbage collected run and memory cleaned up

—perf-basic-prof-only-functions

Node.js, you need to use the —abort-on-uncaught-exception flag to tell the operating system to create a core dump when the program crashes

  • Basic monitoring tools like the top command or task manager
  • Modules like “memory usage” to track memory consumption
  • Heap snapshots, which show all the objects in the JavaScript heap, and who is holding on to them. Heap snapshots can be used to compare different states of the heap and identify what objects are leaking. However, heap snapshots can have a performance impact and can contain sensitive data.
  • Allocation timelines, which are similar to heap snapshots, but focus on a small time period to better understand what is being allocated.
  • Sampling profilers, which are low-overhead tools that sample the heap and show which functions are allocating memory. The speaker notes that sampling profilers are suitable for use in production.
  • Tools for debugging native leaks, such as Valgrind and the sampling profiler, which can help to pinpoint leaks in native code

Garbage Collection Basics: GC deals with the heap, where dynamically allocated objects like arrays, functions, and objects are stored. The heap is divided into pages of 512 kilobytes. The garbage collector’s job is to find live objects (those still in use), reclaim the memory of dead objects, and optionally defragment the heap.

Marking Algorithm: The process of finding live objects starts with known root pointers (stack and global object) and follows all references to other objects, marking each one as reachable. Everything not marked is considered garbage. The cost is relative to surviving objects, not all allocated objects.

Generational Heap Layout: The V8 heap is divided into a young generation and an old generation, with three ages for objects: nursery, intermediate, and old. - Nursery: New objects are allocated here. - Intermediate: Objects that survive the first GC are moved here. - Old Generation: Objects that survive another GC are moved here.

Two Garbage Collectors: V8 uses two independent garbage collectors: - Minor GC (Scavenger): Deals with the young generation. It uses an evacuation process where live objects are copied from “from space” to “to space,” and then the spaces are switched. - Major GC (Mark-Sweep-Compact): Deals with the entire heap. It finds gaps where objects are unreachable and puts the space on a free list. It compacts fragmented pages to save space.

Generational Hypothesis: Most objects die young, so the generational layout optimizes for this by only copying objects that survive garbage collection.

Orinoco Project: This project aims to transform the V8 GC from a purely sequential process to a concurrent and parallel one. The goals are to free the main thread of garbage collection work, improve main thread latency, and ensure JavaScript keeps running smoothly.

Key Concepts in Orinoco:

  • Parallel GC: The main thread and helper threads split the work and do it at the same time, reducing pause times, though still stopping the world.
  • Incremental GC: GC work is divided into small pieces and interleaved with JavaScript execution, but requires fix-up steps.
  • Concurrent GC: JavaScript continues to run on the main thread while helper threads do garbage collection work, requiring more synchronization.
  • Modern V8 GC: The V8 GC uses parallel scavenging (up to 7 helper threads) with interleaved marking, evacuation, and pointer updating for the minor GC (resulting in ~5ms pause times). For the major GC, it uses concurrent marking, parallel compaction and updating, and concurrent sweeping with background tasks to reduce main thread pauses (resulting in ~9.8ms pauses).

GC Triggers:

  • Minor GC: Triggered when the new space runs out of space.
  • Major GC: Triggered by a combination of factors, including allocation rate, object size, and survival rate, with the system trying to dynamically compute an optimal limit.

V8

V8 heap statistics

Tools

ssh -L 8080:localhost:8080 admin@example.com Port forwarding to remote machin we can access remote machine 8080 port form local

CPU profileing

V8 file structure

{
  nodes: [ /* array of Node objects */ ],
  nodesById: { /* mapping of Node ID to Node object */ },
  edges: [ /* array of Edge objects */ ]
}

Each Node represents a JavaScript object in memory:

{
  id: 1210027,               // Unique ID of the object
  name: "system / Context",  // Name of the object (from V8 internals)
  type: "object",            // Type (object, array, string, etc.)
  self_size: 6176,           // Memory occupied by this object (bytes)
  trace_node_id: 0,          // Unused in most cases
  references: [ /* Array of Edge objects */ ],  // Objects this node references
  referrers: [ /* Array of Edge objects */ ]    // Objects that reference this node
}

Edges (References Between Objects)

Each Edge represents a reference between objects:

{
  type: "property",       // Type of reference (e.g., property, element, internal)
  name_or_index: "next",  // Name of the reference (e.g., property name, array index)
  fromNode: { ... },      // Object that holds the reference
  toNode: { ... }         // Object being referenced
}

[<process_id>:<thread_id>] ms: () → () MB, ms (average mu = , current mu = ) ;

https://perforator.tech/docs/en/

Graph View