Skip to content

On Profiling

Jump to bottom
Andre Merzky edited this page Oct 12, 2016 · 10 revisions

RP is a research tool first, and user tool second. To support basic CS research and experiments, RP is instrumented and tunable in ways not strictly necessary in an end user context.

This page describes 2 aspects: profiler instrumentation and profile analysis.

Profiler Instrumentation

RADICAL Utils provides a Profile class. The class' main purpose is to take fine grained time events at certain points during the RP execution. An example usage is below:

import radical.utils as ru

prof = ru.Profiler('test')

prof.prof('event 1', msg='test')
prof.prof('event 2', msg='test', uid=123)
prof.prof('event 3', msg='test')
prof.close()

Running that script results in the following profile ./test.prof:

#time,name,uid,state,event,msg
1457141272.44,test:,,,sync abs,1457141272.44:1457141272.44:1457141272.44:sys
1457141272.46,test:MainThread,,,event 1,test
1457141272.48,test:MainThread,123,,event 2,test
1457141272.53,test:MainThread,123,ACTIVE,event 3,test
1457141272.90,test:MainThread,,,QED,

Times are seconds since epoch. The first entry contains a reference point to the global clock (retrieved via an NTP callout to ensure global synchronization).

Each RP component creates one or more profiles. Those can be combined into a single profile in post-mortem operations. In that process, the sync timestamps are used to compute the correct time offset between events from different profiles, no matter from what resource they originate (system clock errors are corrected).

Profiler Precision

The nominal precision of the timings is 0.0001 seconds. The global syncing during recombination introduces an additional error, since NTP has a potential systematic bias for assymetric network routes. That error should not exceed a couple of milliseconds, at most. Further uncertainties arise from the nature of the implementation, ie. from python call stack overhead -- which though should be below the nominal precision by some orders of magnitude.

Profiler Probes

RP has profiler probes throughout its code base, and many of them are subject to change w/o notice. Those usually inform the development process. Others are considered to be stable, and aim to support experiments. Among the latter class are specifically:

  • all unit and pilot state transitions
  • all API level object creations and destructions
  • all component creations and destructions
  • all invocations of callbacks
  • all event communications

Of those items, the pilot and unit state transitions are likely the most important ones.

For the component and event communication, it is likely worthwhile to reference the RP architecture, as it shows the components and event communication channels which are profiled. The relation between the state models and the RP architecture is depicted here.

The central part of RP is densely instrumented -- about 5% of the code consists of profiling probes. However, profiling is lightweight enough that it in itself does not significantly impact overall RP performance nor total application runtime.

Profile Analysis

Profiles are written in CSV format. A number of utility methods exist to retrieve, combine and convert profiles. The profile_analsysis example demonstrates how profiles can be fetched, recombined, transformed into Pandas data frames, and evaluated.

We also provide the radical.analytics module to analyse RP profiles.

Clone this wiki locally