Notes on Runtime Library

Some pieces of the HILTI/Spicy runtime libraries come with some specific constraints and/or design goals. We collect some notes on that here.

Global State & Thread Safety

• Generally, the runtime library should avoid global state wherever possible to remain thread-safe. While we currently are not spawning multiple threads, we will do so eventually.

• To that end, most global state that the library needs is encapsulated in the hilti::rt::Context struct. Each thread gets its own context instance, and receives access to it through hilti::rt::context::get() (which internally uses a TLS variable). That means that all accesses to anything in Context are automatically thread-safe.

• There's a small set of truely global state in {hilti/spicy}/rt/global-state.h.

  • Some of this is read-only configuration information, which is thread-safe.
  • Some is state that needs to be managed acrosss threads; we need to review this, as currently some of that is not used in a thread-safe manner.
  • Generally, we should avoid adding any truely global state wherever we can, unless there's a strong reason for doing so.

Types

Iterable containers (bytes/list/map/set/stream/vector)

• Our iterator interfaces aren't standard compliant at the moment, we should make them so.

• Different from iterators in the C++ standard library, we need iterators to be safe against undefined behavior. We want the following semantics:

  • An existing iterator must generally remain valid as long as the underlying container sticks around.

  • An existing iterator becomes invalid in two cases:

    1. The underlying container gets destroyed.
    2. Due to container changes, the iterator's current position semantically does not "make sense" anymore. For example, a vector iterator pointing to an index i will be invalid once the vector shrinks to less than i+1 elements. (Note the "semantically" here: For example, resizing of a vector's internal array should not invalidate any iterators.)

  • Once invalid, any attempt to deference an iterator must throw an exception.

  • Even when invalid, iterator operations other than dereferencing should remain supported. For example, if one keep incrementing a vector iterator beyond the vector's size, it may eventually become valid again if the vector grows sufficiently. (This property is particularly important for streams; we'll discuss that further down below.)

  • As the previous bullet suggests, it is possible for an invalid iterator to become valid once more if the container changes accordingly.

  • Iterators must not prevent their underlying containers from being destroyed (e.g., they can't store a strong reference to the container, as that would force it to stay around until the iterator dies.)

• These safe properties came with some overhead.

  • Generally we accept that overhead.

  • Internally, however, we should avoid the overhead where we safely can. For example, if another part of libhilti-rt executes an iteration over a bytes instance where iterators can't become invalid while the operation is in progress, we should just skip the safety checks (this is the reason for the current "Safe" vs "Non-Safe" iterators; but not sure that's the best way to approach it).

  • Longer-term, the code generator could make use of such "unsafe" paths as well in cases where it can guarantee that the safety properties are still maintained by the particular code being generated.