Feature: add a clef_map

[leleogere]

part.key_signature_map and part.time_signature_map are very useful, as they allow retrieving the musical context in any part of the score.

However, I feel that a part.clef_map is missing, that would get the current clef in each staff so far. I'm not sure what the best interface would be for that, as multiple values could be returned in case of multi-staff scores (oppositely as with key/time signatures).

Like this, it would allow for example to know that at time 280, there is a treble key on first staff, and a bass key on second staff. It would be very helpful in contexts of retrieving the full context at each point of the score (e.g. see #380).

[sildater]

Hi! Good idea! I don't know whether it fits the map API as these have unique values per time point. Maybe someone else has an idea how to implement this. In the meantime, I added a quick note feature encoding the active clef per note.

[sildater]

It's not yet tested though.

[leleogere]

Currently, the map API has the following:

• time_signature_map returns a 1D array of 3 elements: [beats, beat_type, musical_beats]
• key_signature_map returns a 1D array of 2 elements: [fifths, mode_as_int] (at some point documenting the mode might be a good idea as currently I had to look at the code to get that major=1 and minor=-1)

As there should be as many clefs as there are staffs, a clef_map could return a 2D array of elements [staff, clef_as_int]. For example, a classic piano score with treble/bass clef on staff 1/2 could return [[1, 0], [2, 1]] with treble=0 and bass=1 (clef/int mapping should be documented too, maybe ideally with an Enum).

Returning a dictionary mapping directly staff_id -> clef might be a good alternative, but it would not be consistent with other map methods that return arrays and not dictionaries.

[sildater]

That's a good idea! The actual clef encoding might require some more details because even though there are only few standard clefs, they can have varying properties like the line they are placed on or octave changes. Something like [[staff_no1, staff_no2], [[clef_sign1, clef_line1, clef_octave1], [clef_sign2, clef_line2, clef_octave2]]] or a more complex attribute to integer mapping. It's still an unknown output size. Moving the attribute to integer conversions to globals is ongoing work.

[leleogere]

Yes, adding those other properties make sense. However, why this shape? The example you gave can't be directly converted as a numpy array because of shapes: [ (2,) (2, 3) ] (and therefore interpolated with scipy). Wouldn't it be possible to have a simple:

[
    [staff_number1, clef_sign1, clef_line1, clef_octave1],
    [staff_number2, clef_sign2, clef_line2, clef_octave2],
]

I haven't digged down to how staff numbers are managed in partitura, but if you always number them sequentially, the staff_number property could even be dropped and resulting in

[
    [clef_sign1, clef_line1, clef_octave1],
    [clef_sign2, clef_line2, clef_octave2],
]

with the first line representing implicitly the first staff, and so on.

It's still an unknown output size

Is it a problem? For a 2 staves part, I would expect a shape (2, 3), and for a n staves part, I would expect a shape (n, 3), I do not see this as a problem.

[sildater]

yes that shape is better! Variable output size not a breaking problem, it's something to be aware of and adapt one's code to, especially in the case of bulk processing.

[leleogere]

By the way, I have a more general question about those map functions. Why did you choose to return numpy arrays rather than the most recent TimeSignature/KeySignature object? This would allow to directly use and understand the returned object, rather that having to understand what each dimension represents.

I feel like something similar to this might be easier to understand for the user:

    def time_signature_map(self):

        [......]

        interpolator = interp1d(
            tss[:, 0],
            tss[:, 1:],
            axis=0,
            kind="previous",
            bounds_error=False,
            fill_value="extrapolate",
        )
        
        def map_function(time: int) -> TimeSignature:
            ts = iterpolator(time)
            return TimeSignature(beats=ts[0], beat_type=ts[1])
        
        return map_function

and same for key_signature_map, that would return a KeySignature object:

    def key_signature_map(self):

        [......]

        interpolator = interp1d(
            kss[:, 0],
            kss[:, 1:],
            axis=0,
            kind="previous",
            bounds_error=False,
            fill_value="extrapolate",
        )
        
        def map_function(time: int) -> KeySignature:
            ks = iterpolator(time)
            return KeySignature(fifths=ts[0], mode=pt.utils.music.key_int_to_mode[1])
        
        return map_function

[leleogere]

Here is a simple implementation of what clef_map could look like, taking inspiration from existing time/key_signature_map. As there should be as many clefs as staves, this simply build an interpolator per staff, and then loop over staves, using the staff's interpolator to get the clef on this staff (defaults to a treble key when a staff has no key, maybe better to default to "none"?).

available_clefs = ["G", "F", "C", "percussion", "TAB", "jianpu", "none"]

def clef_name_to_int(clef: str) -> int:
    return available_clefs.index(clef)

def clef_int_to_name(clef: int) -> str:
    return available_clefs[clef]

def clef_map(part: pt.score.Part) -> Callable[[int], pt.score.Clef]:
    """A function mapping timeline times to the clefs at that time. The function can take
    scalar values or lists/arrays of values.

    Returns
    -------
    function
        The mapping function

    """
    clefs_objects = np.array(
        [
            (c.start.t, c.staff, clef_name_to_int(c.sign), c.line, c.octave_change if c.octave_change is not None else 0)
            for c in part.iter_all(pt.score.Clef)
        ]
    )
    
    # clefs: (time, staff_id, clef, line, octave_change)
    interpolators = []
    for s in range(1, part.number_of_staves + 1):
        staff_clefs = clefs_objects[clefs_objects[:, 1] == s]
        if len(staff_clefs) == 0:
            # default treble clef
            staff, clef, line, octave_change = s, clef_name_to_int("G"), 2, 0
            
            warnings.warn(
                "No clefs found on staff {}, assuming {} clef.".format(s, clef)
            )
            if part.first_point is None:
                t0, tN = 0, 0
            else:
                t0 = part.first_point.t
                tN = part.last_point.t
            staff_clefs = np.array(
                [
                    (t0, staff, clef, line, octave_change),
                    (tN, staff, clef, line, octave_change),
                ]
            )
        elif len(staff_clefs) == 1:
            # If there is only a single clef
            staff_clefs = np.array([staff_clefs[0, :], staff_clefs[0, :]])
        elif staff_clefs[0, 0] > part.first_point.t:
            staff_clefs = np.vstack(
                ((part.first_point.t, *staff_clefs[0, 1:]), staff_clefs)
            )
        
        interpolators.append(
            interp1d(
                staff_clefs[:, 0],
                staff_clefs[:, 1:],
                axis=0,
                kind="previous",
                bounds_error=False,
                fill_value="extrapolate",
            )
        )

    def collator(time: int) -> pt.score.Clef:
        return np.array([interpolator(time) for interpolator in interpolators])
    
    return collator

This will return things like:

clef_map_function = clef_map(part)
print("Before clef change on staff 2:")
print(clef_map_function(1919))
print("After clef change on staff 2:")
print(clef_map_function(1920))

# Before clef change on staff 2:
# [[1. 0. 2. 0.]
#  [2. 0. 2. 0.]]
# After clef change on staff 2:
# [[1. 0. 2. 0.]
#  [2. 1. 4. 0.]]

(Not the easiest to read, but might be easily adapted like my previous message to return a list of pt.score.Clef objects rather than the raw numpy arrays.)

If you're interested in this implementation, I can open a PR to add it.

[sildater]

Very nice addition! Yes, please open a PR so we can straighten out some details.

[sildater]

By the way, I have a more general question about those map functions. Why did you choose to return numpy arrays rather than the most recent TimeSignature/KeySignature object? This would allow to directly use and understand the returned object, rather that having to understand what each dimension represents.

I think this is historically motivated, we initially had only time mapping between different (musical) units, then some additions were added for often used score attributes (meter, key) but with the use case being to retrieve values which can be packed into numerical arrays. The note features are in some ways the generalization of that idea. When handling scores itself and their objects rather than using partitura as a feature extractor towards numerical representations, we use the internal iterators, e.g. if you had an individual note or timepoint and want to retrieve its encompassing measure object:

a_note = np.random.choice(part.notes)
its_measure = next(a_note.start.iter_prev(pt.score.Measure, eq=True), None)

a_time = np.random.randint(part.last_point.t)
its_measure2 = next(part.get_or_add_point(a_time).iter_prev(pt.score.Measure, eq=True), None)

but of course that's a bit cumbersome and not easily accessible for library users.

[leleogere]

Very nice addition! Yes, please open a PR so we can straighten out some details.

Done! See #384

its_measure = next(a_note.start.iter_prev(pt.score.Measure, eq=True), None)

Correct me if I'm wrong, but that would iterate over objects until the previous X object (whatever X is) right? Therefore, it could be quite long for large scores (for example getting the time signature at the very end of a long score without time signature change). Wrapping the interpolant in a function to get a TimeSignature object is probably way faster, as once the interpolant is built, you can easily get the TS at any time you want, without re-iterating on all previous objects (I might try to run some tests soon to get an objective metric).