Unify call path wildcard matching mechanics

[akx]

Terminology: "call path":

ruff/crates/ruff_python_ast/src/call_path.rs

Lines 5 to 6 in e9acb99

	/// A representation of a qualified name, like `typing.List`.
	pub type CallPath<'a> = SmallVec<[&'a str; 8]>;

• Ruff's repo uses both the glob and globset crates; it should maybe only use one (likely globset, since it's more capable and has tricks up its sleeve to optimize matching – though at a quick glance, it doesn't have an ergonomic API for matching only one glob – but then again, maybe using a globset in all cases would be fine).
• There are multiple implementations of "does this call path match a given pattern" – isort uses one, and Support glob patterns in pep8_naming ignore-names #5024 implemented another, and I was just about to implement a third one in WIP/RFC: Wildcards for banned_api #8274 before taking a step back, a deep breath, etc.

In a perfect-er world, I think we'd have a nice generic CallPathMatchMap<T> type that would allow a Ruff developer to map from patterns (supplied by an user) to some other data of type T; for isort, the other data would be the import category, for pep8-naming ignore-names it would probably just be true, for banned_api it would be the msg (and maybe other future data). (For additional cleverness & performance, that type could fall back to being just a FxHashMap when it knows no pattern needs wildcard matching; on the other hand,

One issue with using either glob or globset for matching call paths is that the "deep star" (**) element won't be supported as one might expect (IOW, I'd expect foo.*.bar to match foo.x.bar, foo.y.bar but not foo.x.y.bar) since * is supposed to stop at path delimiters (/), not at dots. I can think of two solutions for this: either add support for alternate element delimiters in the upstream glob or globset crates, or represent call paths internally with slash-delimited paths. (This could of course be hidden behind that nice facade; no one would need to know. 🤫)

[akx]

Oh, hey, @BurntSushi, since you're active in this repo too (❤️), would you be open to allowing alternate (non-path-slash) separators in globset? That would solve part of this very neatly.

[BurntSushi]

I'm not sure. I'm actually in the process of rewriting globset (in my free time). Could you open an issue here? (globset doesn't yet have a proper home of its own yet, although I anticipate that it will once I'm done my rewrite.)

I grant that it is an interesting thought. It's not immediately obvious to me that it's plausible to do though. globs are pretty deeply coupled to file paths. One other possible direction here, instead of changing the glob semantics, you define a mapping between a call path and file paths. So, for example, replace . with / in the call path before doing glob matching.

[akx]

Could you open an issue here?

Done! BurntSushi/ripgrep#2703

One other possible direction here, instead of changing the glob semantics [...]

Yeah, that's my last suggestion in my original message here 😅 It would entail copying the Python qualified names to the path-like format, which sounds like unfortunate additional work.

[BurntSushi]

It is additional work, but it's not clear whether it will have a meaningful impact on an end user's experience though.

[MichaReiser]

I haven't thought much about the problem but my thoughts around your proposal (and some related thoughts)

• I would love to have a newtype wrapper for CallPath that hides the internal representation (whether it is a SmallVec or something else)
• A newtype wrapper would also allow us to implement additional functionality on the type, e.g. matching a CallPath
• Having a CallPathMatchSet or similar that allows for fast matching of CallPaths might be interesting.

[charliermarsh]

Agree!

[akx]

There seems to be a whole lot of code that expects CallPath to just implement everything SmallVec does (understandable) – my rust-fu isn't enough to figure out how to make a newtype CallPath implement all of that easily 😮‍💨

[MichaReiser]

Are there common patterns?

I would prefer not exposing the entire SmallVec API and instead provide a limited API with semantic methods (which would also make it easier to replace our internal representation in the future)

[akx]

@MichaReiser A variety of Iterator-related things and mutations such as push, pop, extend_from_slice, etc... I got to about main...akx:ruff:callpath-v2 until I realized that this is probably not the right way to go about this.

[MichaReiser]

That does look reasonable to me. So there are two problems:

1. Construction of CallPaths. We can provide constructor methods or consider a builder pattern CallPathBuilder.
2. Querying of CallPaths: Methods that allow matching on and inspecting CallPaths