Convert raw strings to non-raw when fixes add escape sequences (#13294)

[ThatsJustCheesy]

Summary

This aims to resolve #13294 by implementing raw to non-raw string conversion, as suggested by @dscorbett. The conversion is comprehensive (as far as I'm aware), but it does introduce an unfortunate amount of complexity to the rule.

I'm new to this codebase and not a Rust expert, so my code might not be idiomatic.

Test Plan

I have added the following test cases:

raw_single_singlequote = r'\ \' " �'
raw_triple_singlequote = r'''\ ' " �'''
raw_single_doublequote = r"\ ' \" �"
raw_triple_doublequote = r"""\ ' " �"""
raw_single_singlequote_multiline = r'\' \
" \
​'
raw_triple_singlequote_multiline = r'''' \
" \
�'''
raw_single_doublequote_multiline = r"' \
\" \
�"
raw_triple_doublequote_multiline = r"""' \
" \
�"""
raw_nested_fstrings = rf'\ {rf'\ {rf'\' '}'}'

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

[dscorbett]

It would also be good to test triple-quoted strings whose final characters match the outer quotation marks:

r"""​\""""
r'''​\''''

The raw_nested_fstrings tests don’t include any of the relevant control characters so they are not effective tests for these rules’ fixes.

[ThatsJustCheesy]

Good points, thank you!

[dscorbett]

My previous example was incomplete: quotation marks in triple-quoted strings need escaping when they precede two more of the same quotation mark. This can also happen in the middle of a string:

r"""​\"""..."""
r'''​\'''...'''

[MichaReiser]

Thanks for this contribution.

I remain hesitant adding raw to regular string conversion considering how rare raw strings is. The goal with our fixes is to fix the majority of cases. It's okay if Ruff can't fix all cases. In this case, being able to fix 1-2 raw strings doesn't justify the added complexity.

My suggestion is that we simply don't provide a fix if the string is a raw-string or that we make it an unsafe fix.

[MichaReiser]

Using bytes to get the next characters panics if the next character is a non-ASCII character. We should use the chars iterator instead (they're cheap clonable)

[MichaReiser]

Nit: there can always only be at most one r or R prefix. That's why it should not be necessaryto iterate, instead you can use find (or is it position?) to get the position of the r or R character

[MichaReiser]

You can use text_len and return a TextRange. Returning a text range has the advantage that you can index directly by range

let string_content = &text[range];

Suggested change

	TextSize::try_from(text.len()).unwrap() - string_flags.quote_len(),
	),
	_ => (0.into(), text.len().try_into().unwrap()),
	};
	text.text_len() - string_flags.quote_len(),
	),
	_ => TextRange::new((0.into(), text.text_len()),
	};

[MichaReiser]

We also need to handle the case where the invalid character was the last character before the quotes in a triple quoted strings:

"""test"<invalid>"""

Let's say <invalid> is the invalid character. Removing <invalid> then results in """test"""" which is not a valid non-raw strings.

[ThatsJustCheesy]

In this case, being able to fix 1-2 raw strings doesn't justify the added complexity.

I kind of agree… imho, the fix should be marked unsafe for raw strings. I can open a new PR for that if preferred.

Regardless, this was a good excuse to learn Rust better :)

[dscorbett]

I think the rules should either offer safe fixes for raw strings or no fixes. Keeping the current fixes but marking them unsafe doesn’t seem useful, because they are incorrect.

[MichaReiser]

I agree. We should not offer fixes if we know they're incorrect. We could consider offering fixes for raw-strings if they don't contain any quotes or backslashes.