Python’s new t-strings

The t-string produces a Template object without a __str__() method. You can’t mistakenly use an f-string in its place. Either the code expects a string, in which case passing it a Template would blow it up, or the code expects a Template, in which case passing it a string would blow it up.

And I'm guessing lots of code will expect strings to maintain backward compatibility.

If it's not a completely new library written exclusively around templates, such code currently accepts strings and will most likely continue to accept strings for backwards compatibility.

That's entirely implementation dependent. For existing libraries I would expect something like

    def get(self, query):
        if isinstance(query, template):
            self.get_template(query)
        else:
            self.get_old(query) #Don't break old code!

I'm guessing no existing functions will be extended to allow t-strings for this very reason. Instead, new functions that only accept t-strings will be created.

There's an obvious risk here, same as with strcpy (no, strncpy.. no, strlcpy... no, strcpy_s) that documentation tends to outlive code, and people keep pasting from tutorails and older code so much that the newer alternatives have a hard time cutting through the noise.

I would argue that as bad as some w3schools tutorials were, and copying from bad Stackoverflow answers, going back to MSA and the free cgi archives of the 90s, the tendency of code snippets to live on forever will only be excarbated by AI-style coding agents.

On the other hand, deprecating existing methods is what languages do to die. And for good reason. I don't think there's an easy answer here. But language is also culture, and shared beliefs about code quality can be a middle route between respecting legacy and building new. If static checking is as easy as a directive such as "use strict" and the idea that checking is good spreads, then consesus can slowly evolve while working code keeps working.

That would be a great argument if Python wasn't a language that let you reach into the internals and define __str__() for things you shouldn't be defining it for. And that is something people will definitely do because, you know, they just need something to friggin work so they can get whatever ticket closed and keep some metric happy tracking time-to-close

I think it's way more likely that existing libraries will introduce new methods that use t-strings and are type safe, rather than entirely defeat the purpose of having a t-string API.

It's pretty common for Python libraries to deprecate and remove functionality. It makes people mad, but it's a good thing, for this reason.

Do the python type checkers / linters / whatever have the ability to warn or error on calling certain functions? That would be nice to eventually enforce migration over to the newer functions that only take a t-string template

They sure do, e.g. https://docs.astral.sh/ruff/rules/pandas-use-of-dot-is-null/

it would likely be safer to have a safe (accepting Templates) and an unsafe (accepting strings) interface.

Now whether maintainers introduce `getSafe` and keep the old behavior intact, or make a breaking change to turn `get` into `getUnsafe`, we will see

> or the code expects a Template, in which case passing it a string would blow it up.

That's where the problem is though -- in most cases it probably won't blow up.

Plenty of SQL queries don't have any parameters at all. You're just getting the number of rows in a table or something. A raw string is perfectly fine.

Will sqlite3 really disallow strings? Will it force you to use templates, even when the template doesn't contain any parameters?

You can argue it should, but that's not being very friendly with inputs, and will break backwards compatibility. Maybe if there's a flag you can set in the module to enable that strict behavior though, with the idea that in a decade it will become the default?

    db.execute(t"Select Count(1) from someTable")

It's one extra letter to "force" for an unparameterized query over a "raw string". The t-string itself works just fine without parameters.

There's definitely a backwards compatibility hurdle of switching to a template-only API, but a template-only API doesn't look that much "less friendly" with inputs, when the only difference is a `t` before every string, regardless of number of parameters.

Sure, but it's just I don't have to do that anywhere else.

I never put an f in front of a string if I'm not putting variables within it.

And I'm generally used to Python inputs being liberal. I can usually pass a list if it expects a tuple; I can pass an int if it expects a float; often I can pass an item directly instead of a tuple with a single item. Regex functions take regular strings or regex strings, they don't force regex strings.

Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.

It's safer, I get that. But it's definitely less friendly, so I'll be curious to see how module maintainers decide to handle this.

Er… that’s just not correct? Python can be more liberal but it’s not always. It depends entirely on the tooling. Libraries will take time to catch up but I can definitely see people creating libraries that enforce t-strings, even if they’re deconstructing them under the hood for legacy libraries.

  I never put an f in front of a string if I'm not putting variables within it.

Linters will even complain if you have a f string without variables. I assume it will be the same for t strings.

Programmers being lazy and shit at their jobs is not a reason to not improve the language.

What's not correct? Python inputs usually are liberal. I didn't say always.

Are you claiming it's traditionally common in Python to be strict with inputs, and that being liberal is the exception?

That Python lets you blindly interchange different types for no good reason. It simply doesn’t.

Yes, it’s common for Python to be strict for inputs when the types are different. For example, try:

Decimal(‘3.0’) / 1.5

You’ll get an error and for good reason.

And they could add deprecation warnings gradually

For the reasons discussed above, I'm not sure that it will be the case for t-strings. I think it'll take a little while for frameworks/libraries to adapt (while still maintaining backward compatibility) and a while for best practices to find their way into our linting and other tools.

Linters complain because f"hello" and "hello" are the _exact_ same string. t"hello" isn't even a string.

Yeah. A while back I was poking through some unfamiliar code and noticed that my editor was rendering a use of `datetime.utcnow()` as struck through. When I hovered it with my mouse, I got a message that that function had been deprecated.

Turns out my editor (vscode) and typechecker (pyright) saw that `datetime.utcnow()` was marked as deprecated (I know one can use the `@deprecated` decorator from Python 3.13 or `__future__` to do this; I think it was done another way in this particular case) and therefore rendered it as struck through.

And it taught me A) that `utcnow()` is deprecated and B) how to mark bits of our internal codebase as deprecated and nudge our developers to use the new, better versions if possible.

But... it usually does. For example, try:

    Decimal('3.0') / 2

It works fine. It doesn't work with a float, which is for good reason. That's the whole point -- its general philosophy is to be pretty liberal with types, except when there's a good reason not to be. Heck, you can even do dumb things like:

    4 + True

And get 5 back. If that's not "blindly interchanging different types for no good reason" then I don't know what is. You can even multiply your Decimal object by False and get an answer...

Or it's like my original example -- the Regex module isn't restricted to r-strings. It happily works with regular strings. Python's general philosophy is to handle input liberally. Even type hinting is an add-on. Now, it doesn't go as far as JavaScript in allowing e.g. "4"+1, but it's still awfully liberal. I just don't see how you can claim otherwise.

I know about Decimal/int mixing but that’s for a good reason: it’s fine to intermix here. But not for floats (precision issues). The bool/int mixing isn’t “good.” It’s a bad implementation detail that Python is stuck maintaining forever. I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.

The reason it works is because Python functionally has no bool type. True and False are just integers with names. It’s stupid and shouldn’t work like that but it does for historic reasons.

Your example of regex makes no sense either. There is no difference between strings and r-strings. They’re literally the same thing to the interpreter, so how could the regex functions enforce you use r-strings? Maybe they should be different but, for historic reasons, they can’t be without Python 4.0.

If you can use a string anywhere you can use a t-string, then a non parametrized t-string is a code smell (lining error). If there is a dedicated template-string API, then there is the implicit threat you are breaking backwards compatibility to stop using regular strings.

> If you can use a string anywhere you can use a t-string

You can't; they're different types. t-strings are not `str`

It's up to good framework/API design to take advantage of this.

> I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.

You seem to be having a different conversation than I am.

I'm just describing Python as it is. I'm not defending it. I know why you can add True to a number, or else I wouldn't have come up with the example. And I know perfectly well that r-strings are just strings. Python easily could have made them a distinct object, to force people from ever making backslash errors, and restricted Regex functions to them, but didn't.

My only point has been, "Pythonic" things tend to be pretty liberal in what they accept. Type hints aren't even enforced, when they exist at all. You seem to think it shouldn't be that way. Great! But regardless, claiming it's not that way -- that Python is somehow this strict language -- is just mischaracterizing it.

> My only point has been, "Pythonic" things tend to be pretty liberal in what they accept

Being able to use a string as a string and an int as an int are not “pretty liberal in what they accept,” it’s just programming language theory 101! I think you’re mistaking duck typing for “liberal acceptance,” which are not the same thing. There’s always been an expectation that you should use compatible interfaces, even within the standard library. I’ve been bitten enough times by passing a generator in when a function expects a list, for example.

> Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.

Maybe that's partly the disconnect here? "t-string" is probably a confusing colloquial name because they aren't strings, they are Templates. The runtime type is a Template. It is a very different duck-type from a string. As a duck-typable object it doesn't even implicitly or explicitly act like a string, there's intentionally no `__str__()` method and `str(someTemplate)` doesn't work like you'd expect. It shouldn't be a surprise that there is also no implicit conversion from a string and you have to use its own literal syntax: it isn't a string type, it's a Template type.

Python here is still liberal with respect to Templates (it is still a duck type). If a function expects a Template and you don't want to use the t"" shorthand syntax nor use the Template constructor in string.templatelib, you just need a simple class of object that has an `__iter__()` of the correct shape and/or has `strings` and `values` tuples.

Sure, it may make sense for some types of APIs to support a Union of str and Template as "liberal" options, but it's a different class of liberal support from Union of list and tuple or Union of int and float which are closer "domains" of types. A Template isn't a string and at runtime looks nothing like one (despite how syntactically it looks like one at "compile time"). Given `__iter__()` in Template, it may make more sense/would be more "natural" to Union Template with List or Tuple more than with a single string.

Yeah, "t-string" is possibly a misnomer, because they are in fact at runtime a Template object (from string.templatelib).

I'm not mistaking it at all. Yes, duck typing is very much liberal acceptance, but Python code tends to go much farther. I could give a million examples -- like how in Python's isinstance() the second argument can be a type or a tuple of types, or in sqlite3 that you can run queries on a connection or on a cursor, and don't even get me started on Matplotlib or Numpy. It's just idiomatic in Python to make things easy for the programmer by accepting multiple types when possible. If you don't recognize this as the common Python pattern, I literally don't know what else to tell you.

A library writer ultimately has to decide if they accept both types. For a database cursor, do you take regular strings + parameter arguments and template strings? Or dedicate a new API to the idea?

  cursor.execute(“select * from x where foo=?”, {foo=1})
  # while also allowing
  cursor.execute(t“select * from x where foo={foo}”)
  #Vs 
  cursor.executetemplate(“select * from x where foo={foo}”)

If ‘execute’ takes string and t-string, then I would consider it a problem to use a t-string without parameters. If there is a novel API just for t-strings, then you are implying widespread breaking changes as you have a schism between the two ways of providing parameters.

My point was that library authors will need to consider this carefully. If you're writing a library where injection attacks matter, then -- long term -- you almost certainly do not want a single method that accepts `Union[str, Template]`. You probably either want to avoid accepting `str` entirely, or perhaps provide two separate methods. Some period of deprecation seems inevitable.

> its general philosophy is to be pretty liberal with types, except when there's a good reason not to be.

And there’s a good reason not to be here.

> The reason it works is because Python functionally has no bool type. True and False are just integers with names.

This has not been true since around 2.4 or 2.5. The oldest Python I have available to me currently is 2.7, and this holds then, as it does now in 3.13:

    >>> type(True)
    <class 'bool'>
    >>> type(1)
    <class 'int'>

Prior to having a bool type, Python didn't even have True/False keywords.

The reason something silly like `4 + True` works is because the bool type implements `tp_as_number` [0]. The reason it works this way is intentional because it would been a Python 3 str-style debacle if ints and bools were not interchangeable.

[0] https://github.com/python/cpython/blob/main/Objects/boolobje...

Can you do it for functions defined by other people, or only for functions that you defined?

I'm thinking in the general case, but motivated by this example of a 3rd party function that accepts a SQL query as a string, and we'd like everywhere in our codebase to stop using that and instead use the 3rd party function that accepts the query as a t-string

In that case I don’t understand the security regression that t-strings cause (see GP). Before it was all just strings, but you had to make sure to use them in the correct place. Now you can still just use strings for backwards compat. but you can also move on to a distinctly-typed solution for SQL and the like.