Python’s new t-strings

Will this allow neat SQL syntax like the following?

    city = 'London'
    min_age = 21
    # Find all users in London who are 21 or older:
    users = db.get(t'
        SELECT * FROM users
        WHERE city={city} AND age>{min_age}
    ')

If the db.get() function accepts a template, it should, right?

This would be the nicest way to use SQL I have seen yet.

Thanks, I hate it. While it's nice syntactic sugar, the difference between an SQL injection vulnerability and a properly parametrized query is now a single letter that's easily missed

Type checkers to the rescue ahaha I think db.get could also raise if the type does not match?

I guess that is a misunderstanding on your side, about how templates work. Less hate and more love might help to avoid this type of hotheaded misconception ;-)

Why do you think changing a letter would cause a vulnerability? Which letter do you mean?

OP is referring to swapping t with f.

f'' vs t'' probably.

The SQLite extension for Tcl offers something similar:

    db1 eval {INSERT INTO t1 VALUES(5,$bigstring)} 

https://sqlite.org/tclsqlite.html#the_eval_method

That would result in a string passed to get() and raise an error as get() operates on a template, not on a string.

The t-string produces a Template object without a __str__() method. You can’t mistakenly use an f-string in its place. Either the code expects a string, in which case passing it a Template would blow it up, or the code expects a Template, in which case passing it a string would blow it up.

Wow that's only slightly better than using the lowercase letter L vs the digit 1 or letter O vs zero to convey a significant difference.

Those are two different types

except if get() can also accept a raw string, which is likely

Why would it?

This definitely seems like one of the motivations for implementing this feature in the first place - https://peps.python.org/pep-0750/#motivation.

Having more control over the interpolation of string values is a win IMO.

Yes, it's quite delightful.

No sane library is going to do that. If they do let you pass a raw string it should be a different function with the risks clearly documented.

The thing this replaces is every library having their own bespoke API to create a prepared statement on their default/safe path. Now they can just take a template.

> This would be the nicest way to use SQL I have seen yet.

EF/EF Core has existed for years :)

https://learn.microsoft.com/en-us/ef/core/querying/sql-queri...

And I'm guessing lots of code will expect strings to maintain backward compatibility.

That’s the sort of thing people have built with the equivalent feature in JavaScript, so it should do. Eg https://github.com/andywer/squid is a nice example.

I've used it for years. In order to generate the models you had to use the visual designer which was slow and buggy.

Generally annoying experience if you have to clock in and out every day to watch that UI break your database relations whenever you click save.

No one uses it today, or in the last 5 years or so I presume. You use https://learn.microsoft.com/en-us/ef/core/modeling/#use-data...

This was a completely separate, legacy extension of VS, not EF let alone EF Core.

If it's not a completely new library written exclusively around templates, such code currently accepts strings and will most likely continue to accept strings for backwards compatibility.

Also I wonder how easy it will be to shoot oneself in the foot. It may be easy to accidentally make it to a string too soon and not get the proper escapeing.

That's entirely implementation dependent. For existing libraries I would expect something like

    def get(self, query):
        if isinstance(query, template):
            self.get_template(query)
        else:
            self.get_old(query) #Don't break old code!

I'm guessing no existing functions will be extended to allow t-strings for this very reason. Instead, new functions that only accept t-strings will be created.

There's an obvious risk here, same as with strcpy (no, strncpy.. no, strlcpy... no, strcpy_s) that documentation tends to outlive code, and people keep pasting from tutorails and older code so much that the newer alternatives have a hard time cutting through the noise.

I would argue that as bad as some w3schools tutorials were, and copying from bad Stackoverflow answers, going back to MSA and the free cgi archives of the 90s, the tendency of code snippets to live on forever will only be excarbated by AI-style coding agents.

On the other hand, deprecating existing methods is what languages do to die. And for good reason. I don't think there's an easy answer here. But language is also culture, and shared beliefs about code quality can be a middle route between respecting legacy and building new. If static checking is as easy as a directive such as "use strict" and the idea that checking is good spreads, then consesus can slowly evolve while working code keeps working.

That would be a great argument if Python wasn't a language that let you reach into the internals and define __str__() for things you shouldn't be defining it for. And that is something people will definitely do because, you know, they just need something to friggin work so they can get whatever ticket closed and keep some metric happy tracking time-to-close

I think it's way more likely that existing libraries will introduce new methods that use t-strings and are type safe, rather than entirely defeat the purpose of having a t-string API.

It's pretty common for Python libraries to deprecate and remove functionality. It makes people mad, but it's a good thing, for this reason.

Do the python type checkers / linters / whatever have the ability to warn or error on calling certain functions? That would be nice to eventually enforce migration over to the newer functions that only take a t-string template

Completely separate is pushing it since it was recommended by Microsoft, but yes, I am old and times have changed.

They sure do, e.g. https://docs.astral.sh/ruff/rules/pandas-use-of-dot-is-null/

Isn't the actually proper way to use prepared statements anyway? If we are doing that properly, then what does this t string business buy us for SQL usage from Python?

it would likely be safer to have a safe (accepting Templates) and an unsafe (accepting strings) interface.

Now whether maintainers introduce `getSafe` and keep the old behavior intact, or make a breaking change to turn `get` into `getUnsafe`, we will see

> or the code expects a Template, in which case passing it a string would blow it up.

That's where the problem is though -- in most cases it probably won't blow up.

Plenty of SQL queries don't have any parameters at all. You're just getting the number of rows in a table or something. A raw string is perfectly fine.

Will sqlite3 really disallow strings? Will it force you to use templates, even when the template doesn't contain any parameters?

You can argue it should, but that's not being very friendly with inputs, and will break backwards compatibility. Maybe if there's a flag you can set in the module to enable that strict behavior though, with the idea that in a decade it will become the default?

How about every library that wants to preserve backwards compatibility?

Or are you suggesting that e.g. every database module needs to implement a new set of query functions with new names that supports templates? Which is probably the correct thing to do, but boy is it going to be ugly...

So now you'll have to remember never to use 'execute()' but always 'execute_t()' or something.

    db.execute(t"Select Count(1) from someTable")

It's one extra letter to "force" for an unparameterized query over a "raw string". The t-string itself works just fine without parameters.

There's definitely a backwards compatibility hurdle of switching to a template-only API, but a template-only API doesn't look that much "less friendly" with inputs, when the only difference is a `t` before every string, regardless of number of parameters.

You don't have to remember it, you can use deprecation warnings and lint tools to remind you. (Until eventually the safe API is the only API and then you really have nothing to remember.)

Sure, but it's just I don't have to do that anywhere else.

I never put an f in front of a string if I'm not putting variables within it.

And I'm generally used to Python inputs being liberal. I can usually pass a list if it expects a tuple; I can pass an int if it expects a float; often I can pass an item directly instead of a tuple with a single item. Regex functions take regular strings or regex strings, they don't force regex strings.

Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.

It's safer, I get that. But it's definitely less friendly, so I'll be curious to see how module maintainers decide to handle this.

As I understand, that's less powerful, because you can do:

    t"INSERT INTO mytable VALUES ({s}, {s[::-1]})"

but you can't do:

    mydb eval {INSERT INTO mytable VALUES ($s, [string reverse $s])}

Instead, you have to write:

    set t [string reverse $s]
    mydb eval {INSERT INTO mytable VALUES ($s, $t)}

There's no reason you couldn't have such power in Tcl, though: it's just that the authors of SQLite didn't.

I’d assume their current safe function isn’t taking a string, and is taking some sort of prepared statement? So they could have it take either their prepared statement or a template, and deprecate their prepared statement.

If a library has functions taking a string and executing it as SQL they probably shouldn’t make that take a template instead, but I’d hope that’s a separate explicitly unsafe function already.

For sqlite3, it absolutely takes a regular string.

If you want to substitute parameters, you put a '?' in the string for each one, and provide an additional (optional) tuple parameter with the variables.

So no, there's no explicitly unsafe function. That's my point.

Because, as the article states, people aren’t using prepared statements. Instead, they pass f-strings because they’re more convenient.

Gotcha. I’d guess they’d want to deprecate that function and create a new one that only accepts a template then, which is definitely annoying! I figured they’d already have more separation between prepared and raw strings which would make it easier.

Er… that’s just not correct? Python can be more liberal but it’s not always. It depends entirely on the tooling. Libraries will take time to catch up but I can definitely see people creating libraries that enforce t-strings, even if they’re deconstructing them under the hood for legacy libraries.

  I never put an f in front of a string if I'm not putting variables within it.

Linters will even complain if you have a f string without variables. I assume it will be the same for t strings.

Programmers being lazy and shit at their jobs is not a reason to not improve the language.

That’s a library author problem, so it’s less likely since library authors tend to be fewer in number and, for popular libraries, get a reasonable number of eyes on this type of change.

What's not correct? Python inputs usually are liberal. I didn't say always.

Are you claiming it's traditionally common in Python to be strict with inputs, and that being liberal is the exception?

That Python lets you blindly interchange different types for no good reason. It simply doesn’t.

Yes, it’s common for Python to be strict for inputs when the types are different. For example, try:

Decimal(‘3.0’) / 1.5

You’ll get an error and for good reason.

And they could add deprecation warnings gradually

f strings are syntax rather than a type, the resulting templates look like a reasonable way to specify a prepared statement.

For the reasons discussed above, I'm not sure that it will be the case for t-strings. I think it'll take a little while for frameworks/libraries to adapt (while still maintaining backward compatibility) and a while for best practices to find their way into our linting and other tools.

Linters complain because f"hello" and "hello" are the _exact_ same string. t"hello" isn't even a string.

Yeah. A while back I was poking through some unfamiliar code and noticed that my editor was rendering a use of `datetime.utcnow()` as struck through. When I hovered it with my mouse, I got a message that that function had been deprecated.

Turns out my editor (vscode) and typechecker (pyright) saw that `datetime.utcnow()` was marked as deprecated (I know one can use the `@deprecated` decorator from Python 3.13 or `__future__` to do this; I think it was done another way in this particular case) and therefore rendered it as struck through.

And it taught me A) that `utcnow()` is deprecated and B) how to mark bits of our internal codebase as deprecated and nudge our developers to use the new, better versions if possible.

Except to maintain backwards compatibility we're probably going to get new methods that only accept templates, completely circumventing any effort to stop people passing in strings.

Prepared statements were the recommended way to run SQL queries when I was starting with PHP 15 years ago, anyone writing code vulnerable to SQL injection at this point should not be writing code.

But... it usually does. For example, try:

    Decimal('3.0') / 2

It works fine. It doesn't work with a float, which is for good reason. That's the whole point -- its general philosophy is to be pretty liberal with types, except when there's a good reason not to be. Heck, you can even do dumb things like:

    4 + True

And get 5 back. If that's not "blindly interchanging different types for no good reason" then I don't know what is. You can even multiply your Decimal object by False and get an answer...

Or it's like my original example -- the Regex module isn't restricted to r-strings. It happily works with regular strings. Python's general philosophy is to handle input liberally. Even type hinting is an add-on. Now, it doesn't go as far as JavaScript in allowing e.g. "4"+1, but it's still awfully liberal. I just don't see how you can claim otherwise.

I know about Decimal/int mixing but that’s for a good reason: it’s fine to intermix here. But not for floats (precision issues). The bool/int mixing isn’t “good.” It’s a bad implementation detail that Python is stuck maintaining forever. I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.

The reason it works is because Python functionally has no bool type. True and False are just integers with names. It’s stupid and shouldn’t work like that but it does for historic reasons.

Your example of regex makes no sense either. There is no difference between strings and r-strings. They’re literally the same thing to the interpreter, so how could the regex functions enforce you use r-strings? Maybe they should be different but, for historic reasons, they can’t be without Python 4.0.

Well yes but the alternative is to never make language improvements because legacy code exists.

If you can use a string anywhere you can use a t-string, then a non parametrized t-string is a code smell (lining error). If there is a dedicated template-string API, then there is the implicit threat you are breaking backwards compatibility to stop using regular strings.

> If you can use a string anywhere you can use a t-string

You can't; they're different types. t-strings are not `str`

It's up to good framework/API design to take advantage of this.

> I’m actually stunned that you think to use this as an example when I think I’d fire any programmer that did that in my team for gross negligence.

You seem to be having a different conversation than I am.

I'm just describing Python as it is. I'm not defending it. I know why you can add True to a number, or else I wouldn't have come up with the example. And I know perfectly well that r-strings are just strings. Python easily could have made them a distinct object, to force people from ever making backslash errors, and restricted Regex functions to them, but didn't.

My only point has been, "Pythonic" things tend to be pretty liberal in what they accept. Type hints aren't even enforced, when they exist at all. You seem to think it shouldn't be that way. Great! But regardless, claiming it's not that way -- that Python is somehow this strict language -- is just mischaracterizing it.

> My only point has been, "Pythonic" things tend to be pretty liberal in what they accept

Being able to use a string as a string and an int as an int are not “pretty liberal in what they accept,” it’s just programming language theory 101! I think you’re mistaking duck typing for “liberal acceptance,” which are not the same thing. There’s always been an expectation that you should use compatible interfaces, even within the standard library. I’ve been bitten enough times by passing a generator in when a function expects a list, for example.

> Being forced to use a single specific type of string in all cases is just very different from how Python has traditionally operated.

Maybe that's partly the disconnect here? "t-string" is probably a confusing colloquial name because they aren't strings, they are Templates. The runtime type is a Template. It is a very different duck-type from a string. As a duck-typable object it doesn't even implicitly or explicitly act like a string, there's intentionally no `__str__()` method and `str(someTemplate)` doesn't work like you'd expect. It shouldn't be a surprise that there is also no implicit conversion from a string and you have to use its own literal syntax: it isn't a string type, it's a Template type.

Python here is still liberal with respect to Templates (it is still a duck type). If a function expects a Template and you don't want to use the t"" shorthand syntax nor use the Template constructor in string.templatelib, you just need a simple class of object that has an `__iter__()` of the correct shape and/or has `strings` and `values` tuples.

Sure, it may make sense for some types of APIs to support a Union of str and Template as "liberal" options, but it's a different class of liberal support from Union of list and tuple or Union of int and float which are closer "domains" of types. A Template isn't a string and at runtime looks nothing like one (despite how syntactically it looks like one at "compile time"). Given `__iter__()` in Template, it may make more sense/would be more "natural" to Union Template with List or Tuple more than with a single string.

Yeah, "t-string" is possibly a misnomer, because they are in fact at runtime a Template object (from string.templatelib).

I'm not mistaking it at all. Yes, duck typing is very much liberal acceptance, but Python code tends to go much farther. I could give a million examples -- like how in Python's isinstance() the second argument can be a type or a tuple of types, or in sqlite3 that you can run queries on a connection or on a cursor, and don't even get me started on Matplotlib or Numpy. It's just idiomatic in Python to make things easy for the programmer by accepting multiple types when possible. If you don't recognize this as the common Python pattern, I literally don't know what else to tell you.

A library writer ultimately has to decide if they accept both types. For a database cursor, do you take regular strings + parameter arguments and template strings? Or dedicate a new API to the idea?

  cursor.execute(“select * from x where foo=?”, {foo=1})
  # while also allowing
  cursor.execute(t“select * from x where foo={foo}”)
  #Vs 
  cursor.executetemplate(“select * from x where foo={foo}”)

If ‘execute’ takes string and t-string, then I would consider it a problem to use a t-string without parameters. If there is a novel API just for t-strings, then you are implying widespread breaking changes as you have a schism between the two ways of providing parameters.

You would need triple-quotes to span multiple lines, but yes, that is exactly how it's intended to work. `db.get` will receive a Template instance, which stores string parts something like `('\n SELECT * FROM users\n WHERE city=', ' AND age>', '\n')` and interpolated parts like `(Interpolation('London'), Interpolation(21))`. It's then responsible for assembling and executing the query from that.

My point was that library authors will need to consider this carefully. If you're writing a library where injection attacks matter, then -- long term -- you almost certainly do not want a single method that accepts `Union[str, Template]`. You probably either want to avoid accepting `str` entirely, or perhaps provide two separate methods. Some period of deprecation seems inevitable.

> its general philosophy is to be pretty liberal with types, except when there's a good reason not to be.

And there’s a good reason not to be here.

> The reason it works is because Python functionally has no bool type. True and False are just integers with names.

This has not been true since around 2.4 or 2.5. The oldest Python I have available to me currently is 2.7, and this holds then, as it does now in 3.13:

    >>> type(True)
    <class 'bool'>
    >>> type(1)
    <class 'int'>

Prior to having a bool type, Python didn't even have True/False keywords.

The reason something silly like `4 + True` works is because the bool type implements `tp_as_number` [0]. The reason it works this way is intentional because it would been a Python 3 str-style debacle if ints and bools were not interchangeable.

[0] https://github.com/python/cpython/blob/main/Objects/boolobje...

Can you do it for functions defined by other people, or only for functions that you defined?

I'm thinking in the general case, but motivated by this example of a 3rd party function that accepts a SQL query as a string, and we'd like everywhere in our codebase to stop using that and instead use the 3rd party function that accepts the query as a t-string

In that case I don’t understand the security regression that t-strings cause (see GP). Before it was all just strings, but you had to make sure to use them in the correct place. Now you can still just use strings for backwards compat. but you can also move on to a distinctly-typed solution for SQL and the like.