Python’s new t-strings

> In addition, I hope that the tooling ecosystem will adapt to support t-strings. For instance, I’d love to see black and ruff format t-string contents, and vscode color those contents, if they’re a common type like HTML or SQL.

This is such a strange take on t-strings. The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string, which can only be done in an ad-hoc fashion and has nothing to do with the template string feature.

The way the feature has been designed there is no indication in the string itself what type of content it is or what it will eventually be converted to. It’s all handled by the converting function.

As others have added, something like sql”select * from {table}” would have been able to do this, but there’s not even any guarantees that something that is in a template that will be converted into valid sql by a converting function should be any type of valid sql prior to that conversion. For all you know t“give me {table} but only {columns}” might be a converted into valid sql after the template is processed.

The original PEP and the original discussion had this in scope. We removed it to let this emerge later. There are different ways to signal the language -- some more friendly to tooling, some more robust.

Python has had type annotations for a decade, and modern IDEs can interpret them.

Writing `query: SQL = t"SELECT ..."` is a small price to pay for such a DX boost.

Couldn’t you do this with a type annotation? e.g. SQLAlchemy could have a SQL type so tools like mypy could see a Template instance and confirm you’re using it safely but Black, Ruff, or SQLFluff could look for the more specialized Annotated[Template, SQL] to realize that the template could be formatted as SQL, and something like Django could even have Annotated[Template, Email], Annotated[Template, HTML], or Annotated[Template, JSX] to indicate what context the same templating syntax is targeting.

> The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string

Not the only thing. You can also look at how it is used. Your editor could know of how some popular libraries use t-strings, track which t-strings get passed into functions from those libraries, and use that to assume what grammar the t-string should follow.

Is that cheating? In some sense, yes, but it also is useful and likely will be worth it for quite a few programmers.

This is what we discussed in the first revision of the PEP (the use of `Annotated`.) But we found out: linters don't know anything about the Python type system.

We hope to get a community around all of this, stuff at PyCon US, EuroPython, etc. and work some of this out. The JSX/TSX world really has good tooling. We can provide that for those that want it, perhaps better on some aspects.

I think you'll just see a pattern like:

    html(t"<h1>Hello</h1>")

And highlighters and static analyzers will key off of this.

JavaScript's tagged template literals are actually about as flexible as this, since you can dynamically choose the tag function, it's just very rare to do so, so tools assume a lot based on the name of the function. Python tools can basically do the same thing, and just not support t-strings that aren't nested inside a well-named processing function.

> This is such a strange take on t-strings

I understand why this seems strange at first!

As Paul mentioned, we spent quite a lot of time considering these issues as PEP 750 came together. In the end, we concluded (a) the PEP leaves open quite a few potential approaches for tools to adopt (not just the one you suggest, as others here have pointed out), and (b) it's ultimately something that the broader tooling community needs to rally around and should probably be out of scope for the PEP itself.

So, with that background in mind, I am indeed hopeful we'll see the ecosystem adapt! :-)

Interesting, thanks for the background. I’ve been curious what Astral is going to do in the space but also worry about what happens when their funding runs out.

Another option would be type hints, something like `title: HTMLTemplate = t"<h1>Hello</h1>"`.

PyCharm (as well as the other JetBrains IDEs) has for at least 10 years supported language injection in Python strings using a comment [0]. Worst case scenario there's no reason whatsoever that that couldn't be used by formatters to apply auto-formatting in a structured and deterministic way.

But that's far from the only option, either! IntelliJ for Java also supports annotations on arguments [1] that then mean that you get syntax highlighting everywhere you use a string literal with the given function:

    public void query(@Language("SQL") String sql)

In Python, typing.Annotated appears to have been specifically designed for purposes like this [2]:

> If a library or tool encounters an annotation Annotated[T, x] and has no special logic for the metadata, it should ignore the metadata and simply treat the annotation as T. As such, Annotated can be useful for code that wants to use annotations for purposes outside Python’s static typing system.

So something like this should be perfectly viable:

    SQL = Annotated[Template, "language", "SQL"]
    
    def query(sql_query: SQL):
        # do stuff with Template to sanitize
    
    query(t"SELECT * FROM foo WHERE bar={bar}")

Now, where you're right is that we shouldn't actually require template strings to accomplish this! As noted, JetBrains has been doing this since forever. But maybe template strings will be useful enough for purposes like this that the tooling will actually evolve to support it in formatters and other editors (and maybe PyCharm can get some of the better support that Java has from JetBrains).

[0] https://www.jetbrains.com/help/pycharm/using-language-inject...

[1] https://www.jetbrains.com/help/idea/using-language-injection...

[2] https://docs.python.org/3/library/typing.html#typing.Annotat...

I'm one of the PEP authors and also with JetBrains. I'm talking to the PyCharm team about this.

I get that. But that “ecosystem might adapt” has absolutely nothing to do with t-strings is what I’m saying. Anyone who wants to validate sql will have to just look through strings guess that it’s sql and validate and highlight it. This could be done before and after just the same. The template strings by nature of just being “strings that will turn into something else at some later point” do nothing to further this in any way.

> And highlighters and static analyzers will key off of this.

Then the t is redundant and we don’t need t strings involved in any way. This would work just as well. In fact better by using html("<h1>Hello</h1>") and have it return a html object instead of returning a just a string which is what t-strings will do. So literally the only contribution from t-strings in that context is to make things worse.

If they had just standardized some way of putting semantics into the template everything would be better. Let us define a “sql = temlate(….)” and use sql”…”

If you replace each mention of t-strings in your post with strings or f-strings it’s still true. So that means t-strings add nothing in that regard.

I mean yes, so t-strings add nothing.

You could already do everything you are proposing t-strings will make easier with just strings or f-strings.

‘query: SQL “Select …”’

“Wait” you’ll say “then the function taking the string has to parse the string and will have to add ugly logic grabbing variables from globals()!” Ah yes. That’s such an ugly solution… let’s instead codify just that as the official best practice and sell people on it by prefixing a t to the string so they don’t notice…