←back to thread

73 points pgjones | 9 comments | | HN request time: 1.448s | source | bottom

SQL-tString is a SQL builder that utilises the recently accepted PEP-750, https://peps.python.org/pep-0750/, t-strings to build SQL queries, for example,

    from sql_tstring import sql
    
    val = 2
    query, values = sql(t"SELECT x FROM y WHERE x = {val}")
    assert query == "SELECT x FROM y WHERE x = ?"
    assert values == [2]
    db.execute(query, values)  # Most DB engines support this
The placeholder ? protects against SQL injection, but cannot be used everywhere. For example, a column name cannot be a placeholder. If you try this SQL-tString will raise an error,

    col = "x"
    sql(t"SELECT {col} FROM y")  # Raises ValueError
To proceed you'll need to declare what the valid values of col can be,

    from sql_tstring import sql_context
    
    with sql_context(columns="x"):
        query, values = sql(t"SELECT {col} FROM y")
    assert query == "SELECT x FROM y"
    assert values == []
Thus allowing you to protect against SQL injection.

As t-strings are format strings you can safely format the literals you'd like to pass as variables,

    text = "world"
    query, values = sql(t"SELECT x FROM y WHERE x LIKE '%{text}'")
    assert query == "SELECT x FROM y WHERE x LIKE ?"
    assert values == ["%world"]
This is especially useful when used with the Absent rewriting value.

SQL-tString is a SQL builder and as such you can use special RewritingValues to alter and build the query you want at runtime. This is best shown by considering a query you sometimes want to search by one column a, sometimes by b, and sometimes both,

    def search(
        *,
        a: str | AbsentType = Absent,
        b: str | AbsentType = Absent
    ) -> tuple[str, list[str]]:
        return sql(t"SELECT x FROM y WHERE a = {a} AND b = {b}")
    
    assert search() == "SELECT x FROM y", []
    assert search(a="hello") == "SELECT x FROM y WHERE a = ?", ["hello"]
    assert search(b="world") == "SELECT x FROM y WHERE b = ?", ["world"]
    assert search(a="hello", b="world") == (
        "SELECT x FROM y WHERE a = ? AND b = ?", ["hello", "world"]
    )
Specifically Absent (which is an alias of RewritingValue.ABSENT) will remove the expression it is present in, and if there an no expressions left after the removal it will also remove the clause.

The other rewriting values I've included are handle the frustrating case of comparing to NULL, for example the following is valid but won't work as you'd likely expect,

    optional = None
    sql(t"SELECT x FROM y WHERE x = {optional}")
Instead you can use IsNull to achieve the right result,

    from sql_tstring import IsNull

    optional = IsNull
    query, values = sql(t"SELECT x FROM y WHERE x = {optional}")
    assert query == "SELECT x FROM y WHERE x IS NULL"
    assert values == []
There is also a IsNotNull for the negated comparison.

The final feature allows for complex query building by nesting a t-string within the existing,

    inner = t"x = 'a'"
    query, _ = sql(t"SELECT x FROM y WHERE {inner}")
    assert query == "SELECT x FROM y WHERE x = 'a'"
This library can be used today without Python3.14's t-strings with some limitations, https://github.com/pgjones/sql-tstring?tab=readme-ov-file#pr..., and I've been doing so this year. Thoughts and feedback very welcome.
1. 90s_dev ◴[] No.44005841[source]
Your library looks great. But a tangential rant about t-strings, using lexical scope for placeholder lookup is just a terrible, terrible design. It should be explicitly passed in a dictionary. I'm not sure why they made this decision.
replies(2): >>44005924 #>>44006033 #
2. jitl ◴[] No.44005924[source]
If they’re gonna do that why bother making a new concept? You could already build(normalString, someDict)

Like why make me state “A goes here, also the value of A is 1” when I can just say “1 goes here”? When I build an array or map, I just write the expression

{ key1: value1 }

I don’t need to write

build({ key1, value1 }, { “key1”: key1, “value1”: value1 })

Why should an sql literal be any different from an array or dictionary literal?

replies(1): >>44006095 #
3. ◴[] No.44006033[source]
4. 90s_dev ◴[] No.44006095[source]
Yeah in retrospect it's identical to what JavaScript does with string literals. I don't know what I was thinking.
replies(2): >>44007947 #>>44008355 #
5. 90s_dev ◴[] No.44007947{3}[source]
Oh wait I know why. It's because the PIP had no specialized syntax highlighting to show that it was getting the variables from scope. So I started reasoning about it differently than I do about JS string literals, rather lazily too, and ended up thinking of something like emacs's dynamic scope or something. Amazing what syntax highlighting does to how we think.
6. williamdclt ◴[] No.44008355{3}[source]
No I think your point is valid, and is valid in JavaScript too.

Designing the “right” approach to look like the “wrong” approach (string concatenation) is a bad idea, however cute it is.

It’s annoying that the wrong thing is the more ergonomic one, but at least it jumps out at any dev with any experience, they know what sqli risk looks like. With templated strings, it’s not so obvious anymore.

replies(1): >>44008734 #
7. 90s_dev ◴[] No.44008734{4}[source]
`...` and fn`...` in JavaScript are just syntactic sugar for function calls, the former for array.join(...) and the latter for fn(...) so there's no issue with these utilizing the current scope since that's what all function calls do.
replies(1): >>44009001 #
8. williamdclt ◴[] No.44009001{5}[source]
I might have misunderstood your point, but scoping isn’t related to what I’m trying to say.

What I’m saying is that, regardless of how it works, I don’t think string templating for SQL is a good idea because it looks almost exactly like string concatenation. It makes more difficult to distinguish beteeen the right approach and the wrong approach (or learn about it)

replies(1): >>44009134 #
9. 90s_dev ◴[] No.44009134{6}[source]
No it was me who misunderstood you. And I kind of agree. I've never been a fan of tagged template literals. It gives you no autocompletion, no type checking, no syntax highlighting, nothing. And it requires a runtime string parser. I get why people like it, and maybe it's fine if you don't need those things and don't mind the cost of runtime parsing, but I need them and I do mind.