Happy 20th Birthday, Django

In a very real sense, I have Django to thank for my entire career. As an undergraduate, my first academic job in a research lab had me building websites to promote the research in a lab. Django was brand new, and I was uninterested in petty concerns like stability and security, so I did everything in Django.

Years later (2009), I got to do interesting work in a cutting edge machine learning lab due to the expertise I developed in Django -- I was accepted into the lab specifically to clean up the mess phd students had made trying to build a complex front end using Django's ORM with physically separate per-user MySQL database servers.

All the things that came after -- being the first full time employee at a machine learning spinout from the lab, getting acquired by a big company and scaling up sensor-driven ML in the real world, quitting to co-found an ML-centered VC fund, starting a (now 10 year old) AI company -- none of it would have happened without Django.

Does/did Django have a reputation for instability or insecurity?

Also, how on Earth did the ML PhDs decide physically segregated databases for each user were a requirement?

It has neither reputation, and to my knowledge, has never had serious incidents of either. It powers huge websites like pokemon.com

And if you spend any time working with a group of PhDs, you'll be shocked at some of the harebrained schemes they come up with. Dumb people are dumb in simple ways. Smart people are dumb in genius ways.

Same here! Django was not only the first piece of software that allowed me to do real freelancing and software development, but also exposed me to high-quality Python source code and development practices taken from the development team.

It's not that Django has/had any reputation for those things -- it's that Django had no reputation at all since it was a nascent project with no track record. That means it had no proof points for being stable or secure. At the time I adopted it (~Nov 2005), the public release of the Django project was only a few months old, and indeed breaking changes followed soon after (the so-called "magic removal branch"). As I recall, there were also various template escaping security issues at that time.

> Also, how on Earth did the ML PhDs decide physically segregated databases for each user were a requirement?

I worked at several labs at top academic institutions, on everything from supercomputer MPI work for multi-agent sims to image pipelines for large weather simulations, and one thing I learned is that being a good coder is orthogonal to being a good researcher. In that particular case, the person who wrote the code made the assumption that the "customers" would not allow their sensor data to be stored alongside the data from other customers, and separate databases with separate passwords was the solution they came up with. Somehow they did not notice that the terrible ergonomics of this solution meant there was probably a better way. Once I ripped out MySQL in favor of Postgres (since it had proper security) and removed the cumbersome middleware layer performance improved by over 100x.

> trying to build a complex front end using Django's ORM with physically separate per-user MySQL database servers.

didnt they hear about sqlite, great for this setup

how does one get to be a coder in a research project

That early Python community was really nice. There were a lot of helpful strangers on the internet. Same with Ruby.

Same here. Built first web application in Django, have tried all the other frameworks over the years, and am now back to building another webapp in Django...

I don't think Django could have talked to sqlite over the network, or would have really solved any problems. When I say physically separate, I mean it in the pre-cloud sense: one front end server (Django) dynamically connecting to a database on a remote server (based on the user account), which had a database continuously logging streams of sensor data from local IoT devices.

What I did was change the architecture to support centralized aggregation in a large co-located Postgres server, implementing security to managed permissions for writers (sensor gateway) and readers (user accounts).

Eventually we had to write a pg backend similar in concept to timescaledb to handle the massive scale of sensor data we were ingesting...but that's another story entirely.

As a student, ask around or check with your advisor. Most labs will have positions for students interested in coding "grunt work" because students are cheap.

As a professional, apply to Masters/PhD programs in your area of interest, or be young and an expert in an obscure technology the lab uses. Some labs will hire out contract work, but can't pay well (hence be young). Other labs have grant money to bring in non-phd researchers, but to get the job you'll both have to be an expert in a required technology, and be able to contribute actively to the research area of the lab. At Yale, I did this, as second author on a conference paper showing novel methods for applying machine learning in multi-agent, sensor -driven environments. It justified my paycheck as a researcher while I spent the other half of my time fixing broken code for other researchers and writing the first version of the software for our lab spinout.

Edit: I just want to clarify that my experiences with this don't go past 2010, so YMMV. Getting old is tough sometimes, it still feels like almost yesterday.

Django has supported database routing for ages. I've abused it for setups like yours (at least as I vaguely understand from the post details) to avoid using a datalake

I also started my programming career thanks to Django, and I started using it while working at a local newspaper, so bonus points there! We built a system that our sales people could book ads, and then we could layout the newspaper through a canvas based tool (used Fabric.js for that), and then send the pages + ad stack to InDesign to be built. Was great to work with the whole process and Django was really never a limitation. I ended up moving on, but it'll always have a place in my heart.

It was added over a year after we needed it (back in 2008/2009). I'm pretty sure our use case was taken as an input into the design of Django's database routing. We were certainly loud enough about it.

> to my knowledge, has never had serious incidents of either.

The magic removal branch in 2006 was one such case (of API instability). I recall another case circa 2007(?) with a "new admin". But, importantly, pre-1.0 Django made no guarantees of API stability. Post-1.0 Django has had a pretty rock-solid reputation for stability and security.

Haha well in that case thanks!

Indeed it was.