I hacked a dating app (and how not to treat a security researcher)

This is exactly why I think software engineering should require a licensing requirement, much like civil engineering. I get that people will complain about that destroying all sorts of things, and it might, yes, but fight me. Crap like this is exactly why it should be a requirement, and why you won’t convince me that the idea is not in general a good one.

Agreed. My stance on this changed over the course of some years after a close family member married an actual engineer (structural) and I got a lot of insight into that world.

It's astonishing to me the ease of which software developers can wreak _real_ measurable damage to billions of lives and have no real liability for it.

Software developers shouldn't call themselves engineers unless they're licensed, insured and able to be held liable for their work in the same way a building engineer is.

> This is exactly why I think software engineering should require a licensing requirement, much like civil engineering.

Civil engineering requires licensing because there are specific activities that are reserved for licensed engineers, namely things that can result in many people dying.

If a major screwup doesn't even motivate victims to sue a company then a license is not justified.

Yes, I will happily fight against authoritarian takes cloaked in vagueries.

I don’t believe engineering licensing is authoritarian, and I’d be interested in hearing why you believe that to be the case (especially, considering, most “real” engineering field have had licensing requirements for a century, without any real complaints against that process).

Conversely, it's the scale, not magnitude. A single physical infrastructure failure can usually only harm a very limited number of people. A digital infrastructure breach can trivially harm millions.

Observing that each individual harm may not be worth the effort of suing over is evidence that the justice system is not effective at addressing harm in the aggregate, not evidence of lack of major harm.

You don't see how gate-keeping who can create software is authoritarian?

The distinction between creating virtual software and physical structures is fairly obvious.

Of course physical engineers that create buildings and roads need to be regulated for safety.

And there are restrictions already for certain software industries, such as healthcare.

Many other forms of software do not have the same hazards so no license should be needed, as it would be prone for abuse.

While the idea is good, I'm not sure how this would get implemented realistically. The industry standards/audits are silly checkbox exercises rather then useful security. The biggest companies are often terrible as far as secure design goes. The government security rules lag years behind the SotA. For example how long did it take NIST to stop recommending changing passwords?

Civil engineering works well because we mostly figured it out anyway. But looking at PCI, SOX and others, we'd probably just require people to produce a book's worth of documentation and audit trail that comes with their broken software.

I would say the risk of identity theft for over 150 million people justifies some preventative measures. And yes, there also were hundreds of lawsuits.

https://en.wikipedia.org/wiki/2017_Equifax_data_breach

Or how about four suicides and 900+ wrongful convictions?

https://en.wikipedia.org/wiki/British_Post_Office_scandal

Not to mention the various dating app leaks that led to extortion, suicides and leaking of medical information like HIV status. And not to forget the famous Therac-25 that killed people as direct result of a race condition.

Where's the threshold for you?

I agree creating software in general shouldn't be gatekept, but requiring that app developers who process PII have more to show than vibe-coding experience would probably be beneficial.

I don't think anyone is proposing that Flappy Bird or Python scripts on Github should be outlawed. Just like you can still build a robot at home but not a bridge in the town center.

There are pretty major exceptions to what require engineering licenses, and it is pretty unclear where software should fall in.

You can sign a liability waiver and do all sorts of dangerous things.

>most “real” engineering field have had licensing requirements for a century, without any real complaints against that process).

Most newer engineering fields are trending away from licensing, not towards it. For example, medical device and drug engineering doesn't use it at all.

I'm curious how you think this would be implemented. Do you think you should need a license to publish on GitHub? Write code on your own computer and run it? Because this was just a startup that some kids founded so saying that a license would have to be a prerequisite to hiring somebody would not cut it. You'd have to cut off their ability to write/run code entirely.

Some engineers like to go on about this, but the reality is they offload the work to marginally qualified techs and unlicensed engineers and stamp the document, just like in software.

There are all sorts of failures in the structural space. How many pumped reinforced concrete buildings are being built in Miami right now? How many of them will be sound in 50-75 years? How likely is the architect/PE’s ghost to get sued?

PE’s are smart professionals and do a valuable service. But they aren’t magic, and they all have a boss.

I mean, kind of? You can't really start any kind of trade business without credentials (other than low-paying under the table work for people who don't care).

You can't stop someone from doing electrical repairs on their own home but if the house burns down as a result, the homeowners' insurance will probably just deny the claim, and then they risk losing their mortgage. Basically, if you make it bureaucratically difficult to do the wrong thing, you'll encourage more of the right thing.

OP didn't qualify the statement "This is exactly why I think software engineering should require a licensing requirement".

No mention of PII or any specifics.

SWE already has regulations. I see no need for a license requirement...

Concerning PII, it's kind of hypocritical for the gov to regulate when the NSA was proven to be collecting data on everyone against their will or knowledge.

I worked on a project that was using federal tax information and had IRS 1075 compliance requirements. Those follow some version of NIST that was out of date at the time.

We had two security teams. Security and compliance. It was not possible to be secure and compliant, so the compliance team had to document every deviance from the IRS standard and document why, then self-report us and the customer to audit the areas where we were outside the lines. That took a dozen people almost a year to do.

All of that existed because a US state (S Carolina iirc) was egregiously incompetent and ended up getting breached. Congress “did something” about it.

well software generally harmless until you integrate in your car (see: Tesla)

I think there defo a line where bug in your puzzle app don't need a license vs AI that drive your 50k+ tesla

I mean this is Tech industry, everyone here gather data big tech or not,

I'm not saying I'm pro identity theft or data breach or something, but the industry culture is vastly different

people here are pro on move fast break things some of idea, I think you just cant tbh

" Crap like this is exactly why it should be a requirement, and why you won’t convince me that the idea is not in general a good one."

If you're looking for a regulatory fix, I would prefer something like a EU-style requirement on handling PII. Even the US model--suing in cases of privacy breaches--seems like it could be pretty effective in theory, if only the current state of privacy law was a little less pro-corporate. Civil suits could make life miserable for the students who developed this app.

There's no governing body that continually researches, vets and updates standards of security. There should be, honestly, but there isn't. Thats not true of professional engineering organizations, or medical boards, or the Bar Association etc.

They all update their recommendation and standards routinely, and do a reasonably good job at being professional organizations.

The current state of this as regards to the tech sector doesn't mean its impossible to implement.

Thats why all the usual standards (PCI, SOC2 in particular) are performative in practice. There's nothing that holds industry accountable to be better and there is nothing, from a legal stand point, that backs up members of the association if they flag an entity or individual for what would be effectively malpractice.

This is why delegated authorities should be managing things instead of congress itself. Because congress has no idea what they're doing on technical topics generally.

Everyone in business is move fast and break things and let people die if it's cheaper until regulations force them not to be. Software is just new enough that mostly doesn't exist yet.

They believe any regulation is authoritative overreach so I doubt you're gonna get anywhere.

Check their comments there's screeds about compelling labor over like basic concepts.

I feel like people who suggest governing bodies for this kind of stuff always imagine some perfect unicorn organization that makes perfect recommendations where as I usually imaging every UX turning into the worst possible 20x step process because of "regulations" and it will actually just be theater and not actually solve whatever problems it claims to.

I’m happy to discuss specifics, so long as they don’t start with the premise “regulation is authoritarianism” and also are in good faith. Kids don’t have to have an engineering license to build a bridge out of popsicle sticks, I doubt you think that someone saying “building a bridge should require a civil engineering license” should apply to that. I’m not unreasonable. I just think there has been entirely too much demonstrated harm to start with the premise of “anyone can build any software they want at any time, with zero liability”.

These students may be liable for things after the fact, but that is hardly any consolation to the people that may have had their intimate personal data leaked. Even if they are successfully sued by everybody on the site, how much money could they possibly squeeze out of a bunch of college students? I don’t know how you can prevent this without some up front thing, such as a license, rather than making them liable after the fact.

I can buy that. If I were dictator of the world, I wouldn’t say “making pong clones requires a license”. Even if you grossly negligently screw up the scoring system in your clone, I wouldn’t say you should be liable for anything. I think there are probably more cases where liability should exist, even without processing of personal data of any sort, and I don’t have an easy “one size fits all” regulation in mind either, it’s surely not going to be that easy, and I fully acknowledge that. I just wish we as an industry would start having that conversation in good faith.

> medical device and drug engineering

is a special case exception, where rather than requiring licensing for the engineers building the product, we put detailled restrictions and regulations on what needs to be done (extensive testing, detailled evidence, monitoring programs, etc) before the product can be sold or marketed.

That is hardly an example of a field where risk-taking is encouraged and unlicensed persons are able to unleash their half-developed ideas on the public.

Do you have any other examples of fields which are "trending away" from licensing?

I mean, bridges collapse sometimes. It’s not really about making things perfect from the get go, it’s about making sure that the industry as a whole learns from mistakes. And I agree that some of the existing standards and audits are checkboxes at best, and actively suggesting problems at worst. But, we need to be evolving those actively anyways, that has to be baked into the DNA of whatever this licensing scheme ends up being.

Anyways, I’m not the one who should be deciding the specifics here, it should be a collaboration between lots of different parties, even if I may have a seat at that table. But we have got to get away from the notion (as seen in other comments in this thread) that any sort of attempt to prevent this kind of harm before it happens is authoritarianism.

That is the 20th century innovation. Unfortunately, the king doesn’t like it.

You haven't thought thought this through. What happens with open source? I need a license to make a PR on github. It will also push all software engineering to places where there isn't a license requirement or onto the darknet.

You aren't thinking through the broader implications.

Will I need a license if Flappy Bird has a online function for uploading high scores to a leader table stored online somewhere?

Will I need a license to put a PR on Github?

You and many others (reading through the comments) are pretending that an information leak is on the same scale and severity as major safety concerns people may have about the safety of physical structure. It is obviously asinine comparison and why you will always get such push-back and people instinctively know they fundamentally different.

You also haven't thought about how many unintended consequences it will have. It will affect things like open source, hiring and how it will affect smaller niche cultures that rely on pseudo-anonymity or just want to do fun things.

Just off the top of my head:

Am I going to need a license to build a EDuke32 package for AUR?

Am I going to need a license to add a plugin to a piece of software?

Will I need a license to stick a gist on github?

Many people that currently make the laws in industry (just look at the UK online safety act) don't understand/won't care about any of the nuance.

>I just think there has been entirely too much demonstrated harm to start with the premise of “anyone can build any software they want at any time, with zero liability”.

Actually it is the opposite. I and many others could argue that it has improved the world immensely. I can talk to people that share my interests from all around the globe, I have the ability to work internationally and never leave my home. I've just recently I've taught myself how to fix many of my own vehicle problems at home using Youtube and do some basic maintenance around the house.

I can get any niche product delivered to my door in a matter of days. All of these are massively positives that have benefited the world immeasurably.

> These students may be liable for things after the fact, but that is hardly any consolation to the people that may have had their intimate personal data leaked. Even if they are successfully sued by everybody on the site, how much money could they possibly squeeze out of a bunch of college students? I don’t know how you can prevent this without some up front thing, such as a license, rather than making them liable after the fact.

A license will guarantee nothing. You should assume that anything you put online can be leaked. I can control the amount of information I put on most sites by either giving them false information or being pseudo-anonymous / anonymous.

However regulation in my country is going to force photo ID for platforms such as Discord (and many others) under the guise of age checks. This will mean that I have to give a third party my ID which has all my data or not use the service. This will tie my identity on Discord (which is pseudo-anonymous) to my Discord account.

So licensing/regulation actually guarantees more data leaks. Because I can't vet the company that deals with the ID check, not can I easily circumvent information gathering. Sure I will probably be able to defeat most of this with a VPN. But it is more of a PITA.

Yes, I have. You aren’t allowed to build a faulty bridge, even free of charge.

Maybe you are allowed to build that faulty bridge in, I dunno, Laos or whatever, and if people go to Laos specifically to drive on your bridge, then that’s on them if it collapses. But countries can and do successfully regulate how software is handled in their jurisdiction, see GDPR for example. It’s not an unsolvable problem, and even if there are cracks (like there are with GDPR), the solution isn’t to throw our hands up and say “welp, nothing to be done, just have to accept that sometimes people’s intimate personal details gets leaked.”

If you think my suggestion is bad (which it very well may be), happy to hear your take on how to prevent things like this and and other negligent software.

[flagged]

Systematically violating people's privacy while not caring about protecting their data is not culture, it's called a problem.

Perhaps they could move even faster and scale better by collecting and storing less data. Moving forward fast instead of moving frantically while looking for things to break seems more reasonable to me. But then again I'm not the kind of person to become a billionaire tech CEO who's unironically bragging about being called the Eye of Sauron, so what do I know.

This was beautifully constructed, and I wish you got a reply.

I appreciate the effort.

Thanks. I think it is very easy for people to focus on a lot of the negatives about the tech over the last 25 years and demand regulation, without recognising the huge amount of innovation that took place because people were allowed to try things.

Aerospace and Automotive engineering would be more examples, and then the obvious case of software and hardware engineering.

As you point out, the trend is for self certification and government review, like is done for medicine, aircraft.

I don't think these are special cases, but the norm for any field developed after the 60's or so.

>risk-taking is encouraged and unlicensed persons are able to unleash their half-developed

That's your hostile strawman, not mine.

I don't imagine some perfect unicorn organization myself.

I do imagine a technical organization that strives to do its best and would have sufficient scope to protect its members legally if need be, so members would be empowered to make the best decisions possible.