Most active commenters
  • jacquesm(8)
  • (6)
  • rubyfan(5)
  • kjs3(4)
  • pixl97(4)
  • IshKebab(4)
  • grammie(4)
  • dun44(3)
  • wredue(3)
  • noduerme(3)

127 points Anon84 | 185 comments | | HN request time: 2.749s | source | bottom
1. ◴[] No.38508672[source]
2. discmonkey ◴[] No.38508712[source]
Classic misunderstanding of programming languages. The only thing stopping cobol from being "known" is these companies paying for it.
replies(3): >>38508797 #>>38508990 #>>38509315 #
3. tw04 ◴[] No.38508735[source]
Plenty of folks know COBOL - the problem is you need to entice someone YOUNG to learn it, which basically means overpaying them. If I'm an 18-year-old why would I focus on learning a programming language that puts me in a job with 0 chance of advancement? Sure I'm irreplaceable to the company, but I've also only got a handful of other places I could go if they treat me poorly or sunset their mainframe.
replies(3): >>38508805 #>>38509372 #>>38510305 #
4. jalk ◴[] No.38508782[source]
There seems to be plenty of cobol transpilers - anybody know why they can’t use those. Immediate thought was it’s just beating on the dead Watson horse.
replies(2): >>38508946 #>>38511891 #
5. dun44 ◴[] No.38508793[source]
Regarding IBM's use of technology widely known for hallucinating to translate sensitive source code: it sounds weird you think the primary objective here is to deliver something that properly translates from one language to another. But it is not - the primary objective is to get money from customers and investors. Delivering something that works is secondary, and honestly, optional.

Many things in IT start making a whole lot more sense once you reexamine your beliefs about their purpose.

replies(3): >>38508983 #>>38512573 #>>38516931 #
6. SillyUsername ◴[] No.38508797[source]
Chicken and egg, companies stop paying, say no devs, devs not interested as no companies paying for it. Tell me that COBOL to Java porting pays $150K p/a and I'll be happy to take the job, at least for a year or 3.
replies(1): >>38511095 #
7. tobyjsullivan ◴[] No.38508805[source]
I’d be curious to know how we’re defining “overpaying” here. These companies have effectively accrued 50+ years of technical debt (they could have migrated off COBOL decades ago). Meanwhile, based on my limited experience with these industries, I’d be surprised if they even offer market rates to new hires.
replies(3): >>38508933 #>>38509381 #>>38510716 #
8. elteto ◴[] No.38508815[source]
These systems have not been running untouched for 60 years. Regulatory environments change almost constantly and the code has to be updated accordingly. I don’t personally know anyone writing COBOL but I’m certain there’s plenty of people doing it. And to change the code you have to know it. So there’s really no 60 year old code that no one knows anymore.

Also, at this point, if you are running on a dead platform and language and you know it, and haven’t addressed it, then it’s on you. I’ve been seeing these COBOL articles since the 2000s I believe.

replies(2): >>38509002 #>>38509163 #
9. hn_throwaway_99 ◴[] No.38508859[source]
The flawed "economic analysis" in this article is so commonly brought out, but at this point we should just call it what it is: lying.

There is no "shortage of COBOL programmers." Businesses are simply choosing what they will pay, and the market is responding appropriately. And this is also not just a case of naively shouting "duh, just pay them more" for a market that can't bear the increase in costs (i.e. some industries can only survive if there are available low cost workers - most of us aren't willing to pay $40 for a fast food burger, for example. Whether those industries should be around if they can only survive on low cost labor is another story...) It's not exactly like the financial world is scrounging for money. Businesses can either choose to pay their CEOs a hundred million or so, or they can choose to spread some of that money to COBOL programmers. The choice is theirs, but there is no "shortage".

replies(4): >>38509265 #>>38509328 #>>38509376 #>>38509535 #
10. mperham ◴[] No.38508913[source]
We see this same article every year or so. Same song for 20+ years now.
11. mlhpdx ◴[] No.38508930[source]
I learned COBOL from curiosity (never using it professionally) in around 2010. It’s simple enough to read and use, but without having seen these legacy systems I don’t know what I don’t know.

This is just one of many examples of organizations squeezing an impressive but foolhardy lifespan out of some systems. Some CEO or another should have come to the conclusion (sooner) that they run a business, not a museum.

12. nobodyandproud ◴[] No.38508933{3}[source]
Let me up the ante. Why do we believe COBOL is yech-debt?
replies(2): >>38509208 #>>38510346 #
13. kjs3 ◴[] No.38508946[source]
If they worked as effectively as they were touted, they'd have cleared out the cobol.
replies(2): >>38509273 #>>38511306 #
14. lucidguppy ◴[] No.38508968[source]
This is a management problem. Lack of long-term planning. People knew COBOL was "feature complete" DECADES ago.

"If it ain't broke - don't fix it." does not mean "don't maintain the things you own - just let them run until they break - then fix them."

replies(1): >>38509058 #
15. datadrivenangel ◴[] No.38508983[source]
Especially for IBM! See IBM Watson for details.
replies(1): >>38509185 #
16. mlhpdx ◴[] No.38508990[source]
It’s more than that. If you spend a couple or more years learning to maintain those systems, where else will you work? Will you find a co-founder or investor for a COBOL based system when you go “entrepreneur”? Maybe, but it better be a damn strong niche.

Not moving forward with the software industry is a weird kind of conceit that separates these companies from the mainstream by far more than money.

replies(2): >>38509423 #>>38509434 #
17. kjs3 ◴[] No.38509002[source]
I’ve been seeing these COBOL articles since the 2000s I believe.

Since the '80s for me. And certainly the '90s, since these sort of zero-content articles were very popular in the runup to Y2K: "the entire world is going to end because there are no Cobol programmers anywhere in the world! Everyone panic!". It's still bullshit.

replies(1): >>38514775 #
18. kjs3 ◴[] No.38509058[source]
I hate to break this to you (large financial here, large COBOL installed base) but we're not just sitting around watching it run. We have hundreds of COBOL programmers, and they actually touch the code. I bet every day even. Crazy, right?

Just because some low-knowledge 'journalist' in a low-content rag decorates their ad-delivery with a low-effort 'you should have outrage about something' articles doesn't mean you should buy into it.

replies(1): >>38509395 #
19. ufmace ◴[] No.38509082[source]
The article title is clickbaity, but the actual point is the proposal of using LLMs to translate large amounts of legacy COBOL systems to more modern languages like Java. Doesn't seem terribly useful to me. I expect you could get a 90% solution faster, but the whole challenge with these projects is how to get that last bit of correctness, and how to be confident enough in the correctness of it to actually use it in Production.

But then all of this has been known for decades. There are plenty of well-known techniques for how to do all that. If they haven't actually done it by now, it's a management problem, and no AI tech is going to fix that.

replies(11): >>38509198 #>>38509418 #>>38509802 #>>38509995 #>>38510231 #>>38510273 #>>38510431 #>>38511157 #>>38511186 #>>38512486 #>>38512716 #
20. pard68 ◴[] No.38509106[source]
I work for a company that develops and maintains code that "no one knows anymore" which helps run the US banking system. This article is ill informed fear mongering and the proposed "solution" is a joke.
replies(1): >>38509397 #
21. pixl97 ◴[] No.38509163[source]
I've seen some currently used COBOL that had it's like change date in the early 80s in the finance industry, so there some rather ancient programs out there. Of course this also means that these files have met the needs of the industry that long and bug free so why go messing with things that move 100s of millions/billions of dollars per day.
replies(1): >>38510919 #
22. ProjectArcturis ◴[] No.38509185{3}[source]
I guess translating COBOL is somewhat less embarrassing than Watson being used to predict fantasy football scores.
23. matthewdgreen ◴[] No.38509198[source]
How hard is it to actually learn COBOL? It seems like a fairly simple language to pick up, but maybe the idiomatic COBOL used in these legacy systems is particularly nasty for some reason.
replies(5): >>38509221 #>>38509476 #>>38509483 #>>38510105 #>>38510187 #
24. pixl97 ◴[] No.38509208{4}[source]
Hell, after seeing both kind of applications (java vs cobol) by these bank/financial companies I am of the opposite mind.

Those COBOL applications will keep working until the sun burns up or inflation makes the numbers so long they no longer fit in bounds.

The java applications they write should fill you with terror.

replies(1): >>38510578 #
25. the_only_law ◴[] No.38509221{3}[source]
Learning COBOL is the easy part. My understanding is the hard part is becoming familiar with insanely expensive, proprietary mainframe platform that’s you’ll find in most COBOL work. I know IBM has some sort of self training material, but I’m not sure if it’s enough to go from zero to qualified. Most work I see in the area seems to want established domain experts, not hackers who learn just enough to be dangerous.
replies(3): >>38509507 #>>38511265 #>>38516948 #
26. ◴[] No.38509265[source]
27. pixl97 ◴[] No.38509273{3}[source]
No really, in fact I'd say the opposite.

The issue with COBOL is any old idiot can't write it. If someone is touching the COBOL they are going in and making minor changes/expansions to an application that is older than the average HN user and those changes and expectations are well defined.

With Java and/or language of the day your cheapest run of the mill contractor given exceptionally poor instructions will crank out X LOC per day where X is the required output for them to keep their job. Maintaining that code will be some other bastards problem.

replies(1): >>38510310 #
28. buro9 ◴[] No.38509304[source]
https://www.glassdoor.co.uk/Salaries/cobol-programmer-salary...

If it's so valuable to the industry that they have good people who know it, that should surely be reflected in the salary / comp for roles... And if that were high, people would learn it, but it's not high, it's arguably worse than just learning a little JavaScript

replies(2): >>38509917 #>>38513921 #
29. moritzwarhier ◴[] No.38509314[source]
What prevents future employees from learning this language? Its development might have stalled, but as long there is documentation, whenever COBOL code breaks, there is are humans who can learn to fix the issue. Even when all "COBOL cowboys" have died out.
30. ajross ◴[] No.38509315[source]
Pretty much exactly. There are wizards at Project Zero doing reverse engineering of systems at the hardware microcode level. Our own kens shows up every few weeks with transistor level explanations of devices whose designers are long since retired.

And we're supposed to believe that no one in the world can figure out a batch-processed COBOL accounting system? Please. The talent is there, it's just that no one responsible for these systems wants to hire at that level. But in extremis they will, and the world will survive.

31. nonrandomstring ◴[] No.38509328[source]
I agree with you on the myth of "a shortage", and respect your strongly pro-market argument. Therefore "some industries can only survive if there are available low cost workers" suggests to me that those industries should simply die. Let more efficient industries that return more value to people thrive in their place?
32. apstats ◴[] No.38509335[source]
I feel like this solution was written by someone who hasn’t built software before. Doing a rewrite for the sake of a rewrite is not a good idea.
replies(1): >>38509391 #
33. rini17 ◴[] No.38509372[source]
How do you decide which language has better chances for advancement anyway? Pick some hot bleeding edge Microsoft tech instead and learn it only to get obsolete in few years.

You will have to eventually relearn regardless what you pick. And I can't believe having some COBOL in the CV will be an issue for anybody.

replies(1): >>38514364 #
34. high_5 ◴[] No.38509376[source]
It makes more sense to invest 100m $ in AI assistant for a year and then pay the CEO 120m/year the next one after firing the rest of the COBOL devs.
35. rabuse ◴[] No.38509381{3}[source]
So let the market decide their fate, as their outdated crap comes to a halt with no one to maintain it.
36. high_5 ◴[] No.38509391[source]
So... Having no COBOL devs left is a better one? What's the alternative then?
replies(2): >>38509551 #>>38510999 #
37. justahuman74 ◴[] No.38509395{3}[source]
Curious if you happen to have a ball-park figure on what 2023 compensation looks like for a COBOL programmer
38. rabuse ◴[] No.38509397[source]
Care to elaborate on this a bit? Some details would've been useful rather than just a "no u" comment.
replies(2): >>38510238 #>>38511290 #
39. lozenge ◴[] No.38509418[source]
Like in the video.. they are interpreting the COBOL and Java code with enough test cases and comparing their behaviour.
40. rightbyte ◴[] No.38509423{3}[source]
It still comes down to paying enough money for someone to do it. Pay me twice what I make now and I'll switch to coding absurd legacy all-caps COBOL systems in a heart beat. Probably less then that too.
41. serallak ◴[] No.38509434{3}[source]
I don't think it'd take two years to learn cobol, it should be less than that.

But it will take much more than that to learn the gnarly codebase in use in those shops...

And that is a skill that is certainly not transferable.

replies(1): >>38510771 #
42. vbezhenar ◴[] No.38509476{3}[source]
Language is easy, spaghetti code written without any discipline 60 years ago and modified in haste since is hard.
replies(3): >>38510023 #>>38510254 #>>38510732 #
43. SoftTalker ◴[] No.38509483{3}[source]
Not hard. It's a bit old-fashioned and sort of verbose but it's nothing difficult especially if you already know any other imperative languages. My first job out of school in the early 1990s was with one of the "big" consulting firms. We learned COBOL in a four-week boot camp and were then dispatched to a client site to write code.
replies(1): >>38511476 #
44. briHass ◴[] No.38509507{4}[source]
Not really much different from today: sure, you can 'learn' a new language in a few days, but you won't know the build tooling, deployment strategies, environment specifics, convention over config, and typical patterns/practices that will allow others to understand what you wrote.
45. gopher_space ◴[] No.38509535[source]
I cut my teeth on financial COBOL back in the 90s and from my perspective the article isn't "lying", it's "completely divorced from reality". For the past three decades it's been cheaper to encapsulate these systems and extend functionality by working in a modern language, and so that's what everyone's been doing.

The prime issue (imho) is that it's less expensive and better for everyone if you rewrite these systems from scratch, but that cost is still so prohibitive that it might not make sense to keep operating the company.

The second issue is hardware availability, a bizarre omission on the author's part. Your org lifespan might entirely, predictably depend on how many spares a forward-thinking dev bought in the 70's.

The third issue is fintech. Someone else has already raised the cash needed to rewrite from scratch, and what you can do about it is sell them your client list.

From a personal perspective COBOL is such a pain in the ass to work with that I'd need a ruinous salary to be enthusiastic about the job.

46. apstats ◴[] No.38509551{3}[source]
Yes - if we need them to people can learn COBOL it's not any harder to learn than many other programming languages.
replies(1): >>38510833 #
47. drewcoo ◴[] No.38509802[source]
> I expect you could get a 90% solution faster, but the whole challenge with these projects is how to get that last bit of correctness, and how to be confident enough in the correctness of it to actually use it in Production.

Is the goal to get working systems or to generate support activity? Or to tank the systems and replace them?

> it's a management problem, and no AI tech is going to fix that

What if we replace middle managers with LLMs?

replies(1): >>38511056 #
48. bluedino ◴[] No.38509820[source]
Not sure what's worse, depending on LLMs, or IBM to fix your COBOL project.
replies(2): >>38511234 #>>38512462 #
49. halfdan ◴[] No.38509917[source]
You’re looking at salaries for COBOL programmers in the UK which is a small subset of an already small market.
replies(1): >>38509985 #
50. IshKebab ◴[] No.38509985{3}[source]
I've checked in America before and came to the same conclusion. There's an idea that COBOL programming pays vastly more than other languages, but it's just a myth. It might pay a little more, but more like 20% more. Certainly not enough to warrant a COBOL career.

And I'm sure some consultants can earn vast sums fixing old COBOL code, but that's just because they're consultants. Consultants always earn vast sums.

replies(1): >>38512258 #
51. IshKebab ◴[] No.38509995[source]
I'm pretty sure there's already a system to transpile COBOL to Java without resorting to LLMs.
replies(3): >>38510301 #>>38511197 #>>38516136 #
52. timbit42 ◴[] No.38510023{4}[source]
COBOL code I saw in the 80's and 90's wasn't spaghetti code. COBOL is pretty structured. If it's not, that would be a prior step in porting it to another language.
53. setr ◴[] No.38510105{3}[source]
I don’t think cobol is that difficult; the problem is that it runs on the mainframe, and that’s a whole different beast. Everything in the mainframe world is both expensive and vendor-supplied, so it’s difficult to learn outside the company, and sufficiently proprietary that you’re probably not going to transfer it well elsewhere.

The language itself also encourages troublesome patterns, like all variables essentially being globally defined and untyped, procedure line numbers matter because of things like PERFORM A THROUGH B (which will execute all logic found between paragraph A and paragraph B)

replies(1): >>38510213 #
54. jacquesm ◴[] No.38510187{3}[source]
COBOL is pretty easy to learn. The problem is that it is so full of archaic nonsense (less so with the more recent versions) that you will be tearing your hair out and wishing for something more modern.

COBOL's main value is in maintaining a pile of legacy codebases, mostly in fintech and insurance that are so large and so old that rewriting them is an absolute no-go. These attempts at cross compiling are a way to get off the old toolchain but they - in my opinion - don't really solve the problem, instead they add another layer of indirection (code generation). But at least you'll be able to run your mangled output on the JVM for whatever advantage that gives you.

With some luck you'll be running a hypervisor that manages a bunch of containers that run multiple JVM instances each that run Java that was generated from some COBOL spaghetti that nobody fully understands. If that stops working I hope I will be far, far away from the team that has to figure out what causes the issue.

It is possible that someone somewhere is doing greenfield COBOL development but I would seriously question their motivations.

replies(2): >>38510508 #>>38512334 #
55. jacquesm ◴[] No.38510213{4}[source]
PERFORM PERFORM UNTIL ...

The reason for that is that originally the 'WITH TEST BEFORE' bit wasn't there so what looked like the test was done afterwards actually would just exit the loop immediately without executing the loop body at all.

So the syntax totally wrong-footed you into believing that your loop body would always be executed at least once but never did...

56. rubyfan ◴[] No.38510231[source]
> If they haven't actually done it by now, it's a management problem, and no AI tech is going to fix that.

This. Further, it’s a failure to continue to disincentivize roles that will support or port this business critical logic to something else. I worked at a large insurer where they slowly laid off mainframe talent over the last decade. Those mainframe salaries were counter to the narrative they were promoting around cloud being the future. Unfortunately in their haste to create optics they failed to migrate any of the actual code or systems from mainframe to cloud.

replies(2): >>38510542 #>>38513728 #
57. jacquesm ◴[] No.38510238{3}[source]
It's spot on. This is such a crazy end run around the real issue that it won't solve anything, it replaces COBOL that apparently the people that need this don't understand with Java that they also won't understand because it will be translated from idiomatic COBOL, which has about as much to do with Java as it does with Prolog, Visual Basic or C. You'd need to be an expert in both Java and COBOL to make soup of it.
replies(1): >>38514797 #
58. rubyfan ◴[] No.38510254{4}[source]
I don’t buy that vintage of code is an indicator of its quality.
59. itsoktocry ◴[] No.38510273[source]
>If they haven't actually done it by now, it's a management problem

How could you possibly know that? Do you think businesses have so few problems to deal with that moving off of Cobol should always be the priority, even if it functions and they are managing it?

It's not possible to resolve everything you have to do all at once.

60. rubyfan ◴[] No.38510301{3}[source]
Judging by the branding this is just an attempt to capitalize on the mindshare around LLM and GPT. Recall about 5-8 years ago they tried to sell the notion of huge cost savings replacing humans with the jeopardy champion and tech executives ate it up for a while.
replies(1): >>38512515 #
61. Nextgrid ◴[] No.38510305[source]
The problem isn't knowledge of the programming language. The problem is the developer experience not just within the language but the runtime environment it's executed in, as well as the corporate environment where such systems are used.

The developer experience is terrible, it's a completely parallel world that diverged about 25 years ago from what we're used to, and it's not particularly good. Furthermore, it's so obscenely priced that it will be never used for anything new (and forget personal projects), so career-wise it's a dead-end and confines you to only ever work in existing legacy deployments.

The "corporate experience" for the lack of a better word is terrible too. Companies that run this have zero engineering culture (if they did they would've finished their migration off COBOL long ago) and developer positions have about the same amount of respect and political influence as the janitor. There are much better options pretty much anywhere else, so the only remaining people are too mediocre to get those better opportunities, so it's not a good environment to learn from either.

Migrating off COBOL (or any legacy system) is possible - after all, startups have built similar systems from scratch. The problem is that this requires a competent engineering team and you won't get that without a good engineering culture and embracing engineering as one of the key pillars of your business and giving it the resources and respect it deserves.

62. kjs3 ◴[] No.38510310{4}[source]
I have no idea how that's the opposite of what I said. Transpilers haven't worked not because of how much or little the replacements are paid, or how well they do their job, but because there's more to moving from one computational ecosystem to another than just the language.
replies(1): >>38511330 #
63. tobyjsullivan ◴[] No.38510346{4}[source]
If there’s a shortage of developers who understand the language, then the team cannot properly understand the application (as per TFA).

If nobody can understand the application, it cannot be modified.

If an application cannot be modified, its value will continuously decrease with time.

That decrease in value, as well as the projected cost of building a replacement, is tantamount to accruing interest on a debt.

They could have reduced that debt by moving to functionally equivalent but more commonly taught languages back when their team could still understand the system.

It wasn’t necessarily wrong for them to accrue this debt - after all, banks and these monolithic institutions are masters at effectively utilizing debt. However, the practice may have become habitual and it looks like they might be in pretty deep at this point.

replies(1): >>38510722 #
64. victor106 ◴[] No.38510431[source]
> Skyla Loomis, IBM’s Vice President of IBM Z Software adds, “But you have to remember that this is a developer assistant tool. It's AI assisted, but it still requires the developer. So yes, the developer is involved with the tooling and helping the customers select the services.” Once the partnership between man and machine is established, the AI steps in and says, ‘Okay, I want to transform this portion of code. The developer may still need to perform some minor editing of the code that the AI provides, Loomis explains. “It might be 80 or 90 percent of what they need, but it still requires a couple of changes. It’s a productivity enhancement—not a developer replacement type of activity.”
replies(2): >>38510757 #>>38510835 #
65. Nextgrid ◴[] No.38510508{4}[source]
> that rewriting them is an absolute no-go

Rewriting and expecting 100% feature-parity (and bug-parity, since any bugs/inconsistencies are most likely relied upon by now) is realistically impossible.

However, new banking/insurance startups proved you can build this stuff from scratch using modern tooling, so the migration path would be to create your own "competitor" and then move your customers onto it.

The problem I see is that companies that still run these legacy systems also have a legacy culture fundamentally incompatible with what's needed to build and retain a competent engineering team. Hell, there's probably also a lot of deadweight whose jobs are to make up for the shortcomings of the legacy system and who'd have every incentive to sabotage the migration/rebuild project.

replies(3): >>38510763 #>>38511195 #>>38512426 #
66. credit_guy ◴[] No.38510542{3}[source]
> it's a management problem, and no AI tech is going to fix that.

There are no absolute "management problems". Something that is a management problem when the effort required is 1000 man-years, may stop being so when it's only 100 man-days.

replies(4): >>38511912 #>>38512414 #>>38514071 #>>38516240 #
67. trealira ◴[] No.38510578{5}[source]
Why do you say the Java applications at banks and financial institutions so bad? I'm not trying to contradict you or say you're wrong; I just want to know why.

Not having ever seen this kind of software, I would assume that the existing COBOL applications are good simply because all the bad COBOL code was probably scrapped decades ago, an example of survivorship bias.

replies(1): >>38511089 #
68. exhilaration ◴[] No.38510716{3}[source]
The answer is India. Just Google India and COBOL, there's tons of schools teaching it. So if Watson doesn't work out for you, IBM is happy to sell you offshore consulting services the bridge the gap! This is all win-win for IBM -- you can can keep your $200 million mainframe service contract or hire IBM to (kinda? maybe?) move you to something more modern.
replies(1): >>38513084 #
69. nobodyandproud ◴[] No.38510722{5}[source]
Many systems are obscure or niche, but that isn’t tech debt.

What you’re describing is a management and leadership failure.

In US tech we have an overabundance of Type A people in it for the money, but there are plenty of smart Type Bs who just want stability and a life, for a fraction of the going SV engineering pay.

You have to wonder why the latter is impossible for American companies to achieve.

70. JackFr ◴[] No.38510732{4}[source]
That is not any COBOL I’ve seen. Straightforward, well documented and comprehensively specified and tested.

When we needed changes (this was back office clearing stuff for a bank) they wouldn’t even talk to us until we specced out the changes we wanted in writing and often the specs we submitted would come back with requests for clarification. This was like the opposite of agile, but I don’t recall any bugs or defects making it into production.

replies(1): >>38511913 #
71. alexwasserman ◴[] No.38510747[source]
Having had to support and migrate COBOL in a big financial the problems weren’t really related to the COBOL, but instead were:

1. The mainframe costs and support. These can be mitigated with migration to a platform like Microfocus to emulate it, but be careful you don’t replace your ultra reliable mainframe with some flakey Windows servers.

2. The embedded business logic. Within the 50-60 years of code there’s a ton a specific edge cases encoded for long forgotten business reasons. These make rewrites really hard as bugs and edge cases have long since become features or behaviors dependent apps are coded to rely on. It takes a ton of extra analysis work to understand what to keep and what to trash.

3. The cobol apps run in a challenging environment that’s also not well understood today. All the jobs in JCL, ISPF to manage things, and a system like CICS for input/output. It’s a huge change beyond just writing code in a regular IDE.

replies(2): >>38511269 #>>38520216 #
72. koenigdavidmj ◴[] No.38510757{3}[source]
That seems to be the spot humans are weakest at—reviewing something where we think the computer did a good job 90% of the time, but quickly noticing when something goes wrong. Similar to level 3 self-driving—requiring full attention, able to instantly snap into full unassisted driving.
replies(1): >>38510941 #
73. jacquesm ◴[] No.38510763{5}[source]
That happens, but what also happens is that everybody is painfully aware of the situation and they do the best they can. Just like you or I would.

And of course, if you start a bank today you'd do the whole cycle all over again, shiny new tech, that in a decade or two is legacy that nobody dares to touch. Because stuff like this is usually industry wide: risk adversity translates into tech debt in the long term.

I suspect that the only thing that will cure this is for technology to stop being such a moving target. Once we reach that level we can maybe finally call it engineering, accept some responsibility (and liability) and professionalize. Until then this is how it will be.

replies(3): >>38511884 #>>38512547 #>>38512694 #
74. exhilaration ◴[] No.38510771{4}[source]
I don't think it'd take two years to learn cobol, it should be less than that.

There's a comment in this thread from a former consultant that completed a 4 week COBOL bootcamp before being sent to a client to write code!

replies(1): >>38511384 #
75. ◴[] No.38510833{4}[source]
76. ◴[] No.38510835{3}[source]
77. uxp8u61q ◴[] No.38510919{3}[source]
Even if it's not handling billions of dollars a day... Why fix what isn't broken? Software isn't an end into itself, it's a tool to solve a business need. If the business need is met, there's not anything to fix.
replies(3): >>38510995 #>>38511161 #>>38512051 #
78. ◴[] No.38510941{4}[source]
79. rafaelmn ◴[] No.38510995{4}[source]
Because you lose the ability to fix/update it if you don't exercise it ? Depends on how much you value the ability to change the software.
80. stonogo ◴[] No.38510999{3}[source]
There are plenty of COBOL devs; they're just not in the American labor market.
replies(1): >>38514232 #
81. dun44 ◴[] No.38511056{3}[source]
90% of middle management could be replaced with simple shell scripts; LLM would be a vast overkill.
82. pixl97 ◴[] No.38511089{6}[source]
Survivorship bias is bad if you're making a statement like "All old houses were good because we still see some old houses around".

But if you already have an old house that's working it's worth considering survivorship bias if you're building a new house to replace it. That new house is much less likely to survive that bias filter. Getting the new house as good as the old house will likely take you far more time and money than you expected.

And the same is with software. For example including outside libraries in your application makes the code much easier to write. You don't have to reproduce functionality. But libraries typically work like kitchen sinks. You may not have wanted a garbage disposal, but your library came with it and for the next X years your application is around you have to make sure that disposal doesn't catch fire. From the security side you have to make sure there is no code interaction with disposal or any of the other 'household' functions its including that you're not testing for.

At least in what I do for a living which is in the field to code security reviews (I don't do these reviews myself, but I work with the teams that do), the amount of security issues that get caught in these new applications in a multi round process, at least to me is staggering. It has to be a multi round process as we see the issues being reintroduced we'd call out in previous sessions. Quite often in these larger organizations the programming teams are so unstable that between our first calls and last calls the entire team working on it will have rotated out.

83. adra ◴[] No.38511095{3}[source]
You work that as a contractor easy if you can find them. The problem is they're rarely hiring one offs to do this because these companies don't have engineering cultures, which is why they contract this shit to system integrators of the world like IBM, then they outsource the work to shoddy consultancy shops to maximize their profits doing basically zero work. If everything goes exceptionally bad (which it did at least in the case I was involved in), then they hire competent contractors to come in and save them from frighteningly large law suits. That's the new dev cycle of these legacy modernizations.. ahh
84. Stratoscope ◴[] No.38511116[source]
One correction on the article: "watsonx" (yes, with a lowercase "w" - ugh) is an umbrella brand for several of IBM's "AI" related products.

The actual product name for this one is "watsonx Code Assistant for Z", as the author could have found with a simple web search:

https://www.google.com/search?q=watsonx+cobol+java

https://newsroom.ibm.com/2023-08-22-IBM-Unveils-watsonx-Gene...

Disclosure: I work for IBM in "watsonx Orders", an automated order taker for restaurant drive-thrus.

replies(1): >>38514212 #
85. keep_reading ◴[] No.38511157[source]
I'll never get tired of these overly confident armchair expert comments on HN
86. kstrauser ◴[] No.38511161{4}[source]
One reason is that it’s harder to execute by the year. Mainframes might be fast for some things, but they won’t/can’t scale as well as a gazillion VPS instances.

I bet there are plenty of shops that have software they’d love to run on a cheap cloud server, so that they could retire their hyperexpensive big iron.

87. Cthulhu_ ◴[] No.38511186[source]
AI is just one tech, there's been "language X to Y" converters for a long time, including Cobol to Java. To the point where it will compile and at least seem to do the same thing, but... that's the thing with these codebases, verifying that it does the same is the challenge.

I have 0 experience in this field, but I'm willing to take a guess that the majority of a Cobol to X developer's work is not (re)writing code, but figuring out what the original code does, what it's supposed to do, and verify that the new code does the same thing. More testing than programming.

88. Cthulhu_ ◴[] No.38511195{5}[source]
This is the way, however, integrating with legacy systems then becomes a challenge; a bank's software is never isolated, it has to interface with others, cough up reports for the authorities, etc etc etc.

The green field isn't everything.

89. grammie ◴[] No.38511197{3}[source]
Heirloom computing where I am cto does this using transpilers with 100% automated transpilation. Using LLMs for an entirely deterministic domain borders on the insane. This is just marketing bs but we get asked about it and what our plan is to counter it all the time. Explaining that using Gen-ai and LLMs for what is a well understood compiler/transpiler problem that is already solved just seems to be too difficult for some people to understand.
replies(2): >>38511253 #>>38512820 #
90. curious_cat_163 ◴[] No.38511234[source]
I would hazard that depending on LLMs is worse.
91. IshKebab ◴[] No.38511253{4}[source]
In fairness I imagine an LLM could maybe transpile to more idiomatic code. For example when you transpile FORTRAN to C you get a load of +1s and -1s everywhere to deal with FORTRAN's 1-based indexing. An LLM could avoid that.

But I agree, it doesn't make sense to risk bugs just for that.

replies(1): >>38511603 #
92. kochbeck ◴[] No.38511265{4}[source]
It’s not that the mainframe is hard to learn. In fact, the environment is pretty easy to understand once you get past the archaic naming (but let’s not kid ourselves: on the POSIX side we’re still running t[ape]ar[chive] and other archaic tools too).

In a way, the ease IS the problem: the runtime environment for COBOL (and other stuff on the mainframe) assumes that the underlying platform and OS deal with the really hard stuff like HA and concurrent data access and resource cost management. Which, on the mainframe, they do.

Now, contrast that with doing the same thing in, say, a Linux container on AWS. From the stock OS, can you request a write that guarantees lockstep execution across multiple cores and cross-checks the result? No. Can you request multisite replication of the action and verified synchronous on-processor execution (not just disk replication) at both sites such that your active-active multisite instance is always in sync? No. Can you assume that anything written will also stream to tape / cold storage for an indelible audit record? No. Can you request additional resources from the hypervisor that cost more money from the application layer and signal the operator for expense approval? No. (Did I intentionally choose features that DHT technology could replace one day? Yes, I did, and thanks for noticing.)

On the mainframe, these aren’t just OS built-ins. They’re hardware built-ins. Competent operators know how to both set them up and maintain them such that application developers and users never even have to ask for them (ideally). Good shops even have all the runtime instrumentation out there too—no need for things like New Relic or ServiceNow. Does it cost omg so much money? Absolutely. Omg you could hire an army for what it costs. But it’s there and has already been working for decades.

God knows it’s not a panacea—if I never open another session of the 3270 emulator, it’ll be too soon. And a little piece of me died inside every time I got dropped to the CICS command line. And don’t even get me started on the EBCDIC codepage.

Folks are like, “But wait, I can do all of that in a POSIX environment with these modern tools. And UTF-8 too dude. Stop crying.” Yup, you sure can. I’ve done it too. But when we’re talking about AI lifting and shifting code from the mainframe to a POSIX environment, the 10% it can’t do for you is… all of that. It can’t make fundamental architectural decisions for you. Because AI doesn’t (yet) have a way to say, “This is good and that is bad.” It has no qualitative reasoning, nor anticipatory scenario analysis, nor decision making framework based on an existing environment. It’s still a ways away from even being able to say, “If I choose this architecture, it’ll blow the project budget.” And that’s a relatively easy, computable guardrail.

If you want to see a great example of someone who built a whole-body architectural replacement for a big piece of the mainframe, check out Fiserv’s Finxact platform. In this case, they replaced the functionality (but not the language) of the MUMPS runtime environment rather than COBOL, but the theory is the same. It took them 3 companies to get it right. More than $100mm in investment. But now it has all the fire-and-forget features that banks expect on the mainframe. Throw it a transaction entry, and It Just Works(tm).

And Finxact screams on AWS which is the real miracle because, if you’ve only ever worked on general-purpose commodity hardware like x86-based Linux machines, you have no clue how much faster purpose-built transaction processors can be.

You know that GPGPU thing you kids have been doing lately? Imagine you’d been working on that since the 1960s and the competing technology had access to all the advances you had but had zero obligation to service workloads other than the ones it was meant for. That’s the mainframe. You’re trying to compete with multiple generations of very carefully tuned muscle memory PLUS every other tech advancement that wasn’t mainframe-specific PLUS it can present modern OSes as a slice of itself to make the whole thing more approachable (like zLinux) PLUS just in case you get close to beating it, it has the financial resources of half the banks, brokerages, transportation companies, militaries, and governments in the world to finance it. Oh, and there’s a nearly-two century old company with a moral compass about 1% more wholesome than the Devil whose entire existence rests on keeping a mortal lock on this segment of systems and has received either first- or second-most patents every year of any company in the world for decades.

It’s possible to beat but harder than people make it out to be. It makes so many of the really hard architectural problems “easy” (for certain definitions of the word easy that do not disallow for “and after I spin up a new instance of my app, I want to drink poison on the front lawn of IBM HQ while blasting ‘This Will End in Tears’ because the operator console is telling me to buy more MIPs but my CIO is asking when we can migrate this 40-year old pile of COBOL and HLASM to the cloud”).

Mainframes aren’t that hard. Nearly everyone who reads HN would be more than smart enough to master the environment, including the ancient languages and all the whackado OS norms like simulating punchcard outputs. But they’re also smart enough to not want to. THAT is the problem that makes elimination of the mainframe intractable. The world needs this level of built-in capability, but you have to be a bit nuts to want to touch the problem.

I have been to this hill. I can tell you I am not signing up to die on it, no matter how valuable it would be if we took the hill.

replies(3): >>38511861 #>>38514038 #>>38516855 #
93. DebtDeflation ◴[] No.38511269[source]
>The embedded business logic. Within the 50-60 years of code there’s a ton a specific edge cases encoded for long forgotten business reasons.

Honestly, this should be the top comment in the thread.

The issue isn't COBOL being a hard language to learn or to translate to Java or not enough programmers or companies not being willing to pay people enough to work with it.

The issue is the 50 years worth of business logic, added incrementally, over the years, with no documentation, blended into the original source, for reasons no one still working there remembers, as you stated. It's IF-ELSE statements all the way down and no one wants to touch a single one of them for fear of breaking something whose conditions might not even manifest themselves for months or years with no real way of even regression testing it.

replies(2): >>38511831 #>>38512035 #
94. pard68 ◴[] No.38511290{3}[source]
There isn't much to elaborate on. We are not hard up for COBOL or RPG devs, I'm sure some municipalities are but they are probably hard up for anyone who has the skills to make six figures.

COBOL is used still for far more reasons than technical debt, there is good reason for the language and I doubt Java is even capable of replacing it. Even if an AI could write a 100% perfect Java version of COBOL, Java would fall flat on its face. COBOL and other languages like it are very performant languages, optimized over more than a half century.

95. grammie ◴[] No.38511306{3}[source]
Heirloom Computing, I am CTO, have transpilers for cobol and pl/1. code transpilation is the easy part. Create a Grammer for the source language, generate the ast, transform the ast to the target language ast and generate the code. Now make that generated code run and behave the same way as on the mainframe and give it all the subsystems that exist on the mainframe for transaction, file handling, security etc. This is the part that is difficult. Fortunately it's already solved and in production with our clients.

So transpilers do work and are in production but our biggest competitor is inertia.

96. grammie ◴[] No.38511330{5}[source]
Correct, language transpilation is the easy bit, ensuring that execution and behavoir is the same as on the mainframe is tough, but also solved.
97. wombatpm ◴[] No.38511384{5}[source]
It’s not the language, it’s the business rules and history.

Why is this input file loaded and rechecked 3 times? Because 30 years ago a file load failed, breaking end of quarter reports. This was the fix: if we can read that file three times and it doesn’t change then we know it’s good

98. grammie ◴[] No.38511420[source]
It's a demo but the code that is transformed is trivial in the extreme. The whole approach of selecting small pieces is just not scalable. One small project i worked in was 22 million lines of cobol code, spread over 4000+ programs imagine having to select bits of code out of that.
99. spindle ◴[] No.38511433[source]
I believe that practically the whole barrier to getting rid of legacy COBOL systems is testing the replacement system. Of course it's possible but it's very very expensive ... I don't have numbers for how expensive, but I've worked on financial software in COBOL on a mainframe, and nothing was the least bit abstruse or scary or difficult about it except the worry that our test suite might not be comprehensive, no matter how much we worked on it (and we worked on the test suite about the same amount as we worked on all the rest of the code put together). But at least our test suite appeared to cover all the cases that eventuated when we ran our COBOL code. How confident would be be that it would cover all the cases that eventuated when we ran translated code? THAT is the problem (or at least was for us). Nothing at all to do with whether translating COBOL can be made easier.
replies(1): >>38512215 #
100. foobarian ◴[] No.38511476{4}[source]
Haha that explains a lot :-)
101. gumballindie ◴[] No.38511509[source]
Another marketing piece by IBM aimed at impressing inexperienced managers and developers. If they wanted to come up with a tool to rewrite Cobol they’d have come up with a standard transpiler.
102. dun44 ◴[] No.38511603{5}[source]
You could do it with a trivial C macro instead.
replies(2): >>38512172 #>>38514564 #
103. hedora ◴[] No.38511721[source]
The implicit assumption here is that developers will charge less to maintain code that was machine translated from cobol to java than they will to maintain the original code.

I’d certainly want to be paid more to deal with the LLM output than the original source code.

I don’t know cobol and I do know java, but that doesn’t enter into it from my perspective.

replies(1): >>38511750 #
104. delecti ◴[] No.38511750[source]
> I don’t know cobol and I do know java, but that doesn’t enter into it from my perspective.

Shouldn't it though? Maybe you (as an individual) would charge (or expect to be paid) more to maintain LLM code than bespoke code, banks know they won't have to pay as much to you (as a member of the collective of java engineers) than they would to a cobol dev.

Really the implicit assumption is that cobol devs are fewer and more expensive than java devs.

105. Closi ◴[] No.38511831{3}[source]
But there is one way it can be done - lots of painstaking analysis of the Cobol source code to attempt to extract the business logic (which is incredibly slow).

At least this is how we did it at my old work where we were replacing a mainframe system.

In principle though, this is stuff that an LLM could do - assist with extracting some of this business logic and speed this process up (while porting it to a language that is easier to work with). While the LLM doesn't necessarily understand the business context, it can do basic analysis which will speed it up massively. It can also generate test cases across the whole codebase.

106. hnlmorg ◴[] No.38511861{5}[source]
Well said! This echos my, admittedly somewhat limited, experiences as well.

I remember one mainframe I supported, there was an explosion on the same block which took out most of the buildings in the area. It was bad enough that the building which housed the mainframe was derelict. But that mainframe chugged along like nothing happened. I can't remember what hardware nor TSS it was running but I woul guarantee that none of the platforms I've supported since would have faired nearly as well (though I did have some SPARC boxes in one company that survived more than 10 years continual use and zero downtime -- they were pretty special machines too).

107. nradov ◴[] No.38511884{6}[source]
Why would software technology ever stop moving? To a first approximation it is unconstrained by physical reality (unlike other engineering disciplines) so I expect it will keep moving at roughly the same rate. Maybe even accelerate in some areas.

Individual organizations can consciously choose to slow down. Which works for a while in terms of boosting quality and productivity. But over the long run they inevitably fall behind and an upstart competitor with a new business model enabled by new software technology eventually eats their lunch.

replies(1): >>38514608 #
108. rsynnott ◴[] No.38511891[source]
Well, I mean, if they were to do that, they wouldn’t be able to issue a press release containing the word ‘AI’, now, would they? Like, that is what this is about.
109. hobs ◴[] No.38511912{4}[source]
Given the decision would shrink a kingdom or put someone powerful out of a job it cannot be solved with AI making it easier.
replies(1): >>38512099 #
110. hnlmorg ◴[] No.38511913{5}[source]
This was my experience too.

Modern software engineers would hate that kind of red tape because we've been conditioned to want shorter feedback loops. Heck, I hated it back then too and I wasn't even accustomed to seeing my results instantly like I am now. It takes a special kind of person to enjoy the laborious administrative overhead of writing detailed specs before you write even a single line of code.

replies(1): >>38512991 #
111. wredue ◴[] No.38511988[source]
>it might be 80 or 90% of what they need

Yeah. Not good enough. For the life of me, I cannot remember the name, but I remember we used another IBM tool to translate cobol to Java, and what it generated was horrific dogshit Java that also was 80-90% of the way there. Thing is that, unless you have very comprehensive testing laid out (lol. Find me a mainframe shop with comprehensive testing and I’ll smash my testicles in a hydraulic press), it’s impossible to know what it missed, and thus the 80-90% of the way there was effectively 0.

We never did push forward with this tooling, but I strongly suspect that had we, the ROI would have been vastly net negative after horrendous business impacts.

112. wredue ◴[] No.38512035{3}[source]
>it’s if-else statements all the way down

My experience with cobol at a few companies is that you’ve forgotten the mighty GOTO.

113. wredue ◴[] No.38512051{4}[source]
Because all the data and processing being on the mainframe poses loads of problems which are causing competitive disadvantages today.

As a shining example of the grief: 8 letter passwords.

114. janalsncm ◴[] No.38512099{5}[source]
That kingdom is shrinking already as people retire. Converting COBOL into Java means your kingdom can expand.
115. wokkel ◴[] No.38512172{6}[source]
I have the pleasure of supporting a transpiled rpg (system i) to Java codebase. Shit that's come up: almost everything is a global. State machine are used for crud logic which is implicit in the rpg runtime and explicit in the Java codebase. Magic constants all over the place. Magic blobs of screen configuration mapping. Transpiling and directly supporting the transpiled codebase is basically masochism.
116. wokkel ◴[] No.38512215[source]
No, getting a 100% equivalent working Java system is trivial. But the goal is having something that looks like it was Java from the start not Java as written by a cobol expert. So proper use of classes. Not al globals. Sensible naming etc. The real cost is in maintenance of the code: having in idiomatic Java could help. Translating to idiomatic Java is very hard
replies(1): >>38514169 #
117. cowsandmilk ◴[] No.38512258{4}[source]
> Certainly not enough to warrant a COBOL career.

I don't understand why people are claiming this sets your career to only ever write COBOL. I know plenty of people who started their careers with perl, but have been Java, JavaScript, TypeScript, Go, or Rust programmers at different points as their careers progressed.

Someone overseeing a project migrating a COBOL monolith over time to a set of decoupled services in some other language would easily have a strong story for the timeless need of improving application architecture.

118. phasetransition ◴[] No.38512334{4}[source]
Son of a COBOL dev... All this virtualization mess, minus the extra Java layer, started back in the 90s, courtesy of Unisys. I remember my dad pulling his hair out when I was in High School, though I did not understand why back then.
replies(1): >>38512622 #
119. ahejfjarjnt ◴[] No.38512414{4}[source]
It is not that difficult a problem code wise. It's a very difficult problem culturally though, in that the people that know mainframes and also C/C++ or Java or whatnot and could help do the migration slowly and safely and mostly in-place, are not the people that want to or are able to touch the COBOL. And outside of the systems programming crowd (who mostly don't do COBOL), the applications people at a lot of mainframe shops tend to wind up being heavily siloed, often attached directly to a particular business unit, with one foot in the Computer Science/IT fields and one foot in Finance. As in, they might literally only have ever written COBOL, and might not understand the high level architecture well enough to actually reconstruct anything. It would be like working on a single app that gets packaged in a docker container your whole career, and then somebody comes and starts asking questions about how you would reimplement Kubernetes and container runtimes. That's really how a system like CICS (where a lot of application developer code ends up, and is mostly transparent to them) actually functions. It was containers way before they were called containers, and usually involved some pretty strict restrictions imposed on the runtime environment.

So you've got a situation at a lot of places where you'd need to replace 50%+ of your staff if you wanted to convert to modern tooling, while at the same time it is harder and harder to replace the staff that leave and know the old set of tools. That cannot continue indefinitely. And until you cross some invisible threshold in the number of people that are comfortable in a modern software development environment, things can continue like they are for a long time.

Ultimately this is driven by the strange (for the tech industry) demographics of mainframe teams, in that they are often still dominated by people that entered the industry before microcomputers were ubiquitous. They may only know "the mainframe way" to do things, because they entered the industry the old-school way by stacking tapes in a data center or moving up from accounting, back before CS degrees and LeetCode and all that. It's only like that in mainframe shops because everywhere else has had massive growth (as in from 0 -> however many programmers they have now) and skews much younger as a result of computing not even having been adopted in any significant way until (for example) the late 80s/early 90s.

It's this cultural aspect of Mainframes (due to the history of being the oldest computing platform still running in production) that remains mostly unexamined and is not well understood by those on the outside. These are unique challenges that mostly don't have to do with the well known and arguably solved problem of "how can I convert some COBOL code into logically equivalent <any other language> code".

One final point is that COBOL is ruthlessly efficient (equivalent in speed to C, sometimes with more efficient use of memory for particular kinds of programs it is specialized for) in a way Java is not. Java is only popular on mainframes due to the licensing advantages IBM was forced to give out due to the much higher resource usage, and because their JVM is actually a very good implementation and well-liked by people on non-mainframe teams. If everything starts moving to Java though, I bet those licensing advantages probably go away. So I think more needs to be invested in C/C++ and possibly Go, rather than Java, at least initially. It is possible to write both of those portably and not suffer the huge performance penalty implicit in moving to a Java stack. I suppose this in particular may just be due to me being several years removed from web development, and some of the mainframe attitudes towards code have started to rub off on me.

120. taneq ◴[] No.38512426{5}[source]
If you need 100% parity, that’s a transpiler job.
121. haolez ◴[] No.38512462[source]
I know of a retail company that hired IBM to get rid of its huge Clipper code base (around 1.5MM LOC). IBM told them that they would get rid of all Clipper in 2 years.

5 years later, IBM has a few containers (physical containers) in this company's parking lot, adapted as offices, full of engineers doing maintenance and new features on the Clipper code base.

replies(1): >>38512644 #
122. kolinko ◴[] No.38512486[source]
As far as I understand, the issue with Cobol specifically is that the code is riddled with gotos and global variables - and untangling all that mess is the real issue, not converting into Java itself.

Using traditional algorithms you end up with literal exponential complexity very fast. You also need a human's ability to figure out which new abstractions to create - otherwise you will end up with a code that is just as difficult to maintain as the original.

replies(1): >>38513933 #
123. kolinko ◴[] No.38512515{4}[source]
Well, they essentially had the same idea as OpenAI with GPT, just they failed to really build it because transformers weren't invented yet.

But their business cases were similar to what we see now with LLMs.

124. wongarsu ◴[] No.38512547{6}[source]
> if you start a bank today you'd do the whole cycle all over again

It is worth noting that we now have much better processes and tooling than software developers had in the 60s. Some Cobol systems predate the invention of SQL or database normalization (3NF, BCNF, etc). Never mind the prevalence of unit testing and integration testing, automating those tests in CI, the idea of code coverage and the tooling to measure it, etc. Programming languages have also come a long way, in terms of allowing enforceable separation of concerns within a codebase, testability, refactorability, etc.

Sure, you can still write yourself into a codebase nobody wants to touch. Especially in languages that aren't as good in some of the things I listed (say PHP, Python or JS). But it's now much easier to write codebases that can evolve, or can have parts swapped out for something new that fits new requirements (even if that new part is now in a new language)

replies(1): >>38512607 #
125. rcbdev ◴[] No.38512573[source]
This has been IBM's M.O. ever since acquiring PwC Consulting in the early 2000s.

Even their downfall feels incredibly sluggish and geriatric.

126. RickJWagner ◴[] No.38512578[source]
I don't think it's worth it.

Disclaimer: Started career in COBOL on mainframes. Hold Sun Certified Java Architect and Java Programmer certs. Have worked in professional Open Source (mostly Java middleware) for over a decade.)

I'm not sure the Java will be more readable. Java's dependency/library spiderweb is much more complex and fragile. COBOL on the mainframe is maintained from a central position. COBOL won't have the attack surface Java will (see previous point about dependencies).

I think it might be better to just train newbies on COBOL.

Caveat: Maybe cloud economics can make a compelling budgetary argument.

127. jacquesm ◴[] No.38512607{7}[source]
We have better processes but we don't necessarily have better programmers, and better tooling is no substitute for that.
replies(4): >>38512766 #>>38513424 #>>38515870 #>>38522829 #
128. jacquesm ◴[] No.38512622{5}[source]
I think you get it now though. I've seen this whole industry up close for the last 40 years or so and it's absolutely incredible how we went from a machine with 32 M of RAM and 300 M of storage sufficient to serve 1400 branch offices of a bank to a phone with a very large multiple of that, that can barely serve a single user.
replies(2): >>38514201 #>>38514543 #
129. rcbdev ◴[] No.38512644{3}[source]
During my time at the company we never once talked about what the customer actually needed or wanted. Your story confirms my suspicion that this type of sales-driven development is just how IBM operates anywhere.

The truth is, IBM has been bean counted to death. Their turnover rates are incredibly high, employees have to deal with utilization targets that they can't meaningfully do anything about and lines of communication are incredibly dysfunctional. Their global headcount has been quietly but steadily declining since the dawn of the 2000s and their whole EMEA business is riding on having good ties to the right people in the public sector.

The result is that IBM's customers are increasingly unhappy. I do not see how this company is going to survive in its current form.

replies(1): >>38513104 #
130. noduerme ◴[] No.38512694{6}[source]
Tech debt is can come from risk adversity or from taking risks on new shiny things. I think you're right that as long as technology is a moving target, it's always going to be there. To me, the trick is not cornering yourself in a situation where your whole ecosystem is essentially abandoned and not rewriting for the sake of chasing the latest craze. That means parallel re-development, from scratch, of all the existing features, on something like a 10- or 15-year cycle. You want to pick a technology you're certain won't sunset in the next 15 years (with upgrades and further development along the way, of course), then spend a couple years rewriting everything in parallel while still running your old system, test it in every way possible, then blue/green it. I've done this three times in my life for one company on the same piece of large business software.

Companies should think of their software the way automakers or aircraft manufacturers think of their platforms. Once new feature requests are piling up that are just more and more awkward to bolt onto the old system, you have another department that's already been designing a whole new frame and platform for the next decade. Constantly rolling at a steady pace prevents panic. Where this breaks down is where you get things like the 737 MAX.

replies(1): >>38513276 #
131. rodgerd ◴[] No.38512716[source]
I once did some work looking into whether I could help move some subsystems off a zOS system with the help of an HP pitch that was a combo of auto-translation plus contractors checking. It was an attractive idea. I've also looked at CICS/COBOL environments that run on *ix systems, with a similar view.

The problem is that once you start working through your system, it's a lot harder than people making this sort of pitch would like you to believe. People writing CICS/COBOL now will likely be writing well-structured code with sensible names, taking advantage of threading and with proper interfaces between different programs. They're shipping data on and off the mainframe with maybe REST maybe SOAP maybe MQ. They're storing their data in DB2.

But people writing the parts of the codebase 20 years ago were still regularly dropping down to 390 assembler to get it to run fast enough and, guess what, that code is still there and still being extended. Maybe they were using DB2, maybe they were using VSAM files. They were using maybe MQ, maybe SNA, maybe TCP sockets with bespoke protocols to interact with the outside world. Programs might have just been talking to each other by banging on memory regions.

If they were working in the 80s and 90s, they had all that, but they were probably experimenting with 4GLs that promised to let anyone build programs. Some of those 4GLs are still around, probably not running CICS. And there was a lot of assembler. Maybe some non-CICS workloads. Passing things around via memory regions would be as often as not.

Oh, and the people who understood any of the business processes that got turned into code are long, long gone. They got laid off to justify the computerisation. The only people who know why things happen a particular way are the people who wrote the code, if they're still alive and working for you.

And so on and so forth. The reason mainframe code bases are hard to pick apart are because they're 50 years of different languages and runtimes and databases and performance tradeoffs and opinions about what good practise is. This is hard. An "AI" COBOL to Java translator isn't 80 or 90% of the job and honestly IBM should be embarrassed to be the ones suggesting that it is.

132. noduerme ◴[] No.38512766{8}[source]
One thing the parent's post suggests, though, is that we do have better standardization and interoperability. Data normalization is a problem that has largely been solved. So is reading and writing data at scale. Once you've gone to SQL, you presumably won't ever need to go to some other database language or drastically restructure your data in order to rewrite your backend or frontend next time. Similarly, there aren't fifty different schemes for serializing data anymore. JSON is "good enough", every language can now parse it and probably will for the next hundred years. So parts have become more interchangeable. The burden on experienced programmers is less, and it's easier for novice programmers with a shallower base of knowledge to work on pieces of a project even if they don't understand the whole thing. In this sense, tech has become less of a moving target.
133. tootie ◴[] No.38512820{4}[source]
It's almost sad. Watson defeated Ken Jennings at Jeopardy 12 years ago and today IBM are nowhere in the AI race. They bet the farm on the exact right domain ahead of the competition and still failed.
replies(1): >>38514245 #
134. nunez ◴[] No.38512959[source]
Literally today I was checking out of a rental car center and the checkout person loaded up an emulated green screen mainframe program to complete the process.

I know for sure that Avis's reservation system runs on mainframe.

To me, that means that the business logic of rental car checkout and return is so complicated and/or nuanced, it is cheaper for rental car companies to find/retain mainframe developers to keep these running than it would be to re-platform onto commodity hardware.

Also, given that basically every industry is powered by a handful of mainframes, it is surprising that COBOL/Fortran developers aren't making insane money like I thought they were.

replies(3): >>38514222 #>>38514425 #>>38514693 #
135. nunez ◴[] No.38512991{6}[source]
Kind of reminds me of taking pure CS101 (I.e. no programming language; just theory and sacrifice)
136. nunez ◴[] No.38513084{4}[source]
Can confirm. When I was in Hyderabad in 2019, I saw signs for a few schools marketing COBOL and FORTRAN.
137. girvo ◴[] No.38513104{4}[source]
And the R&D guys at IBM are 1) brilliant and 2) smart enough to stay well away from IBM proper as much as they can, at least when I partnered with some, some years ago
replies(1): >>38513628 #
138. jacquesm ◴[] No.38513276{7}[source]
That makes perfect sense. Extra points if you designed the system to be replaced in time.
replies(1): >>38514233 #
139. bradleyjg ◴[] No.38513424{8}[source]
We have significantly worse programmers, that’s what better languages and better processes enable. People that would have washed out when the only option was bit twiddling and pointer chasing can now be productive and add value. That’s what progress looks like.
140. haolez ◴[] No.38513628{5}[source]
The best engineer that I know worked a long time for IBM. Until he couldn't bear with it anymore.
replies(1): >>38514130 #
141. GenerWork ◴[] No.38513728{3}[source]
Did the remaining mainframers demand a higher salary to stay on, or did the insurance company have to hire outside contractors at exorbiant rates to migrate everything over?
replies(2): >>38514695 #>>38515997 #
142. aitchnyu ◴[] No.38513921[source]
Indian companies take fresh graduates and train them into mainframe devs. I was a little sceptical about shrinking pool of highly paid people.
143. dzhiurgis ◴[] No.38513933{3}[source]
Isn’t global variables at some point a requirement for performance? Or is there patterns that can do both?
replies(2): >>38513966 #>>38513968 #
144. ◴[] No.38513966{4}[source]
145. vardump ◴[] No.38513968{4}[source]
Global variables usually hinder performance on modern hardware. Harder to use multithreading and (sometimes) higher probability for false sharing.
146. StillBored ◴[] No.38514038{5}[source]
100% agree, but AFAIK the HW isn't doing lockstep on modern zseries. The nonstop was also HW lockstep many years ago and they converted to a checkpoint restart model, which AFAIK is the same on zos/etc with help from the "HW" which is just "software" running in the LPAR/etc.

Regardless, there have been various clustering/SSI/etc software layers designed to make windows/linux/etc behave as though it were lock stepped as well via software/hypervisor checkpoint restart.

So it is not impossible, but you don't get it out of the box because most of these applications have moved the fault tolerance into an application + database transactional model where the higher level operations aren't completed until there is a positive acknowledgment that a transaction is committed (and the DB configured for replication, logging, whatever if needed, blocks the commit until its done).

So, yes that model requires more cognitive overhead for the developer than Cobol batch jobs, which tend to be individually straightforward (or the ones i've seen the complexity is how they interact). But the results can be the same without all the fancy HW/OS layers if the DB is clustered/etc.

147. sublinear ◴[] No.38514071{4}[source]
Management problems have a tendency to snowball and land on the next manager.

Higher authority doesn't understand it and the subjects don't care.

148. girvo ◴[] No.38514130{6}[source]
I haven't checked, but I would not be surprised if the engineers I'm talking about have left too at this point. I worked on a homomorphic encryption system for some insurance companies to be able to query each others claim data with them, fun project!
149. fbdab103 ◴[] No.38514169{3}[source]
I think those companies would kiss you on the mouth and do any unspeakable acts you requested were you able to promise a 100% equivalent system in Java. There is a ton of industry experience in turning working-but-trash Java into something more maintainable.
150. snotrockets ◴[] No.38514201{6}[source]
That phone does a lot more than the bank mainframe used to do.
replies(2): >>38514345 #>>38514457 #
151. dr_kiszonka ◴[] No.38514212[source]
How do you like the job? (Just curious, no relation with or friends at IBM.)
replies(1): >>38514622 #
152. Exoristos ◴[] No.38514222[source]
It's cheaper to threaten their families.
153. dr_kiszonka ◴[] No.38514232{4}[source]
Would bumping starting salaries for Cobol devs to 150k, resolve it in the US? (I am assuming banks could afford it.)
replies(1): >>38525605 #
154. noduerme ◴[] No.38514233{8}[source]
Hahah. The last one was a close call, since the entire front end of the system from 2009-2020 was a responsive single page app written in Actionscript 3, to replace the old PHP page system... but we saw the deadline looming about a year in advance and accelerated it.
155. snotrockets ◴[] No.38514245{5}[source]
IBM are very good at doing that.
156. g805 ◴[] No.38514345{7}[source]
"Busy" and "useful" are entirely different things. Maybe all that extra capability would be better spent elsewhere.
157. Kwpolska ◴[] No.38514364{3}[source]
COBOL is one end of the spectrum, hot bleeding edge tech is another. Pick a mature yet modern stack and it should be fine for longer.
158. Kwpolska ◴[] No.38514425[source]
I doubt the rental car system is complex. It was probably written decades ago and is now complete and bug-free. Keeping it running on a mainframe with emulated terminals is far cheaper than a modern rewrite, and the age/maintainability of their software is the least of their concerns.
159. jazzyjackson ◴[] No.38514457{7}[source]
> iphone hangs while trying to match "blu" up with "bluetooth settings"

So many background tasks, much computation

160. pelasaco ◴[] No.38514460[source]
I think there are better languages to try to convert to, and as long they run on top of JVM, it should be fine.
161. mrweasel ◴[] No.38514543{6}[source]
That is some thing that I can't stop thinking about. I get that banking software does more, like online banking, larger transaction volume, more account and loan types, but why is it that we can not run a small to medium bank on a single modern CPU and 1TB of memory?
replies(1): >>38522587 #
162. IshKebab ◴[] No.38514564{6}[source]
The classic HN "trivial" :-D
163. tsimionescu ◴[] No.38514608{7}[source]
Software technology moves when we figure out new ways of doing software that bring some kind of advantage. If no one is finding new ways to do software that have any purpose, technology will stop moving. Physical reality doesn't really have anything to do with it - we're limited by human ingenuity, and possibly by the mathematical space of algorithms (though that's likely to be much larger).

For an example of this happening in a field, look at the glacial pace of advancement in theoretical physics for the last few decades, compared to 1900s. Or at the pace of development in physics in general in the centuries before.

replies(1): >>38514840 #
164. Stratoscope ◴[] No.38514622{3}[source]
Like many large companies, IBM discourages us from engaging in "social media" (which I guess includes HN) on work-related questions.

I felt that I could make an exception for the minor correction of the product name.

It's probably not appropriate for me to comment here on how I like the job.

I would be happy to discuss it privately with you or anyone else who is curious. My email address is in my profile; feel free to drop me a note and I will follow up with you.

replies(1): >>38518025 #
165. jillesvangurp ◴[] No.38514693[source]
I helped a client with some dependencies on Avis and a few other car rental companies a while ago. They indeed have some really old, clunky software. At least judging from the data dumps I had to work with. Also some notable differences in the quality of the data between different providers that I had to work with.

There are plenty of new mobility startups managing just fine without any Cobol or other legacy software. The issue companies like Avis have isn't their software but their lack of competitiveness and ambition level. The car rental business is always changing and they have to somehow adapt with it. And they struggle with that. It's not their software that holds them back but their dependence on things not changing. The software is just a reflection of how uncompetitive they are.

166. coldtea ◴[] No.38514695{4}[source]
>or did the insurance company have to hire outside contractors at exorbiant rates to migrate everything over

It would be appropriatingly ironic if some or all of those contractors were the same people previously fired.

167. zoom6628 ◴[] No.38514775{3}[source]
Absolutely. I've seen same since I started as a programmer in COBOL and assembly in early 80s.
168. Closi ◴[] No.38514797{4}[source]
And presumably this is why it requires an LLM rather than a transcompiler.

You want to translate 'idiomatic COBOL soup' which has been added to over decades into clean structured (possibly OOP) code covered by a robust library of tests.

169. pharmakom ◴[] No.38514840{8}[source]
Software trends seems to repeat themselves as we forget the lessons learned a decade ago. It’s more like fashion, in that sense.
170. GoblinSlayer ◴[] No.38515870{8}[source]
You don't need better programmers to maintain a js project, you only need hireable programmers.
171. rubyfan ◴[] No.38515997{4}[source]
Kind of the later I guess. Not much migrated really. When I left they were grappling with which major migration to do next, each to the tune of tens of millions each with no business benefit. In that industry there is momentum toward specialty tech platforms to save the day. Out of the frying pan into the fryer.
172. danmaz74 ◴[] No.38516136{3}[source]
I was thinking the same. It looks to me like a much safer process could be:

* use some kind of deterministic transpilation that creates ugly code which for sure reproduces the same behaviors

* add tests to cover all those behaviors

* refactor the ugly code

From my experience with copilot I guess that a LLM could help a lot with steps 2 and 3, but I wouldn't trust it at all for step 1

173. rubyfan ◴[] No.38516240{4}[source]
I’m not convinced the proposed solution is that. This pitch has been around for a long time, before LLM it was some unnamed proprietary technology and after LLM it’ll be something new. The reality is this stuff is hard even when 80% can be automatically converted.
174. kwanbix ◴[] No.38516431[source]
I want to go back to doiyng COBOL programing, so that I can land a job to keep a system up to date. However, while learning COBOL is not that complicated, learning the mainframe part is, because it is very difficult to find a mainfrane to work on, and I have not yet found an emulator to learn on it.
175. kjellsbells ◴[] No.38516855{5}[source]
If there was an HN equivalent of Reddit's /r/bestof, this comment would deserve to be there. Encapsulates everything about why this problem is so hard.
176. kjellsbells ◴[] No.38516931[source]
Indeed. If you are the board of any publicly traded technology company right now, and you don't have an AI strategy that you are telling the market, your stock is going to take a beating as investors flock to those who do. The strategy doesnt need to have a lot of meat behind it. Once it was crypto, now its AI, tomorrow it'll be something else. But AI has really captured people's imaginations, which means even the most foggy investors are looking at it. That drives leadership behavior more than the tech itself
177. specialist ◴[] No.38516948{4}[source]
I briefly helped with some mainframe/legacy modernization work.

Their hard part was determining which feeds and datasets were still needed by someone, somewhere. Over the decades, 100s were created as needed (ad hoc). No inventory. No tooling (monitoring, logging) to help. It's likely only a handful were still needed, but no telling which handful.

The bosses were extremely risk adverse. eg "We can't turn that off, someone might be using it."

I suggested slowly throttling all the unclaimed (unknown) stuff over time. Wouldn't break anything outright. But eventually someone would squeal once they noticed their stuff started lagging. Then incrementally turn things off. See if anyone notices. Then outright remove it.

Nope. No can do. Too risky.

178. throwaway98797 ◴[] No.38518025{4}[source]
not op

appreciate the comment

this exchange is what i show young people to demonstrate why you should never work for such companies (not signaling out yours specifically)

human want to be human

(tho a buddy of mine worked there with great benefits & $)

replies(1): >>38518510 #
179. exabrial ◴[] No.38518392[source]
COBOL -> Misses the entire point of a legacy codebase. COBOL is not the problem.

The actual problem is the missing knowledge from the engineers the CEO downsized and sent their work overseas. This boils down to three things:

1) figuring out the call patterns

2) figuring out the architecture

3) figuring out the business logic

This is also why cowboy coders are dangerous in your organization, while having corporate standards and making your codebase self-consistent (aka "Clean Code") is the most valuable thing you can spend your time on.

source: I made a bunch of dollar bills working on Java / EIS COBOL systems that needed to interact at wire speed, while not paying Big Blue for their EIS Gateway Software.

180. ukuina ◴[] No.38518510{5}[source]
Why is it bad for a company to have a social media or online comment policy?
181. octokatt ◴[] No.38520216[source]
I worked at an insurance company where a giant COBOL mainframe squatted in the middle of their data systems like a toad*.

During the last major attempt to remove the system, they got close. A team of good engineers worked over two years painstakingly updating documentation, transferring and rebuilding systems, and got to the Go-No-Go meeting. They explained everything to the single SME in their 60's who knew best where the bodies were buried, and she seemed satisfied.

When they got to the end of the Go-No-Go, she asked, "Where did you put our payroll?"

COBOL literally sent everyone their paychecks. It was enough of a deathblow to find something that important had been overlooked, the project was abandoned.

*Oracle Toad was in there too, but it felt snappy instead of squatty compared to the COBOL.

182. jacquesm ◴[] No.38522587{7}[source]
Eye candy. Endless abstraction layers.
183. theamk ◴[] No.38522829{8}[source]
Tooling and processing definitely helps, I have seen it with my own eyes!

(1) Start with experienced programmers who know how to write code

(2) Have them establish good culture: unit testing, end-to-end testing, code coverage requirements, CI (including one on PRs), sane external package policy, single main branch, code reviews, etc...

(3) Make sure that there are experienced programmers have final word over what code goes in, and enough time to review a large part of incoming PRs.

Then you can start hiring other programmers and they will eventually be producing good code (or they'll get frustrated with "old-timers not letting me do stuff" and leave). You can have amazing code which can be refactored fearlessly or upgraded. You could even let interns work on prod systems and not worry about breaking it (although they will take some time to merge their PRs...)

The critical step of course is (3)... If there are no experienced folks guiding the process, or if they have no time, or if they are overridden by management so project can ship faster then the someone disables coverage check or adds crappy PRs which don't actually verify anything. And then formerly-nice project slowly starts to rot...

184. stonogo ◴[] No.38525605{5}[source]
No, since that's not too much higher than starting salaries for Python programming at a medium-sized research institute, and there's no clear career progression for COBOL programmers.