La Basilica Di San Pietro

Absolutely gorgeous imagery, and it seems to have a functional purpose as well, as a digital twin for structural modeling.

Incredible work.

The work in the related stories are equally gorgeous. Thanks for sharing mate.

This reminds me strongly of Microsoft Photosynth. Can't help wondering what the lineage between the two looks like.

https://en.wikipedia.org/wiki/Photosynth

https://medium.com/@dddexperiments/why-i-preserved-photosynt...

dumb question: could we, in the future, use some kind of gen ai to generate a videogame map (i'm thinking quake 3 arena / openarena) of buildings like these ?

(not just the basilica di san pietro)

we could have these today; the difficult part is getting permission to use the building in your work (depending on jurisdiction / the work)

This brings back memories of Microsoft's acquisition of SeaDragon. At the time they had a really compelling demo (at least for me) of reconstructing 3D locations based on a smatterings of photos.

https://en.wikipedia.org/wiki/Seadragon_Software

https://www.youtube.com/watch?v=IFSsTwXLqsc

interesting, where would one have to look to learn and/or get the necessary data to pull that off?

i might just want to do that for my own private use (or i might be okay with law infringement).

probably start with finding out who owns / manages the building you want to use (public record, company reports)

if they like your project and see some value in it for themselves, they might even give you the contact of the designer / architect to get files

What does AI have to do with this? They thoroughly scanned the thing, where's the need for AI?

Seeing a digital version of it in such detail only further reinforces how important it is to experience it in person.

Few sights of man-made things have instilled as much awe in me as La Basilica Di San Pietro and most of them are also in Rome (namely the Pantheon and Moses @ Basilica di San Pietro in Vincoli).

Photogrammetry has rebranded itself to "Spatial AI".

So this is one of the vanity project Microsoft undertakes using the vast amounts of money it makes off of proprietary software?

To further sustain this point: I heard that in the past, someone recreated some parts of Politecnico di Milano (a famous technical university in Italy) as a map of some open source first person shooter. Unfortunately I don't remember which shooter it was.

At the time I really thought that was going to be the next Street View.

Maybe with WebGL and Gaussian Splats that can still be the case. But it’s also a ZIRP kind of project. Awesome, but what’s the business model?

Whether you can make reproductions of buildings and public interiors is known as "freedom of panorama". Wikimedia Commons has a comprehensive list by country [1].

https://commons.wikimedia.org/wiki/Commons:Freedom_of_panora...

It’s frustrating to see that the US added architectural copyrights in 1990. There’s an explicit exemption for photographs, but not models.

What problem did this actually solve? If there is one, we managed to live with it for the prior two centuries.

Chrome-based browsers only I presume :')

I wouldn't care about reproducing existing building as long as the AI can generate credible ones in the same style, then place them on dynamic worlds created by prompting the AI. Having intelligent AI NPCs as long as AI generated scenery would be a killer feature in any game. I'm talking about off line disconnected single player games; cloud ones with these features could be already here, but I want to be in control, and marketing rules are against that: who would buy the new shiny V2.0 with the new worlds and characters if 1.0 could create them just by asking it to? As someone who consumed tons of scifi novels and books as a kid and wants to be immersed in big worlds, enjoying great stories also in games (absolutely loved the Mass Effect saga), I already know what's going to happen when we'll be able to feed Philip K. Dick or Asimov, Sturgeon, Bova, Silverberg, etc. books to an AI and have it create worlds, environments, stories and characters straight out of the book descriptions. Literally drooling over it.

Works fine in Firefox for me.

Direct link to the virtual tour: https://virtual.basilicasanpietro.va/en

Not an explicit field expert, but pretty well into computer graphics(games) and been reading a bunch of papers in the field over the years.

Classic photogrammetry was always a mixed bag in terms of results (especially if trying to construct meshes), but even before NeRFs (Neutral Radiance Fields) and Gaussian Splatting there was a ton of work using neural nets to handle various parts and I doubt that many modern tools avoid using them.

So in a way, these fields actually made use of neutral nets/"AI" (honestly more relevant imho than most of the LLM stuff).

You don't see any value in this project? I certainly do.

True, although most of the snazzy NeRF and Gaussian Splatting papers still rely on good old COLMAP on the backend lol

Thanks for the memory - definitely one of the coolest demos I'd ever seen.

Any idea how accurate the pitch was compared to the reality?

With players in control any jank will be quite obvious, the field did accelerate thanks to neural nets but there seems to have been a lot of focus on NeRFs and GS (This interactive demo seems to use GS) and classic triangle-geometry (especially lower polygon counts) hasn't gotten as much love recently as the impressive GS demos has taken over.

But the success of GS and speeding up should rekindle some interest and let us use some of the advances in making "production ready" methods.

Microsoft was bummed that they couldn't acquire it so they recreated it in 3D.

Okay can we please play quake in this map now?

One such pipeline that already works today is photogrammetry of real place -> voxel data using VoxelPlugin. You can then leave it as a Voxel or bake it to a static mesh.

Example: https://twitter.com/phyronnaz/status/1549869716826689539

https://www.youtube.com/watch?v=WZbG5JTpSCA

Looking at the source code with web inspector it seems to be powered by 3D gaussian splatting and BabylonJS (https://doc.babylonjs.com/features/featuresDeepDive/mesh/gau...).

A lot of advances during the Renaissance happened due to some vanity projects of the Medici family (e.g. funding Galileo Galilei and Brunelleschi's Dome).

It's not preserved until the data and source code are open, I'm sure these corporate exercises are impressive to potential clients, but they have absolutely nothing to do with preserving, studying, or expanding access to art and culture.

You can't understand the scale of it until you experience it in person. The way I thought of it was that it is a cathedral made for giants.

I sometimes wonder why they don't just sell the whole thing to a real estate developer or something.

If inornate churches that look more like strip malls and expo centers are so much better for the laity then imagine how much more good it would do for the top brass, relieving them of the burden of having to look at all that sacred art all day long. What's good for the goose is good for the gander.

I think the biggest challenge is not any of the technical or legal problems already mentioned, but that none of these buildings are laid out with the primary objective of being fun to run around shooting people in. So once the novelty wears off, I expect the actual gameplay experience will be rather clunky, especially with competitive gamers.

Yes! I forgot about this but I knew I've seen something very similar long ago also by Microsoft. I wonder if any Photosynth DNA got into this.

This might actually be an awesome use case.

Interesting that people may experience it differently, but to me it was a bit of a letdown, somehow it felt larger than the human scale, so maybe impressive as a technical feat, but also somewhat boring, more intimidating than moving -- I was more touched by some of the others in Rome that you mention. But to me the ultimate awe-inspiring church was the Basilica of Assisi that felt just perfect in proportion and design.

It's like that kid that got expelled for creating a map of his school in Counter-Strike [1], due to fears of security threats. Not that I blame them, I could see people planning a robbery in Minecraft.

[1] https://arstechnica.com/gaming/2007/05/student-creates-count...

Exactly. And it was a great marketing tool for catholicism, imagine simpler (even if rich) folks came to visit the pope and experienced this marvel of medieval construction. You feel utterly insignificant on purpose, feeling weak and in presence of something much larger is an easy way to more faith, a truth valid for all humans across all time.

But to me, despite all of this, there was a lot of sadness in that experience - because you know how desperately poor common folks were, how instead of building such status mega symbol they could have done some proper good. But not for church of that era, it was busy fighting for power and money of that world and trying to show how above everybody else they were.

You can see miniature scale of this in literally every (also non-) older European village or town - religious buildings have received by far the most funding and care, sometimes overshadowing kings castles themselves. Cathedrals were always built to impress masses, and this one is just on top of the game, by huge margin for good reasons I believe.

> But not for church of that era, it was busy fighting for power and money of that world and trying to show how above everybody else they were.

Kind of like the church in America today.

It's also a symbol of all the money and gold, the real values and the secrets of the church...

If God exists, you think he would want you to sacrifice and spend it all on gold and salaries of locals ?

In fairness, the Catholic church is not the sect building strip mall churches.

I think putting AI front and center in the marketing like this is a public relations move by Microsoft to brush up the image of AI in the general public.

Clearly yes if we go by the Bible. Then we should be happy to get away with just gold and labor and not our children like Isaac.

Yes. Impressive tech but the website is ultimately not a great experience. You don't get the detail, the texture, the light, the human scale etc. Instead you get bits of wire frame, stuttering, odd flying movements, anti-aliasing issues etc. And a forced narrative along the side.

> you know how desperately poor common folks were, how instead of building such status mega symbol they could have done some proper good. But not for church of that era, it was busy fighting for power and money of that world and trying to show how above everybody else they were.

This is a tired caricature. We live in comfortable times. Materially, in many way, we are much more comfortable today than kings were back then. The world was different then, and it is irresponsible to project anachronistic categories onto a period of history that operated differently. And that somehow there exists a conflict between building magnificent churches and dealing with poverty is simply nonsense (indeed, poverty was dealt with through tithing and donations and by convents and monasteries with that charism; the first hospitals, for example, were founded by nuns, hence why in many languages the word for nurse is still "sister"). You can do both, hence the corporal works of mercy and spiritual works of mercy. Magnificent churches were not somehow the private property of some caricaturish class of clerical villains (who had no heirs, legitimate ones, anyway). They were the common patrimony of the Church. They were often constructed over long periods of time by the people in the community. They gave everyone, especially the poor, the possibility of witnessing and experiencing beautiful art and architecture that might otherwise only be accessible to the very richest of the magnates (and I challenge you to find a magnate who owned anything as spectacular as St. Peter's).

(Even today, you hear people ask the silly question "why doesn't the Church sell all its artwork and give the money to the poor?". If you allow that question to sink in for a moment, it becomes clear how preposterously silly it is to ask it. So you sell it. Then what? Now, these artworks are the property of private collectors or state institutions. Is that what you want? And the money: you think that will somehow "end poverty"? After food is digested, one's hunger returns. Far greater sums have been expended on the poor. The poor will always be with us. It is something we must continuously deal with. Robbing them of access to beautiful artwork, and depriving the Catholic faithful of their patrimony, is a pretty shitty solution, if it can even be called that.)

Frankly, what I find shameful is that we are richer than we're ever been, and yet we can't seem to produce anything that approaches the beauty of these old cathedrals. We have monks in Wyoming who are using CNC stone carving to build a gothic monastery[0], for crying out loud! We've never been in a better position to build beautiful things and cheaply at scale. And that's kind of the message of these buildings. It's not the material wealth per se, but the magnanimity of spirit that made this beauty possible and the spiritual awe it continues to inspire to this day. It's a condemnation of our vulgarity, of our consumerism. Even the churches we build today usually look like shit. If that's not cultural decadence, I don't know what is.

[0] https://carmelitegothic.com/cnc-stone-carving/

Why would they build? They're losing members year over year in droves. My parents are at that unfortunate age where a lot of friends and family are starting to die, and one constant refrain in the area is "when can the funeral service be? will a priest even be available at the church that day?" My grandmother died recently and they had to wait for a day for a priest to be available at that particular church because he holds Mass at three different physical locations in the area due to a lack of faithful. And this is Southeastern Louisiana, which is about as Catholic as it gets.

Any way to experience this on a VR headset? I have a Valve index

Because it's a massive waste of collective resources, almost entirely to glorify imaginary beings and enrich a priestly class. Worse it's owned by and financing an institution that breeds mental illness through teaching fallacies if not out right abusing people.

Yup, this is culture-washing financed with MS dollars, who finally found one way of promoting AI for Good™ (literally spelled out as such in the marketing video), probably hoping it would make up for the Gen AI slop humanity will have to deal with for decades to come.

It would be nice if they'd let you see the whole thing at once.

Anecdotally, I see a lot of of dwindling in membership (and, it at least one, even schism) in established protestant congregations (Methodist, Presbyterian, Lutheran, Anglican) - many of those buildings having literally shut down - and converts coming to my local Catholic parish, as well as returning practitioners of the faith.

It's hard to imagine the Catholic church being as prominent in the world as it once was, of course, but as member myself, the last couple of years give me hope that we might again start growing.

> But to me, despite all of this, there was a lot of sadness in that experience - because you know how desperately poor common folks were, how instead of building such status mega symbol they could have done some proper good.

Have you considered the idea that these could be considered jobs programs? It took a lot of masons, carpenters, ox/horse handlers, rope makers, quarrymen, etc, to run the supply chain for building this over the course of many, many decades. Just moving the obelisk to its current location took hundreds of men:

* https://en.wikipedia.org/wiki/Vatican_obelisk#History

Further, now, even centuries later, all of this architecture and art is helping the local with a fairly vibrant tourism industry.

Beauty, in addition to being a good in itself:

* https://en.wikipedia.org/wiki/Transcendentals

also has practical benefits.

> But not for church of that era, it was busy fighting for power and money of that world and trying to show how above everybody else they were.

The other option was to be rolled over by secular powers (princes, kings, emperors). If you think politics is nasty now, it was a (often literal) blood sport back in the day.

You have it all wrong. These buildings were built a) to glorify God, and b) because humans love to have beautiful things. And we are all much, much better off for it. These old, beautiful buildings are one of the great treasures that our ancestors left us. We should be thanking them, not criticizing them.

> Frankly, what I find shameful is that we are richer than we're ever been, and yet we can't seem to produce anything that approaches the beauty of these old cathedrals.

Well said. Beautiful things like these are a massive gift to humanity, and by all rights we should be able to do even more projects like them. Hopefully one day we will rediscover that drive and build new things which will enrich not just ourselves, but future generations.

I agree having worked directly with it (long personal project). Photogrammetry is so error prone (get a sign wrong in your sensor fusion/SLAM code and you're cooked for 2 weeks) and I got into it right in the last years before models that could compete were introduced (reactions from PhDs who spent years working on photogrammetry were not nice as you can imagine). Once the models came out they were set to take over, I remember being blown away from the quality of depth maps alone.

Any modern CPU is far more "beautiful" than this cathedral if you understand the incredible cleverness in how it works and how it is made. And a modern CPU is vastly more useful!

Totally agree.

I was there two weeks ago. The tour guide took us through a route that bypassed the longer lines and through some underground areas—culminating in an entrance that completely blew my mind. I never realized how huge the interior was until I stepped in and saw it firsthand. There are few things in my life that completely took my breath away, this ranks in the top 5 for sure.

Babylon.js is funded (owned?) by Microsoft so anytime you see web 3d on a Microsoft site it's almost certainly going to be using that.

It'd be nice if field of vision was universal and not rate limited, or limited to a narrow band of the en spectrum...

While the decline is not universal, shrinking parishes is itself a reason to sell older and more ornate churches and either build smaller strip mall ones, or merging several parishes into big expo center ones.

Of course it only adds to the decline when the hierarchy builds as if going to Mass is no different from a trip to the mall or seeing a show.

There’s also the hypothesis that if the Church gets prestigious awards for their avant-garde architecture people will finally start liking the Church again.

Agreed. St. Pete’s is so huge and perfectly proportioned that it feels not of this world. To think they fumbled around for generations until Michelangelo took over tells you that he was an every thousand years kind of dude.

And oh my, the Pantheon. My wife saved it for a surprise as the family walked through Rome. We approached it from the back. To me it was just another dirty old building from that angle. She hustled me through the entrance while I was keeping track of the kids so I had no clue. I… I may have wept.