Sourcegraph went dark

Glad you use Sourcegraph! I remember that blog post and thought it was awesome. I am the Sourcegraph CEO, and we haven't changed anything about our public code search at https://sourcegraph.com/search. That's the same product tons of customers use for their internal code, and our public code search is a really important way for us to dogfood, iterate fast, etc.

We just made our own internal codebase private.

Ok, so glad to hear that from you directly! Thank you for all the value you’ve put out there for free!

About the codebase part, I don’t have any need for it so I’m not affected by this, but I wonder if it was possible to keep the current state of the code frozen in a public repository and only make private the future work.

That’s how I did it on Lunar, that’s also how the BetterDisplay dev did, it was a good compromise so as to not steal anything that was already free. But of course we don’t have the same business model or licensing needs so I’m pretty sure I’m missing something.

The way I did it is: - freeze the public code to a new branch “lunar3” - make a private repo LunarPro which works exactly like the previous Lunar repo - but on every commit the private repo syncs the code in an encrypted form to the public repo

That way, permalinks remain valid, everything that was free and accessible before is still available in the future and the branch serves as a “compilable” state without any encrypted files.

But again, I’m just one and you’re many, it might get hard to maintain this structure in a team. And some people might still find things to complain about. I know it was that way for me.

Yes! We took a snapshot of the code and are keeping it at https://github.com/sourcegraph/sourcegraph-public-snapshot.

I see the point about keeping the old repo name so that links work. That's also mentioned in the blog post. That's a good idea. Let me chat with our team about that.

For your approach with syncing an encrypted form of the private code, why did you need/want to sync it back? Why not just keep a public snapshot as of the switchover date?

Oh that’s great to hear then! Good to know there’s a snapshot already.

In my case, I only encrypt the code related to Pro features. There are still plenty of free features and improvements that I add and that I know people will benefit from having them searchable (for example people learned how to use private frameworks like MonitorPanel to change resolutions and presets, how to control Night Shift from swift etc. )

And so I needed to still sync some public source code from the private repo. You might not need that, it could be as easy as moving every dev in the team to using sourcegraph/sourcegraph-private

> I am the Sourcegraph CEO, and we haven't changed anything about our public code search at https://sourcegraph.com/search.

But in this other comment (https://news.ycombinator.com/item?id=41298516), you said you have changed public search in two significant ways:

> We did cull lots of non-GitHub repositories and repositories with less star.

Removing low-star repos (and non-GitHub high-star repos) affects users who are looking for obscure or hard-to-find information that's not found anywhere in "popular" repos. I think most of my searches on GitHub (or via Google) are for things in repos with zero stars.

> If you have repositories you want us to add that are below the star threshold [..]

How would I go about finding which repos to request, if my objective is to search the "long tail" for information? That seems like I would need an automated search engine first, to discover the repos :-)

If I found the repo containing specific, obscure or hard-to-find information I was looking for, what would I gain from writing to SourceGraph asking to add that one repo? By the time I've found the right repo, I've probably found the information I'm going to get from it. Future searches will likely need a different repo, one I don't know about yet. Perhaps that's the nature of long tail searches.

I see this same problem in the search engine space: upstarts need both long-tail sites indexed, and need the ratelimit/compute to actually follow those long tail sites

I wish Presearch <https://nodes.presearch.com/> all the best because that's the world I want to live in, although their current implementation is why I'm just clapping for them and not running nodes:

> This software is currently not open source and you are relying on our assurances that nothing malicious is happening underneath the hood.

Anyway, I mention this because if (e.g.) Sourcegraph supported federated queries, you could actually run a source code indexer to help offset some of the compute, ratelimit, and storage that is presumably jamming up their board members