Most active commenters
  • dataflow(3)
  • haunter(3)

←back to thread

1135 points carride | 51 comments | | HN request time: 1.824s | source | bottom
1. samwhiteUK ◴[] No.32411821[source]
I'm going to put my hand up and say I have absolutely no idea how an ISP works. He runs cables to each house in the area... now where does the other end go?
replies(8): >>32411847 #>>32412399 #>>32412468 #>>32412493 #>>32413311 #>>32414510 #>>32414816 #>>32420345 #
2. the_only_law ◴[] No.32411847[source]
Not sure if it’s what the person in question did, but there’s a whole guide that pops up on here occasionally regarding building a wireless ISP.

https://startyourownisp.com/

replies(2): >>32412013 #>>32412060 #
3. dataflow ◴[] No.32412013[source]
I can't find any section of that guide that talks about peering or whatever ISPs are supposed to do to connect to the broader internet. Do you see any step that explains this?
replies(3): >>32412058 #>>32412178 #>>32419009 #
4. haunter ◴[] No.32412058{3}[source]
It's the 2nd step

https://startyourownisp.com/posts/fiber-provider/

If you just Google then it's usually called leased or dedicated internet

Just some (US) examples

https://www.business.att.com/products/att-dedicated-internet...

https://business.comcast.com/learn/internet/dedicated-intern...

https://www.verizon.com/business/products/internet/internet-...

replies(1): >>32412093 #
5. kevmo314 ◴[] No.32412060[source]
From https://startyourownisp.com/posts/fiber-provider/, doesn't this site basically say connect to another ISP?
replies(1): >>32412223 #
6. dataflow ◴[] No.32412093{4}[source]
So they're leasing ("buying"?) fiber from the same ISPs they're trying to displace and relying on that payment to provide them with continued internet access? This doesn't sound like a real first-class ISP, but something akin to an MVNO where they're at the mercy of the same companies they're competing with. I get the initial sale might seem fine, and the established ISPs might be fine with this as long as the company is small, but why wouldn't these companies shut them off (or raise the prices, etc.) when they grow too big to become dangerous?
replies(6): >>32412183 #>>32412211 #>>32412242 #>>32412322 #>>32413459 #>>32414285 #
7. bombcar ◴[] No.32412178{3}[source]
As a small ISP you don't peer - you just buy transit from a bigger ISP. So the basic steps are:

1. Buy a 1G/1G or 10G/10G whatever link to a building you own.

2. Resell that link in parts to customers.

Or you can get yourself into a POP (point of presence) somewhere that multiple providers are also in, and get transit that way. Depends on where you are and what you can get access to.

replies(3): >>32412515 #>>32412609 #>>32414505 #
8. q3k ◴[] No.32412183{5}[source]
You can lease fibre/lambda/L2 transport to an IXP (and there peer with other local ISPs and get global transit from Tier 1 providers) from many companies that don't even have any residential offering.

Or if (technically/financially/legally) possible, even run your own fibre to a PoP housing an IXP on your own.

Once you're in multiple PoPs and on multiple IXPs and with multiple upstreams/peers you're pretty much independent from the whims of a single ISP.

9. bombcar ◴[] No.32412211{5}[source]
Because it's all business to them, and if they did it overtly they could get sued.

But also because once you're in a single location, you can pretty easily get multiple providers to that location for a Price, so there's really no point. Even small rural towns usually have multiple internet connections from different companies, and if they don't you can pay to run fiber if you really wanted to.

People find it hard to believe, but Comcast et al are actually businesses, not Satan's marketing department; and they happily take money even from "competitors".

replies(1): >>32412658 #
10. moffkalast ◴[] No.32412223{3}[source]
Well there three tiers of ISPs, each one buying service from the one above them. It's ISPs all the way down, and the higher up you go the more expensive the hardware to run it gets.
replies(3): >>32412417 #>>32412438 #>>32412531 #
11. haunter ◴[] No.32412242{5}[source]
>This doesn't sound like a real first-class ISP

I'm not an expert but afaik you can't just be a Tier 1 network member https://en.wikipedia.org/wiki/Tier_1_network

Even Tier 2 very limited https://en.wikipedia.org/wiki/Tier_2_network

In this guide's case yes you will be akin to an MVNO, you won't peer but just buying transit traffic. That's why most of these guides are also focusing on making the network wireless only (easier to build infrastructure)

12. mbreese ◴[] No.32412322{5}[source]
Because there are different business units in the upstream company handling the dedicated access vs consumer sides. The dedicated business side have their own sales goals and if you compete with the consumer side, that’s not a problem for them. I’m sure there are some regulatory/anti competitive measures at play here too, but economically, the two sides of the business will act more or less independently.
13. wil421 ◴[] No.32412399[source]
Depending on how close they are he could run cables (ethernet) or fiber. Single mode fiber can go 10km according to some Ubiquiti spec sheets I found on google. Ubiquiti also sells AirMax products that can do PTP or PTMP over the air, although some will be affected by rain. They could even rent space from a radio/cell tower. There are probably a decent amount of other products out there I am only familiar with Ubiquiti.
replies(1): >>32412590 #
14. Macha ◴[] No.32412417{4}[source]
At the T1 level it's more completely a mesh type setup, but even lower tier ISPs might set up peering agreements to bypass their main higher tier ISP where it makes sense for cost or service quality reasons. Or refuse to to extract more money as in the comcast vs level1 disputes over netflix traffic a while back
replies(1): >>32419565 #
15. ◴[] No.32412438{4}[source]
16. beezlebroxxxxxx ◴[] No.32412468[source]
There is a very good Ars Technica article on how an ISP works. It traces the whole network, from submarine cable through to last mile into a house. It was written in 2016, but I imagine it's still relevant:

https://arstechnica.com/information-technology/2016/05/how-t...

replies(1): >>32414197 #
17. Bloating ◴[] No.32412493[source]
There are wholesalers that provide "dark fiber", then you buy data services from another "wholesaler". When I looked into it, dark fiber was available through some utilities and through a government funded non-profit. Data to light-up the fiber was available through several different data centers that connected to that dark fiber.

You still had to build-out the last mile though, and thats what will get you. You either need private easements, or be a registered telecom utility to use public utility easements. That last mile is $20k +/-, depending on your circumstances. If your semi-rural or less, there's ROI sucks. Hence, many smaller ISPs are wireless.

At least in area, there are already a number of wISPs, 5G is rolling out, Starlink eventually. and lots of gov't funding going to the big players to expand their networks (and drive the start-ups out of business.)

There some other business models out there too that look interesting. Underline in Co Springs, for example. They provide a basic tier of service, in order to qualify as a telecom, install the fiber and then allow multiple competing ISPs to use their network.

IMHO, any utility that has the benefit of government privilege should be required to allow competors to use the infrastructure that the taxpayers funded.

I'm waiting on one of you brilliant folks to defy the laws of physics to create a decentralized, wireless mesh internet.

replies(2): >>32414186 #>>32415350 #
18. nixgeek ◴[] No.32412515{4}[source]
As a small ISP you definitely can peer and many do, you just aren’t going to get settlement-free peering with any of the big eyeball networks like Comcast.

Something like Seattle IX is a good example of where lots of peering sessions could be established (although I haven’t looked at Jared’s ASN in any detail to see where it’s present).

https://www.seattleix.net/home

Any traffic you’re able to offload via peering you wouldn’t be paying an IP transit to haul, so it’s worth seeing if networks like Netflix are on the Route Servers (https://www.ams-ix.net/ams/documentation/ams-ix-route-server...) at any IX nearby your network, seeing if you can negotiate a session over the IX even if they don’t participate in the RS, or seeing if you can do PNI (sling a cable between your networks in a facility you’re both located in).

Edit: Jared’s on Detroit IX. https://www.peeringdb.com/net/20268

replies(2): >>32414736 #>>32414827 #
19. MerelyMortal ◴[] No.32412531{4}[source]
I feel like this could be made into an ISP Tycoon game.
replies(2): >>32413211 #>>32413770 #
20. nixgeek ◴[] No.32412590[source]
You can shoot light over SM at distances up to 200km (several important caveats at this distance) and it’s very usual to see spans of between 50-80km.
replies(1): >>32412785 #
21. twothamendment ◴[] No.32412609{4}[source]
Yes, it can be pretty simple. Back in the day when DSL and comcast were the options and all of the connections were things like UP TO 5 or even 20 Mbps, but speeds were rarely that - I paid for a dedicated 2Mbs up and down ($180/month) with no restrictions on use and started sharing/reselling it to others in my apartment building, not with wireless, but with cat5 out the window, up the gutter, back inside, etc. Across the parking lot another guy was sharing his comcast with another building - but comcast was starting to be so slow they couldn't use it. We merged our empires by stringing some cat 5 across the parking lot, around a pole and to his place. Later we added more nearby buildings, all wired until we had 5 buildings and about 20 "subscribers". Even with 2Mbps, everyone on the network was happier with a guaranteed speed than their flaky "up to" speeds they used to have. Did I run an ISP? I had subscribers, had to maintain a network, had a proxy server to reduce requests out of the network, had to deal with abuse and collect money - so I'd say yes, a small one, but yes.
replies(1): >>32414130 #
22. themoonisachees ◴[] No.32412658{6}[source]
To expand on that:

Comcast would much rather sell a dedicated fiber to a business with capital and guarantees.

Selling to the individual consumer doesn't make a lot of sense business-wise, because of the deployment costs and continued support costs.

Comcast is also abusing their status as oligopoly to gouge costumers financially and qos-wise, but if they're selling to a business that buys large quantities and has staff who's job it is to handle network problems, they actually like that (right up until that business threatens to compete with them in areas where said oligopoly is in place, of course)

23. wil421 ◴[] No.32412785{3}[source]
Looking further you can get a UFiber OLT Terminal for $1,799 that can run 20km and support 1024 clients or 128 ONU CPEs per port.

How much would a 200km switch run?

replies(2): >>32413814 #>>32416721 #
24. haunter ◴[] No.32413211{5}[source]
It actually does exist lol! Made by Cisco as an e-learning tool

Gameplay https://www.youtube.com/watch?v=Foa34qoRzjs

https://web.archive.org/web/20150317144142/https://cisco.edu...

replies(1): >>32414599 #
25. southerntofu ◴[] No.32413311[source]
As the other commenters have pointed out, a possibility is simply to "resell" transit from other providers. However, on the Internet all peering networks are somewhat equal and it's entirely possible to extend the "other end" over time to establish dedicated peering with other networks, so that for example traffic from your network to Youtube doesn't have to go through (paid-for) 3rd parties.

There's good chances there are Internet eXchange Points around where you live where for a small maintenance fee anyone can come and place their router and cables to interconnect with others.

So the likely steps are:

1) Find a transit provider, that will serve your trafic to any other network, and where to connect with this provider 2) (Optional) If you don't have the necessary infrastructure, find another provider to get from your last-mile network to your transit provider 3) (Optional) Find other networks to peer with so that you can significantly reduce your transit bill and provide better routes (therefore better service)

Some non-profit ISPs take the problem from the other side, and build a core network without necessarily owning any last-mile infrastructure, which is leased from other operators (opérateurs de collecte) with whom they interconnect at some datacenter/IXP. The most famous example of that in France is FDN.fr which has been operating since early 90s. That approach is more cost-effective in high-density area where the local infrastructure is already quite good, and construction jobs to lay new cables is very costly, but will still set you back 10-30€/month/line.

26. wins32767 ◴[] No.32413459{5}[source]
He's not trying to displace the majors. In rural areas, owning and maintaining a bunch of fiber to service less than a thousand customers isn't a business Comcast really wants to be in unless they get paid a ton for it.
27. cptcobalt ◴[] No.32413770{5}[source]
Dang, as someone that enjoys Tycoons, Tactics, and Management Simulators, this really sounds fun.
28. Nikbul ◴[] No.32413814{4}[source]
At this distance you would want a good repeater at about half point instead. Don't forget that data has to travel back and other side might not have such a strong signal
29. smeyer ◴[] No.32414130{5}[source]
Out of curiosity did you do things above board from a business standpoint (taxes etc.) or was this more of a blackmarket setup?
replies(1): >>32421495 #
30. wyager ◴[] No.32414186[source]
Last mile subsidies are super weird. I was looking at a property in montana in the middle of nowhere that had no electricity nearby, but had gigabit fiber. I called the ISP and it was cheaper to get phone+Gb than just Gb due to subsidy rules.

Basically everyone out there (including me) is on starlink now. Turns out the subsidies were not only inefficient, but pretty pointless.

replies(2): >>32414257 #>>32415804 #
31. digdugdirk ◴[] No.32414197[source]
Thank you, that was a great link for us uninitiated folks.

Also another great plug for ArsTechnica (even though the main article is them as well, and I'm sure most of this audience is well aware of them) and the excellent technical writing they do.

32. aftbit ◴[] No.32414257{3}[source]
Why would you be using Starlink if you have gigabit fiber available? Or was it still quite expensive to install even with the subsidy?
replies(1): >>32426466 #
33. dboreham ◴[] No.32414285{5}[source]
You're misunderstanding this market. There's a wholesale market, which he is buying from. There's a retail market which he is selling into. Some providers service both the wholesale and retail markets, but typically with different divisions, people, tech, resources. It's like saying that if you build a gas station and buy your gas from Exxon then that's bad because Exxon also operates gas stations. It's not like an MVNO where all you're doing is sending the customer a bill, and provisioning API requests to Verizon.
replies(1): >>32414383 #
34. dataflow ◴[] No.32414383{6}[source]
> You're misunderstanding this market. There's a wholesale market, which he is buying from. There's a retail market which he is selling into. Some providers service both the wholesale and retail markets, but typically with different divisions, people, tech, resources.

The difference in divisions/people/tech/resources doesn't explain anything for me. They're both the same company with the same CEO, whether it's one business unit or a dozen. It's not like the executives are oblivious to how much money each unit is making and whether another unit could make more in place of it. If you're the CEO and see you could charge twice as much by doing retail instead of wholesale then you'd obviously try to do that.

Rather, the explanations I'm getting from the other comments seems to be that (a) regulators require some kind of reasonable wholesale to exist to third parties, (b) the big ISPs aren't planning to serve those markets anyway, so they're not missing out on any income by taking money from the last-mile ISPs. And as long as those last-mile ISPs don't try to compete for the same customers then they're fine.

replies(1): >>32416660 #
35. spmurrayzzz ◴[] No.32414505{4}[source]
You definitely can (and should) peer as a small ISP, even if you are buying transit from other providers. This is especially true if you're running an MPLS headend as you'll still have choke points at L2 circuits in your own network. Owning your own peering can be a great way to offload traffic to other circuits that share destinations, most commonly traffic destined for VOD/streaming CDNs.

(N.B. — This is what has worked well for the WISP I cofounded, but YMMV depending on headend infra).

36. andix ◴[] No.32414510[source]
I think you more or less just buy connections from bigger ISPs, so for example you get a 100 Gbps connection to one location and distribute it to your end users from there.

Most of the equipment you can buy, you can even get a lot of the needed things as a service. You just need to organize all those hardware and software things, and get the economic and legal part right too. And in the end it needs to tie together in a way, that your earnings are bigger then your expenses.

I think it’s not so different to opening a car repair shop for example. Just more nerdy.

37. RockRobotRock ◴[] No.32414599{6}[source]
Oh god, it seems to essentially be packet tracer under the hood. It's a great tool, but I HATED using it in school.
replies(1): >>32418473 #
38. 2Gkashmiri ◴[] No.32414736{5}[source]
Wait. The poster above said in point 1 to buy a line,1G, 10g depending on your upstream seller. Why do you need peering then?

If I have 1Gbps line for example and 10 users each are using equal amount 100% of time, it shouldn't matter they send the data to Alaska or Russia or Australia ? Or does it?

Do you buy the pipe and the data itself also?

replies(1): >>32419021 #
39. notRobot ◴[] No.32414816[source]
There's "Start Your Own ISP":

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

40. toast0 ◴[] No.32414827{5}[source]
> Any traffic you’re able to offload via peering you wouldn’t be paying an IP transit to haul

When you're small enough, the difference in price between transit and what it takes to get you to an IX is likely to be pretty small. But, you probably want to be at an IX sooner or later anyway (easier to get multiple transit offers at an IX than on the side of the road), so might as well peer while you're there.

41. thedougd ◴[] No.32415350[source]
https://www.segra.com/

These guys have dark fiber right in front of my neighborhood. They service cell sites for Dish Network near me as well. It's interesting to look through their services. For example, you can get fiber service with layer 2, where you're responsible for adding your IP stack over top of it. Or you can buy at layer 3, where Segra is already running a stack, and establish mesh connectivity. So if a fiber is cut, you'll get another working path. Build your network over the top.

Pretty interesting to understand what's available.

42. daniel-cussen ◴[] No.32415804{3}[source]
They helped a bit, for a while. Gigabit fiber is less maintenance than electrical power, and it's easier to roll-your-own electrical power than it is to get a Gb connection like how would you do that before Starlink, buy an insane amount of radio spectrum? I heard one HN user who did exactly that in Brazil, got a 20-meter tower to connect by radio to the internet some distance away, and it was a very solid high bandwidth connection. But still much harder than a generator and solar panels, or a tiny little hydropower generator on a stream (a great option in places like Southern Chile, not a joke by any means). Or wind.
43. fragmede ◴[] No.32416660{7}[source]
> The difference in divisions/people/tech/resources doesn't explain anything for me. They're both the same company with the same CEO, whether it's one business unit or a dozen.

Then you've not worked in large B2B companies before. Eg Apple pays Google money and Google pays money to Apple, any perceived public rivalry goes out the window as far as business between the two is concerned.

If you're the CEO of Comcast, you've never even heard of this small time ISP, you have far bigger things to spend your time on, and the "upstream" business unit of Comcast really doesn't care what you're doing, so long as your money's green. It's all business. See also: Netflix using AWS despite Amazon having a streaming video service of their own.

44. iptrans ◴[] No.32416721{4}[source]
The switch does not care what kind of optics you use. You can use a $50 switch is you like.

The 200 km optics, however, cost about a grand each.

45. jethro_tell ◴[] No.32418473{7}[source]
gns3 perhaps? I haven't had that setup for a while but I loved it. I had my whole small ISP in it at some point to work as a test/lab env for testing things out. It's a trick to get going but was kinda fun. I took a copy of that when I left and every now and then I fire it up and mess around with my old dsl/dialup ISP from back in the day.
46. mananaysiempre ◴[] No.32419009{3}[source]
If you want to try your hand in a playground for the software (routing) parts, DN42[1] is essentially one.

[1] https://dn42.us/, https://dn42.eu/, https://dn42.dev/

47. icedchai ◴[] No.32419021{6}[source]
You don't "need" peering but it offloads your upstream (transit) links, which are generally much more expensive. In the old days, I worked for couple ISPs and we typically had 3 or 4 upstreams (generally UUNet, Sprint, MCI...) This was back when a T1 was still considered fast.
48. carlhjerpe ◴[] No.32419565{5}[source]
Who is T1, T2 and T3? I really don't see it. Seems like that tiering has aged about as well as the OSI model.
49. dbelson ◴[] No.32420345[source]
Also very much worth reading on this topic: Tubes by Andrew Blum (https://www.andrewblum.net/tubes-2)
50. twothamendment ◴[] No.32421495{6}[source]
Taxes? I was a college student, so I didn't make enough to owe any taxes, but it was mostly cash. I wouldn't have done it if I had to hop through the business hoops.
51. wyager ◴[] No.32426466{4}[source]
I didn't get that property sadly.