←back to thread

Show HN: Get any domain's brand data via API

(brand.dev)
32 points ICodeSometimes | 4 comments | 15 Nov 24 23:19 UTC | HN request time: 0.803s | source
Show context
someuser54541 ◴[15 Nov 24 23:45 UTC] No.42152624[source]▶
What's the tech stack? I assume you're doing quite a bit of scraping.
replies(1): >>42153669 #
ICodeSometimes ◴[16 Nov 24 02:41 UTC] No.42153669[source]▶
Yep scraping alot of sites, what specifically would you like to know?
replies(1): >>42159662 #
1. someuser54541 ◴[16 Nov 24 21:50 UTC] No.42159662[source]▶
What's the tech stack? How did you get around issues with your scrapers IP getting blocked?
replies(1): >>42159849 #
2. ICodeSometimes ◴[16 Nov 24 22:14 UTC] No.42159849[source]▶
High quality proxies, depending on the site i'll use anything from data center proxy up to residential :)

Almost all requests goes through a proxy to be honest.

You pay per GB transferred basically which works ok for me at the moment.

replies(1): >>42159903 #
3. someuser54541 ◴[16 Nov 24 22:22 UTC] No.42159903[source]▶
Does that wipe out your margins? Most proxies I'm aware of are relatively expensive and if you're charging per API request than there's not much margin to work with.

What was the scraper written in? Python? Node? Go?

replies(1): >>42160147 #
4. ICodeSometimes ◴[16 Nov 24 22:50 UTC] No.42160147{3}[source]▶
It indeed does.

My reasoning for creating this are the alternatives are WAY TOO EXPENSIVE (5-10x per api call).

Since it's a one man show, i don't need much margin and i'm happy to just keep it alive since my friends and i use it for other projects that have more significant margin.