←back to thread

PDF to Text, a challenging problem

(www.marginalia.nu)
357 points ingve | 1 comments | 13 May 25 15:01 UTC | HN request time: 0.203s | source
Show context
90s_dev ◴[13 May 25 18:18 UTC] No.43975996[source]▶
Have any of you ever thought to yourself, this is new and interesting, and then vaguely remembered that you spent months or years becoming an expert at it earlier in life but entirely forgot it? And in fact large chunks of the very interesting things you've done just completely flew out of your mind long ago, to the point where you feel absolutely new at life, like you've accomplished relatively nothing, until something like this jars you out of that forgetfulness?

I definitely vaguely remember doing some incredibly cool things with PDFs and OCR about 6 or 7 years ago. Some project comes to mind... google tells me it was "tesseract" and that sounds familiar.

replies(10): >>43976023 #>>43976086 #>>43976796 #>>43977155 #>>43977580 #>>43977605 #>>43978171 #>>43979324 #>>43980118 #>>43981115 #
hallman76 ◴[14 May 25 02:32 UTC] No.43980118[source]▶
We will never get back the collective man-decades of time that has been burned by this format. When will the madness stop?
replies(3): >>43980258 #>>43980692 #>>43984513 #
1. GuB-42 ◴[14 May 25 13:48 UTC] No.43984513[source]▶
PDF is effectively digital paper, and it works really well for this. When I made PDFs 20 years ago, I knew they will always look the same on every device, including on paper, and they did, and they still do. In addition, a document is a single file, reasonably compact, looks good on any resolution, and is generally searchable. Even if not ideal, it can also support scans of paper documents in a way that can be sent to a printer on the other side of the planet and you will get the same result as if you had used a copier.

Data extraction is hard, but that's not what it is designed for, it is for people to read, like paper documents.

Far from being "mad", it is remarkably stable. It has some crazy features, and it is not designed for data extraction (but doesn't actively prevent it!). But look at the alternative. Word documents? Html? Svg? One of the zillion XML-based document formats? Markdown? Is any one of these suitable for writing, say, a scientific paper (with maths, tables, graphics...) in a way that is readable by a human on a computer or in print and will still be in decades and that is easier to process by a machine than a PDF?