←back to thread

OpenAI charges by the minute, so speed up your audio

(george.mand.is)
666 points georgemandis | 1 comments | 25 Jun 25 13:17 UTC | HN request time: 0.219s | source
Show context
pbbakkum ◴[25 Jun 25 21:55 UTC] No.44382153[source]▶
This is great, thank you for sharing. I work on these APIs at OpenAI, it's a surprise to me that it still works reasonably well at 2/3x speed, but on the other hand for phone channels we get 8khz audio that is upsampled to 24khz for the model and it still works well. Note there's probably a measurable decrease in transcription accuracy that worsens as you deviate from 1x speed. Also we really need to support bigger/longer file uploads :)
replies(2): >>44382203 #>>44384158 #
1. georgemandis ◴[26 Jun 25 04:21 UTC] No.44384158[source]▶
I kind of want to take a more proper poke at this but focus more one summarization accuracy over word-for-word accuracy, though I see the value in both.

I'm actually curious, if I run transcriptions back-to-back-to-back on the exact same audio, how much variance should I expect?

Maybe I'll try three approaches:

- A straight diff comparison (I know a lot of people are calling for this, but I really think this is less useful than it sounds)

- A "variance within the modal" test running it multiple times against the same audio, tracking how much it varies between runs

- An LLM analysis assessing if the primary points from a talk were captured and summarized at 1x, 2x, 3x, 4x runs (I think this is far more useful and interesting)