Most active commenters
  • oefrha(6)
  • crazygringo(3)
  • derefr(3)

←back to thread

Students are failing AP tests because the College Board can’t handle HEIC images

(www.theverge.com)
669 points danso | 59 comments | 20 May 20 19:36 UTC | HN request time: 0.867s | source | bottom
1. crazygringo ◴[21 May 20 16:55 UTC] No.23260987[source]▶
I thought iOS was supposed to convert HEIC images to JPEG automatically behind-the-scenes in any file transfer situation where HEIC isn't supported. The article itself even says:

> iPhones convert HEICs to JPEGs automatically when they’re attached to emails in the Mail app

I'm just curious technically why the same didn't happen with the testing portal? If you have a webpage that accepts image uploads, is iOS Safari not smart enough to do the same conversion?

Or was the portal programmed badly or in a non-standard way that that couldn't happen? Or is there a way to do it that the developers ignored?

Just curious for the technical details of who's more to blame here -- Apple not providing enough backwards compatibility, or the testing portal being designed poorly.

Because blaming students for not following obscure instructions to change their phone's overall configuration is not the right path. A national testing portal ought to support the default image format taken by the world's most popular phone, period.

replies(10): >>23261070 #>>23261216 #>>23261225 #>>23261243 #>>23261256 #>>23261511 #>>23261741 #>>23262179 #>>23262549 #>>23264304 #
2. somehnguy ◴[21 May 20 17:01 UTC] No.23261070[source]▶
Probably not. If I Airdrop photos taken on my phone to my Macbook they arrive in HEIC format. I have to use an external tool (iMazing HEIC Converter) to manually convert them for a bunch of places.
replies(2): >>23261284 #>>23261631 #
3. oefrha ◴[21 May 20 17:08 UTC] No.23261216[source]▶
Tried a standard input tag with the proper accept attribute

  <input type="file" accept="image/jpeg,image/png" />
Selected a HEIC file from Photos in Safari, the selected image was automatically converted to JPEG.

Ten bucks says College Board programmer(s) failed to do the most basic and standard filtering.

Edit: Like a sibling comment said, the accept attribute actually isn't necessary; even PNG images (e.g. screenshots) from Photos are converted to JPEG automatically. This is true on both macOS and iOS Safari (latest). To be clear, on macOS you need to select from Photos instead of the filesystem for this to take effect.

In case anyone's interested, source code you can use to test for yourself (a Flask app):

app.py:

  import flask
  
  
  app = flask.Flask(__name__)
  
  
  @app.route("/", methods=["GET", "POST"])
  def index():
      if flask.request.method == "POST":
          image = flask.request.files["image"]
          return f"uploaded {image.filename!r} ({image.mimetype})"
      return flask.render_template("index.html")
templates/index.html:

  <html>
    <head>
      <meta charset="utf-8" />
      <meta name="viewport" content="width=device-width, initial-scale=1" />
    </head>
    <body>
      <form method="post" enctype="multipart/form-data">
        <input name="image" type="file" required />
        <input type="submit" />
      </form>
    </body>
  </html>
replies(5): >>23261508 #>>23261519 #>>23261524 #>>23262200 #>>23262967 #
4. gsnedders ◴[21 May 20 17:09 UTC] No.23261225[source]▶
> If you have a webpage that accepts image uploads, is iOS Safari not smart enough to do the same conversion?

AIUI, Mobile Safari always re-encodes the image. However, if you transfer the image to a device that supports HEIC (e.g., recent macOS) then you'll find out that (desktop) Safari never re-encodes an image when uploading a file.

replies(1): >>23261699 #
5. pdonis ◴[21 May 20 17:09 UTC] No.23261243[source]▶
> A national testing portal ought to support the default image format taken by the world's most popular phone, period.

What if it can't because the format is proprietary and Apple wants to charge for its use? As I understand it, MS and Google have to pay licensing fees to enable support for it in Windows 10 and Android.

replies(3): >>23261337 #>>23261358 #>>23261441 #
6. acdha ◴[21 May 20 17:10 UTC] No.23261256[source]▶
Historically, Safari always converted to JPEG and I don't think that required something like setting the accept attribute (e.g. <input type="file" accept="image/jpeg">). It's not clear whether they're using something like the JavaScript blob APIs to get the file contents without some implicit conversion.
7. crazygringo ◴[21 May 20 17:12 UTC] No.23261284[source]▶
I've experienced that with AirDrop too, but that's because the iPhone knows the Mac can view them.

Also you don't need an external tool -- Preview will convert HEIC to JPEG for you relatively instantly, even in batch. (Though I really do wish the phone would ask first if you want to convert, when Airdropping.)

replies(1): >>23262276 #
8. crazygringo ◴[21 May 20 17:14 UTC] No.23261337[source]▶
Support it by performing client-side conversion to JPEG. The device obviously already has the license.
9. sokoloff ◴[21 May 20 17:15 UTC] No.23261358[source]▶
Have you seen what the College Board charges for AP testing? I think it's a cost of doing business and they can afford to pay a reasonable licensing fee to process the test results. If they can't, increase the test cost by another $5.
10. guptaneil ◴[21 May 20 17:19 UTC] No.23261441[source]▶
You're thinking of HEVC (video encoding, like h.265): https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#P...

HEIC (image encoding) is available without royalties: https://en.wikipedia.org/wiki/High_Efficiency_Image_File_For...

Regardless, in this case, as others noted, iOS would have just converted to jpg if College Board was using a standard image upload form.

replies(3): >>23261546 #>>23263460 #>>23264335 #
11. diebeforei485 ◴[21 May 20 17:23 UTC] No.23261508[source]▶
Is this the case on both Desktop and Mobile Safari?
replies(1): >>23261684 #
12. danso ◴[21 May 20 17:23 UTC] No.23261511[source]▶
Would it be correct for iOS Safari to quietly reencode to jpeg when a HEIC file is uploaded via a webform? How would Safari know that the site's backend didn't want an HEIC file?

I agree the HEIC thing is very confusing (having set up my parents' phones recently), but I can't see how Apple is to blame. For starters, the College Board could've done a much better job emphasizing this step for iOS users in its instructions page. But for me, the overriding factor that places blame squarely on the College Board is this from the article:

> Senior Dave Spencer took a demo test before his Calculus AB exam to make sure he understood the process for uploading photos. He Airdropped an iPhone image of his responses to his Mac and tried to convert it by renaming the HEIC file to PNG. Changing a file’s extension does not guarantee that it will be converted, but Spencer was still able to submit the demo test with no problem.

> Spencer used the same process on the real exam and thought it went through, but he received an email the next day saying the files were corrupted and that he needed to retake the test.

So it seems that students had access to a demo before test day. If I'm reading the story right:

- During the demo, Dave's phone produced a HEIC file

- The demo upload initially failed. So Dave renamed the file extension to PNG.

- Dave uploaded the PNG (in name only), and the demo did not return an error.

- Dave assumed, quite understandably, that the renaming trick would work on test day too.

So the onus in the College Board here: they provided a demo in which the photo upload function appears to have been stubbed (e.g. "if 'PNG|JPG|JPEG' is filename, print "Success"), giving students and teachers false assurance that photo uploads would work on test day.

replies(4): >>23261655 #>>23264606 #>>23265511 #>>23267743 #
13. winter_blue ◴[21 May 20 17:23 UTC] No.23261519[source]▶
Wow, I didn't know browsers even supported automatic image format conversion (this is pretty impressive IMO). Terrible that their programmers didn't specify that tag attribute.
replies(1): >>23261803 #
14. zhengyi13 ◴[21 May 20 17:24 UTC] No.23261524[source]▶
"Be strict in what you send; be liberal in what you accept" seems a relevant maxim here, but I wonder if it's reasonable to assume given the utter ubiquity of PNG and JPEG that of course that's what other people would be using?
replies(2): >>23261710 #>>23263627 #
15. ungzd ◴[21 May 20 17:25 UTC] No.23261546{3}[source]▶
> When containing images and image sequences encoded in a particular format (e.g. HEVC or AVC) its use becomes subject to the licensing of patents on the coding format.
replies(1): >>23261623 #
16. guptaneil ◴[21 May 20 17:29 UTC] No.23261623{4}[source]▶
I'm not sure what you're trying to say with that quote? As I mentioned, HEVC is subject to licensing costs. If you try to wrap your HEVC video content in an HEIC container, you don't suddenly get to skip paying the original licensing costs.
replies(1): >>23264445 #
17. ◴[21 May 20 17:30 UTC] No.23261631[source]▶
18. fortran77 ◴[21 May 20 17:31 UTC] No.23261655[source]▶
Does the HTTP "accept" tag play a role here?

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Ac...

replies(2): >>23261814 #>>23261861 #
19. oefrha ◴[21 May 20 17:33 UTC] No.23261684{3}[source]▶
Yes, see edit.
20. oefrha ◴[21 May 20 17:34 UTC] No.23261699[source]▶
> (desktop) Safari never re-encodes an image when uploading a file.

Not really: https://news.ycombinator.com/item?id=23261216

Images are converted if you select from Photos (instead of the filesystem).

21. oefrha ◴[21 May 20 17:35 UTC] No.23261710{3}[source]▶
Sure, and turns out you don't even need the accept attribute (see edit). They're doing something funny.
22. NikolaNovak ◴[21 May 20 17:37 UTC] No.23261741[source]▶
FWIW, as is the norm, my experience is that it works brilliantly --as long as you stay within apple defaults.

As soon as I stray from golden path - text using anything but Apple MEssages, email using anything but apple mail, post using anything but safari, use/process with anything but Apple Photos - HEIC has been an absolute positive pain in my keister ;-(

Other people may have more luck; note that my work iPhone is an isolated island - my other devices are either Android, or Windows/Linux - which probably explains my troubles to a large degree. Apple's selling point has always been the well-integrated ecosystem so I understand I'm an outlier.

replies(1): >>23263690 #
23. dave5104 ◴[21 May 20 17:40 UTC] No.23261803{3}[source]▶
I don't believe that's what's happening here. The accept attribute just filters what files show up in the file picker. And iOS devices will take an extra step, and on the operating system side convert HEIC files to JPEG.
replies(1): >>23261879 #
24. Aqua_Geek ◴[21 May 20 17:40 UTC] No.23261814{3}[source]▶
I don't follow. How does the "Accept" header factor in here?
replies(1): >>23262050 #
25. detaro ◴[21 May 20 17:43 UTC] No.23261861{3}[source]▶
That's a header the client sends to a server to indicate which formats it would like in the response to it's requests, and not relevant to the problem.
26. arijun ◴[21 May 20 17:44 UTC] No.23261879{4}[source]▶
Thanks for that explanation, I was wondering how it worked.
27. tobylane ◴[21 May 20 17:54 UTC] No.23262050{4}[source]▶
Rule of least astonishment would be that the file is converted unless the accept header lists the heif file. At least by the expectations Apple set up, which don’t match up with standards.
28. scott_s ◴[21 May 20 17:59 UTC] No.23262179[source]▶
snazz posted the actual code downthread: https://news.ycombinator.com/item?id=23261598

My guess is that the JavaScript manually checking the filename's extension is screwing up the automatic conversion.

29. dragontamer ◴[21 May 20 18:00 UTC] No.23262200[source]▶
> Ten bucks says College Board programmer(s) failed to do the most basic and standard filtering.

Why is it a failure of the college board to put "accept="image/jpeg"", instead of iOS which failed to default to the more standardized jpeg format when HEIC was not specified?

HEIC is a newer format which fewer systems support. iOS / Safari should default to JPEG in this case.

replies(9): >>23262291 #>>23262309 #>>23262430 #>>23262568 #>>23262681 #>>23263513 #>>23264517 #>>23266964 #>>23268483 #
30. somehnguy ◴[21 May 20 18:03 UTC] No.23262276{3}[source]▶
I'll try Preview next time instead, thanks! iMazing works fine for the actual functionality but has some sort of bug where it won't re-open if I closed it without force quitting it. Minor but annoying.
replies(1): >>23264092 #
31. bjtitus ◴[21 May 20 18:04 UTC] No.23262291{3}[source]▶
The comment was edited to show that iOS does in fact do this.

But even if it did send HEIC, if you're going to fail to handle "any" format, then you better specify the one you can handle. By the same token, why shouldn't my computer convert every .docx to PDF, or some other "more universal" format?

Instead of specifying the requirement programmatically, they spent their time sending out tweets and writing help docs.

replies(2): >>23262490 #>>23264563 #
32. oefrha ◴[21 May 20 18:05 UTC] No.23262309{3}[source]▶
1. input type=file can be used to upload any file format (or any non-format, for that matter). If you want specific file formats, you make your intention clear.

2. Please try to read the entire comment. iOS does default to JPEG when uploading, the accept attribute isn't needed.

33. salamander014 ◴[21 May 20 18:11 UTC] No.23262430{3}[source]▶
Generally I agree with your point but this isn't your uncle's homemade public photo gallery. These are people's lives on the line. These kids have no say in whether they think they should take these exams. And because of that yes it is absolutely College Board's responsibility to

1. Correctly label their expected stream types. 2. Test all of the most popular devices before having something this broken in production.

What about a kid that's using a cheap android tablet from aliexpress? Are you arguing that the company nobody's ever heard of is at fault for not programming their tablet to use the correct image format? I agree that they should probably be using JPEG but that doesn't mean College Board is off the hook for making it easy for a user to not be able to use their product for something this important.

34. nitrogen ◴[21 May 20 18:14 UTC] No.23262490{4}[source]▶
When was the accept attribute added to the input tag?
replies(1): >>23262807 #
35. wlesieutre ◴[21 May 20 18:18 UTC] No.23262549[source]▶
The article reads to me like the people who had problems sent the images from the phone to their computer, which may not trigger the automatic conversion? One air dropped it to the computer and renamed it PNG, another renamed it JPEG, but they were probably HEIC files with the wrong file extension.
replies(1): >>23262735 #
36. pwthornton ◴[21 May 20 18:19 UTC] No.23262568{3}[source]▶
Not being able to accept common image formats is a failure of the programmers, not Apple, which is looking to use more modern image formats that have better compression and quality.

It really bothers me when programmers make excuses for laziness or shoddy work that results in user's being harmed. Start caring about your users.

The iPhone is the most popular phone model in the United States. This is on the College Board. It takes very little work to make this work properly, which ANY BASIC QA PROCESS WOULD HAVE CAUGHT.

37. scott_s ◴[21 May 20 18:26 UTC] No.23262681{3}[source]▶
Check the actual source code: https://news.ycombinator.com/item?id=23261598

I don't think the failure is with the accept type. I think the failure is with the bespoke JavaScript checking the filenames. That puts the failure on the College Board, not Apple.

38. hundchenkatze ◴[21 May 20 18:29 UTC] No.23262735[source]▶
I believe those students had received noticed to use png/jpeg, and their solution was to change the file extensions. The website accepted them without issue, but these students later received emails telling that their photo was corrupt. The other, unaware students tried to upload the HEIC directly from their phones and the website stopped responding causing their test time to expire.
replies(1): >>23262958 #
39. bchociej ◴[21 May 20 18:32 UTC] No.23262807{5}[source]▶
I was surprised by this, but at least as early as September 2008: https://github.com/whatwg/html/commit/c67bb312ac44737c6c175b...
40. wlesieutre ◴[21 May 20 18:41 UTC] No.23262958{3}[source]▶
Ah gotcha, I think you're right. Sheesh.
41. oefrha ◴[21 May 20 18:42 UTC] No.23262967[source]▶
Turns out I overlooked some details in the article. Some students (after the issue was known) airdropped photos to their computers and attempted to rename .heic to .png/.jpeg to bypass the extension check (who would have guessed extensions don't have to correspond to file types), they went through but were rejected a day later. The article isn't clear on what exactly was happening before, but I suppose students were trying to submit airdropped .heic directly?

In any case, sounds like absolutely horrid QA and communications of what's accepted (and of course, very bad idea not supporting the format in the first place when half your test takers are taking pictures in that format).

(A bit more context on my original comment: I thought they had some working system where test takers could upload photos directly from their phones — e.g., scan a QR code to open a page with a unique ID with an image upload form. Turns out that's too much to ask; their process is "get the image from your camera to your test-taking device whichever way you can think of, not our problem." Not surprisingly that's not a very foolproof process.)

replies(1): >>23264735 #
42. pdonis ◴[21 May 20 19:10 UTC] No.23263460{3}[source]▶
> HEIC (image encoding) is available without royalties

Ah, ok, thanks!

43. mbreese ◴[21 May 20 19:14 UTC] No.23263513{3}[source]▶
When you are accepting photos that will be taken from a phone, why wouldn’t you test to make sure that your site worked with the default settings for a large (if not largest) proportion of your users? If you are the College Board, it seems irresponsible to not have tested this better. Or at least saved the data to attempt to convert later (which it doesn’t seem like they did either).

And if they really needed to control the file formats this much, they should have written this functionality into an app (for iOS and Android) and handled taking the picture themselves.

They really screwed things up for a bunch of kids here.

44. adrianmonk ◴[21 May 20 19:20 UTC] No.23263627{3}[source]▶
If you're going to make an assumption, that one is definitely understandable.

However, taking a step back, you should have enough representative test cases (QA snaps a photo with iPhones, tries to upload) that whether assumptions are reasonable doesn't enter into it because you aren't making them.

45. beezle ◴[21 May 20 19:23 UTC] No.23263690[source]▶
Which is why IOS should default to sending JPG or PNG to/between any non-bespoke Apple product until such a time that HEIC is universally common. That should be transparent to both the end-user as well as the receiving end.

One can moan about the test site not being more robust/careful but it is Apple/Samsung that are using the new, non-standard format and expecting others to accomodate in short order.

46. CamJN ◴[21 May 20 19:51 UTC] No.23264092{4}[source]▶
That's likely an issue with iMazing not properly handling apple's saved ui state mechanism. You could try checking if holding the option key reveals a "quit and close all windows" option in the app's eponymous menu item, or press and hold the Shift key while opening the app to forget previous UI state.

If you don't care for apps to remember their previous windows and ui state, you can disable this mechanism in the settings app: https://support.apple.com/en-ca/HT204005#appresume

47. mokus ◴[21 May 20 20:05 UTC] No.23264304[source]▶
Wow, I have been using iOS (and keeping it up to date) and macOS and Photos.app and uploading images to sites regularly for quite some time and this is the first time I’ve ever even heard of HEIC. Whatever they are doing has been beyond seamless for me.
48. noisem4ker ◴[21 May 20 20:07 UTC] No.23264335{3}[source]▶
>HEIC (image encoding) is available without royalties

As stated by that Wikipedia article, HEIF is the royalty-free container. HEIC means the content is encoded with HEVC, which makes it not free.

49. noisem4ker ◴[21 May 20 20:14 UTC] No.23264445{5}[source]▶
He's trying to remark the fact that HEIC = HEIF(HEVC) => non-free. See my other reply to your first comment. Note that HEVC doesn't necessarily mean video. It can be used to compress still images, as it's the case of Apple devices.
50. derefr ◴[21 May 20 20:18 UTC] No.23264517{3}[source]▶
Because a lack of an accept attribute on the file-input element has a defined meaning in HTML, and that meaning is accept="⋆/⋆".

Which, in turn, means that the form isn't putting any constraint on what's being uploaded at all, and so there's no reason to think that the form is asking for an image in the first place. At that point, it's asking for a file. Any file. And so it has the semantics of taking whatever file you provide it as-is, with no transformation necessary. Just like when a client requests `Cache-Control: no-transform` from a server. It wants the thing on the other side sent to it as-is.

And, it's important to support those semantics (and that particular implicit meaning for leaving out the `accept` attribute), because such "as-is" uploads have many use-cases. The form might be e.g. a computer-forensics or antivirus-signature portal, that wants an evidence file; or the interface to a hex editor or decompiler, that expects a raw binary with arbitrary extension in unknown format; or the SCP component of an old web SSH terminal emulator. All these existed on the web, and used <input type="form">, before the `accept` attribute was introduced. So "not using the `accept` attribute" has to retain semantics that are backward-compatible with that legacy.

51. derefr ◴[21 May 20 20:21 UTC] No.23264563{4}[source]▶
> Instead of specifying the requirement programmatically, they spent their time sending out tweets and writing help docs.

Point of order: separate people, with separate/independent/concurrent job responsibilities. It very likely wasn't the programmers tweeting and writing docs. (Heck, I expect that the College Board doesn't even retain any full-time programmers, only contracts with firms for projects.)

replies(1): >>23267626 #
52. derefr ◴[21 May 20 20:24 UTC] No.23264606[source]▶
> they provided a demo in which the photo upload function appears to have been stubbed (e.g. "if 'PNG|JPG|JPEG' is filename, print "Success"), giving students and teachers false assurance that photo uploads would work on test day.

I don't think it was exactly that it was stubbed out, per se; I would guess it was that the whole "backend" process involved either a human or an async batch process opening the file; and so in both the demo and prod environments, the upload of a file can succeed while the parsing of said file can fail (much later on, after the test is completed, during the grading phase.)

replies(1): >>23265875 #
53. frei ◴[21 May 20 20:31 UTC] No.23264735{3}[source]▶
If they were submitted in time, but then rejected a day later, it seems like those cases can be resolved by some manual work by collegeboard to convert them
54. hinkley ◴[21 May 20 21:18 UTC] No.23265511[source]▶
So they did have a dress rehearsal, it just wasn't high fidelity, and on perhaps the most critical part of the process.
55. danso ◴[21 May 20 21:50 UTC] No.23265875{3}[source]▶
Yeah, I agree with you that the production app likely had an async process, i.e. queuing submissions and processing them first-come-first-serve, which would help explain why Dave didn't get the failure email until well into the next day. Seems safe to assume there was no process for (good) input validation upon upload.

And now that I reread the passage about Dave, I realize it's not necessarily true that Dave tried to upload HEIC, then got an error message, which led him to then rename the file. But he may have actually read the instructions about PNG/JPG being required, then thought he could just rename his file, then uploaded it as his first and only upload to the demo app. And whether the demo app actually did the upload, it apparently didn't do the parsing (I still think that was stubbed out; I can imagine an engineer thinking it'd save a headache if they disabled the parsing module for the demo server)

56. mr_toad ◴[21 May 20 23:40 UTC] No.23266964{3}[source]▶
If I wanted to upload raw photos from my camera to a generic file storage site and they all got magically converted to jpeg because the browser thought that was the best image format, I’d be pretty pissed off. Browsers should follow standards.
57. braythwayt ◴[22 May 20 01:03 UTC] No.23267626{5}[source]▶
Separate people, yes, and we should not blame the people sending out the tweets. But we can hold the organziation they belong to responsible.

It's the same as if Apple shipped me an empty box instead of my iPad Pro. Of course I can't blame the customer support person who takes my call, and I shouldn't be rude to them. But I would still blame Apple the corporation for such a thing, had it happened.

58. MayorMonty ◴[22 May 20 01:20 UTC] No.23267743[source]▶
Yes there was a demo, but it didn't actually submit anything at all. It just mocked through what the exam would look like with timers.

The demo is still up and you can still look at it

59. zodiac ◴[22 May 20 03:52 UTC] No.23268483{3}[source]▶
Imagine trying to upload a heic file to (say) Amazon s3 through their web UI and having it converted to JPEG.