Credit to Claude Code for showing me how to search the HN database through Google BigQuery and for writing the HTML for the leaderboard.
Feature request 2: Em-dash regular-dash ratio.
The em-dash used to be a slightly snooty way for Mac users to announce themselves. Sad that the polarity of perception has reversed.
I’ve been typing em-dashes since I got my first MacBook in 2006 and I’m not going to let the AI companies take my beautiful punctuation away from me.
But I'm not on a Mac right now so I don't know how to even make a real one at the moment other than that LaTeX method.
Whether this is interesting or not, well…
I guess on Windows it's Alt+0,1,5,1 on a numpad. Or you copy+paste from Character Map.
J/k:)
Is the amount of em dashes counted or the comments that have at least one em dash inside them?
You know, I am asking for...science(?).
I also wanted to point out that these could be Kantonese/Mandarin/Japanese/SouthEast Asian users that use their local keymapping software because a lot of them use the idiom symbols (e.g. the dot character, too) when they switch to the English keymaps.
Check out how laptops usually look like over there, a lot of manufacturers build that right into the firmware.
On the other hand, I don’t think o3 was ever a common choice among people copying from LLMs, so en dashes remain infrequent regardless.
A lot of symbols can be accessed with Alt Gr compared to Windows
≤ ≥ ≠ × — – “ ” ’ ° … ¹ ² ³ ™ • ♣ ♢ ♡ ♠
If you work in languages other than English but have a standard English keyboard layout, a compose key is handy for typing accents and non-English letters/ligatures too.
What's a “regular dash”?
Hyphen-minus (which isn't even a dash at all)? En-dash? Figure dash?
- you can’t make a ?.. or !.. with it
- the spacing between the dots is awful in a lot of fonts
- it is hideous in monospace
- typing ellipsis properly is a very easy gesture (triple-tap the dot key), arguably easier than Alt Gr + . (depending on the keyboard)
(Like now)
It’s become a weird kind of witch hunting regarding blogs, too, and I have a 20+ year old site that renders all of its content using Markdown extensions that do the same (and that also convert dual hyphens to em dashes—something I’ve been typing for about as long).
But an ellipsis is separate from and doesn't mmerge with sentence-terminal punctuation, whether its a period or somethig else (when it replaces words at the end of a sentence, the terminal punctuation follows the ellipsis, when at the beginning of a sentence that follows another, the ellipsis follows the punctuation.) The constructs you say can't be formed with it aren't needed.
[0] The AltGrDead variant just means that the regular dead keys on the US Intl are flipped; e.g. ' is now no longer dead per default: I have to hit altgr+' to make it dead (i.e. an acute accent (´)).
I remap a key to the right of Space to Compose, and add various custom sequences. Before long, I was completely comfortably and casually typing dashes and curly quotes and more, and in fact it takes conscious effort for me to limit myself to ASCII when typing prose. (Writing code, writing *, /, -, ' and " is easy. But writing prose, I genuinely will write ×, ÷ if it feels the right one in that place, −, ‘/’ and “/”.)
On one previous laptop keyboard I mapped Menu, on my current one RAlt is more suitable.
When on Windows, I use WinCompose. On Linux, I used to just use it bare, which had advantages and disadvantages—apps implement a Compose key inconsistently, some messing things up related to includes and some handling overlapping sequences differently. More recently I wanted to be able to type Telugu and installed fcitx5 which is no longer mostly broken under Wayland like it was last time I tried, so now fcitx5 is handling the Compose sequences across the entire system, and working more consistently. Also I can use Ctrl+Alt+Shift+U and get a popup where I can search Unicode by code or description. Now if only that pesky popup would handle Shift+Space and Ctrl+Backspace itself rather than letting them fall through to the parent…
In my ~/.config/sway/config:
input * {
xkb_options "caps:backspace,compose:ralt"
}
For my ~/.XCompose, I start with the defaults and one good set of additions, https://raw.githubusercontent.com/kragen/xcompose/master/dot...:
include "/usr/share/X11/locale/en_US.UTF-8/Compose"
include "/home/chris/.XCompose-kragen"
When all’s said and done:
• Compose - - - gets me — EM DASH (stock)
• Compose - - . gets me – EN DASH (stock)
• Compose - - = gets me − MINUS SIGN (custom)
• Compose - - w gets me ⸺ TWO EM DASH (custom; w for wide)
• Compose - - W gets me ⸻ THREE EM DASH (custom; W for Wider)
The last two I use occasionally, the other three I use very frequently. I went through a phase of using HYPHEN and SOFT HYPHEN, now I seldom use them.
I also like to write &c. (italic where supported) for et cetera.
For quotation marks, I also use custom mappings:
<Multi_key> <semicolon> <semicolon> : "‘" U2018 # LEFT SINGLE QUOTATION MARK
<Multi_key> <apostrophe> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK
<Multi_key> <colon> <colon> : "“" U201c # LEFT DOUBLE QUOTATION MARK
<Multi_key> <quotedbl> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK
—⁂—
(Oh yeah, that one’s <Multi_key> <h> <r> : "—⁂—".)
Now, I have one question I’d like answered. Overlapping sequences. If you have -> → and <- ← you’re fine, but when you add <-> ↔, I can’t find any way of using the <- sequence any more. Before fcitx5, some apps would ignore one or the other (in ways difficult to explain which I think involved the fact that some definitions came from includes), and some would let you terminate the sequence early and match the shorter one (e.g. Compose < - Enter). Is there some proper solution I’ve missed?
I have plans for an article on my keyboard arrangements, including sharing a full .XCompose, but I’m going to finish my next major revision to my website first. Because then I’ll be able to draw things instead of just writing.
—⁂—
On mobile, I think I use FUTO keyboard at present, which lets me access most of these things, but not elegantly. I want to make my own keyboard layout that lets me access the good stuff more easily, but I haven’t got to it yet.
Also: anyone want to join me in advocating for completion dictionaries and libraries to replace their ' apostrophes with ’, or at least to support both approaches equally? I’m fed up with not having this stuff, Vim is the only place where it was straightforward to get it about right, and mobile is just a mess.
https://www.gally.net/miscellaneous/hn-em-dash-user-leaderbo...
This second version was vibe-coded with Codex CLI. I also tried Gemini CLI, but it didn’t work very well. The SQL scripts I ran at BigQuery were by Claude.
I am not a programmer or web designer, so I will leave these pages as they are, warts and all. It was a fun project, though. I never would have attempted something like this pre-vibe-coding.
What's needed is a writing comparison before/after 2022 for these users. If there's a sudden 200% increase in the use of em-dashes from one month to the next, it's a very strong indicator that the user started LLMing their posts.
Probably some autocomplete related software release.
Meanwhile there are a lot of languages and cultures. Somewhere all those characters were useful for something. My Atari had a very fun utility that gave you a compose-key that could combine just about everything on the keyboard to access all those weird characters of the extended ascii table. <compose>+ao would give you "a" with a ring on top (å), <compose>+ae gave the danish welded together character that I can't even type any more on windows.
The idea came from some unix thing I believe.
Compose ` e produces è
" a produces ä
v s produces š
v S produces Š
a e produces æ
C = produces €
l - produces £
- > produces →
( 1 ) produces ①
^ 1 produces ¹
_ 1 produces ₁
1 8 produces ⅛
- - - produces —
- - . produces –
. . produces …
. - produces ·
| - produces †
| = produces ‡
" < produces “
x x produces ×
m u produces µ
> = produces ≥
I have also configured Shift+Compose to send the code 'dead_greek' using ~/.Xmodmap:
keycode 135 = Multi_key dead_greek Multi_key Multi_key
https://daringfireball.net/2018/02/ios_messages_smart_punctu...
But British usage – instead – uses spaces, so an en-dash or an em-dash is acceptable.
SELECT
EXTRACT(YEAR FROM timestamp) AS year,
SUM(CASE WHEN text LIKE '%—%' THEN 1 ELSE 0 END) AS withDash,
COUNT(*) AS total,
SUM(CASE WHEN text LIKE '%—%' THEN 1 ELSE 0 END) / COUNT(*) AS fraction
FROM `bigquery-public-data.hacker_news.full`
WHERE type = 'comment'
GROUP BY year
ORDER BY year;
year with— total frac
2006 0 12 0.000
2007 13 70858 0.000
2008 461 247922 0.001
2009 1497 491034 0.003
2010 3835 842438 0.005
2011 4719 1044913 0.005
2012 5648 1246782 0.005
2013 7881 1665185 0.005
2014 8400 1510814 0.006
2015 9967 1642912 0.006
2016 12081 2093612 0.006
2017 14530 2361709 0.006
2018 19246 2384086 0.008
2019 23662 2755063 0.009
2020 27316 3243173 0.008
2021 32863 3765921 0.009
2022 34657 4062159 0.009
2023 36611 4221940 0.009
2024 32543 3339861 0.010
2025 30608 2231919 0.014
Querying for the users who use "—" most as a proportion of all their comments:
SELECT
`by`,
SUM(CASE WHEN text LIKE '%—%' THEN 1 ELSE 0 END) / COUNT(*) AS fraction,
COUNT(*) AS total,
MIN(timestamp) AS minTime,
MAX(timestamp) AS maxTime
FROM `bigquery-public-data.hacker_news.full`
WHERE
type = 'comment' AND
timestamp < '2022-11-30'
GROUP BY `by`
HAVING COUNT(*) > 100
ORDER BY fraction DESC
LIMIT 250;
[0] https://console.cloud.google.com/marketplace/product/y-combi...
ChatGPT always uses them without spaces—like this.
Idk, working in the AI space, I've started to write very succinctly and straight to the point, maybe as a counterweight to the often overly flattering, verbose forms of prose that the LLMs employ. I pay close attention to every word and try to never write more than is necessary.
You can't write CO₂ or m², use a fraction like ½, claim © or mention a price in Euros or Pounds Sterling.
You can't even write major American place names (San José, Oʻahu).
- *Less formatting*: I don't start every bullet point with bold text
- *Varying structure*: I don't start each list item with a one or two word summary, followed by a longer description of what I mean
- *Mobile differences*: I actually only use em dashes on my phone, since it's easy to type on Android, but I refrain from their use on desktop.
But what if need more words for complicated idea?
Short message easy if just 'orange man good' or 'orange man bad' but what if want to explain reason also? Dumb down? What if discussion too dumb already?
Of course this doesn't mean they're using ChatGPT either, they could've switched devices or started using them because they felt like it.
# user before_chatgpt after_chatgpt
1 fao_ 9/1777 (1 %) 36/225 (16 %)
2 tlogan 1/962 (0 %) 59/399 (15 %)
3 whynotminot 1/250 (0 %) 36/356 (10 %)
4 unclebucknasty 13/2566 (1 %) 38/378 (10 %)
5 iLemming 0/793 (0 %) 61/628 (10 %)
6 nostrebored 10/1045 (1 %) 32/331 (10 %)
7 freeone3000 0/2128 (0 %) 74/791 (9 %)
8 pdabbadabba 6/932 (1 %) 20/225 (9 %)
9 thebooktocome 4/632 (1 %) 18/208 (9 %)
10 tnecniv 0/671 (0 %) 34/446 (8 %)
11 dkersten 39/5092 (1 %) 24/318 (8 %)
12 stared 8/1565 (1 %) 29/392 (7 %)
13 ETH_start 3/385 (1 %) 75/1029 (7 %)
14 tcbawo 2/792 (0 %) 15/218 (7 %)
15 jbm 2/406 (0 %) 22/350 (6 %)
WITH by_user AS (
SELECT
`by` AS user,
COUNTIF(text LIKE '%—%') AS match_count,
COUNT(*) AS total_count,
(timestamp >= '2022-11-30') AS after_chatgpt
FROM `bigquery-public-data.hacker_news.full`
WHERE type = 'comment'
GROUP BY user, after_chatgpt
),
combined AS (
SELECT
user,
MAX(IF(NOT after_chatgpt, match_count, 0)) AS match_before_chatgpt,
MAX(IF(NOT after_chatgpt, total_count, 0)) AS total_before_chatgpt,
MAX(IF(after_chatgpt, match_count, 0)) AS match_after_chatgpt,
MAX(IF(after_chatgpt, total_count, 0)) AS total_after_chatgpt,
FROM by_user
GROUP BY user
HAVING total_before_chatgpt >= 200 AND total_after_chatgpt >= 200
),
with_fractions AS (
SELECT
*,
SAFE_DIVIDE(match_before_chatgpt, total_before_chatgpt) AS fraction_before_chatgpt,
SAFE_DIVIDE(match_after_chatgpt, total_after_chatgpt) AS fraction_after_chatgpt
FROM combined
)
SELECT
user,
FORMAT('%d/%d (%.0f %%)', match_before_chatgpt, total_before_chatgpt, ROUND(fraction_before_chatgpt*100)) AS before_chatgpt,
FORMAT('%d/%d (%.0f %%)', match_after_chatgpt, total_after_chatgpt, ROUND(fraction_after_chatgpt*100)) AS after_chatgpt
FROM with_fractions
WHERE fraction_before_chatgpt < 0.01
ORDER BY fraction_after_chatgpt DESC
LIMIT 15
[2] https://console.cloud.google.com/marketplace/product/y-combi...
* If it was not widely used before where/how did (chat)GPT picked it up?
* If it was widely used, then it shouldn't be a topic at all. But, there seems to be informal agreement that it wasn’t widely used.
* Or, could GPT have inferred that even though it's not widely used, it's the better way to go (to use it). Which then makes one wonder about the whole probability of next token idea. Maybe this line of thinking falls too short of what might be really going on internally.
* If it had picked up something that is widely used but in the wrong way, it should make us pause (again) about the future feedback loops these LLMs, which aren't going away, are already creating. Not just in terms of grammar and spelling but also in terms of way of thinking and seeing the world.
People are more casual on the web. It's sort of like how people can often tell when it's me in IM without my name because I properly use periods while that's unusual in that medium. ChatGPT is so correct it feels robotic.
Very effective way to summarize reports, recommendations, or analysis. IME well-received and appreciated by those consuming complex info for the first time.
Still love the style, though one does need to soft-shoe it so as to not scream "this is LLM copypasta!"
For me writing is a way to capture a stream of consciousness so I don’t really see the advantage of using an LLM.
When I see some trivial mediocrity I simply stop reading. It’s just not interesting.
@dang - can we add it to the HN guidelines that we should not or should call out AI when we see it? On one hand people might get defensive and the threads get out of hand. On the other hand, we don’t want AI slop.
More generally any measurable feature of writing that underwent a significant change in frequency around that time would be interesting to look at. Looking at frequencies across the entire post dataset would suggest likely candidates, which individual people could then be tested against. There would be lots of confounding factors and red herrings though -- like the word "ChatGPT" itself!
At least I tended to use em-dash always with spaces surrounding it — like so. I know the anglospace-convention is to use it without spaces, but I just don't like that visually. At least one way to tell me apart from typical LLM-generated text.
On my desktop, the two hyphens remain literal. But on iOS, it turns into an em dash I think. Although it seems like I get the smart quotes more often than the em dash
X11 is likely walking a tree of .XCompose entries with each keypress. Once it gets to '<' and '-' it finds '←' and does not continue to consider your next '>'. So, you need to provide a way to walk a different path.
This works for me.
<Multi_key> <less> <period> <greater> : "↔"
In general we must consider the entirety of .XCompose when choosing new compose key bindings. Maybe there is some utility to help with that. For me, I removed 98% of the default Compose file entries which makes manual checking feasible.
Some only let Compose < - (←) work, stopping and preventing Compose < - > (↔) from working. Others, if I remember correctly, let Compose < - Enter work to get ←.
Once an Input Method is involved, it can handle the Compose key, and that’s what fcitx5 is doing for me now, so that everything’s behaving the same… but that “same” is not what I reckon it should be.
EDIT: There's a second ranking linked at the top that shows this.
It does need some normalization for people who post very few comments, but it feels more fair this way.
curl -s "https://hn.algolia.com/api/v1/search?tags=comment,author_sjs382&hitsPerPage=10000" \
| jq -r '.hits[].comment_text' \
| grep -o "—" \
| wc -lhttps://console.cloud.google.com/bigquery?p=bigquery-public-...
Click on the `+` (white over blue background) in the tab bar at the top that says "SQL query" on popup, and type the following (I use the GoogleSQL pipe syntax (https://cloud.google.com/bigquery/docs/reference/standard-sq... / https://news.ycombinator.com/item?id=41347188) below, but you can also use standard SQL if you prefer):
FROM `bigquery-public-data.hacker_news.full`
|> WHERE type = 'comment' AND timestamp < '2022-11-30'
|> AGGREGATE COUNT(*) AS total, COUNTIF(text LIKE '%—%') AS with_em GROUP BY `by`
|> EXTEND with_em / total AS fraction_with_em
|> ORDER BY fraction_with_em DESC
|> WHERE total > 100 AND fraction_with_em > 0.1
Edit: As you also asked about timestamps:
FROM `bigquery-public-data.hacker_news.full`
|> WHERE type = 'comment' AND timestamp < '2022-11-30'
|> EXTEND text LIKE '%—%' AS has_em
|> AGGREGATE
COUNT(*) AS total,
COUNTIF(has_em) AS with_em,
MIN(timestamp) AS first_comment_timestamp,
MIN(IF(has_em, timestamp, NULL)) AS first_em_timestamp,
TIMESTAMP_SECONDS(CAST(AVG(time) AS INT64)) AS avg_comment_timestamp,
TIMESTAMP_SECONDS(CAST(AVG(IF(has_em, time, NULL)) AS INT64)) AS avg_em_timestamp,
GROUP BY `by`
|> EXTEND with_em / total AS fraction_with_em
|> ORDER BY fraction_with_em DESC
|> WHERE total > 100 AND fraction_with_em > 0.1
https://news.ycombinator.com/item?id=45074990
Also see the relevant XKCD:
I'm probably not alone here in being a longtime Linux user who started using a Macbook after the Apple Silicon transition, late 2022.
On Windows and Linux, inserting an em-dash is a laborious alt-code process. But on MacOS with an Apple keyboard, the `option` key acts like a tertiary shift, so an `–` em dash is just <option><->.
I didn't start using em-dashes (typing -- is just second nature to me and I'm still on Linux most of the time) when I got a Macbook, but I imagine some people in my shoes did.
And the continuations… Honestly? They'll never <|im_end|>.
// • Chronic option-dash and option-shift-dash user, option-[ or option-shift-[ as well as option-] and option-shift-] — not to mention option-8 and option-; …
On Linux, you can set up a Compose key, after which an em-dash is compose, three hyphens (Macintosh: shift-option-hyphen), and an en-dash is compose, two hyphens, period (Macintosh: option-hyphen). Also, a left (resp. right) single (resp. double) quote is compose, less-than (resp. greater-than), typewriter single (resp. double) quote. That’s how I enter them.
You can also (alternatively or at the same time) set up a “Level 3 shift” aka “Alternate Characters Key” aka AltGr, which gets you quotes with one of the English International layouts or quotes as well as dashes with an English Macintosh layout.
To get an em dash on an iPhone, long hold the hyphen—it’s the third (longest) option.
(Edit: typo. Using iPhone after all.)
It's like the memory of the jokes about the wacky phrases of gpt2 or the ew at the yellow hue saturated ai generated images.
In the future this sign will be gone and our pattern recognition will adapt and our memory of this will also mostly be gone. Hello to future tech archeologists. The emdash isn't a meme, it will never survive and replicate but it's fun while it's lasting and I'm enjoying it in the meantime!
I mourn also because in the future we may have few or no obvious signs of LLM use. These are the golden years.
So it was no surprise to me that ChatGPT used em dashes (I assume a US bias to its training data) and I immediately told it to stop using them (along with Title Case titles). (Source: professional writer for 30 years.)
https://news.ycombinator.com/item?id=33950747 (Dec 2022)
https://news.ycombinator.com/item?id=24189762 (Aug 2020)
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
Whether to add it to the formal guidelines (https://news.ycombinator.com/newsguidelines.html) is a different question, of course. I'm reluctant to do that, partly because it arguably follows from what's there, partly because this is still a pretty fuzzy area that is rapidly evolving, and partly because the community is already handling this issue pretty well.
One important thing to know: plenty of things not allowed in HN don't show up explicitly in the site guidelines. They are in no way a comprehensive list, and never will be!
It is novel to see which users were producing text with an em dash before the rise of AI slop. User 'derefr' was 5 years ahead of everyone.
I do wonder if there was some journalism CMS involved, or if these users figured out how to produce the character on their own volition.
EDIT: 'lynndotpy' has an explanation in this thread.
Someone recently created some long list of my reddit comments using them as a farcical claim of having used ChatGPT to author many dozens of 2010 comments.
Writing and publishing style guides like Hart's Rules (Oxford Style Guide) & Chicago manual of style have the 'em' dash use as a parenthetical closed or "no spaces" dash.
In British use – Hart's Rules – writers will choose the 'en' dash with spaces as a parenthetical dash, where US writers/publishers choose the closed 'em' dash for the same thing.
Imo, there is a conflation of 'en' dash and 'em' dash going around due to the ease of smart-dashes auto-correction turning (--) into 'em' dash with the 'en' dash and non-auto-correct 'em' dash needing a key-combo.
Common everyday typing online, I think people will simply use what is convenient and "good enough" -- a single hyphen dash as an 'en' dash or 2-hyphen dashes that may or may not auto correct into an 'em' dash. I prefer mixing spaces with a 2-hyphen dash 'em' dash, but I'm not a published writer so I enjoy doing wild things like that
https://github.com/andrewaylett/aylett.co.uk/blob/d338d35a3d...
Also, for what it's worth, UK style guides recommend endash + spaces (but many write emdash + spaces instead), and so do some other languages (eg. German). There are more countries than just America and Russia!
(Also provides access to the Greek alphabet.)
The people who grew up with the em dash are the younger HTML generation of 30 years ago where — was at least a reasonably convenient character entity even if they were using computers with the various 8-bit character sets that did not contain it.
He has a cult following who believe him to be a victim of a lizard jew conspiracy or something and who are quite displeased by people mocking him for continually putting out obvious AI slop. And clearly the people who are accusing him of slopping it up must be doing so themselves... plus then there are people making fun of these people.
Of course, emdash use was not actually a meaningful factor in any of the determination of chatgpt use in court... but it's a signal that even fairly unsophisticated people notice and often presume underlies claims of AI (ab)use.
TLDR: morons consider me their enemy
Makes me wonder if kerning is done correctly, if the em-dash would look like there were spaces before and after when there were not.
Honestly, even if it doesn't make it pretty I find stringing together a few hyphens does the trick in less formal settings.
I guess you weren't there. We did em-dashes on typewriters. We just turned the platen knob down one click, typed _, and turned it back.
I'm a self-taught typist, with all the quirks that comes with (can type programming stuff very accurately at a 100+ WPM; can type normal stuff at a high WPM as well, but the error rate goes up).
*Hyphen (-)* = word-joiner
*En dash (–)* = “to/between”
*Em dash (—)* = pause, punch, drama
1903 edition of The Wizard of Oz — https://archive.org/details/newwizardofoz00baum/page/2/mode/...
A page from Life magazine, 1894 — https://archive.org/details/sim_life_1894-08-23_24_608/page/...
The Illustrated London News, 1843 — https://archive.org/details/illustrated-london-news-v002-184...
The em dash pretty much just joins the two glyphs together. It's supposed to look that way.
It's sad and not at all unsurprising that people who even half-assedly care about typography get this effort attributed to AI use.
In the post-competence workplace we're collectively building now with all the LLM coding tools, I already see people intuitively attributing non-trivial code to AI. It's a projection of own inability, more or less.
At some point any sentence with proper capitalization will be the marker of AI.
Moreover, your home is not representative of the millions of typewriters in businesses around the world.