Fixing JSON

JSON is just javascript, and it leaves out everything else. That's its entire reason for existing, and it caught on because that's all that 99.44% of anyone needed.

Timestamps you can add today, without changing the protocol; just put them in your data if you need them. So I'm not sure what he's even proposing there.

Schemas: OK, he doesn't like JSON Schema's anyOf. Fair enough. There's no real proposal here for how to fix it, so not much to say here.

Replacing commas with whitespace sounds to me like replacing a very minor irritant with a constant full-body rash. Stuff like his example of "IDs": [ 116 943 234 38793 ] would lead to far more confusion and errors than the occasional stray trailing comma.

So I guess I pretty much vote no on this one thanks for asking

He is proposing to add a timestamp type to the grammar. This would have the advantage that there would be one canonical way to have timestamps in your JSON. It would also mean that the parser would already validate them for you and you would not have to do that yourself every time.

I definitely see value in that.

Yeah, this is precisely the kind of stuff people said would cause JSON to "lose" to XML -- how could you ever build anything without schemas and types and bunches of metadata to ensure you were using it correctly?

And this debate is not new; it's been around a long time, and I wrote about it at length literally ten years ago: http://www.b-list.org/weblog/2006/dec/21/i-cant-believe-its-...

Both vivid and accurate.

In order to combat trailing commas I normally place the comma on the following line, e.g.

    { "time": "3 minutes past four"
    , "age": 229
    , "sex": "monoecious species"
    , "appearance": "Tree-like.  It's a tree."
    }

    uint8_t *data    // Yada
          , *buffer  // Ya
          ;

    var javascript
      , variable = 6
      , declarations = [ "This is taking too long", "Yep" ]
      , mix = [ "Flour", "Sugar", "Water" ]
      , types = { 'old'     : (d) => { return d < new Date }
                , 'new'     : (d) => { return d > new Date }
                , 'borrowed': (o) => { return false }
                , 'blue'    : (c) => { return true }
                }
      , regularly = new Date().toISOString()
      ;
    

There is already a canonical way to have timestamps: encode them as defined in RFC 3339.

Done.

The proposals add nothing to JSON and don't make sense. I mean, screwing up with the language just to add a very specific data type that is used in only a specific corner case that isn't even supported in the use case that's provided as an example? Nonsense.

> They’re inessen­tial to the gram­mar, just there for JavaScript com­pat­i­bil­i­ty.

The entire article goes on to suggest extremely terrible solutions to the problems he is pointing out.

In order:

Commas: If comma's are an annoyance, is that to mean there is no annoyance with XML markup? You can forget a closing tag just as easily, probably more easily. Just get a JavaScript or JSON linter. Fixed. If you are really annoyed by commas, move to YAML or another format.

Datetimes: Just store your dates in a standardized timezone and format, how about [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601). You can also use a [JSON-Schema](http://json-schema.org/) if you really want.

Schemas: There's lots of potentials here. However, again, if you're going to change JSON so much, why not just pick another data format that has better schema support.

Ah okay, so you're that guy whose code I have to reformat in every new job.

If you are not suggesting that timestamp values be wrapped in quotes, then wouldn't you have to worry about every existing parser out there tripping on them?

{"item": "lawn chair", "order-timestamp": "2016-08-21T00:58:47Z"}

Today json is not really tied to javascript that much, I deal with it all the time in python that doesn't include or interact with javascript. JSON is just dictionaries and lists (and strings and numbers and nulls) this is pretty easy to express in dynamically typed languages and not that hard in statically typed languages if you give up on a bit of (static) typing. I don't see any reason why it should be limited to a subset of javascript, there are some good reasons for not changing it all though.

Javascript compatibility is the least important part of json, lots of people use json with python or ruby or rust or java or c for things that will never interact with javascript, and anyways you don't want people to parse json with eval or confusing json with things that include string concatenation or function calls adding non javascript syntax helps make that clear.

There is a good reason to not change JSON(backwards compatibility, updating parsers, confusing formats, ect.), since you're not planning on parsing javascript with eval there is no good reason why any changes made in JSON 2.0 have to conform to a subset of javascript.

Why does Javascript compatibility matter in 2016? The original idea for JSON might have been to parse untrusted data using js eval() but I'd hope that nobody is doing that anymore.

However, I'd agree with your conclusions. The minor irritations outlined here are less bad than breaking javascript compatibility.

That's not canonical at all. It's not even a de facto use of timestamps in JSON; most specifications I see call for ISO 8601.

> Done.

Well no, not done, because you then have to wrap it in a string, which essentially hides it from the JSON parser altogether and moves responsibility for parsing it out of the JSON parser into your application. As far as the JSON parser is concerned, it could be any old string, not a timestamp.

> a very specific data type that is used in only a specific corner case

I don't see how you can call timestamps a corner case – the article says "Among the RESTful APIs that I can think of, exactly zero don’t have timestamps." – and my experience is the same. Pretty much every API I've worked with has used timestamps in some form or another. They aren't a corner case at all – aside links, they are probably the most common data type without direct support in JSON.

That's not the same thing at all. The commas in JSON are equivalent to the whitespace between attributes in XML. A missing closing tag in XML would be equivalent to forgetting to close an object literal or array in JSON. So no, there's no equivalent XML annoyance to what he's asking for, XML is already fixed in the manner he suggests.

Nonsense. ISO 8601 is an ISO standard that defines representations of dates and times.

Date and time representations aren't timestamps.

RFC 3339 is based on ISO 8601 but is designed specifically to handle timestamps by including provisions that tackle interoperability problems that ISO 8601 does not handle.

> Well no, not done, because you then have to wrap it in a string

Nonsense.

JSON is just the languge that is used as a superset for other domain-specific languages. How the other language defined (and parsed) isn't handled by the JSON parser, obviously.

Considering Tim Bray's example, If a domain-specific language specifies that a "Capture time" string is followed by a string storing an RFC 3339 timestamp, that's precisely what the parser for the domain-specific language is expected to parse. The document remains valid JSON, and the domain-specific language remains valid.

> I don't see how you can call timestamps a corner case

Because it is. It isn't a primitive data type, nor is it required to implement generic data structures. Hell, some programming languages don't even support any standard date and time container or aggregate data type. Timestamps are a very specific data type whose application is limited to very specific corner cases that are already handled quite well by other means.

So in other words, what I'm saying isn't "nonsense" at all, and they aren't fundamentally different things as you claim – you just prefer a standard that you think is better?

In any case, you're getting away from the point there. The point was not which standard was better, the point was that RFC 3339 is not canonical.

> > Well no, not done, because you then have to wrap it in a string

> Nonsense.

It's not nonsense in the slightest. You can't put an RFC 3339 timestamp into JSON without wrapping it in a string, at which point it is no longer part of JSON. All JSON sees is a string, not a timestamp.

> The document remains valid JSON, and the domain-specific language remains valid.

I never said that it wasn't valid JSON, my point was that as far as a JSON parser is concerned, it's a string, not a timestamp, so parsing a timestamp has to be handled by your application not the JSON parser.

> It isn't a primitive data type

When the subject of discussion is whether or not it should become a primitive data type, that's circular logic.

> nor is it required to implement generic data structures.

No, but it is required to implement a vast number of JSON-based APIs.

> Timestamps are a very specific data type whose application is limited to very specific corner cases

The "very specific corner cases" being every single RESTful API the author can think of? This is an incredibly common use case, I don't see how you can argue that it's a corner case when it's practically ubiquitous.

Which to be honest is what matters. However I would imagine that a lot of the problem grokking it is due to not being used to it.

Of course don't commit reformats where it diverges from the codebase.

Not 1999. 2004. JSON wasn't on anybody's radar in 1999.

>JSON is incredibly easy to read and write by hand, in comparison with XML

Maybe for small shallow objects (and for those XML is also quite readable). Once size or complexity get a little higher, you're done.

  , "age": 229
  , "appearance": "Tree-like.  It's a tree."
  , "sex": "monoecious species"
  { "time": "3 minutes past four"
  }

  {
    "age": 229,
    "appearance": "Tree-like.  It's a tree.",
    "sex": "monoecious species",
    "time": "3 minutes past four",
  }

    {
    , "age": 229
    , "appearance": "Tree-like.  It's a tree."
    , "sex": "monoecious species"
    , "time": "3 minutes past four"
    }

And what is better than JSON, when it gets complex?

http://www.ioccc.org/2015/endoh1/prog.c

It looks easier for me; I'm far more likely to notice a missing comma.

But I'm sure you must have a valid point, if you would elucidate?

Luckily, Douglas Crockford has an explanation and an almost prophetic solution[1] for your use-case: I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.

Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.

1. https://plus.google.com/+DouglasCrockfordEsq/posts/RK8qyGVaG...

And to me, with any decent editor with folding, it is easy to read. And very importantly, allows comments. I have never understood all the XML-hate out there...

Commas were meant to be used in code that you actually type. Using them as separators in a format that is meant to be produced and consumed by computers is clear, simple, and wrong.

Me neither. I think it's truly irrational.

It's nonsense, because a timestamp isn't a date representation. This was already demonstrated. I don't understand why you decided to ignore this.

> The point was not which standard was better, the point was that RFC 3339 is not canonical.

No, the point is that the ISO standard you've quoted doesn't define timestamps. Hence, the example you provided to refute what I've said was nonsense.

> You can't put an RFC 3339 timestamp into JSON without wrapping it in a string

...and you can't encode a date without representing the year as a number, the month as another number, the day as anohter number, etc etc etc.

You, somehow, miss the point that a primitive data type is not required to represent timestamps.

In fact, you can represent timestamps in JSON by defining an aggregate type.

Timestamps as primitive data types doesn't make any sense if it's possible to use the types that are already available to represent it.

> my point was that as far as a JSON parser is concerned, it's a string

Somehow, you don't understand that JSON is only the superset language, and that JSON-based domain-specific languages represent specific subsets of JSON obtained by imposing other parsing rules.

> When the subject of discussion is whether or not it should become a primitive data type, that's circular logic.

You somehow already forgot that this particular data type is already representable by using another primitive data type.

> No, but it is required to implement a vast number of JSON-based APIs.

No, it's not.

Wrap it in a string. Done.

If that's too hard to do, just specify an aggregate data type.

Is it that hard to understand?

> The "very specific corner cases" being every single RESTful API the author can think of?

Somehow, no RESTful API was barred from being implemented in JSON because of this corner case.

I suspect you are, somehow, confusing "convenience" with "necessity".

edit: I was wrong: http://www.lysator.liu.se/c/ANSI-C-grammar-y.html#initialize...

https://news.ycombinator.com/item?id=12368022

  { a, b
  , c
  ...
  , z
  };