Uploaded image for project: 'OASIS Open Data Protocol (OData) TC'
  1. OASIS Open Data Protocol (OData) TC
  2. ODATA-342

Can we relax the requirement of int64 numbers being represented as strings in the JSON Format for integer numbers that don't lose precision in ECMAScript when represented as numbers?

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: V4.0_WD01
    • Fix Version/s: V4.0_WD01
    • Component/s: JSON Format
    • Labels:
      None
    • Environment:

      [Applied]

    • Proposal:
      Hide

      Change the default serialization of number (Int64 and Decimal particularly) as numbers in JSON.
      Add support for the 'IEEE754Compatible' format parameter to the application/json format and require that the producer MUST serialize Int64 and Decimal numbers as strings if this format parameter has been specified.

      Show
      Change the default serialization of number (Int64 and Decimal particularly) as numbers in JSON. Add support for the 'IEEE754Compatible' format parameter to the application/json format and require that the producer MUST serialize Int64 and Decimal numbers as strings if this format parameter has been specified.
    • Resolution:
      Show
      Accepted: https://www.oasis-open.org/committees/download.php/49055/odata-meeting-35_on-20130502-minutes.html#odata-342

      Description

      The current JSON Format specification requires all Int64 numbers to be represented as strings because of the limitation of numbers in JavaScript were these numbers get stored as 64-bit binary format IEEE values and therefore lose precision past 15 digits. However numbers up to that point could still be represented as numbers.

        Attachments

          Activity

          Hide
          sdrees Stefan Drees (Inactive) added a comment -

          This proposal helps in cutting down (uncompressed wire size) by skipping the delimiters of strings, right?
          But the processing on the receiving side is then forced to implement if-and-else logic or must always decorate the so-or-so-value with a stringify() method, correct?
          The proposal looks more elegant, than conservatively wrapping everything in a typesafe transport container, but are there real cool use cases for it, that make it a good move :-?)

          Show
          sdrees Stefan Drees (Inactive) added a comment - This proposal helps in cutting down (uncompressed wire size) by skipping the delimiters of strings, right? But the processing on the receiving side is then forced to implement if-and-else logic or must always decorate the so-or-so-value with a stringify() method, correct? The proposal looks more elegant, than conservatively wrapping everything in a typesafe transport container, but are there real cool use cases for it, that make it a good move :-?)
          Hide
          hubert.heijkers Hubert Heijkers (Inactive) added a comment -

          Stefan, it will indeed cut down the wire size but that's not what I was after. I'd like to be able to use Int64s in my model if there is ever the chance that an Int32 might not be enough but at the same time I'd like to stick to having numbers in my JSON response as long as that doesn't impact the precision.

          Show
          hubert.heijkers Hubert Heijkers (Inactive) added a comment - Stefan, it will indeed cut down the wire size but that's not what I was after. I'd like to be able to use Int64s in my model if there is ever the chance that an Int32 might not be enough but at the same time I'd like to stick to having numbers in my JSON response as long as that doesn't impact the precision.
          Hide
          sdrees Stefan Drees (Inactive) added a comment -

          That's good news for me. The size argument was a honeypot I do not understand scenarios anymore, where JSON is sent uncompressed over the wire and size is seen as a problem.

          When my tools receive JSON over the network, i usally treat all scalar entries as strings to carefully start peeling them off and finally digesting them.
          So there is allways a transcoding layer between my model in javascript/php whatever and the transport containers content.
          Is this direct transfer of typed integers an important part of processing the wild openness inside open data content? (This is a real question, sorry for eventually using funny words).

          Show
          sdrees Stefan Drees (Inactive) added a comment - That's good news for me. The size argument was a honeypot I do not understand scenarios anymore, where JSON is sent uncompressed over the wire and size is seen as a problem. When my tools receive JSON over the network, i usally treat all scalar entries as strings to carefully start peeling them off and finally digesting them. So there is allways a transcoding layer between my model in javascript/php whatever and the transport containers content. Is this direct transfer of typed integers an important part of processing the wild openness inside open data content? (This is a real question, sorry for eventually using funny words).
          Hide
          hubert.heijkers Hubert Heijkers (Inactive) added a comment -

          Updated proposal to use numeric representation by default for all numbers and add a 'IEEE754Compatible' format parameter for the application/json format allowing the client to signal the server that it needs to use string representation for Int64 and Decimal numbers.

          Reasoning: The JSON format itself does not limit the size of the numbers represented by JSON. It is the JavaScript language that is restricted to numbers according to the IEEE-754. It is only the JavaScript clients that presumably have this limitation while all consumers written using other languages do not. It is therefore up to these clients to indicate to the server that it should format Int64 and Decimal numbers to be compatible to IEEE754. The server, if so requested, MUST then serialize all Int64 and Decimal numbers as strings. However a JavaScript consumer is allowed to not specify this parameter and receive all numbers as numbers and take the gamble that none of these are larger then the maximum value supported as specified in IEEE-754.

          Show
          hubert.heijkers Hubert Heijkers (Inactive) added a comment - Updated proposal to use numeric representation by default for all numbers and add a 'IEEE754Compatible' format parameter for the application/json format allowing the client to signal the server that it needs to use string representation for Int64 and Decimal numbers. Reasoning: The JSON format itself does not limit the size of the numbers represented by JSON. It is the JavaScript language that is restricted to numbers according to the IEEE-754. It is only the JavaScript clients that presumably have this limitation while all consumers written using other languages do not. It is therefore up to these clients to indicate to the server that it should format Int64 and Decimal numbers to be compatible to IEEE754. The server, if so requested, MUST then serialize all Int64 and Decimal numbers as strings. However a JavaScript consumer is allowed to not specify this parameter and receive all numbers as numbers and take the gamble that none of these are larger then the maximum value supported as specified in IEEE-754.
          Hide
          sdrees Stefan Drees (Inactive) added a comment -

          Just as an update: Following the formation of the IETF JSON WG there is some discussion on what to change and what to keep while augmenting the JSON RFC to a STD (to break the cyclic reference that seems to have started between the current JSON RFC pointing to ECMAScript, and some edition of ECMAScript pointing to the JSON RFC as normative reference).

          One point is enforcing the ECMAScript limitiations of IEEE754 floating point or not onto the format. See eg. http://www.ietf.org/mail-archive/web/json/current/msg00299.html for the as of now latest mail in this thread.

          As long as the JSON rep of numbers remains unbounded, this issue (ODATA-342) may well remain closed.

          Show
          sdrees Stefan Drees (Inactive) added a comment - Just as an update: Following the formation of the IETF JSON WG there is some discussion on what to change and what to keep while augmenting the JSON RFC to a STD (to break the cyclic reference that seems to have started between the current JSON RFC pointing to ECMAScript, and some edition of ECMAScript pointing to the JSON RFC as normative reference). One point is enforcing the ECMAScript limitiations of IEEE754 floating point or not onto the format. See eg. http://www.ietf.org/mail-archive/web/json/current/msg00299.html for the as of now latest mail in this thread. As long as the JSON rep of numbers remains unbounded, this issue ( ODATA-342 ) may well remain closed.

            People

            • Assignee:
              Unassigned
              Reporter:
              hubert.heijkers Hubert Heijkers (Inactive)
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: