Uploaded image for project: 'OASIS Open Data Protocol (OData) TC'
  1. OASIS Open Data Protocol (OData) TC
  2. ODATA-1468

Unified treatment of paths in aggregate, groupby and transformnested

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: V4.0_CS02
    • Fix Version/s: V4.0_CSD04
    • Component/s: Data Aggregation
    • Labels:
      None
    • Environment:

      Applied

    • Proposal:
      Hide

      Introduce the following definition in [OData-Aggr, section 2.1]:

      data aggregation path consists of one or more segments separated by a forward slash. Segments are names of declared or dynamic structural or navigation properties other than stream properties, or type-cast segments consisting of the (optionally qualified) name of a structured type that is derived from the type identified by the preceding path segment to reach properties declared by the derived type.

      Rephrase [OData-Aggr, section 3.1] (unchanged parts in green):

      The aggregate transformation takes a comma-separated list of one or more aggregate expressions as parameters and returns an output set with a single instance containing one dynamic property per aggregate expression representing the aggregated value of the input set.

      In the following, P is a data aggregation path with single- or collection-valued segments. P is evaluated relative to the input set of the aggregate transformation, this must be well-defined for each instance in the input set.

      The allowed types of aggregate expressions are listed below. To compute the value of the dynamic property for a given aggregate expression, the aggregate transformation first determines a collection U of instances or primitive values, based on the input set of the aggregate transformation and a path P that occurs in the aggregate expression, as explained below. Each type of aggregate expression defines a function f(U) that then yields the dynamic property value.

      In types 1 and 2, the aggregate expression must be followed by the keyword with and an aggregation method g. In types 1, 2 and 3, it must be followed by the keyword as and an alias, which is then the name of the dynamic property. In types 1, 2 and 4, the aggregate expression can be followed by a from expression.

      Types of aggregate expressions:

      1. A path P whose last segment is an aggregatable property.
        f(U) = g(U).
      2. An aggregatable expression W built from arithmetic operations [OData-URL, section 5.1.1.2], unbound primitive function calls, numeric or duration values, and data aggregation paths with single-valued segments whose last segment is primitive.
        f(U) = g(W evaluated relative to each member of U). In this type, P is absent.
      3. A path P/$count with optional prefix P/ where the last segment of P is collection-valued.
        f(U) = cardinality(U). ($count behaves like a property having the value 1 followed by with sum.)
      4. A path P/c consisting of an optional prefix P/ where the last segment of P has a structured type and a custom aggregate c defined on the collection addressed by P.
        f(U) = c(U). The name of the dynamic property is the name of the custom aggregate.

      Determination of U:

      If P is absent, let U = input set.

      If P is present, it must be well-defined for all instances in the input set. Otherwise, let Q be the portion of P up to and including the last navigation property, if any, and let R be the remainder, if any, of P that contains no navigation properties, such that P equals the concatenated path Q/R. The aggregate transformation considers each entity reached via the path Q exactly once. To this end

      • if Q is non-empty, let E be the set of distinct entities reached via Q starting from the input set
      • if Q is empty, let E be the input set (which may contain multiple instances with the same value).

      Then, if R is empty, let U = E, otherwise let U be the collection containing all instances or primitive values (possibly with repititions) addressed via R starting from E.

      Any aggregate expression that specifies an aggregation method MUST define an alias for the resulting aggregated value. The resulting instance contains one dynamic property per parameter representing the aggregated value. The properties of P do not appear in the result (see Example 64, with P = Q = Sales and R = empty).

      Rephrase [OData-Aggr, section 3.10.1]:

      In its simplest form the first parameter of groupby specifies the grouping properties, a comma-separated list of one or more data aggregation paths with single-valued segments that is enclosed in parentheses.

      ...

      If the property path leads to a single-valued navigation property, this means grouping by the entity-id of the related entities. Other navigation properties specified in grouping properties paths are expanded by default.

      Rephrase step 4:

      In each set resulting from the previous step, each entity and complex propertytype, including nested complex and navigation properties, are augmented if necessary to include all primitive property values of the corresponding projection. In case of grouping by entity-id of a related entity, they must include at least the key properties of the related entity (see example 61).

      Rephrase [OData-Aggr, section 3.13]:

      Their first parameter is a data aggregation path with single- or collection-valued non-primitive segments where only the last segment may have an entity type, but it can instead have a complex type. This path is optionally followed by a type-cast segment to expandselect only related entities of that derived type or one of its sub-types. The sub-path starting with the last segment that is not a type-cast is called the final sub-path in this definition.
      This is a generalizationan extension of the definition in [OData-URL, section 5.1.3] in that it does not require a navigation property or entity-valued instance annotation.

      Remove mentions of "entity-valued instance annotation" from sections 3.13.1 and 3.13.2.

      Rephrase example 61:

      Example 61: Grouping by navigation property Customer is treated as groupby((Customer/$ref))

      GET ~/Sales?$apply=groupby((Customer))
      

      may result in

      {
        "@odata.context": "$metadata#Sales(Customer)",
        "value": [
          { "Customer": { "ID": "C1" } },
          { "Customer": { "ID": "C2" } },
          { "Customer": { "ID": "C3" } }
        ]
      }
      

      depending on which properties the server includes when expanding Customer without $select or $ref.

      Add new example 63:

      If both subtypes have a Rating property but their common base type does not,

      GET ~/Products?$apply=groupby((SalesModel.FoodProduct/Rating,
          SalesModel.NonFoodProduct/Rating))
      

      results in

      {
        "@odata.context": "$metadata#Products(
          SalesModel.FoodProduct/Rating,
          SalesModel.NonFoodProduct/Rating)",
        "value": [
          {"@odata.type": "#SalesModel.FoodProduct", "Rating": 5},
          {"@odata.type": "#SalesModel.FoodProduct", "Rating": null},
          {"@odata.type": "#SalesModel.NonFoodProduct", "Rating": "average"},
          {"@odata.type": "#SalesModel.NonFoodProduct", "Rating": null}
        ]
      }
      

      Note that the @odata.type context information is needed to distinguish the two groups with "Rating": null.

      Remove with sum after $count in example 67.

      Merge ABNF pull request #75 and vocabularies pull request #186.

      Show
      Introduce the following definition in [OData-Aggr, section 2.1] : A  data aggregation path  consists of one or more segments separated by a forward slash. Segments are names of declared or dynamic structural or navigation properties other than stream properties, or type-cast segments consisting of the (optionally qualified) name of a structured type that is derived from the type identified by the preceding path segment to reach properties declared by the derived type. Rephrase [OData-Aggr, section 3.1] (unchanged parts in green ): The aggregate transformation takes a comma-separated list of one or more aggregate expressions as parameters and returns an output set with a single instance   containing one dynamic property per aggregate expression   representing the aggregated value of the input set. In the following, P is a  data aggregation path  with single- or collection-valued segments. P is evaluated relative to the input set of the aggregate transformation, this must be well-defined for each instance in the input set. The allowed types of aggregate expressions are listed below. To compute the value of the dynamic property for a given aggregate expression, the aggregate transformation first determines a collection U of instances or primitive values, based on the input set of the aggregate transformation and a path P that occurs in the aggregate expression, as explained below. Each type of aggregate expression defines a function f(U) that then yields the dynamic property value. In types 1 and 2, the aggregate expression must be followed by the keyword  with  and an aggregation method g. In types 1, 2 and 3, it must be followed by the keyword  as  and an alias, which is then the name of the dynamic property. In types 1, 2 and 4, the aggregate expression can be followed by a from expression. Types of aggregate expressions: A path P whose last segment is an aggregatable property. f(U) = g(U). An aggregatable expression W built from arithmetic operations  [OData-URL, section 5.1.1.2] , unbound primitive function calls, numeric or duration values, and data aggregation paths with single-valued segments whose last segment is primitive. f(U) = g(W evaluated relative to each member of U). In this type, P is absent. A path P/ $count  with optional prefix P/ where the last segment of P is collection-valued. f(U) = cardinality(U). ( $count  behaves like a property having the value 1 followed by with sum .) A path P/c consisting of an optional prefix P/ where the last segment of P has a structured type and a custom aggregate c defined on the collection addressed by P. f(U) = c(U). The name of the dynamic property is the name of the custom aggregate. Determination of U: If P is absent, let U = input set. If P is present, it must be well-defined for all instances in the input set.   Otherwise, let Q be the portion of P up to and including the last navigation property, if any, and let R be the remainder, if any, of P that contains no navigation properties, such that P equals the concatenated path Q/R. The aggregate transformation considers each entity reached via the path Q exactly once. To this end if Q is non-empty, let E be the set of distinct entities reached via Q starting from the input set if Q is empty, let E be the input set (which may contain multiple instances with the same value). Then, if R is empty, let U = E , otherwise let U be the collection containing all instances or primitive values (possibly with repititions) addressed via R starting from E. Any aggregate expression that specifies an aggregation method MUST define an alias for the resulting aggregated value. The resulting instance contains one dynamic property per parameter representing the aggregated value. The properties of P do not appear in the result (see Example 64, with P = Q = Sales and R = empty). Rephrase [OData-Aggr, section 3.10.1] : In its simplest form the first parameter of  groupby  specifies the  grouping properties , a comma-separated list of one or more  data aggregation paths  with single-valued segments that is enclosed in parentheses. ... If the  property  path leads to a single-valued navigation property, this means grouping by the entity-id of the related entities.  Other  navigation properties specified in grouping properties  paths are expanded by default. Rephrase step 4: In each set resulting from the previous step, each entity and complex property type , including nested complex and navigation properties, are augmented if necessary to include all primitive property values of the corresponding projection. In case of grouping by entity-id of a related entity, they must include at least the key properties of the related entity (see example 61). Rephrase [OData-Aggr, section 3.13] : Their first parameter is a  data aggregation path  with single- or collection-valued non-primitive segments where only the last segment may have an entity type, but it can instead have a complex type. This path is optionally followed by a type-cast segment to  expand select  only related entities of that derived type or one of its sub-types. The sub-path starting with the last segment that is not a type-cast is called the final sub-path in this definition. This is  a generalization an extension  of the definition in [OData-URL, section 5.1.3] in that it does not require a navigation property  or entity-valued instance annotation . Remove mentions of "entity-valued instance annotation" from sections 3.13.1 and 3.13.2. Rephrase example 61: Example 61: Grouping by navigation property Customer is treated as groupby((Customer/$ref)) GET ~/Sales?$apply=groupby((Customer)) may result in { "@odata.context" : "$metadata#Sales(Customer)" , "value" : [ { "Customer" : { "ID" : "C1" } }, { "Customer" : { "ID" : "C2" } }, { "Customer" : { "ID" : "C3" } } ] } depending on which properties the server includes when expanding Customer  without $select or $ref . Add new example 63: If both subtypes have a Rating property but their common base type does not, GET ~/Products?$apply=groupby((SalesModel.FoodProduct/Rating, SalesModel.NonFoodProduct/Rating)) results in { "@odata.context" : "$metadata#Products( SalesModel.FoodProduct/Rating, SalesModel.NonFoodProduct/Rating)", "value" : [ { "@odata.type" : "#SalesModel.FoodProduct" , "Rating" : 5}, { "@odata.type" : "#SalesModel.FoodProduct" , "Rating" : null }, { "@odata.type" : "#SalesModel.NonFoodProduct" , "Rating" : "average" }, { "@odata.type" : "#SalesModel.NonFoodProduct" , "Rating" : null } ] } Note that the @odata.type context information is needed to distinguish the two groups with "Rating": null . Remove with sum after $count in example 67. Merge ABNF pull request #75 and vocabularies pull request #186 .
    • Resolution:
      Show
      https://www.oasis-open.org/apps/org/workgroup/odata/download.php/69557/odata-data-aggregation-ext-v4.0-wd05.docx Resolution edited on 2022-02-18 because type casts are also allowed as final segment:  https://www.oasis-open.org/apps/org/workgroup/odata/download.php/69610/odata-data-aggregation-ext-v4.0-wd05.docx Inserted example 63 on 2022-02-25:  https://www.oasis-open.org/apps/org/workgroup/odata/download.php/69650/odata-data-aggregation-ext-v4.0-wd05.docx . This resolution was REJECTED by TC 2022-03-10. See new resolution proposal above. https://github.com/oasis-tcs/odata-specs/blob/0a1fd093c816f034f8e7b67a00f2da01becea4df/docs/odata-data-aggregation-ext/odata-data-aggregation-ext.md

      Description

      Original description: The  groupby specification should also allow instance annotations as grouping properties.

      This idea was rejected by TC 2022-03-10. Still, changes to the specification text are necessary.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              heiko.theissen Heiko Theissen
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: