87 Group Aggregation

Group aggregation is query-result shaping over logic-frame extensions. A fold consumes the observed values of a unary path frame, and tuple construction chooses which labels are exposed in the result. When the result is displayed as a relation, that relation is the observed extension of the frame, not the primary denotation of the path expression.

The examples in this chapter compare Predica idioms with Tutorial D SUMMARIZE. They should be read extensionally: the Predica expression and the Tutorial D expression are being compared by the materialised relation shape they produce.

Aggregates in Tuple Results


s.supplier.(id, shipment.qty.Σsum)

has this materialised extension:


{
    (id = 1, qty = 1300),
    (id = 2, qty = 700),
    (id = 3, qty = 200),
    (id = 4, qty = 900),
    (id = 5, qty = 0)
}

In the tuple constructor expression (id, shipment.qty.Σsum) there are two path tails. Unless renamed, the attribute name to be used for a path tail is the final identifier before any aggregation operator. The path expression still denotes a logic frame; the relation display above is the observed extension of that frame.

Implicit Grouping by Receiver

Suppose the required result is the number of shipments per supplier. In Tutorial D this can be written with SUMMARIZE on the shipments relation SP:


SUMMARIZE SP PER SP { S# } ADD COUNT(*) AS NUM_SHIPMENTS

In Predica, the direct object-centric expression is:


s.supplier.(id, shipment.Σcount as num_shipments)

In logic-frame display notation this means:

id=I:Integer, num_shipments=C:Integer
where
  ∃S:Supplier
    s.supplier(S)
  ∧ S.id(I)
  ∧ C = |{ SP:Shipment | S.shipment(SP) }|

This expression includes suppliers having no shipments, with count zero. That is often the useful object-centric result, but it is not the same as summarising only the observed shipment extension. To match the Tutorial D expression above, filter to suppliers with at least one shipment:


s.supplier.[shipment.Σ ≠ ∅].(id, shipment.Σcount as num_shipments)

Explicit Grouping

Explicit grouping groups the observed extension by a key expression. The key expression is evaluated for each member of the receiver, and members with the same key become a group.


s.shipment.groupby(supplier.id)

In the relation view, this can be displayed as a grouped result with a nested relation display, much like Tutorial D's GROUP. The nested display is not a claim that relations are first-class Predica values. It is a way to show the grouped observed extension.

groupby does not compute aggregates. It restructures the observed extension so later folds can aggregate each group.

s.shipment s.shipment.groupby(supplier.id)
S1,P1,300 S1,P2,200 S1,P3,400 S1,P4,200 S1,P5,100 S1,P6,100 S2,P1,300 S2,P2,400 S3,P2,200 S4,P2,200 S4,P4,300 S4,P5,400 S1 P1,300 P2,200 P3,400 P4,200 P5,100 P6,100 S2 P1,300 P2,400 S3 P2,200 S4 P2,200 P4,300 P5,400

The following Predica expression:


s.shipment.groupby(supplier.id).(id, group.qty.Σsum as total_qty)

or, with an explicit group label:


s.shipment.group(g)by(supplier.id).(id, g.qty.Σsum as total_qty)

has the same observed extension as the Tutorial D expression:


SUMMARIZE SP PER SP { S# } ADD SUM(QTY) AS TOTAL_QTY

Group aggregation examples

The following examples are a comparison catalogue. The Predica column shows path expressions whose observed extensions correspond to the Tutorial D results. Examples that collect nested results with Σ should be read as producing set-valued result terms; any nested relation tables in the display are relation-view renderings of those values, not a separate relation-valued denotational semantics.

# Description Tutorial D (S, P, SP) Predica
1 Total quantity shipped by supplier

SUMMARIZE SP 
    PER SP { S# }
    ADD SUM(QTY) AS TOTAL_QTY

s.supplier.(
    id,
    shipment.qty.Σsum as total_qty)
2 Total quantity shipped by part

SUMMARIZE SP 
    PER SP { P# }
    ADD SUM(QTY) AS TOTAL_QTY

s.part.(
    id,
    shipment.qty.Σsum as total_qty)
3 Number of shipments per supplier

SUMMARIZE SP 
    PER SP { S# }
    ADD COUNT(*) AS NUM_SHIPMENTS

s.supplier.(
    id,
    shipment.Σcount as num_shipments)
4 Number of distinct parts shipped by supplier

SUMMARIZE SP 
    PER SP { S# }
    ADD COUNT(P#) AS NUM_PARTS

s.supplier.(
    id,
    shipment.part.Σcount as num_parts)
5 Total quantity and number of shipments per supplier

SUMMARIZE SP 
    PER SP { S# }
    ADD SUM(QTY) AS TOTAL_QTY,
        COUNT(*) AS NUM_SHIPMENTS

s.supplier.(
    id,
    shipment.qty.Σsum as total_qty,
    shipment.Σcount as num_shipments)
6 Total quantity shipped per supplier–city

SUMMARIZE (SP JOIN S) 
    PER (SP JOIN S) { S#, CITY }
    ADD SUM(QTY) AS TOTAL_QTY

s.supplier.(
    id,
    city,
    shipment.qty.Σsum as total_qty)
7 Total quantity and number of suppliers per part

SUMMARIZE SP 
    PER SP { P# }
    ADD SUM(QTY) AS TOTAL_QTY,
        COUNT(S#) AS NUM_SUPPLIERS

s.part.(
    id,
    shipment.qty.Σsum as total_qty,
    shipment.supplier.Σcount 
        as num_suppliers)
8 Total weight of parts shipped per supplier

SUMMARIZE (SP JOIN P) 
    PER (SP JOIN P) { S# }
    ADD SUM(QTY * WEIGHT) 
        AS TOTAL_WEIGHT

s.supplier.(
    id,
    shipment.(qty * part.weight).Σsum 
        as total_weight)
9 List of parts shipped by each supplier

SUMMARIZE SP 
    PER SP { S# }
    ADD { P#, PNAME } AS PARTS

s.supplier.(
    id,
    shipment.part.(id, name).Σ as parts)
10 Number of heavy parts shipped (weight > 15) per supplier

SUMMARIZE (SP JOIN P 
        WHERE WEIGHT > 15) 
    PER (SP JOIN P) { S# }
    ADD COUNT(*) AS NUM_HEAVY

s.supplier.(
    id,
    shipment.[part.weight > 15].Σcount
        as num_heavy)

Nested grouping examples

The following examples illustrate nested grouping, multi-level grouping, tuple-valued grouping keys, and grouping with filters:

# Description Tutorial D (S, P, SP) Predica
11 Total quantity shipped per supplier per part (two-level grouping)

SUMMARIZE SP PER SP { S#, P# }
    ADD SUM(QTY) AS TOTAL_QTY

s.shipment.(
    supplier.id as s#,
    part.id as p#,
    qty.Σsum as total_qty)
12 For each supplier, aggregates grouped by part (nested grouping)

(SUMMARIZE (SP RENAME S# AS OUTER_S#) PER (SP RENAME S# AS OUTER_S#) { OUTER_S# }
    ADD SUMMARIZE SP WHERE S# = OUTER_S#
        PER SP { P# }
        ADD SUM(QTY) AS TOTAL_QTY
        AS PART_SUMMARY
) RENAME OUTER_S# AS S#

s.supplier.(
    id,
    shipment.(
        part.id,
        qty.Σsum as total_qty
    ).Σ as part_summary)
13 For each city, total quantity shipped and number of suppliers

SUMMARIZE (SP JOIN S) PER (SP JOIN S) { CITY }
    ADD SUM(QTY) AS TOTAL_QTY,
        COUNT(DISTINCT S#) AS NUM_SUPPLIERS

s.shipment.(
    supplier.city,
    qty.Σsum as total_qty,
    supplier.id.{ }.Σcount as num_suppliers)
14 For each part, list of suppliers and total quantity per supplier

(SUMMARIZE (SP RENAME P# AS OUTER_P#) PER (SP RENAME P# AS OUTER_P#) { OUTER_P# }
    ADD SUMMARIZE SP WHERE P# = OUTER_P#
        PER SP { S# }
        ADD SUM(QTY) AS TOTAL_QTY
        AS SUPPLIER_SUMMARY
) RENAME OUTER_P# AS P#

s.part.(
    id,
    shipment.(
        supplier.id,
        qty.Σsum as total_qty
    ).Σ as supplier_summary)
15 For each supplier, total quantity of red parts shipped

SUMMARIZE (SP JOIN P WHERE COLOR = 'Red') PER (SP JOIN P) { S# }
    ADD SUM(QTY) AS TOTAL_RED_QTY

s.shipment.[part.color = Color.red].(
    supplier.id,
    qty.Σsum as total_red_qty)
16 For each supplier, number of parts shipped grouped by color

(SUMMARIZE ((SP JOIN P) RENAME S# AS OUTER_S#) PER ((SP JOIN P) RENAME S# AS OUTER_S#) { OUTER_S# }
    ADD SUMMARIZE (SP JOIN P WHERE S# = OUTER_S#)
        PER (SP JOIN P) { COLOR }
        ADD COUNT(*) AS NUM_PARTS
        AS COLOR_SUMMARY
) RENAME OUTER_S# AS S#

s.supplier.(
    id,
    shipment.part.(
        color,
        Σcount as num_parts
    ).Σ as color_summary)
17 For each city, list of suppliers and their total shipments

(SUMMARIZE (S RENAME CITY AS OUTER_CITY) PER (S RENAME CITY AS OUTER_CITY) { OUTER_CITY }
    ADD SUMMARIZE (SP JOIN S WHERE CITY = OUTER_CITY)
        PER (SP JOIN S) { S# }
        ADD SUM(QTY) AS TOTAL_QTY
        AS SUPPLIER_SUMMARY
) RENAME OUTER_CITY AS CITY

s.supplier.(
    city,
    shipment.(
        supplier.id,
        qty.Σsum as total_qty
    ).Σ as supplier_summary)
18 For each part, total quantity shipped by suppliers in London

SUMMARIZE (SP JOIN S WHERE CITY = 'London') PER (SP JOIN S) { P# }
    ADD SUM(QTY) AS TOTAL_QTY

s.shipment.(
    part.id,
    supplier.[city = "London"].qty.Σsum as total_qty)

The following examples illustrate deeply nested grouping, multi-attribute tuple keys, aggregates over aggregates, and multi-level summarization:

# Description Tutorial D (S, P, SP) Predica
19 For each supplier, for each part, total quantity (deep nested grouping)

(SUMMARIZE (SP RENAME S# AS OUTER_S#) PER (SP RENAME S# AS OUTER_S#) { OUTER_S# }
  ADD SUMMARIZE SP WHERE S# = OUTER_S#
      PER SP { P# }
      ADD SUM(QTY) AS TOTAL_QTY
      AS PART_SUMMARY
) RENAME OUTER_S# AS S#

s.supplier.(
    id,
    shipment.(
        part.id,
        qty.Σsum as total_qty
    ).Σ as part_summary)
20 For each city, for each supplier, total shipments (multi-level grouping)

(SUMMARIZE (S RENAME CITY AS OUTER_CITY) PER (S RENAME CITY AS OUTER_CITY) { OUTER_CITY }
  ADD SUMMARIZE (SP JOIN S WHERE CITY = OUTER_CITY)
      PER (SP JOIN S) { S# }
      ADD SUM(QTY) AS TOTAL_QTY
      AS SUPPLIER_SUMMARY
) RENAME OUTER_CITY AS CITY

s.supplier.(
    city,
    shipment.(
        supplier.id,
        qty.Σsum as total_qty
    ).Σ as supplier_summary)
21 For each supplier, total quantity per part color (grouping by derived attribute)

(SUMMARIZE ((SP JOIN P) RENAME S# AS OUTER_S#) PER ((SP JOIN P) RENAME S# AS OUTER_S#) { OUTER_S# }
  ADD SUMMARIZE (SP JOIN P WHERE S# = OUTER_S#)
      PER (SP JOIN P) { COLOR }
      ADD SUM(QTY) AS TOTAL_QTY
      AS COLOR_SUMMARY
) RENAME OUTER_S# AS S#

s.supplier.(
    id,
    shipment.(
        part.color,
        qty.Σsum as total_qty
    ).Σ as color_summary)
22 For each part, total quantity per supplier city (tuple-valued grouping key)

SUMMARIZE (SP JOIN S) PER (SP JOIN S) { P#, CITY }
  ADD SUM(QTY) AS TOTAL_QTY

s.shipment.(
    part.id as p#,
    supplier.city,
    qty.Σsum as total_qty)
23 For each supplier, average quantity per part (aggregate over aggregates)

(SUMMARIZE (SP RENAME S# AS OUTER_S#) PER (SP RENAME S# AS OUTER_S#) { OUTER_S# }
  ADD AVG(
        SUMMARIZE SP WHERE S# = OUTER_S#
            PER SP { P# }
            ADD SUM(QTY)
      ) AS AVG_QTY
) RENAME OUTER_S# AS S#

s.supplier.(
    id,
    shipment.(
        part.id,
        qty.Σsum as total_qty
    ).total_qty.Σavg as avg_qty)
24 For each city, number of suppliers and total shipments (multi-aggregate grouping)

(SUMMARIZE (S RENAME CITY AS OUTER_CITY) PER (S RENAME CITY AS OUTER_CITY) { OUTER_CITY }
  ADD COUNT(*) AS NUM_SUPPLIERS,
      SUMMARIZE (SP JOIN S WHERE CITY = OUTER_CITY)
          PER (SP JOIN S) { CITY }
          ADD SUM(QTY) AS TOTAL_QTY
          AS SHIPMENT_SUMMARY
) RENAME OUTER_CITY AS CITY

s.supplier.(
    city,
    Σcount as num_suppliers,
    shipment.qty.Σsum as total_qty)
25 For each supplier, summary of heavy vs light parts shipped

(SUMMARIZE (SP RENAME S# AS OUTER_S#) PER (SP RENAME S# AS OUTER_S#) { OUTER_S# }
  ADD SUMMARIZE (SP JOIN P WHERE S# = OUTER_S# AND WEIGHT > 15)
        PER (SP JOIN P) { S# }
        ADD SUM(QTY) AS HEAVY_QTY
        AS HEAVY,
      SUMMARIZE (SP JOIN P WHERE S# = OUTER_S# AND WEIGHT ≤ 15)
        PER (SP JOIN P) { S# }
        ADD SUM(QTY) AS LIGHT_QTY
        AS LIGHT
) RENAME OUTER_S# AS S#

s.supplier.(
    id,
    shipment.[part.weight > 15].qty.Σsum as heavy_qty,
    shipment.[part.weight ≤ 15].qty.Σsum as light_qty)
26 For each part, summary of shipments grouped by supplier status

(SUMMARIZE ((SP JOIN S) RENAME P# AS OUTER_P#) PER ((SP JOIN S) RENAME P# AS OUTER_P#) { OUTER_P# }
  ADD SUMMARIZE (SP JOIN S WHERE P# = OUTER_P#)
        PER (SP JOIN S) { STATUS }
        ADD SUM(QTY) AS TOTAL_QTY
        AS STATUS_SUMMARY
) RENAME OUTER_P# AS P#

s.part.(
    id,
    shipment.(
        supplier.status,
        qty.Σsum as total_qty
    ).Σ as status_summary)
27 For each supplier, total shipments and average shipment size

SUMMARIZE SP PER SP { S# }
  ADD SUM(QTY) AS TOTAL_QTY,
      AVG(QTY) AS AVG_QTY

s.shipment.(
    supplier.id,
    qty.Σsum as total_qty,
    qty.Σavg as avg_qty)