Hacker DirectoryHacker Directory
Continue learning
TRSave HTML ↓

WEB / WEB-17.02

Schemas, serialization and API evolution

How do valid JSON, compatible contracts and permitted resource access differ?

CORE · RED · BLUE · ARTIFACT · PRACTICE

1. Core · Mechanism

Read every scenario without motion

Field retained

  1. Request ready

    The reader prepared pageSize=2 for the public list. The server has not received it; parsing/schema, tenant filtering and response outcome are still pending.

    API reader
    pageSize=2 ready; no response
    Atlas API
    Contract v1; no request
  2. Request arrived

    The request reached the server; parsing and schema decisions are still pending.

    API reader
    Request ready; no UI
    Atlas API
    Request arrived; validation pending
  3. Local contract

    pageSize=2 is an integer within 1–3; permission is separately admitted in this example. The server prepares the compatibility response id=book-A, title=Atlas Notları, subtitle=Web rehberi; the reader has not received it.

    API reader
    Request ready; no UI
    Atlas API
    Request accepted; response ready
  4. Response arrived

    The JSON response reached the reader; old-reader validation and field preservation remain pending.

    API reader
    Response arrived; old reader pending
    Atlas API
    Request accepted; response ready
  5. Old reader

    The old reader accepts required string id/title fields. Its preserve policy retains unfamiliar subtitle=Web rehberi without interpreting it; this local contract decision is not actual DOM-render evidence.

    API reader
    title/subtitle retained; UI ready
    Atlas API
    Request accepted; response ready

Rename breaks

  1. Request ready

    The reader prepared pageSize=2 for the public list. The server has not received it; parsing/schema, tenant filtering and response outcome are still pending.

    API reader
    pageSize=2 ready; no response
    Atlas API
    Contract v1; no request
  2. Request arrived

    The request reached the server; parsing and schema decisions are still pending.

    API reader
    Request ready; no UI
    Atlas API
    Request arrived; validation pending
  3. Local contract

    pageSize=2 is an integer within 1–3; permission is separately admitted in this example. The server prepares the compatibility response id=book-A, heading=Atlas Notları; title absent; the reader has not received it.

    API reader
    Request ready; no UI
    Atlas API
    Request accepted; response ready
  4. Response arrived

    The JSON response reached the reader; old-reader validation and field preservation remain pending.

    API reader
    Response arrived; old reader pending
    Atlas API
    Request accepted; response ready
  5. Old reader

    The response parses as JSON but required title is absent; heading cannot replace it. The old reader rejects the result and prepares no new book UI data; an old title is not reused as success.

    API reader
    title missing; no UI
    Atlas API
    Request accepted; response ready

Unknown rejected

  1. Request ready

    The reader prepared pageSize=2, display=wide for the public list. The server has not received it; parsing/schema, tenant filtering and response outcome are still pending.

    API reader
    pageSize=2, display=wide ready; no response
    Atlas API
    Contract v1; no request
  2. Request arrived

    The request reached the server; parsing and schema decisions are still pending.

    API reader
    Request ready; no UI
    Atlas API
    Request arrived; validation pending
  3. Closed request

    The closed request predicate rejects the unknown display member explicitly; no business read occurs.

    API reader
    Request ready; no UI
    Atlas API
    Schema denied; error ready
  4. Error arrived

    The schema rejection reached the reader; there is no valid book response or rendered book.

    API reader
    Schema error received; no UI
    Atlas API
    Schema denied; error ready

Second page

  1. Continuation ready

    Books A/B were received earlier; this bounded scene starts with the second-page cursor-A request.

    API reader
    cursor-A request ready
    Atlas API
    snapshot-A; C remains; no request
  2. Cursor arrived

    The server received the continuation request; cursor binding and resource policy remain pending.

    API reader
    cursor-A request ready
    Atlas API
    cursor-A arrived; binding pending
  3. Local paginator

    Snapshot/filter/order binding and declared permission pass; book C and a terminal response are prepared.

    API reader
    cursor-A request ready
    Atlas API
    C ready; no nextCursor
  4. Final page arrived

    The book-C response reached the reader; absent nextCursor means completion in this selected contract.

    API reader
    C received; no nextCursor; done
    Atlas API
    C ready; no nextCursor

ACTORS · STATE · DATA FLOW

JSON carries a value model

The Atlas book list begins as an application object, is serialized into UTF-8 JSON text, and is parsed into a new value tree at the other end. These are not the same in-memory object. Object member order is not its semantic ordering; array order matters. A top-level value need not be an object. Strings, numbers, booleans and null are distinct: "2" and 2 need not pass the same schema. Dates, class methods and database connections are not JSON value types.

Absence and an explicit null matter during migration. Our response makes subtitle optional and string-or-null: absence means the information was not supplied, while null is a known empty value. Native JavaScript serialization can omit an undefined object member; this is not a universal JSON language rule. Large integer IDs remain strings: converting an already rounded number back to a string cannot restore precision. Duplicate member behavior varies between parsers, so the producer emits unique names.

What question does a schema answer?

The request schema requires an integer pageSize from 1 through 3 and permits an optional string cursor. The response schema defines a books array with required id and title. These are structural checks after parsing; they do not decide which tenant’s books a user may read. In JSON Schema, properties alone does not require a field, and additionalProperties is not closed by default. This fixture rejects unknown request fields while preserving unknown response fields. The two directions need not use the same rejection policy.

OpenAPI describes request and response media together with schemas; publishing that document does not execute runtime validation. The reviewed current edition is 3.2.1 with a 2020-12-based schema dialect; we do not silently carry over older 3.0 nullable conventions. Read the openapi specification version, info.version document version, /v1 business contract and deployment build number separately. The artifact contains selected schema fragments, not a complete OpenAPI description or an implementation of every JSON Schema keyword.

Evolution and pagination belong together

Adding subtitle in a v1.1 response while retaining title:string is compatible for the reader that preserves unknown response fields. Renaming title to heading within v1 behaves like removal plus addition and breaks the old reader’s expectation. This fixture selects its major version through the URI; it is a product choice, not a universal HTTP requirement. Compatibility is more than parseable JSON: ask separately whether old client code still builds, whether wire types still match, and whether users receive the same business meaning.

The collection is paginated from the outset. After the tenant filter, books A, B and C in a fixed snapshot are ordered by id. pageSize=2 returns the first two with opaque cursor-A; the next page ends with C. The cursor is a local continuation reference bound to this snapshot, filter and order, not permission to read books. A short page alone is not proof of completion; this contract ends when nextCursor is absent. Inserts, deletions, stable tie-breakers, snapshot lifetime and cursor expiry need separate design for a live collection; three fixed records do not solve them.

2. Red · Trust & Impact

Read every scenario without motion

Request field

  1. Producer and reader roles were separated

    The request profile accepts only pageSize and cursor. The request row containing display=wide is a negative example of this closed contract.

    Contract change
    Unknown request=reject
    Schema and reader policy
    Schema gate pending
    Consumer decision
    Consumer not accepted
  2. Syntax and schema conditions were separated

    pageSize=2 is valid, but display conflicts with additionalProperties=false. One correct field does not establish acceptance of the other fields.

    Contract change
    pageSize=2
    Schema and reader policy
    Shape under review
    Consumer decision
    Acceptance pending
  3. The contract mismatch was identified

    The unknown-field request row declares expected=false. Its policy is reject, not silently deleting the unknown field and accepting the request.

    Contract change
    Known contract mismatch
    Schema and reader policy
    Expected=false
    Consumer decision
    Consumer gate closed
  4. The transformation policy remained explicit

    This contract rejects unknown request fields while preserving unknown response fields. Both directions are not assumed to share one normalization policy.

    Contract change
    Request/response policies separate
    Schema and reader policy
    No silent data loss
    Consumer decision
    Incompatible branch separate
  5. A version label did not change the decision

    contractVersion=v1 and documentVersion=1.1 describe documents. Those labels cannot automatically accept a rejected shape or grant business permission.

    Contract change
    Versions: v1 / 1.1
    Schema and reader policy
    Shape mismatch retained
    Consumer decision
    Permission separate
  6. The consumer expectation was preserved at the boundary

    The denial concerns a closed request contract. It alone says neither that the actor’s tenant permission was denied nor that the two normal pages failed.

    Contract change
    Contract failure known
    Schema and reader policy
    Denial cause bounded
    Consumer decision
    No incorrect acceptance

Title rename

  1. Producer and reader roles were separated

    The old reader requires id and title. The renamed response contains id=book-A and heading=Atlas Notları, with no title field.

    Contract change
    Required: id, title
    Schema and reader policy
    Schema gate pending
    Consumer decision
    Consumer not accepted
  2. Syntax and schema conditions were separated

    heading may be a string and the JSON may parse. A new field cannot repair the absence of the old required title field.

    Contract change
    heading present; title absent
    Schema and reader policy
    Shape under review
    Consumer decision
    Acceptance pending
  3. The contract mismatch was identified

    The renamed compatibility row declares expected=false. Preserving unknown response fields cannot make an absent required title valid.

    Contract change
    Known contract mismatch
    Schema and reader policy
    Expected=false
    Consumer decision
    Consumer gate closed
  4. The transformation policy remained explicit

    Adding subtitle in the normal response can be accepted because title remains. An additive field and a required-field rename are different compatibility changes.

    Contract change
    Request/response policies separate
    Schema and reader policy
    No silent data loss
    Consumer decision
    Incompatible branch separate
  5. A version label did not change the decision

    contractVersion=v1 and documentVersion=1.1 describe documents. Those labels cannot automatically accept a rejected shape or grant business permission.

    Contract change
    Versions: v1 / 1.1
    Schema and reader policy
    Shape mismatch retained
    Consumer decision
    Permission separate
  6. The consumer expectation was preserved at the boundary

    The denial concerns loss of required title. It does not forbid every new optional field; preserve mode retains an unknown field without interpreting it.

    Contract change
    Contract failure known
    Schema and reader policy
    Denial cause bounded
    Consumer decision
    No incorrect acceptance

ASSUMPTION · BOUNDARY · CONTROL

Drawing the wrong conclusion from format acceptance

A parsed request proves only that the selected parser accepted its text. Passing the schema proves that pageSize and cursor have acceptable shapes. Neither result proves the tenant binding, the subject’s read permission or that a cursor still belongs to a valid snapshot. The risky assumption is compressing every green result into a single valid-request label. The artifact therefore records parsing, schema, continuation and business permission separately so the reader can identify the rejection boundary.

Unknown-field handling can create invisible data loss. If a new response includes subtitle but an adapter copies only id/title and discards the original, the next serialization loses subtitle. This loss is not required by JSON or by a schema; it is the adapter’s transformation. The fixture’s preserve mode retains the extra member through a round trip, whereas its closed request rejects an unknown member explicitly. Either policy can be deliberate, but claiming preservation while silently discarding data breaks the compatibility contract.

A small change can break business meaning

If an old client interprets an empty books array as an empty archive, adding pagination later changes more than one field. A user may see only the first page and mistake an incomplete list for the whole collection. Changing title from a string to an object can also break the old UI while JSON remains valid. Compatibility review therefore separates required names, wire types, field preservation and collection-completion meaning, identifying which old expectation is broken.

An opaque cursor is not authenticated merely because its contents are unreadable. Even valid continuation information must accompany a fresh resource-access decision for each page; cached results need the same tenant and filter binding. Neither a page count nor the presence of nextCursor establishes permission. The two successful pages prove behavior only for the fixture’s three records; they provide no evidence of concurrent updates, storage transactions or a cursor-signature implementation.

3. Blue · Secure development & defense

Read every scenario without motion

Null and absence

  1. The field contract was reviewed

    The book schema allows subtitle as string or null and does not require it. title is a required string. Optionality and allowing null are different conditions.

    Contract observation
    subtitle: string | null
    Reader/page comparison
    title: required string
    Review conclusion
    Reader result pending
  2. The null-response row was selected

    The null-subtitle response retains id and title and has subtitle=null. This explicit null is not the same representation as a producer omitting the field.

    Contract observation
    subtitle present: null
    Reader/page comparison
    Required fields present
    Review conclusion
    Round trip not reviewed
  3. Compatibility was bound to the schema

    This row has expected=true. It is accepted because null is explicitly allowed; the result does not mean the reader understands every unknown field.

    Contract observation
    Expected=true
    Reader/page comparison
    Null allowed explicitly
    Review conclusion
    Interpretation scope bounded
  4. The old reader preserve mode was read

    oldReaderSchema does not define subtitle but allows additional properties. oldReaderUnknownFields declares that subtitle is preserved without interpretation.

    Contract observation
    Old reader: id/title
    Reader/page comparison
    subtitle preserved
    Review conclusion
    Subtitle meaning not inferred
  5. Silent field loss was checked

    A reader that copies only id/title into a new object and discards original fields violates preserve semantics. Accepted unknown fields and fields retained through round trip require separate observation.

    Contract observation
    Acceptance ≠ preservation
    Reader/page comparison
    Round trip evidence required
    Review conclusion
    Silent discard not accepted
  6. The review conclusion remained explicit

    The contract defines null versus absence while the old reader does not interpret subtitle. This is not evidence that a full schema validator or every client ran.

    Contract observation
    Null acceptance known
    Reader/page comparison
    Preserve contract known
    Review conclusion
    All clients unobserved

Cursor binding

  1. The page context was selected

    The profile selects snapshot-A, tenant-A and id ASC order. Records also contain a tenant-B item; continuation cannot be reviewed apart from the tenant filter.

    Contract observation
    snapshot-A / tenant-A
    Reader/page comparison
    Order: id ASC
    Review conclusion
    Page results pending
  2. The first-page record was read

    The pageSize=2 request returns book-A and book-B with nextCursor=cursor-A. Those two records are not reported as the complete collection.

    Contract observation
    Page 1: A, B
    Reader/page comparison
    nextCursor=cursor-A
    Review conclusion
    Collection incomplete
  3. The cursor-ledger bindings were matched

    cursor-A carries snapshot-A, tenant-A, id ASC and nextIndex=2. It selects a continuation position without granting resource permission.

    Contract observation
    Cursor context matched
    Reader/page comparison
    nextIndex=2
    Review conclusion
    Cursor ≠ permission
  4. The terminal-page result was separated

    The second page contains only book-C and no nextCursor. This terminal result applies to the three tenant-A records in the supplied snapshot, not another snapshot.

    Contract observation
    Page 2: C
    Reader/page comparison
    nextCursor absent
    Review conclusion
    Selected snapshot complete
  5. Filter and permission evidence were separated

    The tenant-B record being absent from these pages is a filter result. permissionDeclared=true is a separate precondition; cursor opacity is not authentication evidence.

    Contract observation
    Tenant filter known
    Reader/page comparison
    Permission declared separately
    Review conclusion
    Opaque ≠ authenticated
  6. The snapshot scope was preserved

    These pages describe a fixed snapshot. Concurrent updates, cursor signatures and storage transactions need new evidence; a terminal page does not prove them.

    Contract observation
    Fixed snapshot only
    Reader/page comparison
    Concurrent storage unobserved
    Review conclusion
    Completion bounded

IMPLEMENTATION · SECURITY CONTROL · VERIFICATION

Write producer and reader policies together

Specify each field’s name, wire type, required or optional status, nullability, default and unknown-field policy. A producer can keep emitting title while a reader preserves an unfamiliar subtitle; test that round-trip contract explicitly. A closed request schema returns a clear error for an unknown field rather than silently deleting it during normalization. Define numeric domains, string size and collection limits alongside parser limits; JSON number grammar does not itself guarantee business units or precision.

Run old-request/new-server and new-response/old-reader regressions separately. With the same normal book record, adding subtitle should succeed, renaming title should fail, and an explicit null subtitle should be accepted. Incrementing a document version cannot substitute for those results. Select a validator that understands the supported schema dialect and pin it; compare generated client expectations with actual runtime responses. Enum handling and strict deserialization in generated clients also belong in compatibility review.

Separate the page boundary from the permission boundary

Document pagination defaults and maximums, ordering tie-breakers, the meaning of an absent nextCursor and the filter/order binding. This fixture uses a fixed in-memory snapshot, applies the tenant filter before pagination, and uses cursor-A only to select the next starting position. If its bound filter changes, return a binding error instead of inventing another page. Explain how clients restart after cursor expiry or snapshot loss; the lesson does not supply a production storage implementation.

Blue evidence includes field preservation through a round trip, absence of a terminal cursor, the old UI continuing to read title, and a separate policy result on every page, rather than merely a schema-pass count. Larger systems also need cache-key, cancellation, timeout and changed-authorization behavior. The scene separates request/response arrivals from local server validator/paginator decisions. Parsing JSON text does not establish continuation binding or business permission.

NOW APPLY IT TO THE CASE

Which old expectation must a new field or cursor preserve?

Requests reject unknown fields while responses preserve them. Pagination follows a fixed tenant/snapshot context; shape and continuation do not replace permission.

Examine the case API contract separately from the representation readable by the browser. Discuss sufficient evidence for envelope, data, errors and resource permission.

Open the related case file ↗

READ THE EXAMPLE TOGETHER

Atlas API contract and continuation ledger

Select a field and examine what it supports and where the evidence ends.

Read the raw artifact and field explanations
{
  "profile": {
    "specEdition": "OpenAPI 3.2.1",
    "schemaDialect": "https://json-schema.org/draft/2020-12/schema",
    "contractVersion": "v1",
    "documentVersion": "1.1",
    "snapshot": "snapshot-A",
    "tenant": "tenant-A",
    "order": "id ASC",
    "unknownRequest": "reject",
    "unknownResponse": "preserve",
    "permissionDeclared": true
  },
  "requestSchema": {
    "type": "object",
    "properties": {
      "pageSize": {
        "type": "integer",
        "minimum": 1,
        "maximum": 3
      },
      "cursor": {
        "type": "string"
      }
    },
    "required": [
      "pageSize"
    ],
    "additionalProperties": false,
    "$schema": "https://json-schema.org/draft/2020-12/schema"
  },
  "bookSchema": {
    "type": "object",
    "properties": {
      "id": {
        "type": "string"
      },
      "title": {
        "type": "string"
      },
      "subtitle": {
        "type": [
          "string",
          "null"
        ]
      }
    },
    "required": [
      "id",
      "title"
    ],
    "additionalProperties": true,
    "$schema": "https://json-schema.org/draft/2020-12/schema"
  },
  "records": [
    {
      "id": "book-A",
      "title": "Atlas Notları",
      "tenant": "tenant-A"
    },
    {
      "id": "book-B",
      "title": "Ağ Defteri",
      "tenant": "tenant-A"
    },
    {
      "id": "book-C",
      "title": "HTTP Notları",
      "tenant": "tenant-A"
    },
    {
      "id": "other-book",
      "title": "Başka Arşiv",
      "tenant": "tenant-B"
    }
  ],
  "cursorLedger": {
    "cursor-A": {
      "snapshot": "snapshot-A",
      "tenant": "tenant-A",
      "order": "id ASC",
      "nextIndex": 2
    }
  },
  "pages": [
    {
      "request": {
        "pageSize": 2
      },
      "response": {
        "books": [
          {
            "id": "book-A",
            "title": "Atlas Notları"
          },
          {
            "id": "book-B",
            "title": "Ağ Defteri"
          }
        ],
        "nextCursor": "cursor-A"
      }
    },
    {
      "request": {
        "pageSize": 2,
        "cursor": "cursor-A"
      },
      "response": {
        "books": [
          {
            "id": "book-C",
            "title": "HTTP Notları"
          }
        ]
      }
    }
  ],
  "compatibilityRows": [
    {
      "case": "normal",
      "response": {
        "id": "book-A",
        "title": "Atlas Notları",
        "subtitle": "Web rehberi"
      },
      "expected": true,
      "unknownMode": "preserve"
    },
    {
      "case": "renamed",
      "response": {
        "id": "book-A",
        "heading": "Atlas Notları"
      },
      "expected": false,
      "unknownMode": "preserve"
    },
    {
      "case": "null-subtitle",
      "response": {
        "id": "book-A",
        "title": "Atlas Notları",
        "subtitle": null
      },
      "expected": true,
      "unknownMode": "preserve"
    }
  ],
  "requestRows": [
    {
      "case": "normal",
      "value": {
        "pageSize": 2
      },
      "expected": true
    },
    {
      "case": "unknown-field",
      "value": {
        "pageSize": 2,
        "display": "wide"
      },
      "expected": false
    },
    {
      "case": "wrong-type",
      "value": {
        "pageSize": "2"
      },
      "expected": false
    },
    {
      "case": "integer-number",
      "value": {
        "pageSize": 2.0
      },
      "expected": true
    },
    {
      "case": "fractional",
      "value": {
        "pageSize": 2.5
      },
      "expected": false
    }
  ],
  "jsonFixtures": {
    "text": "{\"id\":\"book-A\",\"title\":\"Atlas Notları\",\"subtitle\":null}",
    "largeId": "9007199254740993",
    "nullDistinctFromAbsent": true,
    "duplicateNamesProducerForbidden": true
  },
  "boundaries": {
    "schemaGrantsPermission": false,
    "cursorGrantsPermission": false,
    "actualFullValidator": false,
    "concurrentStorage": false
  },
  "oldReaderSchema": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "id": {
        "type": "string"
      },
      "title": {
        "type": "string"
      }
    },
    "required": [
      "id",
      "title"
    ],
    "additionalProperties": true
  },
  "oldReaderUnknownFields": "preserve without interpreting subtitle"
}
requestSchema
Selected closed request rules; not a complete validator.
bookSchema
Current producer book schema; subtitle may be string or null.
pages
Two pages of the fixed snapshot after tenant filtering.
compatibilityRows
Old-reader expectations for addition, rename and null.
cursorLedger
Local snapshot/filter/order continuation binding of an opaque alias.
oldReaderSchema
The old reader knows only id/title and preserves unknown subtitle without interpreting it.

Synthetic JSON and selected schema predicates; no endpoint, complete OpenAPI/JSON Schema validator, cursor cryptography, cache or concurrent database.

Two reality checks

A parsed JSON request is valid.

Parsing only produces values from syntax; schema, continuation and resource policy remain separate decisions.

A new response field is compatible with every old client.

That conclusion requires checking the reader’s reject/preserve/discard behavior and round-trip contract.

YOUR TURN

Work through this story.

Predict the outcome, find the evidence boundary, and explain it in your own words.

A request has pageSize string "2" while the contract requires an integer. Does the unknown/type policy accept it?

  1. Yes; every numeric-looking string in JSON is an integer.
  2. No; wrong-type expected=false.
Read the explained answer

No; wrong-type expected=false. — The selected schema does not silently coerce string "2" into an integer; size bounds/type both apply.

What does selected compatibility require when an old reader sees a new subtitle field?

Selected lesson artifact fields; teaching example

oldReaderSchema.additionalProperties
true
oldReaderUnknownFields
preserve without interpreting subtitle
compatibilityRows[0]
{"case":"normal","response":{"id":"book-A","title":"Atlas Notları","subtitle":"Web rehberi"},"expected":true,"unknownMode":"preserve"}
  1. The field is preserved without the old reader assigning it new meaning.
  2. Every unknown field must be automatically deleted in both directions.
Read the explained answer

The field is preserved without the old reader assigning it new meaning. — additionalProperties=true and preserve policy prevent silent data loss; request policy differs in direction.

cursor-A carries snapshot/tenant/order. Why is page continuation not permission, and why is null subtitle not the same as absent?

Read the explained answer

The cursor selects deterministic continuation context; current tenant/resource permission must still be verified. The book schema permits null subtitle, while absence means the field is missing; consumers preserve that distinction according to the contract. This fixture executes neither a full JSON Schema validator nor concurrent storage.

□ I did not count a cursor as a permission capability.

□ I distinguished null from absent semantics.

Open the application console · request / response / browser
Also open request, URL, cookie and DOM examples

Sources and scope