RFC 6901 Extensions for kotlinx.serialization

This library adds JSON Pointer (RFC 6901) extensions to the kotlinx.serialization library.

What is a JSON Pointer?

A JSON Pointer is a way to address a specific location within a JSON document.

For example, given this document (lifted from RFC 4627):

{
  "Image": {
    "Width": 800,
    "Height": 600,
    "Title": "View from 15th Floor",
    "Thumbnail": {
      "Url": "http://www.example.com/image/481989943",
      "Height": 125,
      "Width": "100"
    },
    "IDs": [116, 943, 234, 38793]
  }
}

The value at pointer "/Image/Thumbnail/Width" is "100".

The value at pointer "/Image/Thumbnail/IDs/2" is 234.

Why kotlinx.serialization?

Why add this support to kotlinx.serialization vs other serialization platforms?

1. Official library from JetBrains
2. Multiplatform
3. Common

In practice, I find that in a given app model-based deserialization is the best choice in 80% of non-test scenarios, and a significant impediment in the other 20%. This small extension library allows both modes of operation without requiring a completely separate serialization library.

Why not use models?

You can! Models are a great choice when you have strongly typed structures which you yourself define at both serialization and deserialization points, or which adhere to a strict external specification.

Why add pointers?

Why add support for JSON pointers when kotlinx.serialization supports parsing directly to model classes instead?

1. Less code. Parsing to models requires making models for every intermediate step in a complex structure.
2. Flexibility. Strict types and contracts are ideal, but we don’t live in an ideal world. Sometimes conditional parsing is required, especially when compatibility modes or unreliable input applies.
3. Better tests. When testing the structure and contents of JSON outputs, a test using models

Consider again the example above from RFC 4627. We did not write this document format, but we have to consume and process it. Suppose we are writing some kind of validation component which passes this along to a renderer IFF the thumbnail dimensions are within an acceptable range. How do we check these values?

With this library’s set of extensions, we might extract these values like so:

val document = Json.parseToJsonElement(inputStr)
val thumbnailWidth : Int? = document.ptrGetInt("/Image/Thumbnail/Width")
val thumbnailHeight : Int? = document.ptrGetInt("/Image/Thumbnail/Height")

Without them, we may need to make model classes specific to the purpose.

@Serializable
data class DocumentContainingImage(
    @SerialName("Image") val image: Image?,
)

@Serializable
data class Image(
    @SerialName("Width") val width: Int?,
    @SerialName("Height") val height: Int?,
    @SerialName("Title") val title: String?,
    @SerialName("Thumbnail") val thumbnail: Thumbnail?,
    @SerialName("IDs") val ids: List<Int>?,
)

@Serializable
data class Thumbnail(
    @SerialName("Url") val url: String?,
    @SerialName("Height") val height: Int?,
    @SerialName("Width") val width: Int?,
)

val permissiveJson = Json { ignoreUnknownKeys = true }
val parsedDoc = permissiveJson.decodeFromString<DocumentContainingImage>(DocumentContainingImage.serializer(), inputStr)
val thumbnailHeight : Int? = parsedDoc.image?.thumbnail?.width
val thumbnailWidth : Int? = parsedDoc.image?.thumbnail?.width

We did not define structure and its naming conventions which conflict with Kotlin property name conventions, so we need to add extra annotations as necessary plumbing. We need to use an alternative Json instance because we do not know whether other members might exist other than those in this example, but don’t want to fail. With this in mind, and knowing that we are not using other fields, we could choose to trim the ignored attributes in the model like so…

@Serializable
data class DocumentContainingImage(
    @SerialName("Image") val image: Image?,
)

@Serializable
data class Image(
    @SerialName("Thumbnail") val thumbnail: Thumbnail?,
)

@Serializable
data class Thumbnail(
    @SerialName("Height") val height: Int?,
    @SerialName("Width") val width: Int?,
)

val permissiveJson = Json { ignoreUnknownKeys = true }
val parsedDoc = permissiveJson.decodeFromString<DocumentContainingImage>(DocumentContainingImage.serializer(), inputStr)
val thumbnailHeight : Int? = parsedDoc.image?.thumbnail?.width
val thumbnailWidth : Int? = parsedDoc.image?.thumbnail?.width

…but doing so only illustrates a clarity and maintainability problem in using model classes here: These data classes carry an implication that an Image contains only a Thumbnail, and a Thumbnail is composed of only a width+height. In this example, our code intends to make no such assertions, implementing only partial models strictly as plumbing necessary for accessing specific arbitrary values in the document structure.

Now compare again to the example using this library.

val document = Json.parseToJsonElement(inputStr)
val thumbnailWidth : Int? = document.ptrGetInt("/Image/Thumbnail/Width")
val thumbnailHeight : Int? = document.ptrGetInt("/Image/Thumbnail/Height")

We grab exactly what we need, and nothing more, with no additional plumbing.

Doesn’t kotlinx.serialization Already Support Generic Models?

Sort of. On the golden path, the following two implementations are theoretically equivalent:

val document = Json.parseToJsonElement(inputStr)
val thumbnailWidth : Int? = document.ptrGetInt("/Image/Thumbnail/Width")
val thumbnailHeight : Int? = document.ptrGetInt("/Image/Thumbnail/Height")

val document = Json.parseToJsonElement(inputStr)
val thumbnailWidth : Int? = document.jsonObject["Image"]?.jsonObject?.get("Thumbnail")?.jsonObject?.get("Width")?.jsonPrimitive?.int
val thumbnailHeight : Int? = document.jsonObject["Image"]?.jsonObject?.get("Thumbnail")?.jsonObject?.get("Height")?.jsonPrimitive?.int

However, there are a number of limitations in the latter example without these rfc6901 exceptions. Most significantly, error fallbacks are not clear and perhaps not as expected.

What happens if the root document isn’t a JSON object? Or it doesn’t contain a key called Image? Or the value is null? Or the value under key “Image” isn’t an object?

With this library, .ptrGetInt("/Image/Thumbnail/Width") will return a value if an integer is found at that path, or else it will return null.

Alternatively, .ptrOptInt("/Image/Thumbnail/Width") will return null only if there is a literal JSON null value at that address, and will throw an exception if no such address exists in the document.

Or, take it a step further using, .ptrReqInt("/Image/Thumbnail/Width"), which returns only Int and throws an exception if the address cannot be resolved or the value at that address is null.

val document = Json.parseToJsonElement(inputStr)
val thumbnailWidth : Int? = document.ptrGetInt("/Image/Thumbnail/Width")
val thumbnailHeight : Int? = document.ptrGetInt("/Image/Thumbnail/Height")