What are the core concepts covered in kotlinx.serialization: Custom Serializers & Protobuf Streaming?

The architecture of `kotlinx.serialization`: Descriptors, Encoders, Decoders, and Serializers; Why `kotlinx.serialization` uses compile-time code generation instead of slow JVM reflection (Jackson/Gson); Authoring custom `KSerializer ` implementations with fine-grained error handling; Ultra-compact binary serialization with `ProtoBuf.encodeToByteArray`

Advanced 26 min readModule: Module 15: High-Performance Serialization: Decoders & Protobuf

kotlinx.serialization: Custom Serializers & Protobuf Streaming

Author high-throughput binary and text encoders: the `KSerializer<T>` interface, `SerialDescriptor` metadata trees, implementing custom non-blocking streaming `CompositeDecoder`, and high-speed binary Protocol Buffers (`ProtoBuf`).

What You Will Learn in This Lesson

The architecture of `kotlinx.serialization`: Descriptors, Encoders, Decoders, and Serializers
Why `kotlinx.serialization` uses compile-time code generation instead of slow JVM reflection (Jackson/Gson)
Authoring custom `KSerializer<T>` implementations with fine-grained error handling
Ultra-compact binary serialization with `ProtoBuf.encodeToByteArray`

Introduction & Core Concept

Traditional JVM serialization libraries (such as Jackson and Gson) rely heavily on runtime reflection to inspect fields and construct objects, consuming CPU and generating large amounts of temporary garbage. `kotlinx.serialization` uses a Kotlin Compiler Plugin to generate type-safe serializer implementations at compile time, enabling zero-reflection serialization for JSON, Protocol Buffers, and CBOR across all Kotlin platforms.

WHY DOES THIS MATTER IN THE REAL WORLD?

High-frequency microservices and mobile network requests serialize millions of payloads per second. Compiled Protocol Buffers serialization is 10x faster and produces payloads 70% smaller than JSON.

Syntax & Structure

kotlin

@Serializable
data class Packet(val id: Int, val payload: ByteArray)
val bytes = ProtoBuf.encodeToByteArray(Packet.serializer(), packet)

Custom KSerializer for Instant and Protocol Buffer Encoding

kotlin

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
// High-Performance Custom KSerializer & Binary Protocol Buffers
package com.kwasacademy.serialization
 
import kotlinx.serialization.*
import kotlinx.serialization.builtins.ByteArraySerializer
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encoding.*
import kotlinx.serialization.protobuf.ProtoBuf
import java.time.Instant
 
// 1. Custom Serializer for java.time.Instant (Serializes as Epoch Milliseconds Long)
object InstantEpochSerializer : KSerializer<Instant> {
    override val descriptor: SerialDescriptor =
        PrimitiveSerialDescriptor("InstantEpoch", PrimitiveKind.LONG)
 
    override fun serialize(encoder: Encoder, value: Instant) {
        encoder.encodeLong(value.toEpochMilli())
    }
 
    override fun deserialize(decoder: Decoder): Instant {
        val epochMs = decoder.decodeLong()
        return Instant.ofEpochMilli(epochMs)
    }
}
 
// 2. High-Performance Telemetry Packet
@Serializable
data class TelemetryEvent(
    val eventId: Long,
    val sensorName: String,
    @Serializable(with = InstantEpochSerializer::class)
    val timestamp: Instant,
    val readings: List<Double>
)
 
fun main() {
    println("=== kotlinx.serialization: Custom Serializers & ProtoBuf ===")
 
    val event = TelemetryEvent(
        eventId = 90210L,
        sensorName = "GPU_TEMPERATURE_CORE_0",
        timestamp = Instant.now(),
        readings = listOf(68.5, 70.2, 69.8)
    )
 
    // 3. Ultra-compact Binary Protocol Buffers Serialization (Zero Reflection!)
    val protoBytes: ByteArray = ProtoBuf.encodeToByteArray(TelemetryEvent.serializer(), event)
    println("ProtoBuf Encoded Binary Size: ${protoBytes.size} bytes (Extremely compact!)")
 
    // 4. Binary Deserialization
    val decodedEvent = ProtoBuf.decodeFromByteArray(TelemetryEvent.serializer(), protoBytes)
    println("Decoded Event ID: ${decodedEvent.eventId} | Sensor: ${decodedEvent.sensorName}")
    println("Decoded Timestamp: ${decodedEvent.timestamp}")
    println("✅ Custom serializer and ProtoBuf executed with 100% compile-time type safety!")
}

Line-by-Line Technical Breakdown

1CompositeDecoder Streaming: When deserializing complex objects, `decodeElementIndex(descriptor)` returns the index of the next field in the stream. This allows non-blocking parsing of fields that arrive out-of-order in JSON or ProtoBuf streams without buffering the entire payload in RAM.

Try It Yourself (Interactive Editor)

Modify the code in real-time and click Run to test live browser output and console logs.

Intelligent Code Runner & Live Sandbox[KOTLIN]

KOTLIN SOURCE EDITOR

Interactive Live Code

// High-Performance Custom KSerializer & Binary Protocol Buffers
package com.kwasacademy.serialization

import kotlinx.serialization.*
import kotlinx.serialization.builtins.ByteArraySerializer
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encoding.*
import kotlinx.serialization.protobuf.ProtoBuf
import java.time.Instant

// 1. Custom Serializer for java.time.Instant (Serializes as Epoch Milliseconds Long)
object InstantEpochSerializer : KSerializer<Instant> {
    override val descriptor: SerialDescriptor =
        PrimitiveSerialDescriptor("InstantEpoch", PrimitiveKind.LONG)

override fun serialize(encoder: Encoder, value: Instant) {
        encoder.encodeLong(value.toEpochMilli())
    }

override fun deserialize(decoder: Decoder): Instant {
        val epochMs = decoder.decodeLong()
        return Instant.ofEpochMilli(epochMs)
    }
}

// 2. High-Performance Telemetry Packet
@Serializable
data class TelemetryEvent(
    val eventId: Long,
    val sensorName: String,
    @Serializable(with = InstantEpochSerializer::class)
    val timestamp: Instant,
    val readings: List<Double>
)

fun main() {
    println("=== kotlinx.serialization: Custom Serializers & ProtoBuf ===")

val event = TelemetryEvent(
        eventId = 90210L,
        sensorName = "GPU_TEMPERATURE_CORE_0",
        timestamp = Instant.now(),
        readings = listOf(68.5, 70.2, 69.8)
    )

// 3. Ultra-compact Binary Protocol Buffers Serialization (Zero Reflection!)
    val protoBytes: ByteArray = ProtoBuf.encodeToByteArray(TelemetryEvent.serializer(), event)
    println("ProtoBuf Encoded Binary Size: ${protoBytes.size} bytes (Extremely compact!)")

// 4. Binary Deserialization
    val decodedEvent = ProtoBuf.decodeFromByteArray(TelemetryEvent.serializer(), protoBytes)
    println("Decoded Event ID: ${decodedEvent.eventId} | Sensor: ${decodedEvent.sensorName}")
    println("Decoded Timestamp: ${decodedEvent.timestamp}")
    println("✅ Custom serializer and ProtoBuf executed with 100% compile-time type safety!")
}

>[KOTLIN Runtime Initialized]

>Program compiled and executed with 0 errors.

>Output generated successfully.

Common Mistakes & How to Avoid Them

#1: Using reflection-based serializers (Gson/Jackson) in Kotlin Multiplatform, causing compilation failures on iOS and WebAssembly.

Kotlin/Native has no JVM reflection engine. `kotlinx.serialization` is required for true multiplatform serialization.

Incorrect / Antipattern

Gson().toJson(model) // Fails on Kotlin/Native (no JVM reflection available)

Correct / Professional Solution

Json.encodeToString(model) // Works universally across JVM, iOS, and JS

Industry Best Practices & Professional Standards

Use `ProtoBuf` format for internal microservice RPC and WebSocket telemetry.
Use `@Serializable(with = CustomSerializer::class)` for third-party classes you do not own.
Configure `Json { ignoreUnknownKeys = true; coerceInputValues = true }` for resilient APIs.

Lesson Summary & Core Takeaways

`kotlinx.serialization` provides compile-time code-generated serializers with zero reflection.
`KSerializer<T>` enables complete control over data encoding and schema descriptors.
Protocol Buffers format produces ultra-compact binary streams for high-speed networks.