Working with GlobalKTable

KTables are the way to go when working with state in Kafka Streams. Basically they are a materialized view on a topic where every message is an upsert of the last record with the same key. (Read duality of streams and tables).

KTables eventually contains every change published to the underlying topic.

Normal KTables only contain the data of the partitions consumed by the Kafka Streams application. If you run N instances of your application, a KTable will contain roughly total entries / N entries. Every instance consumes a different set of Kafka partitions resulting in different KTable content.

If you’d like to have the same state in every instance, you’ll have to use a GlobalKTable. Its API is very similar to KTable but it contains the full state. So every instance of your application will read all partitions of the underlying topic and thus eventually have the same state.

Scenario’s to consider a GlobalTable over a normal one:

Some remarks when you work with GlobalKTables:

    .map((k, v) -> new KeyValue(v.getId(), v))