Connect - kafkata to other systems
OOB - best if code is not in our control
APIs, runtime
conn plugins - libraries executed- moves data
REST API - config n manage plugins
conn - tasks(parallel, resources)
worker process-conn data obj - tasks - src/sink
convertors - data stored in diff formats
prodn - sep servers from brokers
standalone, distri
boot, group.id(conn cluster), kv converter
conv - schema
mirror maker 2 - plugin
rebalance - conn del
src - 1 topic, sink - multi topics
conn - githb/custom
conn - how many tasks, split data btn tasks, worker -> config, task
tasks - in & out of kafka
worker - context - task init
worker - container proc, execs conns, tasks, handles http reqs, store conn config, start conn n tasks, auto comit offset, retries
data model, schema
------------------------
cross cluster data mirror - mm2
regionsal & central, redundancy, cloud mign
cross DC comm
1 cluster/DC
consume from remote than produce to remote
--
hub & spoke-multi local+ 1 central
active-active- 2+DCs share data, sync issues
active-passive-disaster - copy
failover - loss/dups
start after failover - begin/end, auto offset
---
stretch cluster - entrire DC fail
mirro btn 2 dcs
docker image
uber ureplicator
--
Sunday, 23 November 2025
Kafka The Definitive Guide Notes - Part 4
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment