How to make partition and replication factor on target cluster with MM2.

[ervikrant06]

Hello,

I am using MM2 configuration. On source cluster my topic is created with 3P/3R after mirroring on target cluster topic is created with target kafka cluster default settings (4P/2R) not with actual topic setting of source cluster. Do we need to take care of this manually?

$ kubectl get kafkatopic -n ns-strimzi-poc | egrep 'NAME|test-app-topic4'
NAME                                                                                               CLUSTER      PARTITIONS   REPLICATION FACTOR   READY
test-app-topic4                                                                                    test-kafka   3            3                    True

$ kubectl get kafkatopic -n ns-strimzi-poc1 | egrep
 'NAME|test-app-topic4'
NAME                                                                                               CLUSTER       PARTITIONS   REPLICATION FACTOR   READY
test-app-topic4                                                                                    test-kafka1   4            2                    True

[ervikrant06]

@scholzj Hello,

I re-ran the whole excercise again. It reflect the correct partition count but not replica count.

Topic spec on source cluster.

  partitions: 6
  replicas: 2
  topicName: graphite-metrics

Topic spec on destination cluster after mm2 replication

$ kubectl get kafkatopic graphite-metrics
NAME               CLUSTER      PARTITIONS   REPLICATION FACTOR   READY
graphite-metrics   test-kafka   6            1                    True

Two places where I believe replication-factor could have been altered.

From MM2 conf:

    sourceConnector:
      tasksMax: 5
      config:
        replication.factor: 1
        offset-syncs.topic.replication.factor: 1
        sync.topic.acls.enabled: "false"
        replication.policy.separator: ""
        replication.policy.class: "org.apache.kafka.connect.mirror.IdentityReplicationPolicy"
        offset-syncs.topic.location: target

From the cluster deployment

  kafka:
    authorization:
      type: simple
    config:
      allow.everyone.if.no.acl.found: false
      default.replication.factor: 1

Would it right to assume it doesn't pick the replication factor from source cluster topic?

[scholzj]

AFAIK this is intentional ... MM2 cannot mirror the replication factor. It does not know how many nodes your cluster has. It does not know how to change the RF if needed etc.

So you have to do that your self. Ideally, most of your topics should follow your clusters defaults for example (that means the default set in the target cluster).