Consumer group reset offset

[hemahg]

Consumer group reset offset

consumer group reset offset page

Dryrun page

[MikeEdgar]

@hemahg , I noticed that the topics were not appearing for empty consumer groups. This is because the members will also be empty. However, the offsets can also be used to obtain the topic information as long as offsets have been committed for a topic/partition. This might improve the experience in those cases. I suggest defining allTopics[1] like this:

const allTopics: { topicId: string; topicName: string; }[] = [];
row.attributes.members?.
    flatMap((m) => m.assignments ?? []).
    forEach(a => allTopics.push({ topicId: a.topicId, topicName: a.topicName }));
row.attributes.offsets?.
    forEach(a => allTopics.push({ topicId: a.topicId, topicName: a.topicName }));

[1] https://github.com/streamshub/console/pull/960/files#diff-b4f415e287dedebced21247b153a6aad3642bf3edd6355587cdddd93065866f0R192-R193

[MikeEdgar]

@hemahg the API doesn't currently support dry-run operations for the consumer group reset. We can add it in this PR or a separate one, what do you think?

[riccardo-forina]

Changes look good. Please do a round of prettify on the changed files, rebase, update the test that’s failing (I didn’t check which one it is), and we can move this to the dev end and test it on the field

[MikeEdgar]

Can we remove the delete button and make the reset offset button usable? Otherwise, both should be removed (or hidden for now).

[MikeEdgar]

A few additional comments.

Another problem that I am having is that the members table does not update after resetting the offsets. I suspect this is a similar problem to the consumer group list page where the initial value when navigating to the page are always out of date and do not update until the refresh function runs. My guess is that this is somehow due to server-side rendering never being updated, but maybe we can figure it out and fix it with this PR? Something for sure will need to be done about the members page because it never updates currently.

cc: @hemahg @riccardo-forina

[MikeEdgar]

This should be the same schema as ConsumerGroupResponse. It looks like all the optional fields are already correctly set in ConsumerGroupResponse. Can we use a single schema definition instead?

[hemahg]

I checked the ConsumerGroupSchema before implementing it and noticed it contains several fields that are not required for the dry run. For example:

typescript
Copy code
offsets: z.array(OffsetAndMetadataSchema).optional()

const OffsetAndMetadataSchema = z.object({
    topicId: z.string(),
    topicName: z.string(),
    partition: z.number(),
    offset: z.number(),
    logEndOffset: z.number().optional(),
    lag: z.number().optional(),
    metadata: z.string(),
    leaderEpoch: z.number().optional()
});

Fields like offset, logEndOffset, and lag are not needed for the dry run, so I created a new response schema for dryrun

[MikeEdgar]

Right, the dry run will return a subset of the data that you would get from the describe/fetch operator. The back-end API uses the same data model for both. Just mentioning it because it might help reduce maintenance in the future and make it clear that they are the same response schema from the API.

[hemahg]

@MikeEdgar, can we merge this PR? If there are more updates I will do them in another PR

[MikeEdgar]

@hemahg I'm doing another round of testing and I'll merge before EOD.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud