Fixing ci flakyness issue in gha

[louis-tw]

Context

Kafka producer instantiation will be attempted up to 5 times with a 500ms delay between each attempt. In some cases, it has been observed that the CI fails to start the Kafka producer because the kafka docker container itself seems to not be fully up & accessible yet.

Behind the last part of the stacktrace:

    Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
    	at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:89)
    	at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:48)
    	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:439)
    	... 162 common frames omitted

I found out after reading kafka's code that this is due to the url being not reachable due to a mere timeout. Thus, a simple retry and wait should fix the root cause of this flakyness.

Stacktrace in CI:

    Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'twTasksTestTasksService': Unsatisfied dependency expressed through field 'tasksExecutionTriggerer': Error creating bean with name 'twTasksTasksExecutionTriggerer' defined in class path resource [com/transferwise/tasks/core/autoconfigure/TwTasksCoreAutoConfiguration.class]: Failed to construct kafka producer
    	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:771)
    	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:751)
    	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:145)
    	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:492)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1416)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:597)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
    	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254)
    	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1417)
    	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1337)
    	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:910)
    	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
    	... 131 common frames omitted
    Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'twTasksTasksExecutionTriggerer' defined in class path resource [com/transferwise/tasks/core/autoconfigure/TwTasksCoreAutoConfiguration.class]: Failed to construct kafka producer
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
    	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254)
    	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1417)
    	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1337)
    	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:768)
    	... 146 common frames omitted
    Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka producer
    	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:465)
    	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:290)
    	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:273)
    	at com.transferwise.tasks.triggering.KafkaTasksExecutionTriggerer.createKafkaProducer(KafkaTasksExecutionTriggerer.java:497)
    	at com.transferwise.tasks.triggering.KafkaTasksExecutionTriggerer.afterPropertiesSet(KafkaTasksExecutionTriggerer.java:137)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1817)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1766)
    	... 156 common frames omitted
    Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
    	at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:89)
    	at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:48)
    	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:439)
    	... 162 common frames omitted

Checklist

• Change meets or does not compromise the Baseline Security Requirements

[lars-janssen-wise]

LGTM

[wise-github-bot-app]

The approval(s) from lars-janssen-wise do(es)n't fullfill the approvers requirements because:

• The approver's cost centre, ENGCJT, maps to the ENGINEERING business function. As the code that was changed is owned by PLATFORM, this approval won't satisfy our separation of duties check. We'll need an additional approval from someone in PLATFORM. This approval may still help satisfy other codeowner requirements.