[Feature Request] Allow ; as a character in TB_KAFKA_ADMIN_CONFIG

[BlessedDisco]

Is your feature request related to a problem? Please describe.
We would like to integrate TBMQ with our own Kafka setup. Our Kafka brokers require SASL authentication, which means we have to set i.e. sasl.jaas.config via TB_KAFKA_ADMIN_CONFIG. The value of sasl.jaas.config requires a semicolon at the end of the value to work (e.g. org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-secret";). Since the TB_KAFKA_ADMIN_CONFIG values are split on semicolon, we cannot successfully pass this environment variable.

Describe the solution you'd like
Allow semicolons within TB_KAFKA_ADMIN_CONFIG to be escaped (e.g. with a \), so that semicolons can be used in the value without being used to split.

Some examples on how TB_KAFKA_ADMIN_CONFIG should be parsed:

Simple example

k1:v1;k2:v2\;;k3:v3

will be interpreted by the Kafka consumer as

k1:v1
k2:v2;
k3:v3

Real life example

security.protocol:SASL_PLAINTEXT;sasl.mechanism:OAUTHBEARER;sasl.jaas.config:org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required clientId=\"exampleClient\" clientSecret=\"supersecret\"\;sasl.oauthbearer.token.endpoint.url:http://example.com/realms/Example/protocol/openid-connect/token

will be interpreted by the Kafka consumer as

security.protocol:SASL_PLAINTEXT
sasl.mechanism:OAUTHBEARER
sasl.jaas.config:org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required clientId=\"exampleClient\" clientSecret=\"exampleSecret\";
sasl.oauthbearer.token.endpoint.url:http://example.com/realms/Example/protocol/openid-connect/token

Describe alternatives you've considered

1. Using another separator character - might lead to problems with other configuration values
2. Setting JAAS configuration by passing a file to the JVM - required a custom docker container

Additional context
See https://docs.confluent.io/platform/7.0/kafka/authentication_sasl/index.html#client-jaas-configurations for more information on the JAAS configuration.

[BlessedDisco]

This issue is also relevant for all other TB_KAFKA_*_CONFIG variables, e.g. TB_KAFKA_HOME_PAGE_CONSUMER_CONFIG, TB_KAFKA_MSG_ALL_ADDITIONAL_CONSUMER_CONFIG and TB_KAFKA_MSG_ALL_ADDITIONAL_PRODUCER_CONFIG

[dmytro-landiak]

hi @BlessedDisco,

Thank you for the feature request! It seems promising at first glance, and we appreciate your input. We’ll review it in more detail and will follow up with an update on the implementation and estimated timeline for availability.

Thank you for helping us improve!