Client Setup for Authentication using SASL - Kerberos

Prerequisite:
1. A kafka cluster setup with Kerberos
2. A keytab file from the administrator

Note: This document shows a client implementation using the information provided in https://docs.confluent.io/2.0.0/kafka/sasl.html#configuring-kafka-clients


Steps
1. Copy the keytab file to resources folder. (In case of production deployment, the file must be managed outside of the project repository)

2. Add the following properties to application.yml

spring:
    kafka:
      bootstrap-servers: # make sure secured port is used      properties:
        schema.registry.url: # make sure secured port is used for e.g. https://abc.def.net:8081        security.protocol: SASL_SSL
        sasl:
          mechanism: GSSAPI
          kerberos:
            service:
              name: kafka

3. Create a jaas.conf file as follows

KafkaClient {
    com.sun.security.auth.login.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    keyTab="./src/main/resources/filename.keytab"    principal="prinicipal_name"    # example of principal - "xyz/a.b.net@abc.def.net"}

4. Add configuration file krb5.conf to the resources folder
Files in step 3,4 can be added to the system path of the application as follows
-Djava.security.auth.login.config=./src/main/resources/jaas.conf
-Djava.security.krb5.conf=./src/main/resources/krb5.conf

Alternate approach to step 4 - (in a spring-boot application). Add a configuration as follows

@Configurationpublic class SecurityConfig {
    @Value("${app.kafka.security.authLoginConfig}")
    private String authLoginConfig;
    @Value("${app.kafka.security.krb5Realm}")
    private String krb5Realm;
    @Value("${app.kafka.security.krb5Kdc}")
    private String krb5Kdc;
    @PostConstruct    public void configureKafkaSecurity() {
        if (!StringUtils.isEmpty(authLoginConfig)) {
            System.setProperty("java.security.auth.login.config", authLoginConfig);        }
        if (!StringUtils.isEmpty(krb5Realm)) {
            System.setProperty("java.security.krb5.realm", krb5Realm);        }
        if (!StringUtils.isEmpty(krb5Kdc)) {
            System.setProperty("java.security.krb5.kdc", krb5Kdc);        }
    }
}

with following properties in applcation.yml

app:
  kafka:
    security:
      krb5Realm: abc.def.NET
      krb5Kdc: abc.def.net
      authLoginConfig: ./src/main/resources/jaas.conf

Setting Up Kafka using Docker Compose

Pre-requisites -
1. Docker
2. Git

Note- This document refers to material presented in  https://docs.confluent.io/current/quickstart/ce-docker-quickstart.html. It only includes the steps required to get started with Kafka development

Steps
1. Clone the Confluent Platform Docker Images GitHub Repository and check out the 5.4.1-post branch. (or whichever is the latest branch)

git clone https://github.com/confluentinc/examples
cd examples
git checkout 5.4.1-post

2. Navigate to cp-all-in-one examples directory.

cd cp-all-in-one/

3. This will have the docker-compose.yml file. Take a backup of file. And remove section and reference related related to connect, ksql-server, ksql-cli, ksql-datagen.  Here is the updated docker-compose.yml file.


4. Navigate to the folder where you have the updated docker-compose.yml

docker-compose up  --> This will download the components and start zookeeper, kafka Broker, schema-registry and control center. Wait for all the components to start successfully.

If the components are already downloaded, 
 docker-compose start --> To start the service 

5. Run the following command to verify the services

docker ps

It will appear as follow

The following url's can be used to access the schema registry and control center
Schema Registry : http://localhost:8081
Control Center : http://localhost:9021

6. To stop the service  --> docker-compose stop

To Stop and remove the service --> docker-compose down