Lately, I have spent quite a bit of time familiarizing myself with Keycloak [1]. This is the first time I have worked thoroughly with the product, and it has been the source of a great deal of frustration. Mostly because it does not seem to be designed to play well with modern software development practices related to Infrastructure and Configuration as Code (IaC/CaC). Do not get me wrong, I do think that Keycloak is great, it just lacks a bit in some departments which makes it unnecessary difficult to deploy in a secure manor. Despite all its shortcomings, it does seem like it is possible to configure it in a satisfactory way, it is just completely undocumented how to do it. So in order to spare other developers from having to go through the same struggle as I had to go through, this article is a writeup on how this configuration is done.
Before we get started however, I want to make some disclaimers. This is by no means a comprehensive guide on how to secure your Keycloak instance, although by applying these tricks, you will have a great starting point for applying other best practices. Also, while I am quite happy with the result, I somehow feel like there should be an easier way to get there. So if you happen to know a better approach, feel free to get in touch via my socials and tell me how.
So, with that out of the way, let us talk about what we want to achieve. There are two main points I would like to have in place:
- Configuring Keycloak with code
- Limit the attack surface by limiting the number of high privileged users
Unfortunately, Keycloak does not out of the box support any of the above-mentioned requirements 😢. However, let us have a look at how we can setup our project to achieve the desired outcomes despite the lack of official support.
Configuring Keycloak with Code
When building software systems, it is common to make use of different environments for testing and production. In a basic case there are only these two environments, but there can of course also be some staging environment, UAT environment or dev environments in play as well. There is not really any limitation on how complicated one can make their setup of alternative environments. Add to that, the possibility of having ephemeral environments that e.g., are spun up in a pipeline to run tests. Maintaining all these environments does not scale well if one has to manually configure everything for every environment. And more importantly, it will highly likely result in configuration drift, i.e., environments will end up with a different configuration than intended because changes accumulate over time and there is no easy way to get a good overview.
Working around the issue about lacking CaC support can be done by introducing the third party tool keycloak-config-cli[2] which is an open source project especially designed for the purpose of configuring Keycloak using configuration files, either JSON or yaml and one nice thing about it is that it uses the same format as Keycloak realm export format. This is great as it is easy to get started. All you have to do is to start Keycloak in a docker container like:
docker run -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin quay.io/keycloak/keycloak:latest start-dev
then you can log in to the GUI (on localhost:8080) and sign in with the admin user (admin/admin), create a new realm and either export it directly or use click ops to make an initial configuration before exporting it. In the current UI while writing this, exports can be done from Realm settings > the “Action” dropdown (upper right corner) > Partial Export > Export. This will, if you did not change anything, give you a neat little json file of +2000 lines, containing the entirety of the configuration that makes up your newly created realm. Now we can take the export file, remove all UUIDs and give it to keycloak-config-cli, which will then be able to recreate the settings starting from a scratch installation. In a docker compose based setup, this might look something like this:
name: keycloak
services:
keycloak:
image: quay.io/keycloak/keycloak:latest
ports:
- "8080:8080"
environment:
KEYCLOAK_ADMIN: admin # Set the username of the admin user
KEYCLOAK_ADMIN_PASSWORD: admin # Set the password of the admin user
KC_PROXY: edge
KC_HOSTNAME_PORT: 8080
KC_HOSTNAME: localhost
KC_HEALTH_ENABLED: true
healthcheck:
test: ["CMD-SHELL", "exec 3<>/dev/tcp/127.0.0.1/9000;echo -e 'GET /health/ready HTTP/1.1\r\nhost: http://localhost\r\nConnection: close\r\n\r\n' >&3;if [ $? -eq 0 ]; then echo 'Healthcheck Successful';exit 0;else echo 'Healthcheck Failed';exit 1;fi;"]
interval: 15s
timeout: 2s
retries: 15
command: start
keycloak-config-cli:
image: adorsys/keycloak-config-cli:latest
environment:
KEYCLOAK_URL: http://keycloak:8080
LOGGING_LEVEL_ROOT: "debug"
KEYCLOAK_USER: admin # Use the admin user to perform actions
KEYCLOAK_PASSWORD: admin # Use the admin user password configured above
IMPORT_FILES_LOCATION: /config/* # Where keycloak-config-cli will look for config files
volumes:
- ./realm-export.json:/config/realm-export.json # Mount the realm export file into the container
depends_on:
keycloak:
condition: service_healthy
In this docker compose file, we configure Keycloak with the admin credentials and we pass the start command to the Keycloak container. This will pass the argument to kc.sh. We could for example also pass start-dev if we wanted.
Next, we configure the keycloak-config-cli-service, and we give it the admin credentials which we configured for the Keycloak admin account. This is of course because since the config cli tool will configure the entire realm, it needs to have the highest permissions. If we run the above docker compose file with:
docker compose up --build --force-recreate
and then sign into the Keycloak instance, we can see that the realm exists as we configured it, even between complete tear downs of the environment using docker compose down.
Disable the admin account
Now we know how to configure a realm with code using keycloak-config-cli. While it is a good start, the above approach still does not solve all our requirements. Firstly, the above approach only works to configure other realms than the master realm. This is because the master realm is special and Keycloak will not accept changes made by keycloak-config-cli. Secondly, applying the above approach require us to leave the admin account in place, which is problematic as anyone who gained access to this account would be able to completely control our Keycloak instance, and e.g., lock us out.
One approach to solving the admin account problem would be to configure the admin account so that it used a stronger authentication method than username/password, e.g., client certificate authentication. Unfortunately, this is not a solution for now as keycloak-config-cli does not currently support this authentication method.
Solving these problems are thus somewhat tricky because, as previously mentioned, Keycloak is not designed to configure the master realm as code. It is however possible to work around this, and we do that by building a custom Keycloak Docker image where we make use of Keycloak’s import command. We create a Dockerfile which looks like:
# Dockerfile
FROM quay.io/keycloak/keycloak:latest
WORKDIR /opt/keycloak
ENV JAVA_OPTS_APPEND="-Dkeycloak.migration.replace-placeholders=true" # Enable variable substitution
COPY ./master-realm.json conf/master-realm.json
COPY --chmod=0555 ./entrypoint.sh data/entrypoint.sh
ENTRYPOINT ["data/entrypoint.sh"]
the key here is that we are modifying the default behavior of the Keycloak image, to instead of starting Keycloak directly, we add our own config. An important line in this file is the line:
ENV JAVA_OPTS_APPEND="-Dkeycloak.migration.replace-placeholders=true"
which will enable variable substitution when importing a realm. By default, Keycloak only enables this functionality when starting and not when importing. Furthermore, we make this docker image run a custom entry point script which looks like this:
#!/bin/bash
/opt/keycloak/bin/kc.sh import --file "/opt/keycloak/conf/master-realm.json" --override true
/opt/keycloak/bin/kc.sh $@
I.e., it starts by importing the realm config before it passes the command, which was passed to the image, along to the kc.sh tool. This approach gives us full control to configure the master realm as code by modifying the content of master-realm.json.
The primary modification we want to make to the master realm is to disable the admin account. Disabling the master account is not that difficult, if we do not pass the KEYCLOAK_ADMIN environment variable to the container, the admin account will not be created. However, in its place, we need to have some other way for config-cli to configure our instance. Since the cli is a system user, we can create a client and configure its access via a client secret. This can be done by modifying the master-realm.json in two places. First, we create a user which we will give admin access (by giving it the admin realm role):
"users": [
{
"username": "service-account-config-cli",
"emailVerified": true,
"createdTimestamp": 1734969736972,
"enabled": true,
"totp": false,
"serviceAccountClientId": "config-cli",
"disableableCredentialTypes": [],
"requiredActions": [],
"realmRoles": [
"default-roles-master",
"admin"
],
"notBefore": 0,
"groups": []
}
]
we do not configure any access credentials for this user, but we say that the serviceAccountClientId will be "config-cli". This means that when we authenticate using that client, we will act using the above user. I.e., we will use the client secret from config-cli and we will get the permissions configured for the user service-account-config-cli, in this case, admin permissions.
We also need to configure the corresponding client. The easiest way to do it is by creating it in the Keycloak UI and export it. This will include a whole lot of configs, but most of it is not needed, and we can get pretty far with this minimal config:
{
"clientId": "config-cli",
"name": "",
"description": "",
"rootUrl": "",
"adminUrl": "",
"baseUrl": "",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "${CONFIG_SECRET}",
"notBefore": 0,
"bearerOnly": false,
"consentRequired": false,
"standardFlowEnabled": true,
"implicitFlowEnabled": false,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": true,
"publicClient": false,
"frontchannelLogout": true,
"protocol": "openid-connect",
"attributes": {},
"authenticationFlowBindingOverrides": {},
"fullScopeAllowed": true,
"nodeReRegistrationTimeout": -1,
"protocolMappers": []
}
where the most notable configs are:
"secret": "${CONFIG_SECRET}" - this will set the client secret to the value passed in the CONFIG_SECRET environment variable. Allowing us to inject it from a secret store, and not store it in clear text in git
"directAccessGrantsEnabled": false - disables the possibility to sign in using username/password
Note: That the variable substitution trick only works because we added the JAVA_OPTS_APPEND environment variable when we built the image. If it is not properly set, the secret will be the literal string ${CONFIG_SECRET} which would be a bit unfortunate.
With all the above pieces in place, we can update our docker compose file to look like:
name: keycloak
services:
keycloak:
build: .
ports:
- "8080:8080"
environment:
CONFIG_SECRET: ConfigCliSecret # The cli will use this value
KC_PROXY: edge
KC_HOSTNAME_PORT: 8080
KC_HOSTNAME: localhost
KC_HEALTH_ENABLED: true
healthcheck:
test: ["CMD-SHELL", "exec 3<>/dev/tcp/127.0.0.1/9000;echo -e 'GET /health/ready HTTP/1.1\r\nhost: http://localhost\r\nConnection: close\r\n\r\n' >&3;if [ $? -eq 0 ]; then echo 'Healthcheck Successful';exit 0;else echo 'Healthcheck Failed';exit 1;fi;"]
interval: 15s
timeout: 2s
retries: 15
command: start
keycloak-config-cli:
image: adorsys/keycloak-config-cli:latest
environment:
KEYCLOAK_URL: http://keycloak:8080
LOGGING_LEVEL_ROOT: "debug"
KEYCLOAK_CLIENTID: "config-cli"
KEYCLOAK_CLIENTSECRET: ConfigCliSecret
KEYCLOAK_GRANTTYPE: client_credentials
IMPORT_FILES_LOCATION: /config/*
volumes:
- ./realm-export.json:/config/realm-export.json
depends_on:
keycloak:
condition: service_healthy
The most notable differences are that we no longer set the KEYCLOAK_ADMIN and KEYCLOAK_ADMIN_PASSWORD variables to the keycloak service and instead pass the CONFIG_SECRET which will be the secret value that the keycloak-config-cli service will use. We also replaced the KEYCLOAK_USER and KEYCLOAK_PASSWORD variables to the keycloak-config-cli service with the variables:
KEYCLOAK_CLIENTID: "config-cli" - the id of the client that the cli will use to authenticate
KEYCLOAK_CLIENTSECRET: ConfigCliSecret - the secret value (in a production setup, this should probably come from a key vault somehow)
KEYCLOAK_GRANTTYPE: client_credentials - tells keycloak-config-cli to use the client_credentials grant type instead of the direct access grant type.
After we have done this, no one is able to log into the admin UI anymore. While this is very secure, we still might want the possibility to log in to the admin console if we need to debug something. Therefore, my suggestion is that you create another user, which can log into the console, but which only has read access.
Organizing the project
After going through the above steps, we can configure every aspect as code. However, the json export files are large and working with them does not scale well. Especially if we have many different realms that should have mostly the same configuration, with only small deviations. It will result in a lot of duplicated configs and it will be difficult to remember to update the config everywhere if something needs to change.
Ideally, one would like to be able to split the config into smaller components so that they could be included piece by piece into the main config. A structure like the below:
.
└── keycloak-config-cli/
├── clients/
│ ├── config-cli.yaml
│ └── clientB.yaml
├── users/
│ ├── user1.yaml
│ ├── user2.yaml
│ └── user3.yaml
├── roles/
│ ├── role1.yaml
│ └── role2.yaml
├── mappers/
│ ├── mapper1.yaml
│ └── mapper2.yaml
├── master-dev.yaml
├── master-test.yaml
└── master-prod.yaml
keycloak-config-cli has some support [3] for string substitution, however, it is too primitive to be of any real use when solving this problem. However, I have found that if the config files are rewritten to yaml, which keycloak-config-cli also supports, it is trivial to write a small python script or use some other yaml preprocessing tool to build the final configuration file. A tip here is that most LLMs does a good job with this, so you do not have to rewrite thousands of lines of json into yaml manually
The approach of building the config into a unified file before deploying has the additional benefit that the config which will be applied to Keycloak is explicitly listed inside an (albeit pretty big) file. Other approaches of splitting the config into smaller files can easily end up in a situation where there is conflicting configuration in different files, in which case the keycloak-config-cli will happily apply one of them in an unpredictable manor, making configuration errors hard to debug. In the worst case, one might end up with a much less secure configuration than intended, e.g., some dev configurations ending up in production.
Final words
After following along in this article, you should have acquired the possibility to configure every aspect of your Keycloak instance using code that can be checked into and tracked by a version control system. It is of course still possible to configure Keycloak into an insecure state, but at least with this approach, it will be explicit and can be easily changed across all your environments when it is discovered.
Finally, I want to again make the reflection that it feels like it should be easier to achieve what we have done in this article and if anyone happens to know how, please get in touch and tell me how to.