Dynamic server configuration management — Spring Cloud Config

Spring Cloud Config Server/Client with S3, Git, and Vault as the source for configuration

Challenge/Problem:

Be it a Monolithic service or microservice, services have to be restarted to apply any configuration changes. This is a challenge and has to be done carefully considering the impact it has on active users. Restarting a service may take a few seconds to a few minutes depending on the size of the service. During this period these services will be inactive and doesn’t respond to any traffic. To have a smooth transition to the new configuration, various deployment strategies can be followed(Blue/Green, Canary, Ramped/Rolling update). But these require additional engineering work that incurs an extra cost. To avoid that, the best way is to update the server configuration dynamically without restarting it. This can be done using Spring Cloud Config Server.

Solution:

Above is the diagrammatic representation of Spring cloud config server, client and configuration sources(Git, S3, Vault). Lets now dive deeper and see how to create an application with the shown setup above.

Configuration Sources Setup:

Git
For this scope of application, local git repository is used.
Create a file with configuration properties as below and name it as my-application-dev.properties

folderPath = /users/documents/config — This is from git file — default

my-application is the application name used in the Spring Config Client application. Spring Config Client uses this name to locate the config properties. dev is the profile name. Ex. Dev, prod, stage,…

Once the file is created, add it to a git repository.

To initialize the folder containing the above file as git,

git init

Add the file to git and commit the code,

git add .
git commit -m "add my-application properties file"

With the above steps, the configuration properties to be served from git is ready.

S3:
Create a file with configuration properties and name it as my-application-dev.properties .

hostName = localhost - This is loaded from s3 - dev

Upload this configuration file to an S3 bucket.

Vault:
Here’s the link to get started with the vault setup.

Once the setup is done, start the vault server.

vault sever -dev

Note: Do not use -dev for production applications.

Now, to add secrets to vault server,

vault kv put secret/my-application,dev secretKey="jQweR23CvbsM - This is loaded from Vault - dev"

my-application is the application name and dev is the profile name.

Vault setup is done and secrets(configuration) are added to vault.

Configuration Sources git, s3 and vault are now ready with configurations for the application my-application with the profile name dev.

Configuration Server Setup

Below are the dependencies to be added in pom.xml in your maven project.

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>

<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
</dependency>

Add these as bootstrap.properties

# active profiles to use
spring.profiles.active=vault,git,awss3

# git base config
spring.cloud.config.server.git.uri=#git-repo-location
spring.cloud.config.server.git.clone-on-start=true

# vault base config setup
spring.cloud.config.server.vault.kv-version=2
spring.cloud.config.server.vault.host=localhost
spring.cloud.config.server.vault.port=8200

# S3 base config setup
spring.cloud.config.server.awss3.region=us-east-1
spring.cloud.config.server.awss3.bucket=cloud-config-arvind

Bootstrap.properties(.yml/.json) is loaded before application.properties (.yml/.json). Bootstrap.properties is loaded by a parent Spring ApplicationContext, which is loaded before the one that uses application.properties.

Spring uses AWS credentials to access the specified S3 bucket. Credentials are found using Default AWS Credentials Provider Chain.

Add @EnableConfigServer annotation to the main method of spring boot application.

@EnableConfigServer
@SpringBootApplication
public class SpringCloudConfigServerApplication {

public static void main(String[] args) {
SpringApplication.run(SpringCloudConfigServerApplication.class, args);
}

}
Configuration Client(s) Setup 

Below are the dependencies to be added to pom.xml in your maven project.

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>

bootstrap.properties

spring.cloud.config.uri=http://localhost:8888

spring.profiles.active=dev

#this is needed for vault
spring.cloud.config.token=#vault-token-goes-here

application.properties

management.endpoints.web.exposure.include=*
management.security.enabled=false;

# name of the application - this is used to fetch from configuration sources - git, s3, vault
spring.application.name=my-application

Once the initial setup with the client application is done, the next part is how to use the configuration values in the application?

ConfigModel.java

@Value annotation is used to access the variable from the environment.

@RefreshScope
@Configuration
public class ConfigModel {

@Value("${hostName: Error loading from aws s3...}")
private String hostName;

@Value("${secretKey: Error loading from vault...}")
private String secretKey;

@Value("${folderPath: Error loading from git file...}")
private String folderPath;

public String getHostName() {
return hostName;
}

public String getSecretKey() {
return secretKey;
}

public String getFolderPath() {
return folderPath;
}
}

To test whether the values are loaded from sources correctly, a rest endpoint is created and used to return all the values as JSON.

@RestController
public class SampleApi {

@Autowired
ConfigModel configModel;

@GetMapping("/variable/all")
public HashMap<String, String> getAllEnvVariable() {

HashMap<String, String> map = new HashMap<>();
map.put("hostName", this.configModel.getHostName());
map.put("folderPath", this.configModel.getFolderPath());
map.put("secretKey", this.configModel.getSecretKey());

return map;
}
}

In the API, ConfigModel is autowired and the values are accessed using it.

Now, all the code setup is done and its time to test the entire setup working.

  1. Start the Config Server. This runs on port 8888.
  2. Start the Config Client application. This runs on port 8080.
  3. Make a GET API call to http://localhost:8080/variable/all using postman or any other rest client.

Output:

Now, we are going to test how the application configuration is modified without the need to restart the server.

  1. Change the property values in configuration sources(git, s3, vault)
  2. Commit the changes to repository in-case of git, update the s3 files and vault secrets.
  3. To apply the configuration changes to the client, call the refresh rest api provided by actuator using post method. http://localhost:8080/actuator/refresh
    This will return a JSON of updated configuration keys as shown below.

4. Make a GET call to http://localhost:8080/variable/all and notice the updated configuration values.

What happens when the refresh endpoint is called?
A proxy object is created and dynamically redirects to the new proxy object.

Here’s the link to the Github repository containing this project. 
You can download the source and get the setup running easily by just setting up configuration sources.

Clap

Leave A Comment