A lot of AppStore modules, and probably your own modules too, require configuration to run properly in your application. A simple example is the AppStore SMTP Email module which needs to know how to access your SMTP server otherwise it cannot do what it is supposed to do. In this module and many other modules this is solved by allowing you to store that configuration data in the database and you can adjust these settings while the application runs.

As the data is stored in the database and it requires an administrator or developer to configure the modules it is a manual process that can introduce errors. And when the deployment documentation is not properly maintained it is hard to know what the desired configuration looks like. It becomes hard to deploy or recover an instance and have it up and running with little effort in configuration.

There are many cases but to illustrate my best practice I will use the AppStore module SMTP Email module as an example here. I prefer to use a mail server for the production instance and another one for all other environments. Consider this case: I have a copy of the production data on the acceptance server and I did not know that I had to make certain configuration adjustments. Due to testing the application, emails are sent to users. Real users! So my real production users receive emails from the acceptance server? This must be prevented at all time and separating services or building in mechanisms to prevent such things are an operational necessity.

Hard code configuration data

Theoretically hard coding configuration data is an option, but least preferred. Hard coding data is per definition a last resort. It is hard to manage and it takes a lot of effort when the data is more complex than just a string or integer. If you have to deal with an API key or decryption key that you want to hide as good as you can, it is an option; not perfectly secure but it helps.

Link configuration data to your environment

I highly recommend to enhance the modules to link their configuration data to environments. Per environment your application can figure out where it runs. Is it a production server, acceptance server, developer laptop, etc. It does not matter what such identification looks like, as long as you make it unique per instance or group of instances. When your application knows where it runs then it can use the configuration data of that environment.

First you define a single constant called Environment where you modify the value per environment to your need. Set the default value to a non-production environment or to a dummy value, see the example below, to prevent out of the box deployments to behave like production.

Now we have to enhance AppStore modules to make this work. Do not change AppStore modules but add a new module and name it AppStoreModuleCustomized so for the SMTPEmailModule module create module SMTPEmailModuleCustomized. Document the version of the original AppStore module so you know afterwards where it was based upon.

In the new module SMTPEmailModuleCustomized add an entity like the screenshot below. It shows a specialization of the original EmailSettings entity where a String attribute Environment is added. This approach prevents making changes to the original module which helps to upgrade the AppStore module in the future.

Additionally, but that is too much for this blog post, you have to make sure that the processes of the original module are using the new environment attribute. Add changes to the new module, and do not change the AppStore module except for excluding superseded pages, microflows and scheduled events to prevent them from being used. For the SMTPEmailModule example the next changes should be made:

• Add management of EmailSettings based on the specialized EmailSettings entity with the new Environment attribute
• Add a scheduled event to send queued e-mails using the proper EmailSettings object
• Add a microflow to send e-mails from other microflows where it uses the proper EmailSettings object

Too bad Mendix does not offer proper inheritance as available in OO languages. This is more or less what we are trying to achieve here. Use the AppStore module as a basis and override parts of it.

When developers use the new microflows then this should work fine. One final hint: when the EmailSettings object for the current environment cannot be found, do not create a new one with default values, proceed and see what happens. At least log a Critical error to help notify developers or operations that things are not going as expected.

You can consider the next choice: allow multiple objects, one per environment, in your database where you typically expect one. The UI of manually changing EmailSettings expects a single object, but you could decide to use a ListView or TemplateGrid to show all available objects. Use validation on save to prevent duplicates per environment.

Import configuration data

When you implemented the recommendations in the paragraph above then you are in a much better position to prevent making operational mistakes.

To take this to the next level:

Store configuration data with your application and import it programmatically into the database instead of relying on a human being to add this manually via management pages.

The configuration data is defined once per environment or per use and the amount of changes to the data are limited. And if it has to change then it is changed by a developer. You could even remove the administration pages to prevent people bypassing the process.

A Mendix directory structure has a folder called resources. It contains folders and files belonging to specific modules. So why not add your own folders and store files with configuration data? A developer manages the content of the files and an import process reads them and populates the appropriate database tables. Changes to those files are made by a developer and changes are committed to SubVersion/TeamServer so you see who changed what over time.

In this post I use JSON as file format. See the example below where you see an array where an EmailSettings object is available for environments Production and DevPaul. The module supports encrypted passwords so you can put the encrypted passwords in this file.

[{
    "Environment": "Production",
    "EmailSettings": {
        "Server": "localhost",
        "Port": 25,
        "UseSSL": false,
        "UseTLS": false,
        "UserName": null,
        "Password": null,
        "From": "from@domain.tld",
        "MaxRetry": 5
    }
},
{
    "Environment": "DevPaul",
    "EmailSettings": {
        "Server": "smtp.gmail.com",
        "Port": 587,
        "UseSSL": true,
        "UseTLS": false,
        "UserName": "my-user@gmail.com",
        "Password": "--my-encrypted-password--",
        "From": "from@domain.tld",
        "MaxRetry": 2
    }
}]

When importing this JSON file you can import directly into the target structure or use an intermediate structure. I prefer the latter to support manipulating data while processing it and to be independent on (future changes to) the target structure; so I want to be able to make my own choices.

All AppStore modules have their own unique non-standardized structures and tools and here is your opportunity to standardize your way of managing configuration data.

Now we have a JSON file and a target structure. What steps to take?

1. First of all, with target structure I mean a result like that in the previous paragraph. So you have to be able to know what environment the data is for. You could import the proper data and put it in the original structure without changes (as per the previous paragraph). But then you trust that the configuration data in the database is up to date immediately; so copying your production database to acceptance is still risky.
2. Add a JSON document and an import mapping to your application model.
3. Create a microflow that knows where to find the JSON file and use the import mapping to import it. Process the imported data properly to get the desired result.

The data model where the imported EmailSettings data is temporarily stored looks like:

Here an example of an import microflow:

It follows these steps:

1. Initialization to know what the current environment is, where the file(s) can be found and which log node to use when logging progress and errors.
2. Delete any existing configuration data to ensure that the import replaces current data. And when the import fails we are sure that there’s no unwanted configuration data. This action has pros and cons and decide for yourself if you want this.
3. Import the JSON file into a FileDocument object.
4. Apply the import mapping to transform the file into data. Delete the FileDocument object because we do not need it anymore.
5. Find the Environment object that matches ours. When found retrieve the EmailSettings associated to it.
6. Store the EmailSettings in the target structure.

It takes some effort to set this up once per AppStore module or your own module, but you will get the benefit later. This is all about managing and controlling your application configuration and you have to take that serious.