24. Function to component mapping¶
Technical Story: Issue #713
24.1. Context and Problem Statement¶
To implement the EAI patterns we use a combination of a generic component which handles the communication with Kafka and a FaaS solution. The business logic of the EAI patterns (message splitting/aggregation or transformation) is provided via functions which are hosted on the FaaS solution. The generic component communicates with Kafka and de/serializes the messages. We need a means to wire the instances of the generic component with the functions. E.g. A user wants to insert a message splitter between two message-based components. To realize this an instance of the generic component in combination with a splitting FaaS function will be used. The generic component needs the address of the FaaS gateway and the function name (e.g. http://address:8080/function/msg-payload-splitter) to call the function. To provide the necessary information to instances of the generic component we considered the following techniques.
24.2. Decision Drivers ¶
- MUST be supported by the language/technology which is used to implement the generic component
- MUST be easy to integrate into MICO
- SHOULD be a well known and proven solution
24.3. Considered Options¶
- Environment variables
- Function registry
- Control Topic
- Function composition
- Function proxy
- Configuration file
24.4. Decision Outcome¶
Chosen option: “Environment variables”, because MICO already supports this and it is easy to implement in the generic component.
24.5. Pros and Cons of the Options¶
24.5.1. Environment variables¶
This option uses environment variables to store the necessary information. These variables are controlled by MICO.
- Good, because MICO already supports setting environment variables
- Good, because most languages support reading environment variables
- Good, because we need environment variables anyway because we need to provide the Kafka configuration
- Good, because it is a well known and proven solution
- Good, because it follows the (twelve-factor app methodology)[https://12factor.net/de/config]
- Bad, because the configuration can not be changed on the fly (do we want this?)
24.5.2. Function registry¶
We implement a separate component which stores the mapping of the functions to the instances of the generic component. We could evaluate if (Apache ZooKeeper)[https://zookeeper.apache.org] or (etcd)[https://github.com/etcd-io/etcd] fits this purpose.
- Good, because the mapping can change in the fly
- Good, because there is support for this approach in most languages
- Bad, because more development/evaluation overhead
24.5.3. Control Topic¶
We already use Kafka so we could follow the dynamic router pattern and use a control channel(topic) to configure the instances. This topic could use the compaction feature to always retain the last known configuration for each message key.
- Good, because we already decided to use Kafka
- Good, because this option provides a log of configuration changes
- Bad, because the configuration for Kafka could not be provided via this method
- Bad, because more development/evaluation overhead
24.5.4. Function composition¶
We don’t implement the generic component as a separate component but as a function which is hosted on the FaaS solution. There is a Kafka connector for OpenFaaS. Then compose the functions with a composer like this or follow the workflow techniques described in the OpenFaaS documentation.
- Good, because the basic components for this approach already exist
- Good, because no separate generic component is necessary
- Bad, because the composer does not seem to be very mature and proven
24.5.5. Function proxy (API Gateway)¶
Each instance of the generic component calls the API gateway which handles the routing/mapping to the functions.
- Good, because each instance of the generic component calls the same gateway and therefore needs the same configuration
- Bad, because we don’t have this proxy and therefore this adds development overhead
24.5.6. Configuration file¶
Generate a configuration file per instance of the generic component and deploy it with the instance. Same pros and cons like environment variables but more complexity for file generation/parsing and deployment.