In earlier articles, I introduced the concept of Design-time Configuration Items. The final missing element is the composition of DSIs into higher-order DSIs. Composition forms the basis for configuration management and is universally well understood by all stakeholders.
During system development, many artifacts (files, database records, stored objects) are created and/or generated in CICD tools, which can be grouped in logical design-time units that capture the different aspects of such a unit, the requirements, the definition, the related test cases, the pipeline definition to create and verify the unit, the test results, the found defects, etc. I call such units design-time configuration items (DCI) and introduced this concept in my earlier articles. Each such unit should have one purpose, result in one or more well-defined and predictable install-time/run-time configuration items.
Looking at other engineering disciplines the basis for product management/configuration management is managing the Bill-Of-Materials (BOM), which reflect the product structure/composition. Terms like assembly, sub-assembly, and parts are used, where parts are the atomic building blocks, and assemblies and sub-assemblies are the defined end-products and intermediate products. Products are managed using systems called Product Data Management (PDM) or Product Information Management (PIM). With such a system different BOMs can be created to satisfy different stakeholders/processes, e.g. purchasing BOM, manufacturing BOM, etc. Note that product management in this context is about managing the product definition and not about product instances.
Product composition view
You could say that the backbone of product management is the product structural view which can be imagined like a hierarchy, a tree structure. Observing closer you see that it is a tree of trees, where each root represents a design-time configuration item which has one level below it, the identified components, where a component is the application of a lower level DCI. This indirection allows the same DCI to be used in many places, it allows design-time reuse and it allows specific configuration at usage level to be defined (see fig 1).
It assumes that each DCI is defined context-free and each component only exists in the context of a particular DCI. Each DCI is identified by a part number (P#) and each component has a context.
The attraction of describing the composition of systems, the composition or structural view, is that is highly universal and is easily understood. You do not have to be an engineer to understand it, so it works well for both engineering and non-engineering stakeholders. Other relations, the more intricate design relations, often are specific to the engineering discipline, functional sub-system, or even to the specific technology used. They require more, specialized and deeper knowledge to understand.
In software engineering, we talk a lot about dependencies and our design diagrams often are graphs, nodes of some form or another (boxes), and the dependencies between them (arrows). We can have compile-time dependencies, load-time dependencies, run-time dependencies all of which can even be specific to technology. Run-time dependencies can be further classified as dependencies within a node/stack and dependencies between nodes/stacks. Non-engineering stakeholders are likely to not understand such relations, and they should not have to, since configuration management should be about serving all stakeholders.
Another reason why the dependencies are not suited as the basis for configuration management is that dependencies do not bother about any boundaries, such as product boundaries and boundaries of responsibility. Within IT and software we often deal with a system-of-systems situation where our diagrams have a lot of boxes and arrows with no consideration for where one digital product starts and another end.
Also from a learning perspective, a composition view is a good starting point for complex systems. The book Documenting Software Architectures says the following about this, where it is called the module view — decomposition style.
A decomposition style view present the functionality of a system in intellectually manageable pieces that are recursively refined to convey more and more details. Therefore, this style is well suited to support the learning process about a system. Besides the obvious benefit for the architect to support the design work, this style is an excellent learning and navigation tool for newcomers in the project or other people who do not necessarily have the whole functional structure of the system memorized.
The essence for me lies in recursively refined to convey more and more details.
Relation between design-time CIs and run-time CIs
The above book focuses on describing/documenting complex systems. The overlap with configuration management is to identify the units with which to do this, called modules in the book. Configuration management adds things like version management, life cycle statuses, and responsibilities. What I think is essential for configuration management of IT systems, is that design-time CIs should always serve the purpose of allowing controlled changes of run-time CIs. Or reasoned the other way around, each run-time CI should be based on a specific version of a design-time CI. A controlled change is applying another version of a design to a run-time CI, then the current applied version. If a unit can not full-fil that role, then it is not a configuration item IMHO. A DCI shall be a unit of installation/deployment/change/replacement.
Also here there is a level of indirection and a run-time CI shall refer back to either a digital product or a component (see fig 2).
The CICD process serves one purpose and that is to instantiate each component in the digital product hierarchy and validate any intermediate product (sub-assemblies) until finally the digital product itself (assembly) is established. So a complex digital product does not have a single pipeline but can be based upon many pipelines, potentially one for each level in the hierarchy, building and validating a single component.
Unit of responsibility, unit of reuse, unit of release and unit of change
Since in the digital world there are no physical boundaries, choosing the boundaries of higher order CIs, those which are not dictated by technology, can seem arbitrary. Considerations can be.
- Responsibility; A single team should be responsible for the definition, verification and release of a CI. Work can be distributed over teams based on CIs.
- Design reuse; Can a unit be defined that can be reused; either within the design of a single digital product, or across multiple digital products?
- Granularity; What is the optimal granularity of a unit when small units have a small change impact, but having many units also introduces a administrative burden?
Note that design re-use can introduce value stream relations between teams, where team A creates the design (Part or Sub-assembly) and team B defines a component that references it. In practical terms, one team creates some template or artifact (e.g. IaC, library) that is reused by many other teams. The quality and release of such items should be controlled via Configuration Management.
Organizational challenges
Product thinking and thinking in product composition/decomposition is currently not common in IT. It can be a challenge to overcome the organizational unawareness of such concepts in those enterprises where engineering was considered not to be a part of the primary process.
Historical perspective; the V-model
In the past, the V-model was created to help structure system engineering projects. The model is based on the two sides of the letter V, the definition/specification side; the step-wise refinement until code, and the test and integration side (see below).
The relation between both sides is that a ‘level’ in the definition leg is validated/verified by a level in the test and integration leg.
Moving from project to digital product thinking configuration management based on DCIs and a digital product BOM is going down in the definition leg and do step-wise refinement of the digital product into its components. Where the V-model suggests that the verification and validation side is somewhat loosely coupled to the definition side, the use of DCIs assumes that each DCI specifies its own verification/validation and the means to automate this, if possible (e.g. pipeline). With DCIs, the relationship and ownership of artifacts (e.g. definitions, test specification, test results, etc) are well defined and managed throughout the product’s lifecycle.
The V-model is a very structured approach to system engineering but it is associated with a top-down/waterfall approach and therefore not very popular these days. With DCIs, a clear structure can still be applied to your digital product data without imposing a methodology.
DCIs to assist Agile at scale
In contrast to top-down/waterfall approach mentioned above these days agile development is mostly applied, which comes with challenges of it’s own. Scaling agile product development is not easy and several frameworks have been created to support agile at scale. Many lively debates pass by on internet whether these framework are truly agile or exist just to generate money from consultancy, courses and certifications.
Configuration management based on DCIs will not solve this problem, but having a clear insight in your digital product structure and consequently having an insight in the up-stream/down-stream responsibilities will certainly help. Whether your digital product is complex or not, cloud and open source software will introduce upstream dependencies. If you want to leverage enterprise scale through design re-use combined with agile, you will need transparency that can only be delivered by structural configuration management at design level.
Virtual composition; assemblies and sub-assemblies
To leverage design-reuse, the organization has to get used to define and work with sub-assemblies, where one team’s product (assembly) is another team’s sub-assembly. Since there are no physical boundaries assemblies and sub-assemblies are virtual compositions. To align design CIs and run-time CIs and be able to connect to the proper responsible team, virtual assemblies needs also to be introduced at run-time. This in contrast to the current practice where a single application or service is represented by a collection or stack of factual/concrete CIs. Introducing these abstract/virtual compositions is something an organization needs to get used to.
Difference between a BOM and a SBOM
The essence of a BOM is that it reflects a single simple and well-understood relationship, the consists-of relationship. These days the term SBOM is mentioned a lot related to the Executive Order on Improving the Nation’s Cybersecurity. It is good to understand that there is a difference between a BOM described above and a list of transitive dependencies, what the SBOM factually is. As said before, dependencies do not bother about boundaries of any sort and one of the most important aspects of composition is to just that, reflect boundaries. System parts often rely on generic shared platforms, and with cloud, we see an abundance of these. A transitive dependency list often crossed the boundary of the unit deployed on a platform and the platform itself. From a digital product perspective, there are two products in this relationship, the platform as a product and the (part of the) product deployed on top of it. As such any dependencies provided by the platform is not part of the product you deploy on this. For Java this can be explicitly defined by the Maven dependency scope. The dependencies provided by the platform should be defined with scope ‘provided’ or ‘system’. These dependencies will not be packaged and distributed as part of the digital product that is deployed on the platform, they are outside the BOM. In that respect, SBOM is the wrong name. To make a silly analogy, the electrical outlet and power plant are also not considered part of a vacuum cleaner, but it will not work without it.
Conclusion
Design-time configuration management of your digital product portfolio supports any development practice/methodology. It can be applied both top-down and bottom-up. It will provide the transparency required by many enterprise processes and will demarcate ownership/responsibility at any level, both on the design/definition side and the run side. The challenge is to find the right granularity to leverage enterprise design-reuse and to not overburden the organization with the overhead. Some types of organizations will have to learn to think in digital products and their virtual compositions, which can be a challenge.
If you have more questions related to this topic, feel free to book a meeting with one of our solutions experts, mail to sales@amazic.com.
