Nearly every (large scale) organization has adopted containers now in a certain shape or form. Containers and Kubernetes are still on the rise. The number of Kubernetes clusters grows rapidly and many of those organizations explore ways to acquire or build a so-called “container platform”. They already see clear benefits to streamlining the containerized initiatives. Nowadays, companies such as Docker Inc, Otomi Pivotal Cloudfoundry offer Container Platforms. While the technology might be ready to roll out applications to production, everything still evolves quite rapidly. Besides the technological challenges, organizations need to be organized very well to implement everything they need. Governance-related aspects are very important here. Container Platforms and governance – the complete picture.
Problem statement
Small initiatives tend to grow too big as soon as they gain momentum. Sometimes this leads to being “out of control”. On the other hand, companies often demand “a total solution” for a specific problem. Buy software product X and your problems are solved. Or not? Everything that comes with container technology and especially Container Platforms is no different here. There are a lot of companies that struggle to position the roles and responsibilities of teams that want to adopt container technology, especially if they want to run it “at scale”.
Think of questions like who will be responsible for the container images, the registry to store them all. Or what about the processes to patch containerized systems, set up security guidelines, enforce standards and guidelines on naming conventions, etc.
In this article, we’ll explore various topics that all have to do with the departments and teams that have a say in the adoption of Container Platforms in large enterprises.
Who is the owner?
One of the most prominent questions to ask yourself is: Who will be the owner of the Container Platform. Will it be a team at the business department that resides close to the release managers or perhaps the internal IT department that offers IT services to the entire organization? In fact, the positioning of your new Container Platform is extremely important since it determines the direction of discussions around relevant topics and the potential outcome of those discussions. It might also determine what is in scope and what is not.
Business department or Internal IT group
If your internal IT department is the owner, they might have a strong focus on each and every technological aspect, but they constantly need to keep an eye on what the business wants and needs. And on the other hand, if a team at the business department is the owner, things can also be complex. Often they lack visibility of the complete problem area since there are so many (tech) decisions to make. They need to have a good understanding of the various topics. IT departments can help them to get that knowledge but it’s difficult to step into each others’ shoes to bridge that gap.
Responsibilities
In the end, at least the following responsibilities need to be governed and the owner is there to push it in the right direction:
- Accept end-responsibility for every aspect of the Container Platform.
- Provide sufficient budget to acquire the platform or build it (and maintain it afterward). Training and education are part of the package.
- Approve or reject a business case for the platform. Perhaps that person should also be involved actively when the business case is created in the first place.
- Align the direction of the container platform with the tactical and/or strategic direction of the company.
There are many more aspects that are relevant here, it’s to get you started.
Core platform team
Suppose the organization transforms to a “business technology company”. The (senior) management positions the so-called “core team” at the business department. Then what should that core team do?
First of all, the core team should be an Agile team that requires team roles such as a Product Owner, a Scrum Master, and a bunch of developers that also know a lot about cloud, infrastructure, containers, and security. The SPOC can be the Product Owner of the team, but it can also be a senior manager who is the budget holder for the initiatives.
Vital is their “dot on the horizon” and the upper management should support this as well. It’s this dot that should steer every discussion and clearly scope things to reasonable proportions. Everyone is involved needs to be aware of it.
Responsibilities
The following list represents some of the responsibilities of the core team:
- Create a functional and technical design for the Container Platform and its connecting services.
- Conduct stakeholder management for every team in the organization that is affected by the container platform. Be sure to include teams and individuals that benefit as well as those that are negatively impacted. Your good old friend the RACI charts can help here.
- Create the first container (base) images that are hardened and secured. Store and distribute them in such as way that other developer teams can find them.
- Propose standards and guidelines on how to effectively use the container platform, how to design and build applications that benefit most of it. Be sure to share these with the Agile teams, otherwise, their architectures might not fit and that’s a loss when they are ready to onboard.
- Actually build and maintain the infrastructure-related aspects as well as everything that comes with it: CI/CD pipelines, Service interfaces, artifact repositories, code templates, source code repositories, persistent storage solutions, integration points with ticketing tools, logging and monitoring, security, etc.
There’s more
- Propose a “future roadmap” for the features that are (going to be) supported by the platform.
- Onboard new teams so they can actually use the platform in an early stage. Only offer “best-effort” support from this moment in time. Later on, you can extend.
- Train and educate other teams to get a feeling of the platform. But there’s a catch here: if your platform is too difficult to use, organizational implementation might be delayed, the proposed benefits can be fewer than expected or the entire initiative fail completely.
- Handle incident management: first-line support and processes to conduct follow ups for problem management.
These are just a few aspects that might have some overlap with the Cloud Platform Team in your organization.
Cloud platform team
Assume your Container Platform will be built in the public cloud and it depends on the (core) cloud services which your internal cloud platform team offers. An impression of the aspects and responsibilities that need to be in place:
- The cloud platform team should offer as much “ready to consume” cloud services as needed by the core platform team. Both teams should be as independent as possible.
- Problems arise on topics like who will build or maintain the Virtual Machine images for the nodes that host the containers of the Kubernetes cluster? These images need to be hardened and secured.
- Or what about the requirements for persistent storage. Throughput, data retention, connectivity, integration with the Kubernetes clusters, data security, meta-data, etc.
Security
- Conduct a risk assessment of these cloud services together with the CISO team and offer a future roadmap to DevOps teams to see which feature is ready to be used by which (type) of workload.
- Offer IAM solutions that follow the characteristics of the container-based solutions of the platform. Often they need to work closely together with the IAM team. This team should also be well informed of the scope and characteristics of the containerized workloads.
- Think of the least privilege principle for container workloads especially when they need to access other cloud resources. To complicate stuff, even more, the above-mentioned cloud resources might be present in the same cloud account, in another (companies’) account, or even in a second cloud environment.
Define the boundaries of services and responsibilities together with the core platform team. Often, there is overlap and confusion about who does what. This needs to be sorted out and communicated, especially for the developer teams that interact with both parties.
Enterprise Architects
Every Container Platform requires a thorough architecture that spans all technical aspects that are relevant for the various stakeholders. Enterprise Architects should define standards and guidelines on a high level, whereas specialists implement these. Specialists reside in teams all across the organization. Besides this, tool selection and approvals (the so-called “future state architecture”) is important topic for them to cover.
Networking experts help to outline the internal- and external data flows as well as communication between containers in the cluster itself. Another principle is the topic of “segregation of duties”. Does every team require it’s on Kubernetes cluster? Or is it possible and also allowed to host multiple applications from multiple teams in one cluster?
It’s also important that Enterprise Architects (help to) define which containerized solutions are allowed in the first place. Data classification may help here to use as selection criteria. Besides the Container Platform does the organization also allow other containerized solutions, for example, AWS Elastic Container Services as a cheaper and potentially less complex environment? This could be the case for simpler workloads that demand less advanced options. However, that brings diversification to the architecture landscape and things become more complicated, also the governance-related aspects. Enterprise Architects need to have a strong vote here.
Besides these topics, the following aspects are also food for Enterprise Architects: backward compatibility, versioning of (components) of the Container Platform as well as prescriptions and guidelines on the re-usability of various components. Think of container (base) images, (security) policies and their underlying rules. If Open Source software plays an important role in your organization, Enterprise Architects might support Open Policy Agent as a (cloud) agnostic tool to describe compliance policies as code. Large enterprises often require professional support, this is covered with Styra that offers it.
Legal and compliance
Once the majority of these aspects are covered, the Legal and Compliance department also comes with their requirements. You need to involve this special stakeholder and think from their perspective, you can’t expect them to think for the tech perspective. The teams in this department will assess your proposed future solution and validate in which way it differs from the existing (hosting and deployment) solutions. They will ask you questions like:
- Which (commercial) licenses are needed? Are these in line with the organization itself?
- If you need to acquire a new tool, have the right processes been followed?
- In case you run the Container Platform in the cloud, what is the impact on sensitive data that you or the developer teams aim to process? You would require to fill in a lot of documents such as the PIA, ISRA, and GDPR (in case you reside in Europe). See the article that has 10+ abbreviations that explain all of these.
In the end, the Legal and Compliance team needs to approve the information you present to them and they have a say in how you need to enforce the proposed solutions for the above-mentioned challenges. If your team or this department fails in this area, your business continuity is at risk.
Cloud Approval Board (CAB)
Every large organization has a Cloud Approval Board or a similar department with the same purpose. Simply speaking, the CAB is the highest authority that approves or rejects the entire solution that you want to implement. It consists of members from the pool of Enterprise Architects, the CISO department, the persons that shape the Business Continuity plans as well as members from Change Management and Legal & Compliance. Budget holders might also be present as well other stakeholders such as business representatives.
Once all of the preparation is done and you checked the boxes of all the papers and all requirements that are needed, you can offer your solution to them. Most often, the CAB reviews solutions every 2 – 4 weeks or so. They can come up with new questions and extra requirements which you need to investigate or present alternatives for. This can cause some delay, but if your solution is accepted, you can rush forward. Congratulations, this is an important milestone.
End of part 1
As you can see, there are already a lot of aspects, teams, and individuals involved. These were mainly from the business / organizational perspective. In part two of this article, we’ll explore more topics that cover the technical challenges.
If you have questions related to this topic, feel free to book a meeting with one of our solutions experts, mail to sales@amazic.com.