As you know, there are currently two main distinct approaches for deploying Galaxy: Ansible/Terraform and Kubernetes. An issue here is that much of the work is replicated in developing and maintaining these two approaches. We recognize the administration of our deployments is understaffed, regardless if it’s one of the usegalaxy.* servers or AnVIL for example. We are looking at hiring more help but the reality is that there will always be more work than available effort. To help this situation, we’d like to see more synchronization between the deployment approaches and consolidation on parts of the deployment stack.
With that, we’d like to see the Admin and Deployment working groups merge into a new Systems working group. The aim here is to find synergies between the deployment approaches. All of you as members of these working groups have complementary experience and expertise that can be leveraged to create more robust solutions. We would like to encourage open-minded and strategic thinking on how Galaxy instances are deployed, starting by creating an inventory of the components that make up Galaxy installations. Then, deciding if a single deployment strategy for any individual component is achievable, realizing it, and deploying it across different environments. Even if the number of such components is minimal, engagement within one group will raise the level of understanding between the members for different deployments, strategies, and use cases.
Furthermore, problems encountered during running instances should not be hacked around in deployments, but rather fixed upstream. For this it is essential the Systems working group engage closely with the backend working group. Every problem that is fixed upstream will make your deployments and the ones by the community easier. This is also where long-term strategic thinking comes into play where Galaxy architectural decisions can, and should, be made to ease our deployments.
Let us know what you think or if you have alternative suggestions.