Application <=> Server Management Protocol (ASMP)
While I was in Belgrade for WordCamp Europe, I took part in a meeting of a smaller group of core contributors to discuss the status and future of the Servehappy project. One topic we discussed was that the ideal end goal, from the end user’s perspective, would be to have an automatic updating mechanism that keeps the entire server automatically up-to-date and secure. While this might be a lofty goal, we wanted to nevertheless look into what would be necessary to make this a reality.
While we can already automatically update WordPress Core, as well as plugins and themes, we cannot do the same for the components of the WordPress stack that are installed at the server level. There’s currently no standardized protocol for server applications to request information about or changes to the underlying infrastructure. While we have APIs to provision such servers through code, we have no protocol or API to allow the applications that run on them to make changes after the provisioning.
I hereby suggest creating a communication protocol (tentatively called ASMP) between servers and applications that run on these servers that allows for bidirectional communication so that these two actors can coordinate.
Let’s look at a few examples
As an example, think about regular users managing a WordPress site. On the WordPress “Server setup” page in the admin backend, the end user would see the current PHP and MySQL versions used by WordPress, as well as potential notifications about available updates.
The available updates are calculated by querying ASMP about the 'php' and 'mysql' components and then checking the returned ranges of supported versions against its own requirements.
When the user wants to run such an update, they click the corresponding button, which will have WordPress send an ASMP request to the server to 'set' the version of the 'php' component to the requested version.
Then WordPress can poll the ASMP state of that request to get updates. It could return states like 'queued', 'in-progress', 'completed' or 'rejected'` to let WordPress know how this is moving forward and adapt the user interface to give feedback to the user.
This would give more control to users, reduce support tickets for hosting providers and indirectly improve version distribution of the server components managed in this way.
As another example, think about developers that manage their project through a package manager like Composer or NPM. When you provision a server with a composer install instruction, you could have Composer automatically make requests to the underlying server based on the "require" statements that define the platform and extension requirements.
Composer could just as well throw an error when the server is not capable of providing the requested environment, and let the user know which components failed to resolve properly.
Guiding principles
Planning the ASMP protocol and providing reference implementations should follow a few guiding principles:
- Applications can only make requests, not force any changes.
- There is no guaranteed support, servers are free to only provide partial implementations.
- Requests are asynchronous, they don’t return responses.
- State and capabilities can be queried, so that user interface and expectations can be adapted to the current server.
These guiding principles still allow for more exotic implementations, like a request to update a given component doing nothing more than creating a support ticket with required details. Hosting providers are free to choose the subset to implement and the specific way to implement it. The implementations can then grow from there and be further automated in time.
Technical details
The transport to be used could be regular HTTPS, with a server environment variable providing the host to contact for queries and requests. This environment variable would point to localhost when running on stand-alone servers, but could be deviated to centralized server for hosting providers that have more complex orchestration needs. The server environment should also contain an authentication mechanism, like a token, to make sure queries and requests can only be made from the correct server.
We should provide a reference implementation based on a common server distribution. This makes sure that it is easy for hosting providers to add support to their hosting and helps keep the experience consistent.
Also, focusing on building implementations into popular server control panels would accelerate adoption, as all servers managed through a given control panel would automatically inherit the implementation of the ASMP protocol.
What’s next?
I am in the process of introducing this as a proposal to the Cloudfest Hackathon which will be held on March 23-25, 2019 in Rust, Germany. If this gets accepted, it will probably be an immediate opportunity to collaborate with major hosting/control panel providers to build a working prototype. Anyone who is interested in participating in this endeavor and willing to travel to Germany for the occasion can still apply for the hackathon.
While I admire the initiative, I do think that the majority of app (in your example – WordPress) users should not be responsible for taking this decision. There are several reasons for that:
1) they are paying for a hosting. It’s not their day to day business to decide which PHP/MySQL to run. If PHP version differences are more or less described out there and also all the benefits for WordPress and its plugins, what about MySQL? MySQL 5.7 breaks things for some plugins. Old WordPress (I’ve tested 3.x) itself can’t be installed in a default MySQL 5.7 because of its strict internal rules. There is no way an average user will know that. There is a great possibility that after upgrading to 5.7 she/he won’t even notice the issue until some records won’t be saved into the DB.
2) thus hosting company, that is getting money, should be proactive and notify its clients about the new PHP/MySQL/Apache/Nginx versions that were released and are available for an upgrade.
3) hosting companies should insist on upgrading a server with PHP not further than the latest day of a certain version support. That means that starting from a middle of the PHP support period (not just security, but still regular support) – it should write its clients, and provide deadlines, and actually do scans and server upgrades when the time has come. Currently all hosts are not teaching its clients at all. Some of them are proactive on providing an ability to upgrade, like SiteGround. But even they still allow me to downgrade to PHP 5.5 AND create a site on PHP 5.6 and 7.0.
Hosting companies have power, people and technological knowledge to fix this issue. Most of them just don’t care.
The proposed solution in this article looks a lot like yet another support channel with a single topic: please upgrade my PHP for domain example.com from 5.2 to 7.2. Further automation that was mentioned can already be achieved using the technology that is already there. Also, control panels already have this versions management, and companies like SiteGroup already created solution to change PHP version right from WordPress admin area (see their SG Optimizer plugin and its PHP Config section).
IMO the focus should be changed to hosting companies being proactive, and not their clients.
https://xkcd.com/927/Report
Hey Slava,
Thanks for the comment (and sorry again for messing with your initial tweet ;) )!
I’ll try to address the observations you shared using your bullet points…
1.) They are usually paying for the hoster to keep their site online, which creates a contractual obligation for the hoster. This puts the hoster into a position where they cannot easily do server-side updates, because they might break the site. They need for the user to actually “pull the trigger”, otherwise they breach their contract. I agree that users should not need to decide what versions to use, but they usually need to “sign off” the decision to make a change in some way.
2.) Yes, and that could be managed through the above protocol. As an example, WordPress could show a notice in the admin backend that the hoster recommends to update the PHP version to 7.3, the user can push a button to “agree”.
3.) Hosters are being paid to provide what the clients are paying for, nothing more and nothing less. If a client has software that only runs on PHP 5.2, they will pay the hoster to run a server on PHP 5.2. The hoster can of course refuse to do that. But this will not cause the client to update the code, it will cause the client to choose a different hoster.
Also, I don’t see this as a support channel, but rather as a way to reduce a lot of the friction surrounding the server/application interaction. For example, during a discussion with one of the Google engineers at WCUS, we talked about the use case of having such a protocol drive changes to a Kubernetes cluster: the user pushes a button to update a server component like PHP, and this will trigger an automated deployment of a new container/cluster with that new component and run tests. if the tests were successful, the new infrastructure is automatically used, and it will show a “rollback” button to switch back to the old one in case something looks broken. There’s no support involved, it is just a matter of the application being able to drive the server stack.
In general, I have to say that I completely agree with what you are saying in regards to what best practices would dictate. However, the practical reality does not match these best practices, and you cannot just ignore this difference. I’d say that pretty much any Software Engineer or DevOps Specialist will know how to keep software and server up-to-date and in sync, they won’t need such a protocol. But the long tail of the internet is run in large parts by people that do not belong to this group. Just telling them “you’re doing it wrong” will not improve anything at all. We have to provide solutions that they can actually make use of to help improve the web for all.
Looking forward to meet again and discuss this in more detail! ;)
Cheers,
AlainReport