Given the trouble of recreating sources here in the blog, I have decided to move them into a proper VCS system at Kenai.
I will go through the existing blog entries, move the source and insert links to the VCS instead.
Go to the Buildmeister Kenai project to access the sources.
2009-06-10
2009-06-09
Configuration of Hudson
We use Hudson to continuously build our projects.
We have one Master and six Slaves.
There are 385 projects being built. 128 of these are simple projects where only some XML documents are validated. The other 257 projects are mostly JEE applications, but there are also a few regular Java projects.
Build times span from 12 seconds to 27 minutes.
We have the Hudson configuration expressed by a simply formatted property(ish) file. This file is under CVS control, and is manipulated by a number of support scripts independent of Hudson.
When the configuration is changed, it triggers a build in Hudson on the Master (the only project that is allowed on the Master). This translates the configuration to Hudson-lingo then restarts Hudson.
So we can track and undo changes to the configuration. If the configuration becomes corrupt, it will either fail to translate - or we can roll back to the previously working version.
Specifically, the configuration build process consists of these steps:
(1) tries to avoid having any build process running when Hudson restarts. (2) and (5) ensures that Hudson will resume (if the build fails) or be restarted with the new configuration.
(3) decouples us from Hudson. Should we decide to use a different continuous integration engine in the future, we can continue to use our configuration scripts. We just write a new translation script.
Much more robust, efficient and delicious than what I made at Systematic; there we had only a few (big) projects, so we used a manually configured LuntBuilt. But it was quickly growing unmaintainable by the time I left. Hopefully they are using automatically managed configurations by now :)
Lovely as this is (in my mind, anyway), step (1) does not work as I would have wished.
It could still cause an abort of running projects that take longer than the configuration project to build. And it did on occasion.
Today we use security on Hudson, so the Shutdown/Resume URLs are not accessible. So we almost always accidently terminate project builds when the configuration is changed.
What I think is needed is a special Hudson configuration bootstrap builder (beyond my available time, I'm afraid) that would:
Hopefully someone will make me such a builder :)
We have one Master and six Slaves.
There are 385 projects being built. 128 of these are simple projects where only some XML documents are validated. The other 257 projects are mostly JEE applications, but there are also a few regular Java projects.
Build times span from 12 seconds to 27 minutes.
We have the Hudson configuration expressed by a simply formatted property(ish) file. This file is under CVS control, and is manipulated by a number of support scripts independent of Hudson.
When the configuration is changed, it triggers a build in Hudson on the Master (the only project that is allowed on the Master). This translates the configuration to Hudson-lingo then restarts Hudson.
So we can track and undo changes to the configuration. If the configuration becomes corrupt, it will either fail to translate - or we can roll back to the previously working version.
Specifically, the configuration build process consists of these steps:
- Prepare Hudson for shutdown
- Set up a cron job to resume Hudson in 15 minutes
- Translate the property configuration into Hudson's configuration tree.
- Copy the generated configuration tree on top of the Hudson's active configuration.
- Set up a cron job to restart Hudson Service in 2 minutes
(1) tries to avoid having any build process running when Hudson restarts. (2) and (5) ensures that Hudson will resume (if the build fails) or be restarted with the new configuration.
(3) decouples us from Hudson. Should we decide to use a different continuous integration engine in the future, we can continue to use our configuration scripts. We just write a new translation script.
Much more robust, efficient and delicious than what I made at Systematic; there we had only a few (big) projects, so we used a manually configured LuntBuilt. But it was quickly growing unmaintainable by the time I left. Hopefully they are using automatically managed configurations by now :)
Lovely as this is (in my mind, anyway), step (1) does not work as I would have wished.
It could still cause an abort of running projects that take longer than the configuration project to build. And it did on occasion.
Today we use security on Hudson, so the Shutdown/Resume URLs are not accessible. So we almost always accidently terminate project builds when the configuration is changed.
What I think is needed is a special Hudson configuration bootstrap builder (beyond my available time, I'm afraid) that would:
- Prepare Hudson for shutdown
- Wait for all running jobs to complete
- Build the project
- Resume if the project failed, otherwise restart the Hudson Service.
Hopefully someone will make me such a builder :)
Subscribe to:
Posts (Atom)
