Creating Self Hosted puppetmaster in Wikitech labs

While trying to test an exim-puppet patch ( ) which adds a new router in one of my labs instance, I came across the need to create a self hosted puppetmaster. For starters ( like I was few days before ), puppet is a provisioning language, as they call it and applies pre-written configuration files to various daemons and tools coming under its realm. For example, I can set it to generate a standardized exim4.conf file when its run, this standardizing file need not be on the local machine you are working on. A self hosted puppetmaster will have all the configuration files packed inside a local directory – which is inside /var/lib/git/operations/puppet/
Where it come useful :
* It come into use when you want to play around with the default configuration files of your instance.
* When you dont want your changes not to be overwritten by the default puppet configured files.
Steps to make your Wikitech labs instance as a self hosted puppetmaster:
* make sure that all previously enabled roles were applied and puppet is running completely.

$ sudo puppet agent -tv

* go to Special:NovaInstance and select configure on your preferred instance.
* now, add tick the role role::puppet::self . This is our required self puppetmaster role
* go to your instance shell and give the command:

$ sudo service puppetmaster restart
$ sudo puppet agent -tv 

* now make sure you have the pupept repo in your local instance.

$ cd /var/lib/git/operations/puppet/ 

if the folder exists – Yay!
Now, you can apply your operations/puppet patches to this folder.


Using puppet realm switch to select between beta/prod ( Wikimedia clusters )

Since the BounceHandler extension is currently installed only in the beta clusters ( Official testing servers of Wikimedia- ), writing a custom router in the exim configs of operations/puppet ( configuration repo managed by puppet ) to collect in all the bounce emails and HTTP POST to the extension API seemed risky. This was due to the fact that operations/puppet is meant for production and something beta specific in it, that too continuous API requests dont look sane ( we cant estimate the traffic yet ).
Marius Hoch ( WMF ) came with the idea of an exim-realm switch, which seemed to solve the issue. You can switch variables between beta and production by:
We edited

$ sudo <editor> templates/exim/manifests/mail.pp

and added this before the inclusion of exim4.conf.SMTP_IMAP_MM.erb.

# config - labs vs. production 
case $::realm {
    'labs' : {
	$verp_post_connect_server = ''
	$verp_bounce_post_url = ''
    'production': {
	$verp_post_connect_server = 'appservers.svc."${::mw_primary}".wmnet'
	$verp_bounce_post_url = ''
    default: {
	fail('unknown realm, should be labs or production')

The internal networks selection was done to make sure the bounce gets POSTed to the right wiki. Now, this configuration can be used in exim4.conf.SMTP_IMAP_MM.erb by editing the bouncehandler router by:

command = /usr/bin/curl -H 'Host: <%= @verp_post_connect_server %>' <%= @verp_bounce_post_url %> -d "action=bouncehandler" --data-urlencode "email@-"

Hope it helps someone in beta 🙂