My blog, keeping you up-to-date with my latest news.

 

Using Hiera with Puppet

DevOps • 2014-02-19 • 1 Comment

 

Using Hiera with Puppet

Since a few months, I work essentially with Puppet to make continuous deployment.
When I write manifests, some of them specifically aim one or two servers, others servers must do others things.
For example, by putting roles, the database role does not need an apache server, while the frontend role does. Same applies for Openstack, a keystone server has differents needs than an horizon or a glance server.

In short, it quickly becomes harder and not really aesthetic to continually add conditions in all manifests in order to define if yes or no, such and such services must be installed or running on one server and not on another.

Hiera makes Puppet better by keeping site-specifc data out of your manifests. Puppet classes can request whatever data they need, and your Hiera data will act like a side-wide config file.

Configuration

I use the version 3.2.4, if you use the version 2.7, there's a good chance the following doesn't apply.

First, we must create a directory for Hiera.

$ mkdir -p /etc/puppet/hieradata

After, put hiera.yaml in hieradata/ directory.

---
:backends:
    - yaml

:yaml:
    :datadir: /etc/puppet/hieradata

:hierarchy:
    - %{::environment}/%{::role}
    - %{::role}
    - common

This YAML tells that Hiera will only use YAML backend, but if you want, you can specify a JSON backend. YAML is still easier to read for this example.
Also, we tell that the directory containing YAML files is /etc/puppet/hieradata.
Finally, the hierarchy will define which files will used by Hiera. To define this, we use "facts". Facts allow to dynamically load files depending on the environment or role.

More options can be added to this configuration file: http://docs.puppetlabs.com/hiera/latest/configuring.html#global-settings

For example:
For the development environment, and database role, Hiera will find and execute these files in this order:

  • /etc/puppet/hieradata/development/database.yaml
  • /etc/puppet/hieradata/database.yaml
  • /etc/puppet/hieradata/common.yaml

 

Writing YAML files

We just finished to prepare the hiera.yaml file, we now can write other files.

common.yaml

---
classes:
  - 'common::roles::%{::role}'
users:
  'GoT':
    name: 'got'
    ensure: present
    managehome: true

We imagine that we have a module named common, with classes for roles, and also a list of users that must be deployed on all servers.

development/database.yaml

---
mysql::server::root_password: 'strongpassword'

databases:
  gotcms:
    user: 'got'
    password: 'super_secret_db_password'
    host: 'localhost'
    grant: 'ALL'

Database configuration for the development environment. Defines the root password and which database is used on the server.

database.yaml

---
users:
  'MySQL':
    name: 'mysql'
    ensure: 'present'
    managehome: false

Again a user on each server who have the database role.

Now we have three files defining quickly what we will have on our database server.

Adding facts

In the next example we'll use JSON instead of YAML, just for fun.

The rest of the article will be executed on the target server in masterless.

If the directory facts.d/ is not present, you must add it:

$ mkdir -p /etc/facter/facts.d/

Then add a file /etc/facter/facts.d/gotcms.json in it

{
  "role": "database",
  "environment": "development"
}

You can test if facts are correctly set with the facter command.

root@os-mysql:~# facter role environment
environment => development
role => database

You might change these informations depending on your needs.

Dependencies

As you can see above, we need the MySQL module (https://github.com/puppetlabs/puppetlabs-mysql), so if you want to test, you can install this dependency with the command:

$ puppet module install puppetlabs/mysql

Using Hiera

To try our code, we need two Puppet classes, the first one is the common class located in /etc/puppet/modules/common/manifests/init.pp

class common {
  # Include classes from hiera
  hiera_include('classes')

  # Create user from a hiera hash
  create_resources('user', hiera_hash('users'))
}

The next one, for the database role, is located in /etc/puppet/modules/common/manifests/roles/database.pp

class common::roles::database {
  # Only include mysql::server to configure database, hiera does the rest
  include mysql::server

  create_resources('mysql::db', hiera_hash('databases'))
}

There are some functions that you can use in your manifests to get Hiera data:

  • hiera: Returns a value for a given key, value can be data of any type (string, array, or hash)
  • hiera_array: Returns all matches throughout the hierarchy, not just the first match, as a flattened array of unique values. If any of the matched values are arrays, they’re flattened and included in the results.
  • hiera_hash: Returns a merged hash of matches from throughout the hierarchy. In cases where two or more hashes share keys, the hierarchy order determines which key/value pair will be used in the returned hash, with the pair with the highest priority datasource winning.

In addition, theses functions accept two additionals arguments:

  • a default argument in the second position, providing a hash to be returned in the absence of any matches for the key argument
  • an override argument in the third position, providing a datasource to consult for matching values, even if it would not ordinarily be part of the matched hierarchy. If Hiera doesn’t find a matching key in the named override datasource, it will continue to search through the rest of the hierarchy.

Puppet apply

As I said above, I work in masterless, and this is what you logically get by executing the following command:

$ puppet apply -e 'include common' --hiera_config=/etc/puppet/hieradata/hiera.yaml
Notice: Compiled catalog for os-mysql.numergy.dev in environment production in 0.74 seconds
Notice: /Stage[main]/Mysql::Client::Install/Package[mysql_client]/ensure: ensure changed 'purged' to 'present'
Notice: /Stage[main]/Common/User[MySQL]/ensure: created
Notice: /Stage[main]/Mysql::Server::Install/Package[mysql-server]/ensure: ensure changed 'purged' to 'present'
Notice: /Stage[main]/Mysql::Server::Config/File[/etc/mysql/my.cnf]/content: content changed '{md5}77f15d6c87f9c136c4efcda072017f71' to '{md5}0de5070586e53c379e90f8760ae62f5d'
Notice: /Stage[main]/Common/User[GoT]/ensure: created
Notice: /Stage[main]/Common::Roles::Database/Mysql::Db[gotcms]/Mysql_database[gotcms]/ensure: created
Notice: /Stage[main]/Common::Roles::Database/Mysql::Db[gotcms]/Mysql_user[got@localhost]/ensure: created
Notice: /Stage[main]/Common::Roles::Database/Mysql::Db[gotcms]/Mysql_grant[got@localhost/gotcms.*]/ensure: created
Notice: Finished catalog run in 2.07 seconds

You can read more about Hiera on the puppetlabs documentation: http://docs.puppetlabs.com/hiera/latest/index.html

<< Back to Blog Discuss this post

 

Comments

 
  1. txalamar

    Hello,

    I'm triyng to use mysql::server::grants with hiera to set some grants but I can't figure out how to declare the lookup and how to define the data itself:

    in main.pp:

    create_resources('mysql::server::grants', hiera_hash('grants'))

    and in yaml file:

    ---
    grants:
    user@10.235.4.10:
    ensure: 'present'
    options: "['GRANT']"
    privileges: "['ALL']"
    table: "*.*"
    user: "user@10.235.4.10"

    but does not seem to work

    Colud you pelase help me?

    Thank you

 

Add a comment

 
  • Please verify you are human

Categories