Reading Time: 10 minutes

Introduction

In this article, we will be reviewing the Chef software, how it works, and why it is useful. We will also explore how it is helpful in DevOps. And then, we will install Chef on Ubuntu 18.04. 

What is Chef? 

Chef is a configuration management system written in Ruby and Erlang using the object-oriented language for configurations description. It is used to ease the task of configuring and maintaining multiple servers. It can be integrated into cloud platforms, such as the new Liquid Web Managed Cloud and VMWare platforms to manage the server's configuration processes. 

It is essential to remember that there are two types of configuration management:

  1. Push Configuration - When the central server sends configuration to other servers, e.g., Ansible uses such a scheme. This can be useful when there are devices, and you are unsure whether they could be available at this time.
  2. Pull Configuration - When nodes check servers from time to time and extract information and configuration, e.g., Chef uses this scheme.    

Also, it is essential to keep in mind that it uses the infrastructure as code methodology. This approach uses the software to manage infrastructure description and management through configuration files rather than manual management.

DevOps, along with other internal sectors, begin using it by determining which tasks should be automated. The Recipes and Cookbooks describe those processes. They are then tested using additional tools such as ChefSpec (the platform which checks resources and recipes as part of a simulation run). When everything works as intended, the Recipes and Cookbooks are put on the Chef server and run using the Knife tool. This tool allows users to use the software more seamlessly.

Components

Chef consists of three main components:

  • Chef Workstation - The central server where all changes are initiated, updates are created, and records are kept. Information from workstations is also sent to the central server.
  • Chef ServerInfra - This is the component that provides a connection between all devices. The cookbooks, recipes, and policies created on Master are sent to the server where they are stored and extracted by the worker nodes. 
  • Chef Worker Nodes - The software Server manages these devices. When needed, the internal mechanism lets worker nodes connect to the master server using a client to get updates, new policies, and other changes.

Tools

Chef uses the following tools: 

  • Recipe - This is a configuration file with a set of attributes that are used to manage infrastructure. They are loaded during the client startup and compared with the existing characteristics of the node.
  • Cookbook - This is a collection of recipes, the main building blocks uploaded to the server. When the software is run, it ensures that recipes kept in it will bring the given infrastructure to the state specified in the recipe.     

Advantages of Chef 

  • It completely automates deployment. 
  • It can work well with thousands of nodes.
  • It updates and installs changes quickly.
  • Chef continually checks and monitors the whole system, so all malfunctions are noted and monitored.
  • The Ruby language is used. 

Disadvantages of Chef 

  • One of the significant disadvantages is cookbook management. These need to be continuously checked, edited, and maintained.
  • It requires a solid knowledge of code to write scripts for the tool, and this can make work more complicated. 

How Does Chef Integrate in DevOps? 

First, let’s define the term "DevOps," which we often come across in this article. DevOps is a set of practices and methodologies to improve the efficiency of development processes and software operating at the cost of the continuous integration and interaction of various specialists using automation tools. 

It is essential to understand that DevOps is not just the integration of automation tools, but also methodologies and philosophies, which have paved the way for the collaboration of various departments and groups such as testing, development, and exploitation. In short, the DevOps' main task is to automate most duties and processes in the company. Chef in DevOps is used and applied for server automation for configuration management, which automates processes and tasks on many servers and other company devices.  

These are some advantages which DevOps get using Chef as a tool for servers automation: 

  • Sequence and scalability - All devices continuously get the same installations, updates, and software deployment, providing the same state for devices and standing for stability.
  • Quick deployment - Speed and accuracy are critical features for any project. The software provides a well-configured system that lowers the necessity of manual configuration and lowers the risks of making mistakes.
  • It integrates with multiple systems and cloud technologies. 
  • The software also acts as an infrastructure repository. It can be used for recreating repositories from scratch.
  • It’s unnecessary to describe all recipes from the beginning; many ready-made templates have already been checked and will let you save time.  

Install and Configure Chef

We are going to use three Ubuntu 18.04 servers for installation.

Chef Main Server

  • Hostname: server-chef
  • IP address: 192.168.1.101

Chef Workstation

  • Hostname: workstation
  • IP address: 192.168.1.102

Managed Node 

  • Hostname: client-node
  • IP address: 192.168.1.103

If you don’t know how to find the IP, run the following command to locate the IP address.

root@server-chef:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:54:1c:ed brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 86363sec preferred_lft 86363sec
    inet6 fe80::a00:27ff:fe54:1ced/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:61:75:dc brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.101/24 brd 192.168.1.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe61:75dc/64 scope link
       valid_lft forever preferred_lft forever
root@server-chef:~#

Configure Hosts File

Master

Next, we need to add the IPs to the /etc/hosts file on every server. To do that, run the following. 

root@server-chef:~# tee -a /etc/hosts<<EOF
e> 192.168.1.101 server-chef
> 192.168.1.102 workstation
> 192.168.1.103 client-node
> EOF
192.168.1.101 server-chef
192.168.1.102 workstation
192.168.1.103 client-node
root@server-chef:~#

Verify and Duplicate

Let’s check whether we have succeeded.

root@server-chef:~# cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 server-chef server-chef

# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

192.168.1.101 server-chef
192.168.1.102 workstation
192.168.1.103 client-node
root@server-chef:~#

Workstation

Now let’s run the same command on other servers.

root@workstation:~# tee -a /etc/hosts<<EOF
> 192.168.1.101 server-chef
> 192.168.1.102 workstation
> 192.168.1.103 client-node
E> EOF
192.168.1.101 server-chef
192.168.1.102 workstation
192.168.1.103 client-node
root@workstation:~#

Client-node

root@client-node:~# tee -a /etc/hosts<<EOF
> 192.168.1.101 server-chef
> 192.168.1.102 workstation
> 192.168.1.103 client-node
> EOF
192.168.1.101 server-chef
192.168.1.102 workstation
192.168.1.103 client-node
root@client-node:~#

It is also essential to check whether changes were added to the host's file using the cat command, the same way as we did on server-chef. To update the system on three servers, run the following command. 

root@server-chef:~# apt update && apt -y upgrade
0 upgraded, 0 newly installed, 0 to remove, and 0 not upgraded.
root@server-chef:~#

Chef Server Installation 

The central Server serves as the main connection point between the workstations and nodes. When configurations are changed and edited on the workstation, they are sent to Master Server, and then all nodes extract the changes from that server. 

Download and Install Chef 

root@server-chef:~# wget https://packages.chef.io/files/stable/chef-server/13.1.13/ubuntu/18.04/chef-server-core_13.1.13-1_amd64.deb

After that, we are going to install the software onto the system.

root@server-chef:~# dpkg -i chef-server-core_*.deb
Selecting previously unselected package chef-server-core.
(Reading database ... 105623 files and directories currently installed.)
Preparing to unpack chef-server-core_13.1.13-1_amd64.deb ...
Unpacking chef-server-core (13.1.13-1) ...
Setting up chef-server-core (13.1.13-1) ...
root@server-chef:~#

To run Chef services using the command line, we are going to use the chef-server-ctl utility. Type yes to accept the licensure during the configuration. 

root@server-chef:~# chef-server-ctl reconfigure
+---------------------------------------------+
            Chef License Acceptance

Before you can continue, 3 product licenses
must be accepted. View the license at
https://www.chef.io/end-user-license-agreement/

Licenses that need accepting:
  * Chef Infra Server  * Chef Infra Client  * Chef InSpec

Do you accept the 3 product licenses (yes/no)?

> yes  
...
Running handlers:
Running handlers complete
Chef Infra Client finished, 482/1032 resources updated in 02 minutes 29 seconds
Chef Server Reconfigured!
root@server-chef:~#

Now we check the status of the running services. 

root@server-chef:~# chef-server-ctl status
run: bookshelf: (pid 24765) 182s; run: log: (pid 18093) 272s
run: nginx: (pid 24591) 185s; run: log: (pid 18742) 214s
run: oc_bifrost: (pid 24467) 187s; run: log: (pid 17831) 294s
run: oc_id: (pid 24584) 186s; run: log: (pid 17858) 289s
run: opscode-erchef: (pid 24779) 183s; run: log: (pid 18237) 269s
run: opscode-expander: (pid 24649) 184s; run: log: (pid 17960) 277s
run: opscode-solr4: (pid 24624) 184s; run: log: (pid 17896) 283s
run: postgresql: (pid 24443) 187s; run: log: (pid 17312) 306s
run: rabbitmq: (pid 25526) 165s; run: log: (pid 19009) 211s
run: redis_lb: (pid 18311) 263s; run: log: (pid 18310) 263s
root@server-chef:~#

Set Up SSHKeys

Next, we are going to create an administrator and organization using private SSH keys. First, let’s start a catalog.

root@server-chef:~# mkdir .chef
root@server-chef:~#

To create an administrative user, we use the chef-server-ctl command again. Use your specific values for the variables USER_NAME and FIRST_NAME 'PASSWORD' like the example below.

chef-server-ctluser-createUSER_NAMEFIRST_NAMELAST_NAMEEMAIL'PASSWORD'--filenameFILE_NAME

It’s important to note that we should use a name similar to chefadmin.pem, where .pem is an RSA key in the FILE_NAME variable. Use your values: 

root@server-chef:~# chef-server-ctl user-create chefadmin Katherine Liquidweb liquidweb@gmail.com 'liquidweb' --filename ~/.chef/chefadmin.pem
root@server-chef:~#

Let’s check the list of users on the server.

root@server-chef:~# chef-server-ctl user-list
chefadmin
pivotal
root@server-chef:~#

Now we are going to create an organization file using the chef-server-ctl command.

chef-server-ctl org-create chef-org "YOUR ORGANIZATION" --association_user USER_NAME --filename FILE_NAME

Use your values here as well.

root@server-chef:~# chef-server-ctl org-create chef-org "Liquid Web Chef Infrastructure" --association_user chefadmin --filename ~/.chef/chef-org.pem
root@server-chef:~#

Now let’s check organizations on the server.

root@server-chef:~# chef-server-ctl org-list
chef-org
root@server-chef:~#

We have installed and configured the Chef Server.

Workstation Installation and Configuration

The Chef workstation is where users create Cookbooks. Cookbooks are blocks of configuration settings that create recipes for accomplishing various tasks. Let’s download and install the Workstation software.

Install Workstation

root@workstation:~# wget https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb

root@workstation:~# dpkg -i chef-workstation_*.deb

Set up Repository

Next, run the command to create a repository where the Cookbooks are stored.

root@workstation:~# chef generate repo chef-repo

Create Catalog

Now, to create a catalog where the Knife configuration and RSA keys are stored, run this command.

root@workstation:~# mkdir ~/chef-repo/.chef
root@workstation:~#

Let’s move into the catalog: 

root@workstation:~# cd chef-repo/
root@workstation:~/chef-repo#

Generate SSH Keys

Now we will generate SSH Keys for Workstation authentication to gain access to the Chef Server. 

root@workstation:~/chef-repo# ssh-keygen -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/.ssh/id_rsa.
Your public key has been saved in /home/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:QY5TxH+NJnsNYSR7Ai8Gixu9LmNnAxY6HuxaO6MpXwI root@workstation
The key's randomart image is:
+---[RSA 4096]----+
| .o= ... |
| o B.o oo |
| + = =.+..+ |
| . . + + ooo= . |
|E = + . S = o |
| + + o . . . |
| = = = . |
|.+o= = . |
|=ooo |
+----[SHA256]-----+
root@workstation:~/chef-repo#

Next, copy the key to the Chef Server with Workstation.

root@workstation:~/chef-repo# ssh-copy-id root@192.168.1.101

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/.ssh/id_rsa.pub"
The authenticity of host '192.168.1.101 (192.168.1.101)' can't be established.
ECDSA key fingerprint is SHA256:yEsdeqtMxrUnxc67qj0v15wIgXbSELQQg2qRc2+j0s4.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Password:

Number of key(s) added: 1

Verify and Share SSH Key

Now, log in using this command to ensure the key(s) is present.

root@workstation:~/chef-repo# ssh 'root@192.168.1.101'

Now we are going to copy chefadmin.pem. and chef-org.pem from the Chef Server to the Workstation. The files must have .pem extension.

root@workstation:~/chef-repo# scp root@192.168.1.101:~/.chef/*.pem ~/chef-repo/.chef/
The authenticity of host '192.168.1.101 (192.168.1.101)' can't be established.
ECDSA key fingerprint is SHA256:yEsdeqtMxrUnxc67qj0v15wIgXbSELQQg2qRc2+j0s4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.101' (ECDSA) to the list of known hosts.
chefadmin.pem 100% 1678 2.4MB/s 00:00
chef-org.pem 100% 1674 4.4MB/s 00:00
root@workstation:~/chef-repo#

To verify the files have been copied, run an ls on the .chef folder.

root@workstation:~/chef-repo# ls ~/chef-repo/.chef
chefadmin.pem chef-org.pem
root@workstation:~/chef-repo#

Adding Version Control

While working on the Workstation, there are many changes made to the Cookbooks. To track these changes, we created a version control system using the create git repositories command in the chef-repo catalog. First, we need to exit the catalog. 

root@workstation:~/chef-repo# cd ..

Set Up Git

Now we will add a username and email address to set up git.

root@workstation:~# git config --global user.name liquidweb
root@workstation:~# git config --global user.email liquidweb@liquidweb.com
root@workstation:~#

Add File to .Gitignore

Next, we add the catalog .chef to .gitignore as there are system files needed for Chef to work correctly. To accomplish this, run the following commands.

root@workstation:~# echo ".chef" /chef-repo/.gitignore
.chef /chef-repo/.gitignore
root@workstation:~#
root@workstation:~# cd chef-repo/
root@workstation:~/chef-repo#

Git Add/Commit

Now let’s run git add and git commit.

root@workstation:~/chef-repo# git add .
root@workstation:~/chef-repo#
root@workstation:~/chef-repo# git commit -m "first commit"
root@workstation:~/chef-repo#

Check Git Status

root@workstation:~/chef-repo# git status
On branch master
nothing to commit, working tree clean
root@workstation:~/chef-repo#

The installation and configuration of Chef Workstation are complete. Now we can move to create a Cookbook. 

Creating a Cookbook

First, we need to leave the catalog to create the Cookbook.

root@workstation:~/chef-repo# cd ..
root@workstation:~#
root@workstation:~# chef generate cookbook liquidweb_cookbook
Generating cookbook liquidweb_cookbook
- Ensuring correct cookbook file content
- Committing cookbook files to git
- Ensuring delivery configuration
- Ensuring correct delivery build cookbook content
- Adding delivery configuration to feature branch
- Adding build cookbook to feature branch
- Merging delivery content feature branch to master
Your cookbook is ready. Type `cd liquidweb_cookbook` to enter it.

There are several commands to run to get started developing and testing your cookbook locally. Type `delivery local --help` to see a full list.

We can also review a default test cookbook which is stored at test/integration/default/default_test.rb. A default recipe can be found here.

recipes/default.rb

Generate Cookbook

Now we are going to create a chef-repo for app using Chef. This will also initialize the catalog.

root@workstation:~# chef generate app chef-repo

Configure Knife

Knife is a command-line tool for Chef used to manage Nodes, Cookbooks, and recipes. To configure it, we will need to create a configuration file config.rb and add the following lines.   

root@workstation:~# nano chef-repo/.chef/config.rb

current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name 'chefadmin'
client_key "chefadmin.pem"
validation_client_name 'chef-org-validator'
validation_key "chef-org-validator.pem"
chef_server_url 'https://server-chef/organizations/chef-org'
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]

In this parameter, we will need to add our URL.

chef_server_url 'https://server-chef/organizations/chef-org'

Next, move into the chef-repo catalog.

root@workstation:~# cd chef-repo/
root@workstation:~/chef-repo#

Copy SSL

Now copy the SSL certificates using the Knife command-line tool. It should be noted that Knife has no method to verify if a certificate is correct. Therefore, we should verify the authenticity of these certificates after downloading them.

root@workstation:~/chef-repo# knife ssl fetch
WARNING: Certificates from server-chef will be fetched and placed in your trusted_cert
directory (/home/chef-repo/.chef/trusted_certs).
Adding certificate for server-chef in /home/chef-repo/.chef/trusted_certs/server-chef.crt
root@workstation:~/chef-repo#

Verify Configuration

Now, verify the configuration file is installed and configured correctly.

root@workstation:~/chef-repo# knife client list
chef-org-validator
root@workstation:~/chef-repo#

Install and Configure Bootstrap for Nodes

Bootstrap installs the Chef client on the Workstation to configure the node to communicate with the Chef master server. Nodes can read configuration from the Chef master server using the client node's username and password in its initial run. The username and password used are the ones configured on the server. You can create and configure a special user who will run Chef commands beforehand. 

We are going to connect to the Node using the following command. 

knife bootstrap YOUR_IP_NODES -x NAME_USER -P PASSWORD --node-name NAME_NODE --sudo

Values: 
YOUR_IP_NODES - your IP which Node has;
NAME_USER - user for work of Node;
PASSWORD - for connection; 
NAME_NODE - Node name.

This is how it would look.

root@workstation:~/chef-repo# knife bootstrap 192.168.1.103 -x liquidweb -P liquidweb.com --node-name client-node-1 --sudo
192.168.1.103 Chef Client finished, 0/0 resources updated in 01 seconds
root@workstation:~/chef-repo#

Now, let’s check which nodes have been found.

root@workstation:~/chef-repo# knife node list
client-node-1
root@workstation:~/chef-repo#

To get more information about the node, run this command.

root@workstation:~/chef-repo# knife node show client-node
Node Name: client-node-1
Environment: _default
FQDN: client-node
IP: 192.168.1.103
Run List:
Roles:
Recipes:
Platform: ubuntu 18.04
Tags:
root@workstation:~/chef-repo#

At this stage, the installation and configuration are complete. 

Conclusion

In this article, we have installed and studied Chef in more detail and learned how it functions. We have also discovered how to install the Master Chef Server, the Chef Workstation, the Chef client, Knife, Cookbooks, and utilizing a Git Repository for Chef. We should note that ongoing work is required to master this tool, but we have provided a solid base to begin your journey in working with Chef. 

Avatar for Katherine Kelly

About the Author: Katherine Kelly

Latest Articles

Blocking IP or whitelisting IP addresses with UFW

Read Article

CentOS Linux 7 end of life migrations

Read Article

Use ChatGPT to diagnose and resolve server issues

Read Article

What is SDDC VMware?

Read Article

Best authentication practices for email senders

Read Article