Being Lazy With Fabric - Part 1

I'm lazy when come to do repetitive tasks and it's nice when machines do the work for us, specially boring ones like us sysadmins have to do daily.
Example:

Check if that file exists for me at N servers.  
Install a package on hundreds of servers.  
Delete a user on all servers  
and so on...  

Haven't you heard of fabric yet?

Fabric is a Python library that provides automation using SSH(Paramiko), from copy files, execute tasks(normal or using sudo), prompt the user for input, deploy and whatever you can think of. :)

The fabfile.py used here is on my Github.

First lets install fabric, I will use pip since I'm inside a virtualenv:

(fabric)bicofino@ambush:~/envs$ pip install -U fabric
Downloading/unpacking fabric  
  Downloading Fabric-1.8.3.tar.gz (251kB): 251kB downloaded
  Running setup.py egg_info for package fabric

    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
Downloading/unpacking paramiko>=1.10,<1.13 (from fabric)  
  Downloading paramiko-1.12.3.tar.gz (1.1MB): 1.1MB downloaded
  Running setup.py egg_info for package paramiko

Downloading/unpacking pycrypto>=2.1,!=2.4 (from paramiko>=1.10,<1.13->fabric)  
  Downloading pycrypto-2.6.1.tar.gz (446kB): 446kB downloaded
  Running setup.py egg_info for package pycrypto

Downloading/unpacking ecdsa (from paramiko>=1.10,<1.13->fabric)  
  Downloading ecdsa-0.11.tar.gz (45kB): 45kB downloaded
  Running setup.py egg_info for package ecdsa

Installing collected packages: fabric, paramiko, pycrypto, ecdsa  
  Running setup.py install for fabric

    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    Installing fab script to /home/bicofino/envs/fabric/bin
  Running setup.py install for paramiko
Successfully installed fabric paramiko pycrypto ecdsa  
Cleaning up...

(fabric)bicofino@ambush:~/envs$ fab -V
Fabric 1.8.3  
Paramiko 1.12.3  

On Ubuntu you can install it running apt-get install -y fabric

Now with fabric installed, lets start with a simple task.

Create a file with name fabfile.py on your homedir, remember to always run the fab command where you saved your fabfile.py.

from fabric.api import run

def mem_usage():  
    '''Check free memory'''
    run('free -m')
(fabric)bicofino@ambush:~/envs/fabric$ fab -l
Available commands:

    mem_usage  Check free memory


(fabric)bicofino@ambush:~/envs/fabric$ fab -l
Available commands:

    mem_usage  Check free memory

(fabric)bicofino@ambush:~/envs/fabric$ fab -ubicofino -H server01,server02 mem_usage
[server01] Executing task 'mem_usage'
[server01] run: free -m
[server01] Login password for 'bicofino': 
[server01] out:              total       used       free     shared    buffers     cached
[server01] out: Mem:         15951      11668       4282          0        189       9945
[server01] out: -/+ buffers/cache:       1533      14418
[server01] out: Swap:         6143          0       6143
[server01] out: 

[server02] Executing task 'mem_usage'
[server02] run: free -m
[server02] out:              total       used       free     shared    buffers     cached
[server02] out: Mem:          1877       1562        314          0         15       1210
[server02] out: -/+ buffers/cache:        337       1539
[server02] out: Swap:         4031          0       4031
[server02] out: 


Done.  
Disconnecting from server02... done.  
Disconnecting from server01... done.  

The command is pretty straightforward the parameter -u is the user and -H the hosts that you want to run the task and finally the task mem_usage.
Also the parameter -l list your tasks avaiable.

You can use sudo on a task also:
First we need edit our fabfile.py to look like this:

from fabric.api import run,sudo

def mem_usage():  
    '''Check free memory'''
    run('free -m')

def sudo_test():  
    '''Testing with sudo'''
    sudo('touch /root/myfile.txt')
    sudo('ls -la /root/myfile.txt')

And lets try it.

(fabric)bicofino@ambush:~/envs/fabric$ vim fabfile.py
(fabric)bicofino@ambush:~/envs/fabric$ fab -ubicofino -H server01 sudo_test
[server01] Executing task 'sudo_test'
[server01] sudo: touch /root/myfile.txt
[server01] Login password for 'bicofino': 
[server01] sudo: ls -la /root/myfile.txt
[server01] out: -rw-r--r-- 1 root root 0 Mar 24 17:45 /root/myfile.txt
[server01] out: 


Done.  
Disconnecting from server01... done.  

We can copy files with fabric too using the put function:

from fabric.api import run,sudo,put  
def copy_file():  
    '''Copy a local file to a server'''
    put('/tmp/myfile.txt', '/tmp/myfile.txt')
    run('ls /tmp/myfile.txt')
(fabric)bicofino@ambush:~/envs/fabric$ fab -l
Available commands:

    copy_file  Copy a local file to a server
    mem_usage  Check free memory
    sudo_test  Testing with sudo

(fabric)bicofino@ambush:~/envs/fabric$ fab -ubicofino -H server01 copy_file
[server01] Executing task 'copy_file'
[server01] Login password for 'bicofino': 
[server01] put: /tmp/myfile.txt -> /tmp/myfile.txt
[server01] run: ls /tmp/myfile.txt
[server01] out: /tmp/myfile.txt
[server01] out: 


Done.  
Disconnecting from server01... done.  

To copy a file from a server you use the function get:

from fabric.api import run,sudo,put  
def copy_file():  
    '''Copy a local file to a server'''
    put('/tmp/myfile.txt', '/tmp/myfile.txt')
    run('ls /tmp/myfile.txt')
(fabric)bicofino@ambush:~/envs/fabric$ rm -f /tmp/myfile.txt 
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 get_file
[server01] Executing task 'get_file'
[server01] download: /tmp/myfile.txt <- /tmp/myfile.txt
[server01] run: ls /tmp/myfile.txt
[server01] out: /tmp/myfile.txt
[server01] out: 


Done.  
Disconnecting from server01... done.  
(fabric)bicofino@ambush:~/envs/fabric$ cat /tmp/myfile.txt 
lalalalalallalala  

Pretty simple isn't?

Now lets do something usefull?
New server but no SSH key? Lets automate it!

I have two functions, one to read the keyfile and other is the proper task. What's new here is the function append as the name says append text to a file.

from fabric.contrib.files import append,exists

def read_key_file(key_file):  
    key_file = os.path.expanduser(key_file)
    if not key_file.endswith('pub'):
        raise RuntimeWarning('Trying to push non-public part of key pair')
    with open(key_file) as f:
        return f.read()

def push_key(key_file='~/.ssh/id_dsa.pub'):  
    run("rm -rf /home/bicofino/.ssh")
    run("mkdir /home/bicofino/.ssh")
    key_text = read_key_file(key_file)
    run('touch ~/.ssh/authorized_keys')
    append('~/.ssh/authorized_keys', key_text)
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 push_key
[server01] Executing task 'push_key'
[server01] run: rm -rf /home/bicofino/.ssh
[server01] Login password for 'root': 
[server01] run: mkdir /home/bicofino/.ssh
[server01] run: touch ~/.ssh/authorized_keys
[server01] run: echo 'ssh-rsa Iwillnotprintmykeyhere bicofino@ambush
' >> "$(echo ~/.ssh/authorized_keys)"

Done.  
Disconnecting from server01... done.  

Now lets work with arguments on tasks, you can pass the argument as task:argument or task:foo=bar. I used warn_only=True to not abort the execution of the task if something fails.

def start(service):  
    run("/etc/init.d/{0} start".format(service),warn_only=True)

def status(service):  
    run("/etc/init.d/{0} status".format(service),warn_only=True)

def stop(service):  
    run("/etc/init.d/{0} stop".format(service),warn_only=True)
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 status:nginx
[server01] Executing task 'status'
[server01] run: /etc/init.d/nginx status
[server01] out: nginx dead but pid file exists
[server01] out: 


Warning: run() received nonzero return code 1 while executing '/etc/init.d/nginx status'!


Done.  
Disconnecting from server01... done.

(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 start:nginx
[server01] Executing task 'start'
[server01] run: /etc/init.d/nginx start
[server01] out: Starting nginx:                         [  OK  ]
[server01] out: 
[server01] out: 


Done.  
Disconnecting from server01... done.

(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 stop:nginx
[server01] Executing task 'stop'
[server01] run: /etc/init.d/nginx stop
[server01] out: Stopping nginx:                         [  OK  ]
[server01] out: 
[server01] out: 


Done.  
Disconnecting from server01... done.  

Also you can use prompt to type any kind of information, in the example below the directory:

def prompt_test():  
    local = prompt('Type dir to list:')
    dir = '/{0}'.format(local)
    with cd(dir):
        run('ls')
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 prompt_test
[server01] Executing task 'prompt_test'
Type dir to list: /root  
[server01] run: ls
[server01] out: backup    configuration  myfile.txt  skeletons    swap  trust
[server01] out: 


Done.  
Disconnecting from server01... done.  

So far so nice? Now a example of roles, a role is a decorator used to hook up host lists.

Add to your fabfile.

env.roledefs = {  
    'webservers': ['web01','web02','web03','web04'],                                         # My web servers
}
Now you can pass a task to a bunch of servers using fab -R:

(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -R webservers status:httpd
[web01] Executing task 'status'
[web01] run: /etc/init.d/httpd status
[web01] Login password for 'root': 
[web01] out: httpd (pid  31886) is running...
[web01] out: 

[web02] Executing task 'status'
[web02] run: /etc/init.d/httpd status
[web02] out: httpd (pid  19000) is running...
[web02] out: 

[web03] Executing task 'status'
[web03] run: /etc/init.d/httpd status
[web03] out: httpd (pid  22912) is running...
[web03] out: 

[web04] Executing task 'status'
[web04] run: /etc/init.d/httpd status
[web04] out: httpd (pid  28713) is running...
[web04] out: 


Done.  
Disconnecting from web02... done.  
Disconnecting from web01... done.  
Disconnecting from web04... done.  
Disconnecting from web03... done.  

This is a beginner tutorial, fabric has much more use and options, maybe I will cover that in another post. (That's why I'm using Part 1. :P)

Feel free to leave a comment if you have any questions or suggestions.
For more information please visit http://docs.fabfile.org/