I’m lazy when come to do repetitive tasks and it’s nice when machines do the work for us, specially boring ones like us sysadmins have to do daily. Example:
Check if that file exists for me at N servers.
Install a package on hundreds of servers.
Delete a user on all servers
and so on...
Haven't you heard of fabric yet?
Fabric is a Python library that provides automation using SSH(Paramiko), from copy files, execute tasks(normal or using sudo), prompt the user for input, deploy and whatever you can think of. :)The fabfile.py used here is on my Github.
First lets install fabric, I will use pip since I’m inside a virtualenv:
(fabric)bicofino@ambush:~/envs$ pip install -U fabric
Downloading/unpacking fabric
Downloading Fabric-1.8.3.tar.gz (251kB): 251kB downloaded
Running setup.py egg_info for package fabric
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no previously-included files matching '*.pyc' found under directory 'tests'
warning: no previously-included files matching '*.pyo' found under directory 'tests'
Downloading/unpacking paramiko>=1.10,<1.13 (from fabric)
Downloading paramiko-1.12.3.tar.gz (1.1MB): 1.1MB downloaded
Running setup.py egg_info for package paramiko
Downloading/unpacking pycrypto>=2.1,!=2.4 (from paramiko>=1.10,<1.13->fabric)
Downloading pycrypto-2.6.1.tar.gz (446kB): 446kB downloaded
Running setup.py egg_info for package pycrypto
Downloading/unpacking ecdsa (from paramiko>=1.10,<1.13->fabric)
Downloading ecdsa-0.11.tar.gz (45kB): 45kB downloaded
Running setup.py egg_info for package ecdsa
Installing collected packages: fabric, paramiko, pycrypto, ecdsa
Running setup.py install for fabric
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no previously-included files matching '*.pyc' found under directory 'tests'
warning: no previously-included files matching '*.pyo' found under directory 'tests'
Installing fab script to /home/bicofino/envs/fabric/bin
Running setup.py install for paramiko
Successfully installed fabric paramiko pycrypto ecdsa
Cleaning up...
(fabric)bicofino@ambush:~/envs$ fab -V
Fabric 1.8.3
Paramiko 1.12.3
On Ubuntu you can install it running apt-get install -y fabric
Now with fabric installed, lets start with a simple task.
Create a file with name fabfile.py on your homedir, remember to always run the fab command where you saved your fabfile.py.
from fabric.api import run
def mem_usage():
'''Check free memory'''
run('free -m')
(fabric)bicofino@ambush:~/envs/fabric$ fab -l
Available commands:
mem_usage Check free memory
(fabric)bicofino@ambush:~/envs/fabric$ fab -l
Available commands:
mem_usage Check free memory
(fabric)bicofino@ambush:~/envs/fabric$ fab -ubicofino -H server01,server02 mem_usage
[server01] Executing task 'mem_usage'
[server01] run: free -m
[server01] Login password for 'bicofino':
[server01] out: total used free shared buffers cached
[server01] out: Mem: 15951 11668 4282 0 189 9945
[server01] out: -/+ buffers/cache: 1533 14418
[server01] out: Swap: 6143 0 6143
[server01] out:
[server02] Executing task 'mem_usage'
[server02] run: free -m
[server02] out: total used free shared buffers cached
[server02] out: Mem: 1877 1562 314 0 15 1210
[server02] out: -/+ buffers/cache: 337 1539
[server02] out: Swap: 4031 0 4031
[server02] out:
Done.
Disconnecting from server02... done.
Disconnecting from server01... done.
The command is pretty straightforward the parameter -u is the user and -H the hosts that you want to run the task and finally the task mem_usage.
Also the parameter -l list your tasks avaiable.
You can use sudo on a task also: First we need edit our fabfile.py to look like this:
from fabric.api import run,sudo
def mem_usage():
'''Check free memory'''
run('free -m')
def sudo_test():
'''Testing with sudo'''
sudo('touch /root/myfile.txt')
sudo('ls -la /root/myfile.txt')
And lets try it.
(fabric)bicofino@ambush:~/envs/fabric$ vim fabfile.py
(fabric)bicofino@ambush:~/envs/fabric$ fab -ubicofino -H server01 sudo_test
[server01] Executing task 'sudo_test'
[server01] sudo: touch /root/myfile.txt
[server01] Login password for 'bicofino':
[server01] sudo: ls -la /root/myfile.txt
[server01] out: -rw-r--r-- 1 root root 0 Mar 24 17:45 /root/myfile.txt
[server01] out:
Done.
Disconnecting from server01... done.
We can copy files with fabric too using the put function:
from fabric.api import run,sudo,put
def copy_file():
'''Copy a local file to a server'''
put('/tmp/myfile.txt', '/tmp/myfile.txt')
run('ls /tmp/myfile.txt')
(fabric)bicofino@ambush:~/envs/fabric$ fab -l
Available commands:
copy_file Copy a local file to a server
mem_usage Check free memory
sudo_test Testing with sudo
(fabric)bicofino@ambush:~/envs/fabric$ fab -ubicofino -H server01 copy_file
[server01] Executing task 'copy_file'
[server01] Login password for 'bicofino':
[server01] put: /tmp/myfile.txt -> /tmp/myfile.txt
[server01] run: ls /tmp/myfile.txt
[server01] out: /tmp/myfile.txt
[server01] out:
Done.
Disconnecting from server01... done.
To copy a file from a server you use the function get:
from fabric.api import run,sudo,put
def copy_file():
'''Copy a local file to a server'''
put('/tmp/myfile.txt', '/tmp/myfile.txt')
run('ls /tmp/myfile.txt')
(fabric)bicofino@ambush:~/envs/fabric$ rm -f /tmp/myfile.txt
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 get_file
[server01] Executing task 'get_file'
[server01] download: /tmp/myfile.txt <- /tmp/myfile.txt
[server01] run: ls /tmp/myfile.txt
[server01] out: /tmp/myfile.txt
[server01] out:
Done.
Disconnecting from server01... done.
(fabric)bicofino@ambush:~/envs/fabric$ cat /tmp/myfile.txt
lalalalalallalala
Pretty simple isn’t?
Now lets do something usefull? New server but no SSH key? Lets automate it!
I have two functions, one to read the keyfile and other is the proper task. What’s new here is the function append as the name says append text to a file.
from fabric.contrib.files import append,exists
def read_key_file(key_file):
key_file = os.path.expanduser(key_file)
if not key_file.endswith('pub'):
raise RuntimeWarning('Trying to push non-public part of key pair')
with open(key_file) as f:
return f.read()
def push_key(key_file='~/.ssh/id_dsa.pub'):
run("rm -rf /home/bicofino/.ssh")
run("mkdir /home/bicofino/.ssh")
key_text = read_key_file(key_file)
run('touch ~/.ssh/authorized_keys')
append('~/.ssh/authorized_keys', key_text)
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 push_key
[server01] Executing task 'push_key'
[server01] run: rm -rf /home/bicofino/.ssh
[server01] Login password for 'root':
[server01] run: mkdir /home/bicofino/.ssh
[server01] run: touch ~/.ssh/authorized_keys
[server01] run: echo 'ssh-rsa Iwillnotprintmykeyhere bicofino@ambush
' >> "$(echo ~/.ssh/authorized_keys)"
Done.
Disconnecting from server01... done.
Now lets work with arguments on tasks, you can pass the argument as task:argument or task:foo=bar. I used warn_only=True to not abort the execution of the task if something fails.
def start(service):
run("/etc/init.d/{0} start".format(service),warn_only=True)
def status(service):
run("/etc/init.d/{0} status".format(service),warn_only=True)
def stop(service):
run("/etc/init.d/{0} stop".format(service),warn_only=True)
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 status:nginx
[server01] Executing task 'status'
[server01] run: /etc/init.d/nginx status
[server01] out: nginx dead but pid file exists
[server01] out:
Warning: run() received nonzero return code 1 while executing '/etc/init.d/nginx status'!
Done.
Disconnecting from server01... done.
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 start:nginx
[server01] Executing task 'start'
[server01] run: /etc/init.d/nginx start
[server01] out: Starting nginx: [ OK ]
[server01] out:
[server01] out:
Done.
Disconnecting from server01... done.
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 stop:nginx
[server01] Executing task 'stop'
[server01] run: /etc/init.d/nginx stop
[server01] out: Stopping nginx: [ OK ]
[server01] out:
[server01] out:
Done.
Disconnecting from server01... done.
Also you can use prompt to type any kind of information, in the example below the directory:
def prompt_test():
local = prompt('Type dir to list:')
dir = '/{0}'.format(local)
with cd(dir):
run('ls')
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -H server01 prompt_test
[server01] Executing task 'prompt_test'
Type dir to list: /root
[server01] run: ls
[server01] out: backup configuration myfile.txt skeletons swap trust
[server01] out:
Done.
Disconnecting from server01... done.
So far so nice? Now a example of roles, a role is a decorator used to hook up host lists.
Add to your fabfile.
env.roledefs = {
'webservers': ['web01','web02','web03','web04'], # My web servers
}
Now you can pass a task to a bunch of servers using fab -R:
(fabric)bicofino@ambush:~/envs/fabric$ fab -uroot -R webservers status:httpd
[web01] Executing task 'status'
[web01] run: /etc/init.d/httpd status
[web01] Login password for 'root':
[web01] out: httpd (pid 31886) is running...
[web01] out:
[web02] Executing task 'status'
[web02] run: /etc/init.d/httpd status
[web02] out: httpd (pid 19000) is running...
[web02] out:
[web03] Executing task 'status'
[web03] run: /etc/init.d/httpd status
[web03] out: httpd (pid 22912) is running...
[web03] out:
[web04] Executing task 'status'
[web04] run: /etc/init.d/httpd status
[web04] out: httpd (pid 28713) is running...
[web04] out:
Done.
Disconnecting from web02... done.
Disconnecting from web01... done.
Disconnecting from web04... done.
Disconnecting from web03... done.
This is a beginner tutorial, fabric has much more use and options, maybe I will cover that in another post. (That’s why I’m using Part 1. :P)
Feel free to leave a comment if you have any questions or suggestions. For more information please visit http://docs.fabfile.org/