OSU DevOps BootCamp Documentation

OSU DevOps BootCamp Documentation
OSU DevOps BootCamp
Documentation
Release 0.0.1
OSU OSL
OSU LUG
August 23, 2017
Contents
1
Ready to Learn DevOps? Lesson 0: Start Here
3
2
Schedule
2.1 Fall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Winter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Spring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
5
5
5
3
Donate
3.1 Lesson 0: Start Here . . . . . . . . . . .
3.2 Lesson 1: First Steps . . . . . . . . . . .
3.3 Lesson 2: Operating Systems . . . . . .
3.4 Lesson 3: Shell Navigation . . . . . . . .
3.5 Lesson 4: Users, Groups, Permissions . .
3.6 Lesson 5: Files . . . . . . . . . . . . . .
3.7 Lesson 6: Packages, Software, Libraries .
3.8 Lesson 7: Questions, Answers, Docs . .
3.9 Lesson 8: Version Control . . . . . . . .
3.10 Lesson 9: Programming . . . . . . . . .
3.11 Lesson 10: Frameworks . . . . . . . . .
3.12 Lesson 11: Testing . . . . . . . . . . . .
3.13 Lesson 12: Continuous Integration . . .
3.14 Lesson 13: Security . . . . . . . . . . .
3.15 Lesson 14: Databases . . . . . . . . . .
3.16 Lesson 15: Dev Processes & Tools . . .
3.17 Lesson 16: DNS . . . . . . . . . . . . .
3.18 Lesson 17: Configuration Management .
3.19 Lesson 18: Application Isolation . . . . .
3.20 Lesson 19: Cloud Infrastructure . . . . .
3.21 Lesson 20: Contributing to Open Source
3.22 About . . . . . . . . . . . . . . . . . . .
3.23 Schedule . . . . . . . . . . . . . . . . .
3.24 Running DOBC . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7
7
10
14
17
20
23
26
28
33
38
45
49
52
54
58
66
70
76
82
85
87
89
91
92
i
ii
OSU DevOps BootCamp Documentation, Release 0.0.1
DevOps BootCamp (DOBC) is a free course hosted by the OSU Open Source Lab. The course is dedicated to teaching
core software development and systems operation skills to passionate OSU students and community members.
DOBC is always 100% free for in-person and online students.
Contents
1
OSU DevOps BootCamp Documentation, Release 0.0.1
2
Contents
CHAPTER 1
Ready to Learn DevOps? Lesson 0: Start Here
DevOps Bootcamp’s curriculum is available for you to learn at your own pace. Get started now!
3
OSU DevOps BootCamp Documentation, Release 0.0.1
4
Chapter 1. Ready to Learn DevOps? Lesson 0: Start Here
CHAPTER 2
Schedule
The DevOps BootCamp content is available for free but meet-space guided lectures are offered throughout the year.
Check the schedule below for our in-person lectures; each lecture covers a different part of the curriculum covering
the entire course during the OSU academic school year.
Warning: If you are working ahead be aware that the schedule and slides may be subject to change. Check back
regularly.
Fall
Lesson
Fall Kickoff
Date/Time
Oct 7, 2017 9:30am-3:30pm
Location
OSU KEAR 112
Description
DevOps BootCamp Fall Kickoff
Winter
Lesson
TBA
Date/Time
Location
Description
Date/Time
Location
Description
Spring
Lesson
TBA
5
OSU DevOps BootCamp Documentation, Release 0.0.1
6
Chapter 2. Schedule
CHAPTER 3
Donate
We appreciate the help! To donate, go to http://osuosl.org/donate.
Lesson 0: Start Here
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
7
OSU DevOps BootCamp Documentation, Release 0.0.1
About the Program
DevOps
DevOps is a field which takes skills from Software Development and Operations Engineering to create
and run applications more effectively.
TLDR: Development + Operations == Better Services
What DevOps BootCamp (DOBC) is
TLDR: Couch to DevOps in 1 school year
DOBC is a free education program offering:
• Mentors teaching DevOps related tools and concepts.
• A challenge for anybody willing to put in the effort.
• One-on-one Apprenticeship.
• Hands-on training and lectures
• Free and Open Source course materials!
What DOBC is not
DevOps BootCamp is not:
8
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• A for-credit OSU class
• A Student job
• Easy
Why DOBC Exists
DOBC was created because the OSU OSL:
1. Merged with the school of EECS.
2. Wanted to help students meet Company demands and expectations of recent graduates.
3. Needed to bridge the “Skills Gap” of the OSU EECS curriculum.
4. Wanted to build a DevOps Learning community.
What You Will do
You will Learn:
• Linux systems
• Networking
• Software development
• Tools and why they matter
You will build:
• Functioning applications on the cloud
• Cloud infrastructures
Who Teaches DOBC
The teachers of DOBC include:
• OSL Students
• OSL Faculty
• Guests from The Industry
• You!
The ‘Agreement’
You get out what you put in. DOBC is not meant to be easy. Stick with it, persistence is rewarded.
Student Benefits: A free education on industry topics, tools, and concepts
Student Responsibilities: Show up if you can, keep up if you cannot, put forth effort, and don’t forget to have fun.
Give us feedback.
• There will be a survey you, should take it.
• Honesty is the best policy.
3.1. Lesson 0: Start Here
9
OSU DevOps BootCamp Documentation, Release 0.0.1
Getting Involved
Where To Ask Questions
• Internet Relay Chat
• Mailing lists
• During Lecture and Hand-on Lessons
• More on the About page...
How To Ask Questions
• Always be respectful to those helping you.
• Stay calm and articulate.
• Explain you are trying to achieve and be thorough.
The OSU Open Source Lab is Hiring
For more information check the OSL Hiring Page regularly.
Lesson 1: First Steps
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Vocabulary
A 10,000ft view of the world
General Topics:
• Software: A program that runs on a computer.
• Operating System: Computer software that manages other software.
10
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• GNU/Linux: A free Operating System.
• Computer Security: Like physical security but harder to solve with a baseball bat.
• Virtual Machine: A computer emulated in software.
Development:
• Version Control: A way to track changes and contributions to a project.
• Continuous Integration: Releasing updates continuously.
Buzzwords:
• FOSS: Free (and Libre) Open Source Software. Free as in Speech, not Free as in Pizza (but that too
usually).
• ‘The Cloud’: Computers somewhere else.
• Containers: Not virtual machines, but basically virtual machines.
TODO: What Vocabulary Do You Know?
What other vocabulary can you think of related to DevOps?
What about Silicon Valley, Programming, System Administration, etc?
Note: This is a TODO. It’s basically an exercise or activity but with a cheeky name. Try them out if you don’t feel
confident in a topic.
Notation
• Variable (use whatever word you like here e.g., foo, bar, baz)
– $ONE_VARIABLE_NOTATION
– <another notation for variables>
• Literal (copy this exactly)
– copy_me_exactly
• Comments (parts of the code just for humans)
– this_is(code) # everything after the octothorp is a comment!
– other_code(line) // This can also be a comment.
langauge!
It depends on the
• Code-block:
#! /usr/bin/env python
# This is a code block.
# Most of the time you can copy this code and run it exactly as is.
# It should be clear Where it ’goes’ and how to run it based on context.
print(’Hello world!’)
$ echo Hello World
# Copy the text after ‘$‘ into your termianal and press enter.
3.2. Lesson 1: First Steps
11
OSU DevOps BootCamp Documentation, Release 0.0.1
TODO: Reading Examples
Trick question: how would you read this
#!/bin/python
dogs = [’$BREED_ONE’, ’$BREED_TWO’, ’$BREED_THREE’]
for breed in dogs:
print(breed)
Answer: Reading Examples
Replace the $BREED_N with actual dog breeds.
#!/bin/python
dogs = [’corgie’, ’pug’, ’french bulldog’]
for breed in dogs:
print(breed)
Getting Setup on Linux
Lecture Setup
1. Get login credentials from your lecturer.
• You will be provided a username, password, host, and port.
12
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Linux/Mac:
2. Open a terminal and verify you have ssh installed by entering the command ssh --version.
3. Run ssh -p <port> <username>@<host> and enter the password when prompted (it will hide
your password in the terminal).
Windows:
2. Install an SSH Client (install Putty)
3. Log into your remote Linux environment using the credentials given to you.
(a) Under Host Name (or IP address) enter <user>@<host>, under Port enter <port>.
(b) You will be prompted for your password in new window, it will hide the password as you type it.
Home Setup
We suggest you install Vagrant, a tool which makes it easy to run and acquire Virtual Machines.
You
may
also
need
to
install
VirtualBox,
a
tool
necessary
for
Vagrant
to
function.
TODO: Change Your Password!
Challenge Change your password on your Linux machine.
$ passwd
Changing password for user <user>.
Changing password for <user>.
(current) UNIX password: # Enter old password, hidden
New password:
# Enter new password, also hidden
Retype new password:
passwd: all authentication tokens updated successfully.
Don’t forget: when you login next time, use the new password you just set.
Further Reading
• More information on Virtual Machines.
• Install Putty if you want to access a remote Linux box.
• Install Vagrant if you want to run a local Linux Virtual machine.
• Install VirtualBox in addition to Vagrant for local virtual machines.
3.2. Lesson 1: First Steps
13
OSU DevOps BootCamp Documentation, Release 0.0.1
Lesson 2: Operating Systems
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
What an Operating System is
14
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Anatomy of an OS
• User Interface: What you interact with. Window Managers for instance.
• Application Layer: What developers use to make software run.
• Kernel: The Core of the OS. Makes communication between hardware and applications sane.
• Hardware: What does the actual computations. The thing your keyboard is plugged into.
Types of Operating Systems
Popular Operating Systems
• UNIX
– Linux
* Android
* Debian
* RHEL
– MacOS / Darwin
– FreeBSD
• Windows
GNU/Linux
Welcome to the Family
3.3. Lesson 2: Operating Systems
15
OSU DevOps BootCamp Documentation, Release 0.0.1
Flavors of Linux
• Debian
– Ubuntu
* LinuxMint
• RedHat
– RHEL
– Fedora
– Centos
• Gentoo
– ChromeOS
• Slackware
• ArchLinux
TODO: Pop Quiz
1. What are some different types of Operating Systems?
16
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
2. What constitutes a ‘Distribution’ of Linux?
3. How is Linux different from Windows? OSX?
4. How is Debian different from Gentoo?
Further Reading
OSU Courses:
CS 344: Operating Systems I
• Required course for all CS Students at OSU.
• Covers fundamentals of low-level programming concepts.
– Multi-threaded programming
– Read / Write operations
– Socket programming
CS 444: Operating Systems II
• Required course for all CS Students at OSU.
• Covers kernel hacking and low-level OS design.
– IO / Process scheduling
– Building kernel modules
– Memory management
Free Online Resources: OSDev.org is a wiki dedicated to helping people develop their own operating systems. It’s
a big leap from this lesson, but great if you’re interested in learning the nitty-gritty.
Operating Systems Design and Implementation by Andrew S. Tanenbaum is a classsic in the world of OS
Development. It’s also a big leap, but can teach you more about how Operating Systems work than you ever
thought there was to know.
Lesson 3: Shell Navigation
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
The Shell
A shell is a text-based user-interface for a computer.
3.4. Lesson 3: Shell Navigation
17
OSU DevOps BootCamp Documentation, Release 0.0.1
Shell Examples
sh Required by all POSIX Operating Systems.
bash Default on most GNU/Linux-based Operating Systems.
csh Default shell on most BSD (Unix) based Operating Systems
fish Useful but not sh compliant shell.
zsh The hip new shell on the block.
Navigation Concepts
Basic Shell Commands
$
$
$
$
$
$
$
$
$
$
pwd
# Prints the current working directory (where you are)
ls
# Prints the contents of the current working directory
cd <path/to/other/directory>
# Navigates to a new directory.
echo "some thing $AND_VARS"
# Prints a string to the screen.
cat foo.txt bax.txt # Prints the contents of a file(s) to the screen.
grep foo file.txt
# Searches ‘file.txt‘ for the string ‘foo‘
less file.txt
# Prints a file to the screen so you can arrow up/down.
env
# Prints environment variables to the screen.
whoami # Prints out current user
help
# When in doubt, always type help.
Shell Scripts
about_me.sh
#!/bin/sh
if [ $(whoami) == "root" ]; then
echo "You’re root!"
else
echo "Your username is $(whoami)"
echo "Your home-directory is $HOME"
echo "Your current directory is $PWD"
echo "Your computer’s host-name is $HOSTNAME"
fi
Invoke with:
18
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
$ chmod +x about_me.sh
$ ./about_me.sh
# Tell Linux that this can be run as a program.
# Invoke the script.
File Paths
. The current directory.
.. The parent directory.
~ Alias for your home directory.
/ Separates directories: one_dir/another_dir/last_dir
Alone, or at the start of a path, it is the root directory.
$ tree -F
.
|-- bar/
|
|-- one
|
‘-- two
|-- baz/
‘-- foo/
‘-- a/
‘-- b/
5 directories, 2 files
Special Characters
Wildcard (*) Used as a stand-in for any character(s).
Example: cat *.log cats all files in the current working directory ending in .log.
End of line ($) Used to specify the end of a regex. We’ll cover what regex is later.
Curl braces ({ }) Used to specify a set.
Example:
ls {foo,bar,baz}ley-thing expands to ls fooley-thing barley-thing
bazley-thing
Escape special characters (treat them as normal characters) with the escape character (\).
Type Less, Tab More
Pressing the tab key auto-completes a command, file-path, or argument in your shell.
Pressing tab multiple times completes the command to the best of the shells ability and then lists the possible completions (if there are any).
$ ls b
# <tab>
$ ls ba
# <tab>
bar_thing/ baz_thing/
$ ls bar # <tab>
$ ls bar_thing
3.4. Lesson 3: Shell Navigation
19
OSU DevOps BootCamp Documentation, Release 0.0.1
TODO
Further Reading
BASH Programming - Introduction HOW-TO A free online resoruce of learning bash programming. Covers some
concepts we’ll get to later in DOBC, but a good resoruce to have on hand.
Running rm -rf / on Linux This video demonstrates what happens when you ‘delete your hard-drive’ on Linux. A
fun watch!
Lesson 4: Users, Groups, Permissions
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
The User
You... ish.
$
$
$
$
whoami
who
w
id
#
#
#
#
your username
who is logged in?
who is here and what are they doing?
user ID, group ID, and groups you’re in
Sometimes robots are users too: Apache, Mailman, ntp.
What a User has
/etc/passwd:
root:x:0:0:root:/root:/bin/bash
username:password:uid:gid:uid info:home directory:shell
What Users Can Do
• Change Passwords with the passwd command.
• Act as Another user with su.
$
$
$
$
su $USER
su
sudo su sudo su $USER
#
#
#
#
become user, with
become root, with
use your password
become $USER with
THEIR password
root’s password
instead of root’s
your password
• Act as themselves.
– ls -l to see file permissions.
– Check the file’s group and user.
– Check the file’s read, write, and execute bits.
20
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Managing Groups and Users
$ cat /etc/passwd
# username:x:UID:GID:GECOS:homedir:shell
$ useradd <user_name>
$ userdel <user_name>
$ passwd
$
$
$
$
# vs adduser, the friendly Ubuntu version
groupadd
usermod
groupmod
cat /etc/group
root:x:0:
Examples of Non-Human Users
• mailman: For the mailing list program.
• apache: For the HTTP Server.
• postfix: For the other mail program.
3.5. Lesson 4: Users, Groups, Permissions
21
OSU DevOps BootCamp Documentation, Release 0.0.1
Root and Sudo
Root: Basically god on Linux.
[[email protected] ~]$ yum install httpd
# Runs command as ‘foo‘ user
Loaded plugins: fastestmirror, ovl
ovl: Error while doing RPMdb copy-up:
[Errno 13] Permission denied: ’/var/lib/rpm/__db.002’
You need to be root to perform this command.
[[email protected] ~]$ sudo yum install httpd
# Runs command as ‘root‘ user.
password:
Loaded plugins: fastestmirror, ovl
[... installs correctly ...]
Warning: Acting as root is dangerous! You can accidentally delete your filesystem, forcing you to completley
re-install your OS! Type carefully.
TODO
• Create a user on your system for yourself, with your preferred username.
• Give your user sudo powers.
• Use su to get into your user account.
• Change your password.
• Create a directory called bootcamp in your home directory.
• Create a group called devops.
22
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Further Reading
• Understanding Linux File Permissions
Lesson 5: Files
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
About Files
Everything in Linux is a file... except the things that aren’t.
Files have:
• Owners
• Permissions (what different people can do with it)
• An inode (a low-level description of the file)
• Size
• Filename
$ ls -il
total 8
2884381 drwxrwxr-x 5 test test 4096 Nov 6 11:46 Documents
2629156 -rw-rw-r-- 1 test test
0 Nov 13 14:09 file.txt
2884382 drwxrwxr-x 2 test test 4096 Nov 6 13:22 Pictures
Everything is a file!?
Yes. Except the things that aren’t..
int read_medical_device_data(int device_file_pointer) {
// Open a connection to the device
int * stream = open(device_file_pointer);
// Write the stream of data to the screen
write(STDOUT, stream);
// Do some other stuff with that data
// Close the data stream
close(stream);
return EXIT_SUCCESS;
}
File Extensions
.jpg, .txt, .py
3.6. Lesson 5: Files
23
OSU DevOps BootCamp Documentation, Release 0.0.1
Not necessary, more of a recommendation.
$ ls
some_text_file
squirrel
$ file some_text_file
some_text_file: ASCII text
$ file squirrel
squirrel: JPEG image data, JFIF standard 1.01
Hidden Files
Any file starting with . is called a hidden file and is not listed by default.
Adding the -a flag to ls command includes hidden files in your output.
$ ls
Documents
file.txt
$ ls -a
. .. Documents
Pictures
file.txt
.hidden_file
Pictures
.vimrc
Finding Metadata with ‘ls -l’
$ ls -l
drwxrwxr-x
5
test
test
4096
Nov 6 11:46 Documents
-rw-rw-r-1
test
test
0
Nov 13 14:09 file.txt
drwxrwxr-x
2
test
test
4096
Nov 6 13:22 Pictures
---------------- ------- -------- ------------ ------------|
|
|
|
|
|
|
|
|
|
|
File Name
|
|
|
|
+--- Modification Time
|
|
|
+------------- Size (in bytes)
|
|
+----------------------Group
|
+-------------------------------Owner
+---------------------------------------------- File Permissions
Editing Metadata
$ chown root myfile
# Change the owner of myfile to "root".
$ chown root:staff myfile
# Change the owner of myfile to "root" and group to "staff".
$ chown -hR root /mydir
# Change the owner of /mydir and subfiles to "root".
$ chgrp -R devops /home/$yourusername/bootcamp
# Make the group devops own the bootcamp dir
24
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
chmod and Octal Permissions
+-----+--------+-------+
| rwx | Binary | Octal |
+-----+--------+-------+
| --- | 000
| 0
|
| --x | 001
| 1
|
| -w- | 010
| 2
|
| -wx | 011
| 3
|
| r-- | 100
| 4
|
| r-x | 101
| 5
|
| rw- | 110
| 6
|
| rwx | 111
| 7
|
+-----+--------+-------+
• u, g, o for user, group, other
• -, +, = for remove, add, set
• r, w, x for read, write, execute
Executing a File?
For instance:
$ ls -alh my-script
-r-xr-xr-x 1 username username 1.9K Sep 27 09:44 my-script
$ cat my-script
#!/bin/bash
# The above line tells Linux how to invoke the script on my behalf.
echo ’This is a script being run without using bash!’
$ ./my-script # my-script is invoked just like a compiled binary!
This is a script being run without using bash!
Types of Files
• - is a normal file
• d is a directory
• b is a block device
• l is a symlink
Directories
Directories are also files!
• +r allows you to read the contents of the directory.
• +w allows you to add files to the directory.
• +x allows you to use the directory at all.
3.6. Lesson 5: Files
25
OSU DevOps BootCamp Documentation, Release 0.0.1
$ ls -alh | grep foobarbaz
drw-rw-rw- 2 voigte
voigte
4.0K Sep 29 10:47 foobarbaz
$ ls -alh foobarbaz
# Below is the literal output, not psuedo-output
ls: cannot access foobarbaz/.: Permission denied
ls: cannot access foobarbaz/..: Permission denied
total 0
d????????? ? ? ? ?
? .
d????????? ? ? ? ?
? ..
TODO: Messing with Files
$ touch foo # create empty file called foo
• Create an empty file in /home/$yourusername/bootcamp.
• Who can do what to the file?
• Change the group to devops.
• Make a file called allperms and give user, group, and world +rwx.
• Make more files and practice changing their permissions.
Further Reading
• Permission Mishaps
• Access the Linux kernel using the /proc filesytem
Lesson 6: Packages, Software, Libraries
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Software
Everything that isn’t hardware.
• Code that is run on a Computer.
• Binaries.
• Scripts.
Libraries
• Often used to make development easier.
• Rarely run on it’s own.
26
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• Shared code.
Package Management
• Automagically manage software and libraries on your system.
• Examples:
– Android Play Store
– Apple App store
– Steam
Core Package Management Functionality
TLDR: To take care of installation, removal, and updates of software.
Popular Linux System Package Managers
Popular Linux Package Managers:
Apt (.deb, dpkg) Used by default on the Debian, Ubuntu, Linux Mint operating systems.
Yum (.rpm, rpm) Used by default on the RedHat, CentOS, Fedora operating systems.
Programming Langauge Package Managers
Examples:
• Python: pip
• Ruby: gem, rubygems
• Haskell: cabal
• NodeJS: npm
• ... and so on forever ...
Other Package Managers
Portage The Source-based package manager for Gentoo.
Yaourt An Arch User Repository wrapper for Pacman, the Arch Linux Package manager.
Nix A ‘Fully Functional/Transactional’ package manager.
Brew An Open Soruce package manager for OSX.
Chocolatey A package manager for Windows.
3.7. Lesson 6: Packages, Software, Libraries
27
OSU DevOps BootCamp Documentation, Release 0.0.1
Installation from Soruce
How to install a package from source:
Using grep as an example:
$
$
$
$
$
$
wget http://mirrors.kernel.org/gnu/grep/grep-2.25.tar.xz
tar -Jxvf grep-2.25.tar.xz
cd grep-2.25
./configure --prefix=$HOME/bin/
make
make install
TODO: Install sl
• Install the git, gcc, make, ncurses-bin,
libncurses5-dev packages via package manager.
ncurses-base,
libncurses5-dev,
and
$ sudo apt install git gcc make ncurses-bin ncurses-base libncurses5-dev libncurses5-dev
[...]
• Install sl from source into the directory ~/bin/.
$ git clone https://github.com/mtoyoda/sl.git
[...]
$ cd sl
$ make
gcc -O -o sl sl.c -lncurses
$ mkdir ~/bin
$ ln sl ~/bin/
$ echo "export PATH=$PATH:$HOME/bin" >> ~/.bashrc
$ source ~/.bashrc
$ whereis sl
sl: /home/username/bin/sl
$ sl
Further Reading
• More about APT
Lesson 7: Questions, Answers, Docs
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
When in doubt
$ <program> --help
$ <program> -h
28
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Most programs allow you to pass a help flag which will print out basic usage. This is useful as a quick reference for
how to use the program.
Man Pages
$ man <program>
• Type / and then enter a keyword to see where that word appears.
• Press n to go to the next (and p to go to the previous) occurrence of that word.
$ man man
MAN(1)
Manual pager utils
MAN(1)
NAME
man - an interface to the on-line reference manuals
SYNOPSIS
man [-C file] [-d] [-D] [--warnings[=warnings]] [-R encoding] [-L
locale] [-m system[,...]] [-M path] [-S list] [-e extension] [-i|-I]
[...]
DESCRIPTION
man is the system’s manual pager. Each page argument given to man is
normally the name of a program, utility or function. The manual page
[...]
Anatomy of a Man Page
Most Man Pages include:
• Name
• Flags
• Description
• Basic Usage
• Authors
If you’re lucky they will also include:
• A Good description
• Advanced Usage.
• Examples
• History
• See Also
Sections of Man
1. Executable programs or shell commands
2. System calls (function provided by the kernel)
3.8. Lesson 7: Questions, Answers, Docs
29
OSU DevOps BootCamp Documentation, Release 0.0.1
3. Library calls (functions provided from within libraries)
4. Special files (usually found in /dev)
5. Files formats and conventions eg /etc/passwd
6. Games
7. Miscellaneous (including macro packages and conventions), e.g., man(7), groff(7)
8. System administration commands (usually only for root)
9. Kernel routines [Non standard]
Note: Some distros use info instead of man. To learn more about the info command, see Further Reading.
Project Docs
Where to look:
• http://docs.some-random-project.io/
• http://some-random-project.io/docs/
• http://organization.com/some-random-project/
Questions and Answers
• Stack Overflow
• Forums
• Mailing Lists
• Blogs
30
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
How to Talk to People
• Chatrooms
• Meetups
• Face to Face (!?)
IRC
Quick Facts:
• Internet Relay Chat (IRC)
• Very old (RFC 1459, May 1993)
• Works on everything (Terminal, GUI, Web-browser, etc)
• The people you want to listen to are there
• Oregon State ran one of the first servers ever!
TODO Getting on IRC
To get on IRC, Use irssi or weechat in screen:
# This step is optional, but persistent IRC is cool
$ ssh <username>@<a remote linux server>
# start screen with the name ’irc’
$ screen -S irc
#
$
#
$
start your client in the 0th window of the screen session
irssi
or
weechat-curses
#
#
#
$
$
exit irc screen with CTRL+a, CTRL+d
exit ssh session with CTRL+d or ’exit’
to get back to irc:
ssh <username>@<preferred shell host>
screen -dr IRC
Connecting and Setup
In the IRC client run these commands
/connect irc.freenode.net
/nick <myawesomenickname>
/msg nickserv register <password> <email>
/nick <myawesomenickname>
/msg nickserv identify <password>
/join #devopsbootcamp
3.8. Lesson 7: Questions, Answers, Docs
31
OSU DevOps BootCamp Documentation, Release 0.0.1
Commands and Tips
• /list: Reports all the channels on a server.
• /topic: Reports current channel topic.
• /names: Reports nicks of users in channel.
• /join <channel>: Join a new channel.
• /whois <nick>: Learn about a person.
• /msg: Directly message an individual.
• /help <command>
• Tab-completion works with nicks
• You get a hi-light when your name is said.
• Symbols (@, +) are not part of names, show status in channel.
• chanserv and nickserv are robots.
– /msg nickserv help to get nick help.
– /msg chanserv help to get channel help.
IRC Jargon
• ping/pong: “I would like to tell you something”/”I’m here, tell it to me.”
• tail: “~”
• hat: “@” Denotes admin status in a channel.
• nick: Your name.
• netsplit: When the IRC servers lose connection with eachother.
32
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• kick/ban/k-line: GTFO
Asking for Help
It’s okay to ask for help. Here are some things to keep in mind:
1. Ask yourself what should be happening?
2. Ask yourself what is actually happening?
3. Google the problem(s).
4. Skim the manuals of each component.
5. Identify a friend, mentor, or IRC channel who could help.
6. When they’re not busy, give them a quick synopsis of points 1 and 2, mentioning what possibilities you’ve ruled
out by doing steps 3 and 4.
Contributions = expertise + time
Further Reading
• About info: info is an alternative to man that some distros use instead.
Lesson 8: Version Control
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
3.9. Lesson 8: Version Control
33
OSU DevOps BootCamp Documentation, Release 0.0.1
Text Editor: Nano
• User types like normal.
• Arrow keys used to to navigate the cursor.
• ^ + <key> Commands (control + key)
Version Control Systems
VCS is how one tracks changes, modifications, and updates to source files over time. Creating a history of changes
for a project over time.
Used for:
• Documentation
• Code
• Configuration
• Collaboration
Other Names Include:
• Source Control Management (SCM)
• Version Control Software
• Revision Control Software
What VCS Solves
Version control solves a lot of problems:
34
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• I have changes I want to integrate (merge) into the main project.
• I want to track the state of this project over time.
• I want to make some changes without possibly breaking what I have.
• ... and much more.
Principles of VCS
Types of VCS
Git
Git is a Free and Open Source distributed version control system designed to handle everything from small
to very large projects with speed and efficiency. ( https://git-scm.com )
Setting up Git
$
$
$
$
sudo yum install git
git config --global user.name "My Name"
git config --global user.email "[email protected]"
git config --global core.editor "nano"
TODO: Use Git Locally
Create a project with Git:
$ mkdir my-project
$ cd my-project
# Always run ‘git init‘ inside of a project folder!
$ git init
# Never inside of your home directory.
Add and commit a file to your project with Git:
$ touch newfile.txt
$ git add newfile.txt
$ git commit # Edit message in Nano, save the file, exit to commit.
3.9. Lesson 8: Version Control
35
OSU DevOps BootCamp Documentation, Release 0.0.1
To see which files are staged, unstaged, or untracked:
$ git status
To look through your repository history:
$ git log
To create and checkout a branch:
$ git branch
# Shows your branches and current branch
* master
$ git checkout -b <new-branch> # Switches to new branch ‘<branch name>‘
$ git branch
master
* new-branch
$ git checkout master
# Switches to existing branch ‘<branch name>‘
TODO: Working With a Git Repository
• Checkout a new feature branch on your repository.
• Create/Edit files on the new branch.
• Create a diff between the two.
• Locally merge the changes from your new branch into Master.
36
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
What not to do with Git
Workflow(s)
Everybody uses VCS differently.
Choose the workflow that works best for everybody involved.
Centralizing Git
Gitlab Open Source, free to run, feature rich.
Github Very popular. Not Open Source but free for Open Source projects.
Bitbucket Also popular, similar to Github, unlimited free private and public repositories.
Gitolite Bare-bones. Fewer feature than the previous three. Open Source, useful for learning the nitty-gritty Git really
works.
Cloning a Repository
$
$
$
$
cd /path/to/my/projects
git clone <some git url>
cd <new repo directory>
ls
3.9. Lesson 8: Version Control
37
OSU DevOps BootCamp Documentation, Release 0.0.1
TODO: Cloning Exercise
$ git clone https://github.com/DevOpsBootcamp/tinsy-flask-app.git
See http://git.io/vcVmB for more details about the tinsy-flask-app repository.
$
$
$
$
cd tiny-flask-app
virtualenv venv
pip install -r requirements.txt
python script.py
Now if you go to <your ip address>:<http port> in your web-browser to see a live version of the app!
Further Reading
The Online Git Docs This is a portal to all of the official docs on git-scm.com. It includes everything from Getting
Started to Git Internals. Check it out!
Git workflow tutorial This is the tutorial provided on https://git-scm.com/about/distributed. It is a good high-level
overview of some common git workflows.
A successful Git branching model This blogpost describes a git workflow (git-flow) that the Open Source Lab bases
their workflow on.
Lesson 9: Programming
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Paradigms
Programming is a big topic.
Note: Pseudo-code
function f(x):
# This line is a comment, not run by the computer.
# Comments are only for human eyes.
if x is less than than 5
print "x is less than 5"
else if x is less than than 10
print "x is greater than five and less than 10"
else
print "x is greater than 10"
38
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Variables & Constants
>>> x = "value"
>>> print(x)
value
>>> x = "different value"
>>> print(x)
different value
Data Types
Data types dictate how a piece of data should be handled within a program.
Flow Control
Flow Control allows you to execute code only if certain conditions are met.
Conditionals: If / Else If / Else Conditionals are used to tell the program when to execute commands.
In pseudocode, they usually look something like
if some conditional statement is true
do something
else if some other conditional
do something else
else
do a final thing
Loops: For / While / Do While Loops are used to do multiple things, usually an indefinite number of things.
For instance:
for every element, let’s call it "foo", in a list "my_list"
if foo is greater than five
print(foo)
else
print(foo + " is too small")
While loops execute indefinitely (while something continues to be true).
For loops iterate over a list (array) of elements or to a specific number.
Input & Output
>>>
>>>
>>>
>>>
>>>
>>>
user_input = get_input("Where would you like to go today? ")
-> Where would you like to go today? Nebraska
print(user_input)
-> nebraska
print(reverse(nebraska))
-> aksarben
Functions
3.10. Lesson 9: Programming
39
OSU DevOps BootCamp Documentation, Release 0.0.1
function read_file(x):
# Also check that it exists! How convenient!
if file_exists(x)
v = read_file_to_string(x)
return v
else
print("file does not exist")
return Null
Structs
struct dog {
breed: String
height: Float
color: String
age: Integer
}
spot = struct dog
# Create a new variable of type ‘struct dog‘
spot.breed = "corgie" # Assign each member a variable.
spot.height = 1.5
spot.color = "Blond"
spot.age = 1
print(spot.breed, spot.height, spot.color, spot.age)
Objects
class chair():
function init(material):
self.material = material
function rock():
print("The ", self.material, " chair rocks slowly.")
>>> my_chair = chair.init("plastic")
>>> my_chair.rock()
>>> -> The plastic chair rocks slowly.
Libraries
import math_lib
print(math_lib.pi, math_lib.pow(2, 5), math_lib.tan(79.3))
# prints out "3.14 32 .951"
TODO: Write Pseudo-Code
Write pseudo-code to do the following tasks:
• Count to 20 (hint: for loop).
• Get user input and print it.
40
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• Generate prime numbers.
Hints:
• Break the problem down to the simplest steps.
• Don’t worry about the details.
• This is pseudo-code! Get creative.
Python
$ sudo <apt or yum> install python
Python Datatypes
• You don’t need to declare the type of your variables, Python will assume the type of your variable and type it
for you.
• Python is a duckly-typed language. If it walks like a duck and quacks like a duck, then Python treats it like a
duck. As long as an object implements the proper interfaces, it can act like any type it wants.
3.10. Lesson 9: Programming
41
OSU DevOps BootCamp Documentation, Release 0.0.1
Type
boolean
integer
long
float
string
list
dict (map)
tuple
object
None
Example
True
7
18,446,744,073,709,551,615
12.4
"Hello World!"
[’first’, ’second’]
{’key1’: ’value’, ’key2’, ’value2’}
(’value’,’paired value’)
anObjects.variable == <value>
Python Variables
# This is a comment
boolean = True # boolean
name = "Lucy" # string
age = 20 # integer
pi = 3.14159 # float
alphabet = [’a’, ’b’, ’c’]
dictionary = {"pi":3.14159, "sqrt 1":1}
winter = (’December’, ’January’, ’February’, ’March’)
print(name + " is " + str(age+1) + " this " winter[3])
REPL: Try it out
Open a REPL (Read Evaluate Print Loop):
$ python
>>> print("I’m in a REPL!")
>>> name =
# <Your name>
>>> age =
# <Your age>
>>> print(name + " is " + str(age))
>>> # We need to convert age from int to string so it can print!
Python Control Flow
if name == "Lucy":
for month in winter:
print name + " doesn’t like " + month
else:
print "My name isn’t Lucy!"
Python Functions
def myfunction(arg1, arg2):
return arg1 + arg2
print myfunction(1, 5)
42
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Python Libraries
There are a few ways to use other code in your code:
from math import pi
x = pi
from math import *
x = pi
There are hundreds of Python libraries. If you’re trying to do something and think “This has probably been solved...”,
Google it!
Some libraries to know:
• sys
• os
• dateutil
• future
• And more
3.10. Lesson 9: Programming
43
OSU DevOps BootCamp Documentation, Release 0.0.1
Python (Virtual) Environments
$ sudo apt-get install python-virtualenv
$ sudo yum install
# In each project you work on, you’ll want to run
$ virtualenv venv
$ source venv/bin/activate
(venv)$ pip install <package>
(venv)$ deactivate
TODO: Practicing Python
Formalize the last TODO by writing them in Python.
Prove the program works by running the code!
Further Reading
Python on Learnpython.org The Python programming language’s website offers some good (free) tutorials and reference documentation.
Python on Codecademy Codecademy is a great resource for learning many programming languages and offers a
good (free) beginner’s guide to Python.
CS 160, 161, 162 These OSU courses focus on programming fundamentals covered in this lesson in greater detail.
Python is used in CS 160 and C/C++ is used in CS 161 and CS 162.
44
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Lesson 10: Frameworks
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Frameworks
• Web frameworks
• Game frameworks
• GUI frameworks
The job of a framework
To take care of the boring stuff.
Why and When to use a Framework
Use a framework if you are making a cookie cutter application.
If a framework exists for what you’re doing, consider using it.
Types of Frameworks
• Testing Frameworks
• Web-app Frameworks
• Game Frameworks
Web Frameworks
3.11. Lesson 10: Frameworks
45
OSU DevOps BootCamp Documentation, Release 0.0.1
Static vs Dynamic Sites
There are two types of websites: Static and Dynamic.
Static Site Rarely changes, looks the same for all visitors (Blog, News, Document)
Dynamic Site Changes based on who you are and what you do. (Search Engine, Login)
Popular Web Frameworks
Python
Django Offers many feature out of the box: Admin page, easy database management, simple templating, convenient URL routing. Well documented too.
Flask Sparsely featured, offers very little out of the box and lets you build up the features you need. Well
supported with community libraries and add-ons.
Ruby
Rails Arguably the most popular web-framework out there. Similar to Django in it’s features out of the box.
Sinatra Analogous to Flask on the Python side, very simple and easy to start with, encourages building up the
features you need.
Node.js
ExpressJS A bare-bones NodeJS application, similar again to Flask.
46
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
The Model-View-Controller Pattern
URL Routing
@app.route(’/accounts/<account_name>’, methods=[’DELETE’])
def delete_account(account_name):
if authenticated() and authorized():
database.remove_account(account_name)
return ’Success’, 200
3.11. Lesson 10: Frameworks
47
OSU DevOps BootCamp Documentation, Release 0.0.1
else
return ’Failure’, 401
Templating Engines (mad-libs!)
<!DOCTYPE HTML>
<html>
<head>
<title>Template Example</title>
</head>
<body>
<p>Your lucky number today is {{ number }}!</p>
</body>
</html>
render_template("template.html", number=random.randint(0, 99))
Your lucky number today is 42!
...
<body>
{% for message in messages %}
<p>{{ message }}</p>
{% endfor %}
</body>
...
messages = ["Welcome!", "Test Message", "Vim > Emacs"]
render_template("template2.html", messages=messages)
Welcome!
Test Message
Vim > Emacs
HTTP
GET http://web.site/page.html HTTP/1.1
HTTP/1.1 200 OK
Content-Type: text/html
...
<!DOCTYPE HTML>
...
HTTP Methods
REST
• Servers are stateless
• Resources are self-contained
48
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
• HTTP methods have predictable side-effects
TODO: Dynamic Website
Part One: Writing The Views
Part Two: Writing The Templates
Further Reading
The Flask Microframework Flask is a web framework that is simple enough for beginners to use but configurable
enough to allow more advanced users to have full control over their application. It has a very active community
and fantastic documentation.
Intro to HTTP and REST HTTP is the protocol that web clients and web servers use to communicate with each
other, and REST is a set of web design guidelines that is takes advantage of HTTP’s features and allows different
applications to easily communicate with each other.
Lesson 11: Testing
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Testing
def add_double(x, y):
return 2*(x+y)
def test_add_double():
expect(add_double(1, 2) == 6)
3.12. Lesson 11: Testing
49
OSU DevOps BootCamp Documentation, Release 0.0.1
Why Testing Matters
Structure of a Test
Most tests consist of the same general structure:
Types of Testing
Concept: Mocking
Simulating behavior external to a program so your tests can run independently of other platforms.
You’re testing your program, not somebody else’s. Mock other people’s stuff, not your own.
Testing Frameworks
$ run tests
Finding tests...
Running tests in tests/foo.ext
Running tests in tests/bar.ext
Running tests in misc/test_baz.ext
50
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Frameworks vs ‘The Hard Way’
While you can write tests the hard way:
var = some_function(x)
if var == expected_output:
continue
else
print("Test X failed!")
$ run test
Test 5 failed!
It’s usually easier to use a framework.
def simple_test():
expect(some_function(x), expected_output)
$ run tests
....x.....
Test 5 failed.
Debug information:
...
Teardown and Setup
Useful for:
• populating a test database
• writing and deleting files
• or anything else you want!
def tests_setup():
connect to database
populate database with test data
def tests_teardown():
delete all data from test database
disconnect from database
def some_test()
setup is called automatically
use data in database
assert something is true
teardown is run automatically
TODO: Using Python’s unittest
Let’s suppose that we want to add a new view to the Flask app we created in the Frameworks lesson’s TODO. When
the user enters the url /hello/<name>, where “name” is any string of the user’s choice, the view should return “Hello
<name>!!” BEFORE you actually write this view, write a test that will test the desired functionality first– i.e., test that
your hello.py returns “Hello bob!!” when “bob” is provided as the name variable. AFTERWARDS, implement the
actual view to make your test(s) pass.
Unittesting in Flask Check out the official Flask docs for help with syntax.
3.12. Lesson 11: Testing
51
OSU DevOps BootCamp Documentation, Release 0.0.1
Further Reading
CS 362 This OSU Course covers testing very in depth and even covers types of testing including Random testing and
testing analysis.
Python Unittest Documentation A good reference for using Python’s built-in unit-testing module.
Lesson 12: Continuous Integration
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Continuous Integration
52
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Automated Testing
Tool: Travis CI
Runs test suites for:
• C / C++
• Java
• Javascript
• Python
• Ruby
• Many more on the Travis CI docs!
Tool: Jenkins
• Does pretty much anything you can tell a computer to do, automatically.
• Builds and uploads binaries (releases).
• Runs tests.
• Reports build successes/failures.
• Also has plugins!
3.13. Lesson 12: Continuous Integration
53
OSU DevOps BootCamp Documentation, Release 0.0.1
TODO: Setup Travis on a GH Repo
Further Reading
Jenkins Documentation The Jenkins project documentation. If you need a broad overview read the Getting Started
with... docs.
TravisCI Documentation If you end up working on a large project on GitHub you’re going to interface with TravisCI
sooner or later.
CircleCI Documentation CircleCI is a tool we didn’t get to touch on. It is very similar to Travis.
Lesson 13: Security
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Security
se·cu·ri·ty ( sikyooritē/ ) [ noun ] The state of being free from danger or threat.
The safety of a state or organization against criminal activity such as terrorism, theft, or espionage.
Types of Security
There are three main types of security in computing:
Threat Models
Threat models allow you to focus and limit your security resources on what is necessary instead of what
is possible.
54
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Access Control
Passwords / Passphrases
Problems with Passwords
• People repeat passwords.
• Many passwords are easy to guess.
• Passwords are hard to remember.
Solutions for Passwords
3.14. Lesson 13: Security
55
OSU DevOps BootCamp Documentation, Release 0.0.1
Choosing Pass-phrases
Relevant funny bash.org post
Certificates and HTTPS
56
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Types of Attacks
Code Injection
Code Injection Attacks
<img src="http://example.com/?action="Delete All Accounts">
Code Injection Defenses
3.14. Lesson 13: Security
57
OSU DevOps BootCamp Documentation, Release 0.0.1
Web Server Attacks
Discovering Vulnerabilites
1. Test and document the bug to verify it exists. If you think you encountered a bug, make sure you can replicate it. If you can’t how can you expect the developers to recreate it?
2. Disclose it privately to those responsible for fixing it. Provide examples – it’s basically a bug report, but
through private channels (not public tracker yet!)
3. Give them time to release a patch before announcing it. Google waits 90 days to announce a bug after informing the developers.
Further Reading
codebashing.com/sql_demo Try your hand at actual SQL Injection attacks
OverTheWire Wargames Learn the basics of offensive security by solving challenges and using exploits to gain
access to the password for the next level.
Lesson 14: Databases
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
58
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Databases
Relating Data
Imagine a kitchen cupboard program that stores food currently in stock, where it is, recipes using it,
expiration dates, etc.
Databases and Structure
Structure SQL databases are based on around Relational Algebra
<Table 1>
+---------------+-----------+-----------+
| <Primary key> | <Field 1> | <Field 2> |
+---------------+-----------+-----------+
| 1
| value
| value‘
|
| ...
| ...
| ...
|
+---------------+-----------+-----------+
<Table 2>
+---------------+-----------+--------------------------+
| <Primary key> | <Field 1> | <Foreign key to Table 1> |
+---------------+-----------+--------------------------+
| 1
| val
| 7
|
| ...
| ...
| ...
|
+---------------+-----------+--------------------------+
Concept: Relational Algebra
<Table 1>
+------------------+------------------+
| <Name>
| <Major>
|
+------------------+------------------+
| Linus Torvalds
| Computer Science |
| Richard Stallman | Computer Science |
+------------------+------------------+
<Table 2>
+------------------+--------------+----------------+
| <Major>
| <School>
| <Advisor Name> |
+------------------+--------------+----------------+
| Computer Science | Engineering | Dennis Ritchie |
+------------------+--------------+----------------+
<Table 1> JOIN <Table 2>
+------------------+------------------+-------------+----------------+
| <Name>
| <Major>
| <School>
| <Advisor Name> |
+------------------+------------------+-------------+----------------+
| Linus Torvalds
| Computer Science | Engineering | Dennis Ritchie |
| Richard Stallman | Computer Science | Engineering | Dennis Ritchie |
+------------------+------------------+-------------+----------------+
3.15. Lesson 14: Databases
59
OSU DevOps BootCamp Documentation, Release 0.0.1
When to use a Database
When you have to work with a lot of well structured data.
Databases are useful for two situations:
1. Lots of data.
2. High throughput.
60
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Lots of Data
Note: 1 PB = 1,000,000 GB
Concurrent Read/Writes
• Atomicity: Either the entire transaction succeeds or it fails completely
• Consistency: Transactions always leave the database in a valid state
• Isolation: Concurrent operations look like they took place sequentially
• Durability: Transactions are permanent after they’re committed
When not to use a Database
Databases might not be particularly useful for:
• Storing content for a website that rarely updates
– Alternative: Use a static site generator such as Pelican or Jekyll
• Hosting large individual files
– Alternative: Store the files on disk
3.15. Lesson 14: Databases
61
OSU DevOps BootCamp Documentation, Release 0.0.1
Types of Databases
There are two broad types of databases.
• SQL: Stores data in tables organized by column and field.
• NoSQL: Stores data differently than an SQL database.
• NewSQL: A middle-ground between SQL and NoSQL
SQL
Examples:
• MySQL/MariaDB
• PostgreSQL
• SQLite
NoSQL
Examples:
• MongoDB
• Apache Casandra
• Dynamo
• Redis
Database Concepts
Schemas
CREATE TABLE nobel (
id int(11)
NOT NULL
AUTO_INCREMENT,
yr int(11),
subject varchar(15),
winner varchar(50)
)
ENGINE = InnoDB;
Migrations
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
(’app’, ’0001_initial’)
]
operations = [
62
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
migrations.AddField("Nobel", "topic", models.CharField(80))
]
Raw SQL Syntax
SELECT
SELECT
yr, subject, winner
FROM
nobel
WHERE
yr = 1960 AND subject=’medicine’;
+------+------------+-------------------------------+
| yr
| subject
| winner
|
+------+------------+-------------------------------+
| 1960 | "medicine" | "Sir Frank Macfarlane Burnet" |
| 1960 | "medicine" | "Sir Peter Brian Medawar"
|
+------+------------+-------------------------------+
INSERT
INSERT INTO
nobel
VALUES
(’2013’,’Literature’,’Herta Müller’);
+-----+------+--------------+----------------+
| id | yr
| subject
| winner
|
+-----+------+--------------+----------------+
| ... | ... | ...
| ...
|
| 873 | 2013 | "Literature" | "Herta Müller" |
| ... | ... | ...
| ...
|
+-----+------+--------------+----------------+
UPDATE
UPDATE
nobel
SET
winner=’Andrew Ryan’
WHERE
subject=’Peace’ AND yr=’1951’;
+-----+------+---------+----------------+
| id | yr
| subject | winner
|
+-----+------+---------+----------------+
| ... | ... | ...
| ...
|
| 120 | 1951 | "Peace" | "Andrew Ryan" |
| ... | ... | ...
| ...
|
+-----+------+---------+----------------+
3.15. Lesson 14: Databases
63
OSU DevOps BootCamp Documentation, Release 0.0.1
DELETE
DELETE FROM
nobel
WHERE
yr = 1989 AND subject = ’peace’;
TODO: Crafting Queries!
Craft a query to get the following data out of our Nobel table:
• Who won the prize for Medicine in 1952?
• Who won the 1903 Nobel in Physics?
• Which prize(s) were awarded to Linus Pauling?
• How many people have won more than once? (Difficult)
Don’t worry about getting it exactly right! Craft pseudo-SQL!
Answers
SELECT winner FROM nobel
WHERE yr=1952 AND subject=’medicine’; #(Selman A. Wksman)
SELECT * FROM nobel
WHERE yr=1903 AND subject=’physics’; #(3)
SELECT * FROM nobel
WHERE winner=’Linus Pauling’; #(2)
SELECT COUNT(*) FROM nobel
AS n0 INNER JOIN nobel AS n1 on n0.winner=n1.winner
AND (n0.yr!=n1.yr or n0.subject!=n1.subject); #(16)
TODO: Using a Real Database
Installing MySQL
#
#
#
$
#
$
Install mysql -- hit ’enter’ to name your user root, and then enter
again for password
On Debian-based systems:
sudo apt update && sudo apt install mysql-server
On Red Hat/Fedora based systems:
sudo yum install mysql-server
$ sudo /etc/init.d/mysql start
# Start the mysql service
$ mysql_secure_installation # Use this to set the root password
# Hit ’yes’ or ’y’ for all options
# Add a sensible password which you will remember
# DO NOT MAKE IT YOUR USUAL PASSWORD.
64
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
$ sudo /etc/init.d/mysql status
$ mysqladmin -u root -p ping # Ping the database
$ mysqladmin -u root -p create nobel # Create a table for Nobel prizes
Users
Login to the mysql shell with your root user credentials:
$ sudo mysql -p
mysql> CREATE USER ’me’@’localhost’
IDENTIFIED BY ’password’;
mysql> GRANT ALL PRIVILEGES ON nobel.*
TO ’me’@’localhost’
WITH GRANT OPTION;
mysql> exit
Importing Data
#
$
$
#
$
#
$
Get the database from the osl server
sudo apt install wget
wget http://osl.io/nobel -O nobel.sql
put the database in a file called nobel.sql
sudo mysql -p nobel < nobel.sql
Open up mysql shell to execute queries
sudo mysql -p nobel
# List all the tables
SHOW TABLES;
# Print the layout of the database to the screen
DESCRIBE nobel;
Ways to Use a Database
Raw Queries
mysql> SELECT subject, yr, winner FROM nobel
WHERE yr=1960;
+------+------------+-----------------------------+
| yr
| subject
| winner
|
+------+------------+-----------------------------+
| 1960 | Chemistry | Willard F. Libby
|
| 1960 | Literature | Saint-John Perse
|
| ... | ...
| ...
|
+------+------------+-----------------------------+
3.15. Lesson 14: Databases
65
OSU DevOps BootCamp Documentation, Release 0.0.1
Native Queries
#!/usr/bin/python
import MySQLdb
db = ("localhost","testuser","test123","nobel" )
cursor = db.cursor()
cursor.execute("SELECT subject, yr, winner FROM nobel WHERE yr = 1960")
data = cursor.fetchall()
for winner in data:
print "%s winner in %s: %s " % (winner[0], winner[1], winner[2])
db.close()
Object Relational Mappers
• Maps an Object in an application to a database table or relationship.
• Talks SQL to the database, your favorite language to you.
• Lets you point to different databases with the same syntax.
• Intelligently manages transactions to the database.
# SELECT * FROM nobel WHERE yr = 1960
for subject, yr, winner in session.query(Nobel).filter_by(yr=1960):
print "%s winner in %s: %s " % (subject, yr, winner)
Further Reading
CS 340 The CS 340 course at OSU (titled “Databases”) is a great introduction to this topic. If you have the option to
take it you should!
Lesson 15: Dev Processes & Tools
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Code Analysis
• Static Analysis
• Dynamic Analysis
66
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Debugging Tools
• Print (broken) variables.
• Read and reports error messages.
• Highlight (incorrect) syntax.
CLI Debugging Tools
C/C++ Tools
• GDB
• Valgrind
Python Tools
• PDB
NodeJS Tools
• node debug
• Node Inspector
Web Consoles
3.16. Lesson 15: Dev Processes & Tools
67
OSU DevOps BootCamp Documentation, Release 0.0.1
• Ctrl+Shift+K (Command+option+k) in Firefox
• Ctrl+Shift+I (Cmd+opt+I) in Chrome
Linters
Examples:
• flake8 (Python)
• splint (C)
• jshint (NodeJS)
src/times.js: line 407, col 20, Expected ’{’ and instead saw ’return’.
src/times.js: line 415, col 49, Missing semicolon.
src/times.js: line 407, col 58, ’error’ is not defined.
Code Coverage
Name
Stmts
Miss Cover
Missing
------------------------------------------------------my_program.py
20
4
80%
33-35, 39
my_other_module.py
56
6
89%
17-23
------------------------------------------------------TOTAL
76
10
87%
Integrated Development Environments (IDE)
Examples:
• Netbeans (Java)
• Visual Studio (.NET)
• PyCharm (Python)
• Eclipse (Many)
68
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Style Guides
Example: Real-World Style Guides
The Linux kernel style guidelines are actually fun to read:
“First off, I’d suggest printing out a copy of the GNU coding standards, and NOT read it. Burn them, it’s
a great symbolic gesture.”
—Linux Kernel Coding Style
NASA’s Jet Propulsion Laboratory style guidelines are very short and are concerned with automated tooling to do code
analysis:
“All loops shall have a statically determinable upper-bound on the maximum number of loop iterations.”
—JPL Coding Standard
Dependency Isolation
TODO: Python Virtualenvs
Setup and enter the virtual environment.
$ virtualenv <virtualenv name>
New python executable in /path/to/<venv name>/bin/python
Installing setuptools, pip, wheel...
done.
$ source <venv name>/bin/activate
Install a package. This installs it in the current working directory and so does not ask for root permissions.
(<venv name>) $ pip install flask
[...]
To list all packages in the venv:
(<venv name>) $ pip freeze
click==6.6
Flask==0.11.1
itsdangerous==0.24
Jinja2==2.8
3.16. Lesson 15: Dev Processes & Tools
69
OSU DevOps BootCamp Documentation, Release 0.0.1
MarkupSafe==0.23
Werkzeug==0.11.11
Deactivate (leave) the venv.
(<venv name>) $ deactivate
$
Other Examples
Node.js: Creates a node_modules directory and tracks dependencies in package.json.
Go: Dependencies are tracked via git repositories and using the go get command.
Rust: Dependencies and versions are specified in Cargo.toml. All compiled code (and dependencies) are stored
in a target directory.
Development Servers
A Carbon Copy of the Production Environment(s)
Further Reading
• The Community Ruby Style Guide is a good resource for anybody learning Ruby. It’s the style guide that
Rubocop enforces.
• The Official Python Style Guide (PEP8) is a well respected style guide for Python and is commonly accepted as
the python style guide.
Lesson 16: DNS
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Use it for learning purposes at your own peril.
If you have any feedback, please fill out our General Feedback Survey.
70
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Problems DNS Solves
Obligatory History Lesson
HOSTS.TXT circa 1977:
MIT
Yale
Harvard
ATT
...
1
2
3
4
HOSTS.TXT a few years later:
...
joeBillson
susan-gill
...
14895
15832
How DNS Works
1. Computer A wants to fetch data from devopsbootcamp.osuosl.org. (notice the . at the end of the
address).
2. Computer A checks the local cache.
3.17. Lesson 16: DNS
71
OSU DevOps BootCamp Documentation, Release 0.0.1
3. If the address isn’t in the cache, A contacts the DNS root server. (We’re actually skipping a few layers of
cache. Read up for more info on that.)
4. One of the root nodes tells A to check the org node.
5. The org node is contacted and tells A to check the osuosl node.
6. The osuosl node tells it to check the devopsbootcamp node.
A DNS Request
DNS Records
Acronym
A, AAAA
MX
NS
SOA
PTR
CNAME
Name
IP Addresses
SMTP Mail Exchangers
Name Servers
DNS Zone Authority
Pointers for Reverse DNS Lookups
Domain Name Aliases
A Records
The A record is used to map an IP address to a domain name. This is as close to a ‘regular’ record as you
can get.
osuosl.org.
300 IN
A
140.211.15.183
MX Records
The MX record is for tracking mail servers.
osuosl.org.
osuosl.org.
osuosl.org.
osuosl.org.
72
3600
3600
3600
3600
IN
IN
IN
IN
MX
MX
MX
MX
5
5
5
5
smtp3.osuosl.org.
smtp4.osuosl.org.
smtp1.osuosl.org.
smtp2.osuosl.org.
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
NS Records
Servers with a NS record are allowed to speak with authority on a domain and DNS requests.
osuosl.org.
osuosl.org.
osuosl.org.
86258
86258
86258
IN
IN
IN
NS
NS
NS
ns1.auth.osuosl.org.
ns2.auth.osuosl.org.
ns3.auth.osuosl.org.
SOA (Authority) Records
SOA is the record for proving authority over a site or zone.
osuosl.org.
86400
IN
SOA ns1.auth.osuosl.org. ...
CNAME Records
CNAME is an record for aliasing old names to redirect to new names.
bar.example.com.
86400
IN
CNAME
foo.example.com
NXDOMAIN Records
Tells you there is no answer to a query:
Host something.invalid.osuosl.org not found: 3(NXDOMAIN)
Some ISPs and others never serve NXDOMAINS, instead they point you at themselves.
The Root
$ dig ns .
;; ANSWER SECTION:
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
.
512297
3.17. Lesson 16: DNS
IN
IN
IN
IN
IN
IN
IN
IN
IN
IN
IN
IN
IN
NS
NS
NS
NS
NS
NS
NS
NS
NS
NS
NS
NS
NS
i.root-servers.net.
e.root-servers.net.
d.root-servers.net.
j.root-servers.net.
b.root-servers.net.
a.root-servers.net.
f.root-servers.net.
h.root-servers.net.
g.root-servers.net.
c.root-servers.net.
m.root-servers.net.
k.root-servers.net.
l.root-servers.net.
73
OSU DevOps BootCamp Documentation, Release 0.0.1
The Thirteen
Tool: dig
dig is a command-line tool for performing DNS lookups.
Syntax:
dig @server name type
Examples:
dig @ns1.osuosl.org osuosl.org A
Example: Recursive Request
First we query a NS record for .:
$ dig ns .
;; QUESTION SECTION:
;.
IN NS
;; ANSWER SECTION:
.
518400
.
518400
.
518400
.
518400
.
518400
74
IN
IN
IN
IN
IN
NS
NS
NS
NS
NS
i.root-servers.net.
a.root-servers.net.
l.root-servers.net.
f.root-servers.net.
b.root-servers.net.
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
etc...
Next we query NS for org.:
$ dig ns com. @a.root-servers.net
;; QUESTION SECTION:
;org.
IN NS
;; AUTHORITY SECTION:
org.
172800
org.
172800
IN
IN
NS
NS
a0.org.afilias-nst.info.
a2.org.afilias-nst.info.
etc...
;; ADDITIONAL SECTION:
a0.org.afilias-nst.info. 172800 IN
A
199.19.56.1
etc...
Next we query NS for osuosl.org.:
$ dig ns osuosl.org. @199.19.56.1
;; QUESTION SECTION:
;osuosl.org.
IN NS
;; AUTHORITY SECTION:
osuosl.org.
86400
osuosl.org.
86400
osuosl.org.
86400
IN
IN
IN
;; ADDITIONAL SECTION:
ns1.auth.osuosl.org.
ns2.auth.osuosl.org.
ns3.auth.osuosl.org.
86400
86400
86400
NS
NS
NS
ns3.auth.osuosl.org.
ns2.auth.osuosl.org.
ns1.auth.osuosl.org.
IN
IN
IN
A
A
A
140.211.166.140
140.211.166.141
216.165.191.53
Next we query A for osuosl.org.:
$ dig a osuosl.org. @140.211.166.140
;; QUESTION SECTION:
;osuosl.org.
IN A
;; ANSWER SECTION:
osuosl.org.
300 IN
A
140.211.15.183
;; AUTHORITY SECTION:
osuosl.org.
86400
osuosl.org.
86400
osuosl.org.
86400
IN
IN
IN
NS
NS
NS
;; ADDITIONAL SECTION:
ns1.auth.osuosl.org.
ns2.auth.osuosl.org.
ns3.auth.osuosl.org.
86400
86400
3600
ns1.auth.osuosl.org.
ns2.auth.osuosl.org.
ns3.auth.osuosl.org.
IN
IN
IN
A
A
A
140.211.166.140
140.211.166.141
216.165.191.53
TODO: Traverse the DNS Tree with dig
Can you traverse the DNS tree to get to these websites? Give it a try!
3.17. Lesson 16: DNS
75
OSU DevOps BootCamp Documentation, Release 0.0.1
• github.com
• web.archive.org
• en.wikipedia.org
Further Reading
• Try running dig on some of your favorite websites and see what you find.
• Read the manpage on dig and see what else you can find in the output.
• Try registering your own domain name and run a website using the Github Student Pack resources like Digital
Ocean and DNSimple.
Lesson 17: Configuration Management
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Configuration Management
“Configuration management is the process of standardizing resource configurations and enforcing their
state across IT infrastructure in an automated yet agile manner.”
• Puppet Labs
user { ’audience’:
ensure => present,
}
Short History of CM
In the beginning there were no computers.
Then many years passed and eventually we built the first computer.
Then a few years after that we had more computers than we really had time to manage. Things got out of hand pretty
quick.
Concept: Infrastructure as Code
• Install packages, configure software, start/stop services.
• Ensure/guarantee a specific state of a machine.
• Provide history of changes for a system.
• Repeatable way of rebuilding a system.
• Orchestrate a cluster of services together.
76
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Pull vs Push Models
Pull Model Scales well but difficult to manage.
Push Model Simple to manage and setup but not scalable.
Tools
• Puppet
• Chef
• CFEngine
• Ansible
• Saltstack
3.18. Lesson 17: Configuration Management
77
OSU DevOps BootCamp Documentation, Release 0.0.1
Puppet
• Uses custom CM Language.
• Primary Push Model.
• Widely Adopted.
• Very stable.
• Difficult to get setup.
78
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Chef
• Primarily Push Model.
• Code files are Ruby.
• Widely Adopted.
• Difficult to setup.
CFEngine
• Fast at execution, slow at adaptation.
• Very old.
• Stable.
3.18. Lesson 17: Configuration Management
79
OSU DevOps BootCamp Documentation, Release 0.0.1
Ansible
• Easy to use.
• Easy to setup.
• Does not scale well.
SaltStack
• Easy to use.
• Hard to get started.
Delcaration Configuration
packages [nginx, python, vim]
state installed
update true
service nginx
state enabled
alert service myapp_daemon
80
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Chef Example
• Install apache and start the service
• Configuration is called a ‘recipe’
• Written as pure Ruby code
package "apache" do
package_name "httpd"
action :install
end
service "apache" do
action [:enable, :start]
end
Note: Since chef uses Ruby you can do loops and other cool Ruby-isms in your configuration management. This can
be a gift and a curse.
Puppet Example
• Install apache and start the service
• Configuration is called a ‘manifest’
• Puppet DSL based on Ruby
package { "apache":
name
=> "httpd",
ensure => present,
}
service {
name
ensure
enable
require
}
"apache":
=> "apache",
=> running,
=> true,
=> Package["apache"],
Note: Since Puppet designed its own language you are more limited in what you can express, but this isn’t always a
bad thing. It’s feature rich and can do pretty much anything that Chef can.
Ansible Example
• Install apache and start the service
• Configuration is called a ‘playbook’
• Uses YAML file format for configuration
- hosts: all
tasks:
- name: Install Apache
yum:
name: httpd
3.18. Lesson 17: Configuration Management
81
OSU DevOps BootCamp Documentation, Release 0.0.1
state: present
- name: Start Apache Service
service:
name: httpd
state: running
enabled: yes
Note: Ansible’s langauge is Yaml, which is basically JSON but easier to read and write. This is similar to Puppet
in it limits the possible functionality, but again: these tools all achieve the same result, they just get there in different
ways.
Further Reading
• Ansible’s Documentation is comprehensive and contains an easy-to-follow “Getting Started” guide.
• Kitchen-CI is a Chef oriented testing system
• Puppet Learning VM is a prebuilt VM for learning Puppet’s fundamental concepts
Lesson 18: Application Isolation
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
82
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Application Isolation
Virtual Machines
[vm] # ps aux
USER PID %CPU %MEM
VSZ
root
1 0.0 0.6 110564
root
2 0.0 0.0
0
root
3 0.0 0.0
0
root
5 0.0 0.0
0
[... 120+ more lines ...]
[host] # ps aux
USER PID %CPU %MEM
VSZ
root
1 0.0 0.1 200328
root
2 0.0 0.0
0
root
3 0.0 0.0
0
root
5 0.0 0.0
0
[... 240+ more lines ...]
RSS
3164
0
0
0
RSS
5208
0
0
0
TTY
?
?
?
?
TTY
?
?
?
?
STAT START
Ss
2015
S
2015
S
2015
S<
2015
STAT
Ss
S
S
S<
START
Aug25
Aug25
Aug25
Aug25
TIME
11:17
0:00
3:55
0:00
TIME
0:44
0:00
0:05
0:00
COMMAND
/lib/systemd/systemd --system --deserialize 15
[kthreadd]
[ksoftirqd/0]
[kworker/0:0H]
COMMAND
/sbin/init
[kthreadd]
[ksoftirqd/0]
[kworker/0:0H]
OS Emulation
Containers
[container] $ ps aux
PID
USER
TIME
COMMAND
3.19. Lesson 18: Application Isolation
83
OSU DevOps BootCamp Documentation, Release 0.0.1
1
6
root
root
0:00
0:00
sh
ps aux
Container Technologies
Containers vs VMs
Pros
Virtual Machines
Complete process isolation
‘Battle Tested’
Containers
Fast startup
Little overhead
Cons
Virtual Machines
Slightly more overhead.
Slow startup.
Cross-kernel emulation.
84
Containers
Security concerns.
No cross-kernel emulation.
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Tools
Virtual Machines
VirtualBox
VMWare
Containers
Docker
Rkt
Virtual Machines
VirtualBox An Open Source VM Manager.
Widely used and supported on Linux, Mac, and Windows.
VMWare A closed source VM Manager.
VMWare is a widely used and tends to have better performance than Virtual Box. While it can emulate Linux it
does not work natively on Linux.
KVM The Kernel-based Virtual Machine.
Linux’s native infrastructure for handling Virtual Machines and emulation. Usually used in a larger emulation
program, not alone.
Containers
Docker The defacto CLI tool for creating and using containers.
Very popular and well integrated into other tools.
RKT A competitor to Docker created by CoreOS. Approaches container management from a different angle which
has it’s advantages and disadvantages.
chroot The oldschool way to use containers. Not a container in the modern sense, but achieves similar process
isolation.
Jails The BSD Unix form of containerization. Offers a level of secure isolation not really possible in Linux.
TODO
Further Reading
Docker
RKT
Lesson 19: Cloud Infrastructure
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
3.20. Lesson 19: Cloud Infrastructure
85
OSU DevOps BootCamp Documentation, Release 0.0.1
What the Cloud Looks Like
[...] a model for enabling ubiquitous, on-demand access to a shared pool of configurable computing
resources.
Advantages over Bare Hardware
Private Clouds
Public Clouds
Cloud + Configuration Management
Advantages
Running your software on a cloud is:
• Ephemeral
• Cost effective
• Low startup cost
Disadvantages
Clouds can be great tools, but they have some disadvantages:
• Central Point of Failure
86
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
TODO
Further Reading
AWS provides a lot of services, not all of which are named very well. This article explains what each service does in
plain English.
Lesson 20: Contributing to Open Source
Homepage
Content
Slides
Video
Warning: This lesson is under construction. Learn from it at your own risk. If you have any feedback, please fill
out our General Feedback Survey.
Open Source
• Learn lots of new things, and grow as developers.
• Give back to a community that has given you something.
• You have more to contribute than you may realize!
• Meet amazing people.
• Personal fulfillment.
Community Benefit
Share the Love (and the Code)
Personal Benefit
• ‘Learning the Ropes’ of a substantial code-base
• Working with others
• Getting code reviewed
• Documenting contributions
• Testing your changes
Free?
Free Software: [Free Software] means that the users have the freedom to run, copy, distribute, study, change
and improve the software.
The Four Freedoms:
0. The freedom to run the program as you wish, for any purpose.
1. The freedom to study how the program works, and change it so it does your computing as you wish. Access
to the source code is a precondition for this.
3.21. Lesson 20: Contributing to Open Source
87
OSU DevOps BootCamp Documentation, Release 0.0.1
2. The freedom to redistribute copies so you can help your neighbor.
3. The freedom to distribute copies of your modified versions to others. By doing this you can give the whole
community a chance to benefit from your changes. Access to the source code is a precondition for this.
Assessing a New Community
Elitism vs Nice-ism
Communication style
Documentation and Guides
Things to Look for
• When are the top pull requests time-stamped? Anything older than 3-4 months might not be ideal.
• Open / recent issues (especially with help wanted labels) are good.
• Many contributors means they’re used to people helping out.
How to Get Involved
Finding a Project
In order of perceived usefulness:
• Openhatch
• 24 pull requests
• BugsAhoy
• Showcased github projects
• Trending github projects
I Can’t Find a Project I Like!
That’s okay.
First Steps
0. Find a project
1. Read Contributing and Getting Started docs
2. Look at list of issues
3. Do a thing!
• Write a test
• Fix a typo
• Deploy and update the installation docs
88
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Know your Licenses
Licenses to use:
• MIT
• Apache 2.0
• AGPL/GPL/LGPL 2/3
Licenses to *not* use:
• Public Domain Dedication
TODO: Find an Open Source Project
Further Reading
Choose A License
About
What is DevOps?
DevOps is a hybrid of skills from both Software Development (Dev) and Computer Operations (Ops) intended to
meet the unique demands of cloud computing. Software Developer and Systems Administrator are no longer mutually
exclusive job titles. Devs need more Ops knowledge to understand how their application will run in the real world.
Admins need more Dev knowledge to design infrastructure that fit an app’s needs efficiently and effectively. To top it
off site reliability engineers and many modern security roles require at least a little background in both development
and operations.
Purpose of DevOps BootCamp
DevOps BootCamp is an OSU Open Source Lab program dedicated to teaching core software development and systems operation skills. The program is free and open to any interested OSU students, community member, and online
go-getter. DevOps BootCamp provides a comprehensive Open Source education that is outside the scope of regular
Linux Users Group meetings and OSU Coursework.
3.22. About
89
OSU DevOps BootCamp Documentation, Release 0.0.1
What Students Get
• Mentorship from students and professionals with advanced skills in software development and systems administration.
• Professional connections in the software industry.
• A welcoming environment to start learning, for those who have always wanted to learn about software development and systems administration but were never sure where to start.
• An opportunity to fill in knowledge gaps for self-taught coders or sysadmins.
• The skills to build and deploy Open Source software, or contribute to existing projects
What the Open Source Lab Gets
• The OSL gets a larger pool of candidates to recommend to companies interested in recruiting students.
• The OSL gets to work with a wider variety of students, helping it contribute to the school of EECS.
• The Open Source Community gets more project contributors.
Target Audience
Our goal is to make the DevOps BootCamp program accessible to students and community members from all backgrounds. Students should:
• Want to learn.
• Be willing to ask questions
• Be open to setting apart time to play with the tools you’ll be learning about in the class.
Policies
Attendance
Attendance is not mandatory but highly suggested to get the most out of DBOC; We will not spend class time reviewing
material for those who skip a lecture and each classes curriculum will build on what you learned the previous session.
All curriculum will be available online before and after class sessions to get caught up.
BootCamp mentors will be available at scheduled times outside of regular classes to help answer any questions about
the training program’s content. If you attend a lesson and don’t understand something then you are encouraged to ask
that question during the meeting since others are likely have the same question.
Laptops
As the course progresses, you will need a laptop. We hope and recommend that you decide to set up your laptop to
dual-boot to Linux as the course progresses, but it is not required. If you don’t own a laptop and are an OSU student
you can check out a laptop from the OSU Library for at least 24 hours at a time.
As long as your laptop is new enough to boot from USB and connect to a wireless network the exact specifications do
not matter. You will be provided with a remote virtual machine with which to do all class projects.
If you are not an OSU student and do not have access to a working laptop, contact the DevOps BootCamp (email
devopsbootcamp) organizers and they will see whether one can be loaned out to you.
90
Chapter 3. Donate
OSU DevOps BootCamp Documentation, Release 0.0.1
Get Involved
Mailing list
Join the mailing list for updates.
IRC
Join us on irc.freenode.net in #devopsbootcamp (students will be setting up an IRC network for the
program early in the program).
Website & Curriculum
If you’d like to help edit this site, email devopsbootcamp or ping anyone in #devopsbootcamp on Freenode with
your GitHub username to get access to the web site repo. You’ll also want to learn the ReStructured Text markup
language to edit the site, if you don’t already know it.
Schedule
The DevOps BootCamp content is available for free but meet-space guided lectures are offered throughout the year.
Check the schedule below for our in-person lectures; each lecture covers a different part of the curriculum covering
the entire course during the OSU academic school year.
Warning: If you are working ahead be aware that the schedule and slides may be subject to change. Check back
regularly.
Fall
Lesson
Fall Kickoff
Date/Time
Oct 7, 2017 9:30am-3:30pm
Location
OSU KEAR 112
Description
DevOps BootCamp Fall Kickoff
Winter
Lesson
TBA
Date/Time
Location
Description
Date/Time
Location
Description
Spring
Lesson
TBA
3.23. Schedule
91
OSU DevOps BootCamp Documentation, Release 0.0.1
Running DOBC
If you’re reading this it means you’re interested in running DOBC yourself. It may have been passed on to you, or you
may just like the curriculum and want to use it to start your own DOBC. Either way, thank you for reading this!
This page is a growing checklist, warning, notes, and fables from those teaching and contributing to DOBC. If you
read this, heed it’s warnings and take it’s lessons to heart you will no doubt be on your way to success.
Before You Begin
Meet-Space Lectures
Practice. Just like any public speaking engagement, you should review what you’re going to do and practice it in real
time. This means you should say the things you’re going to say and even do the activities the students will do.
Always have a buddy. Teaching alone can be done, but if at all possible try to have a teaching ‘buddy’. This person
is at least about as expert on the topics you’re covering as you are. Your buddy can field questions, help with
TODOs, and can even take over the lesson if you need them to (you might need to go to the bathroom, who
knows).
Always be taking notes. As a lecturer you won’t always teach perfectly. You won’t get it perfect the first, second,
third, or even last time – but you should always strive for perfection.
Take notes on what could have gone better, questions that were asked, and confusions students had. The DOBC
curriculum can be very dense and sometimes it skims over important stuff. During each lesson be sure to
improve the curriculum based on your notes. Whoever teaches it next time will thank you.
Online Engagement
Adding to Lessons
Attribute Images We do our best to attribute images by linking them to the website we got them from. If you add
images to the website try to do the same.
92
Chapter 3. Donate
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement