null null

Cr
eat
i
ngLAMP
I
nf
r
ast
r
uct
ur
e
f
orDi
gi
t
al
Humani
t
i
es
Pr
oj
ect
s
J
onMar
t
i
n
Thi
spackagei
si
nt
endedf
ort
heper
sonal
,educat
i
onaluseofDHSIat
t
endees.
Por
t
i
onsappearher
ewi
t
hconsi
der
at
i
onoff
ai
ruseandf
ai
rdeal
i
nggui
del
i
nes.
©DHSI2016
Welcome to DHSI 2016!
Home
Courses
Scholarships
Schedule
[email protected] & Events
Visitor Info
People
Archive
Sunday, 5 June 2016 [DHSI Registration, Meetings, Workshops]
9:00 to 4:00
Early Class Meeting: 3. [Foundations] DH For Department Chairs and Deans (David Strong C114,
Classroom)
Further details are available from instructors. Registration materials will be available in the classroom.
12:30 to 5:00
DHSI Registration (NEW LOCATION: MacLaurin Building, Room A100)
After registration, many will wander to Cadboro Bay and the pub at Smuggler's Cove OR the other direction to
Shelbourne Plaza and Maude Hunter's Pub.
1:00 to 4:00
3-hour Workshops
- 45. 3D Visualization for the Humanities [5 June] (MacLaurin D107, Classroom)
- 46. Beyond TEI: Metadata for Digital Humanities [5 June] (MacLaurin D109, Classroom)
4:00 to 5:00
Workshop:
- 44. Twitter Basics: An Introduction to Social Media [5 June] (MacLaurin D105, Classroom)
Monday, 6 June 2016
Your hosts for the week are Ray Siemens and Dan Sondheim.
7:45 to 8:15
Last-minute Registration (MacLaurin Building, Room A100)
8:30 to 10:00
Welcome, Orientation, and Instructor Overview (MacLaurin A144)
Classes in Session (click for details and locations)
10:15 to Noon
1. [Foundations] Text Encoding Fundamentals and their Application (Clearihue A102, Lab)
2. [Foundations] Digitisation Fundamentals and their Application (Clearihue A051, Lab)
3. [Foundations] DH For Department Chairs and Deans (David Strong C114, Classroom)
4. [Foundations] Fundamentals of Programming/Coding for Human(s|ists) Clearihue A103, Lab)
5. [Foundations] Understanding The Predigital Book: Technology and Texts (McPherson Library A003,
Classroom)
6. Out-of-the-Box Text Analysis for the Digital Humanities (Human and Social Development A160, Lab)
7. Geographical Information Systems in the Digital Humanities(Human and Social Development A170, Lab)
8. CloudPowering DH Research (David Strong C108, Classroom)
9. Digital Storytelling (MacLaurin D111, Classroom)
11. Digital Humanities Pedagogy: Integration in the Curriculum (Cornett A121, Classroom)
13. Text Processing - Techniques & Traditions (Cornett A229, Classroom)
14. Issues in Large Project Planning and Management (Hickman 120, Classroom)
15. 3D Modelling for the Digital Humanities and Social Sciences (MacLaurin D010, Classroom)
16. Digital Humanities Databases (Clearihue D132, Classroom)
17. Creating LAMP Infrastructure for Digital Humanities Projects (Human and Social Development A270,
Classroom)
18. RDF and Linked Open Data (Human and Social Development A264, Classroom)
19. Palpability and Wearable Computing (MacLaurin D016, Classroom)
20. Drupal for Digital Humanities Projects (MacLaurin D109, Classroom)
21. Introduction to Electronic Literature in DH: Research and Practice (MacLaurin D115, Classroom)
Lunch break / Unconference Coordination Session (MacLaurin A144)
12:15 to 1:15
Undergraduate Meet-up, Brown-Bag (details TBA)
1:30 to 4:00
Classes in Session
Institute Panel: Perspectives on DH (or, #myDHis ...)
Jason Boyd (Ryerson U): "At Play in the Digital Humanities."
Abstract: This presentation will move from discussion of the Texting Wilde Project and designing playful
interfaces for it to a more general reflection on the place of play in DH, informed by work in games studies,
gameful design, speculative computing, and queer theory.
Liz Grumbach (Texas A&M U): "Modding the Academy: eMOP, ARC, and Emerging (Digital) Humanities
Paradigms."
Abstract: As “research staff” at the Initiative for Digital Humanities, Media, and Culture (IDHMC) at Texas A&M
University, my digital humanities is about collaboration, project development, and finding effective methods for
facilitating research in an academic field that still privileges traditional humanities research methodologies over
emerging forms of scholarly communication and knowledge production. I will briefly describe two
interdisciplinary projects run out of the IDHMC (the Early Modern OCR Project and the Advanced Research
Consortium) to facilitate further discussion of how emerging digital methodologies can remodel our perceptions
of how and where scholarly research is produced.
4:10 to 5:00
Claudia von Vacano (UC Berkeley): "Data Science in the Service of Humanity at Berkeley."
Abstract: The Division of Arts and Humanities at Berkeley is building an infrastructure to buttress digital
humanities in conversation with data science. This includes critical analysis of a data science initiative. In this
short talk, Digital Humanities (DH) at Berkeley Director Claudia von Vacano discusses radical organizational
change within a strongly-established institution. Claudia argues that it is incumbent upon us to bring greater
diversity to data science thinking in order to prevent data science from becoming a monolithic enterprise across
universities. Now more than ever artists and humanists need to ensure that data science is at the service of
humanity in its various expressions. Claudia discusses the specific organizational structures at Berkeley that are
supporting this ambitious endeavor.
Amardeep Singh (Lehigh U): "'Harlem Shadows': A Collaborative, Social Justice-Oriented Digital Edition"
For a Digital Humanities graduate seminar, I developed an assignment that aimed to introduce students to the
basic elements of constructing a digital edition based on a given text (Claude McKay's 1922 collection of poetry,
"Harlem Shadows). The project was fully collaborative; fundamental decisions regarding the purpose of the site,
the intended audience, and site design were left entirely to students, with research input from faculty. This
turned out to be an effective way to get students new to DH oriented in at least one area of the field, while also
keeping questions of social justice and representation of minority writers online a central focus.
Lisa Goddard (U Victoria): “On the Same Page: Models for Library-DH Collaboration.”
Abstract: DH researchers increasingly find themselves in need of research supports that extend beyond the
services offered in traditional humanities departments. These might include server space, programming
expertise, database management, metadata support, and digital preservation services. Libraries need access to
faculty and student subject expertise to enrich online collections and exhibits, and to improve discovery and
research interfaces. There is an enormous amount of complementarity in our needs and interests, but new
approaches are necessary to help us bridge dissimilar governance and funding models. This presentation will
draw on our experience at UVic Libraries to propose models that can foster closer collaboration for the mutual
benefit of the library and its DH partners.
Co-Chairs: Constance Crompton (UBC Okanagan) and Matt Huculak (U Victoria)
(MacLaurin A144)
5:00 to 6:00
Light Reception
(Felicitas, Student Union Building)
Tuesday, 7 June 2016
9:00 to Noon
Classes in Session
12:15 to 1:15
Lunch break / Unconference
1:30 to 4:00
Classes in Session
DHSI Colloquium Session 1 (MacLaurin A144)
4:15 to 5:30
Chair: TBA
Working at the Intersection of Renaissance Studies and DH: An Update on Iter Initiatives. Randa El Khatib (U
Victoria); Lindsey Seatter (U Victoria); William Bowen (U Toronto, Scarborough); Daniel Powell (King’s College
London); Ray Siemens (U Victoria)
The NYU Libraries Web Hosting Pilot: An Update and Lessons Learned. Zach Coble (New York U)
Exploring Place in the French of Italy. Heather Hill (Fordham U)
Digital Representations of Petrarch’s "Rerum Vulgarium Fragmenta". Isabella Magni (Indiana U)
Voices of Southern Patagonia. Gustavo Navarro (U Nacional de la Patagonia Austral)
Wednesday, 8 June 2016
9:00 to Noon
Classes in Session
12:15 to 1:15
Lunch break / Unconference
1:30 to 4:00
Classes in Session
DHSI Colloquium Session 2 (MacLaurin A144)
4:15 to 5:30
Chair: TBA
Transforming Bad OCR into Useful Metadata? Exploring NLP Possibilities. Evan Williamson (U Idaho)
Piloting Linked Open Data for Artists' Books at University of California, Irvine. Emilee Mathews (U California,
Irvine)
Four Words: The Role of Theory in Digital Humanities. Grant Glass (U North Carolina, Chapel Hill)
Rethinking the Exhibition Catalogue: Documentation, Curation, and the Digital Humanities Project. Julia
Polyck-O'Neill (Brock U); Aleksandra Kaminska (Simon Fraser U)
Digital Storytelling for Digital Humanities. John F. Barber (Washington State U, Vancouver)
Thursday, 9 June 2016
9:00 to Noon
Classes in Session
12:15 to 1:15
Lunch break / Unconference
[Instructor lunch meeting]
1:30 to 4:00
Classes in Session
DHSI Colloquium Session 3. Special Theme: Gender and the Digital (MacLaurin Building, Room A144)
4:15 to 5:00
Chair: TBA
A Textual Analysis of Female Renaissance Playwrights using R. Elizabeth Ramsay (Trent U)
The Comparison of Human-Reading and Machine-Reading in Le Système de la Nature. Maryam Mozafari
(Simon Fraser U)
Does Gender Affect How Genre-Conformingly Writers Write? Sayan Bhattacharyya (U Illinois, UrbanaChampaign); Ted Underwood (U Illinois, Urbana-Champaign)
DHSI Colloquium (contd.), Electronic Literature Affiliated Event: Alan Sondheim, "Language, Accident, and
the Virtual"
(MacLaurin Building, Room A144)
5:00 to 5:30
Abstract: This talk engages concepts of blankness, geography, gamespace in virtual worlds, and what I term
edgespace - the limits of the gamespace, where language occurs and seethes. I argue that the phenomenology
of the real comes into play when living spaces are abandoned, where broken geographies are signs of a future
already present. I present instances of digital language production in such spaces, working through virtual
worlds such as Second Life and the Macgrid, as well as self-contained Open Sim software that can be run on
most computers.
The edgespace is always uneasy, tottering, catastrophic; it is the space of the unalloyed digital, where things no
longer operate within a classical or modernist tradition. Increasingly, this space characterizes our current place
in the world, with its fractured media histories and environments of scorched earth, environmental depredation,
and slaughter. We can work through and within such spaces, developing (as perhaps Occupy did) new forms of
production, resistance, and digital culture.
Performance: Vibrant Lives (Jessica Rajko, Eileen Standley, Jacque Wernimont, Stjepan Rajko)
(Just outside MacLaurin Building, Room A100)
5:30 to ~6:15
Critically commenting on this use of personal data, Vibrant Lives is an interactive installation that gives
audiences a real-time sense of their own voluminous “data shed” (the data that we share as a part of everyday
life). In this, Vibrant Lives troubles boundaries erected by ideas of disembodied, abstracted, “immaterial”
metadata and people. By juxtaposing the different ways that we engage with technologies of communication
and preservation, we ask our audiences to consider interplays of value, valuation, technology, information, and
material/matter. Rather than exploring more traditional audio/visual methods for sharing data information, this
work offers haptic (touch-based) feedback to elicit a more visceral understanding of what it means to “shed
data.”
Friday, 10 June 2016 [DHSI; ELO and INKE Opening]
8:00 to 9:30
Electronic Literature Organization + INKE Registration (MacLaurin Building, Room A100)
9:00 to 9:30
ELO Welcome
9:00 to Noon
DHSI Classes in Session
9:30 to Noon
ELO Featured Session A
Electronic Literature Organization + INKE Registration (MacLaurin Building, Room A100)
12:15 to 1:15
DHSI Lunch Reception / Course E-Exhibits (MacLaurin A100)
ELO Gallery Exhibit Opens
INKE Welcome
Joint Institute Lecture, ELO, INKE, and DHSI
Jon Saklofske (Acadia U): “Prototyping Resistance: Wargame Narrative and Inclusive Feminist Discourse.”
Panel Discussants: Stephanie Boluk (Pratt Institute), Diane Jakacki (Bucknell U), Elizabeth Losh (UC San
Diego), and Anastasia Salter (U Central Florida)
Co-Chairs: Dene Grigar (Washington State U, Vancouver), Jon Bath (U Saskatchewan), Ray Siemens (U
Victoria)
(MacLaurin A144)
1:30 to 3:00
3:15 to 4:45
Abstract: Can a feminist war game exist? War, traditionally the sole purview of men, is an essential site for
asking critical questions about masculinist systems and mediated representations, especially since subjects,
objects and agents are all instruments within the ideological narratives that frame the brutal history of armed
conflict. Simply including female “warrior” characters in a war game, for example, continues to normalize the
mechanism of war while extending its “inclusiveness” to groups that have traditionally been marginalized and
victimized by it. However, prototyping complex intersections between mechanisms of war, digital game
narrativities, and inclusive feminist values suggests that feminist discourses can be used to denaturalize and
reframe narratives of war in spaces of programmed play.
ELO Concurrent Session 1: 1.1 Best Practices for Archiving E-Lit; 1.2 Medium & Meaning; 1.3 A Critical Look at
E-Lit; 1.4 Literary Games; 1.5 eLit and the (Next) Future of Cinema; 1.6 Authors & Texts
ELO Action Session 1
Joint Reception: ELO, INKE, and DHSI (University Club)
DHSI Colloquium Poster Session
ELO New Scholars Poster Session
DHSI Colloquium Poster Session Presenters:
Practicums in the Digital Humanities: Four KPU Case Studies. Greg Chan (Kwanten Polytechnic U) and
ENGL 4300 Students (TBD)
The HathiTrust Research Center: Supporting Large Scale Analysis of the HathiTrust Digital Library. Sayan
Bhattacharyya (U Illinois); Nicholae Cline (Indiana U); Eleanor Dickson (U Illinois); Leanne Mobley (Indiana U)
Digital Scholarship in the Institutional Repository. Jeri Wieringa (George Mason U)
Canada's Early Women Writers and All Their Relatives. Karyn Huenemann (Simon Fraser U)
Darwin's Semantic Voyage. Jaimie Murdock (Indiana U); Colin Allen (Indiana U); Simon DeDeo (Indiana U)
Possible Spanish Idiom In A Name At Nootka. Paula Johanson (U Victoria)
5:00 to 6:00
INKE Poster Session Presenter:
Genre-based Approaches to Domain Modeling: A Case Study Involving Rulebooks. Kelly Colht (Information
School, U Washington)
ELO Poster Session Presenters:
Social Media for E-Lit Authors. Michael Rabby (Washington State U Vancouver)
– O True Apothecary! Kyle Booten (UC Berkeley, Center for New Media)
Electronic Literature Production – A Case Study of Korporacja Ha!art. Aleksandra Malecka (Jagiellonian U);
Piotr Marecki (Jagiellonian U)
How to Upset the Contributors? The Responses of Collaborating Readers to the Publication of Crowdsourced
Literature. Heiko Zimmermann (U Trier)
Life Experience through Digital Simulation Narratives. David Núñez Ruiz (Neotipo)
Building Stories. Kate Palermini (Washington State U Vancouver)
Help Wanted and Skills Offered. by Deena Larsen (Independent Artist); Julianne Chatelain (U.S. Bureau of
Reclamation)
Beyond Original E-Lit: Deconstructing Austen Cybertexts. Meredith Dabek (Maynooth U)
Arabic E-Lit (AEL) Project. Riham Hosny (Rochester Institute of Technology / Minia U)
7:30 to 9:00
ELO Readings and Performances (Felicitas, Student Union Building)
Saturday, 11 June 2016 [ELO + INKE + Suggested Outings!]
8:00 to Noon
Electronic Literature Organization + INKE Registration (MacLaurin Building, Room A100)
8:30 to 10:00
ELO Lightning Round
ELO Gallery Exhibit Opens (10:00)
INKE Session 1
ELO Concurrent Session 2: 2.1 Literary Interventions; 2.2 E-Lit in Global Contexts; 2.3 Theoretical
Underpinnings; 2.4 E-Lit in Time and Space; 2.5 Understanding Bots
ELO Action Session 2
INKE Session 2
10:30 to Noon
12:15 to 1:15
Lunch
ELO Artists' Talks
1:30 to 3:00
ELO Concurrent Session 3: 3.1 E-Lit Pedagogy in Global Setting; 3.2 The Art of Computational Media; 3.3
Present Future Past; 3.4 Beyond Collaborative Horizons; 3.5 E-Loops, Reshuffling Reading And Writing In
Electronic Literature Works; 3.6 Metaphorical Perspectives; 3.7 Embracing Bots
ELO Workshops (to 3.30)
INKE Session 3
3:30 to 5:00
ELO Featured Session B
INKE Session 4
6:00 to 9:00
ELO Informal Banquet (University Club)
(INKE participants are invited to hang out with our ELO pals, too!)
9:00 to Midnight
ELO Screenings
Some ideas, for those who'd like to explore the area!
Suggested Outing 1, Botanical Beach (self-organised; car needed)
A self-guided visit to the wet, wild west coast tidal shelf (and historically-significant former research site) at
Botanical Beach; we recommend departing early (around 8.00 am) to catch low tide for a better view of the
wonderful undersea life! Consider bringing a packed lunch to nibble-on while looking at the crashing waves
when there, and then have an afternoon drink enjoying the view from the deck of the Port Renfrew Hotel.
Suggested Outing 2, Butchart Gardens (self-organised)
A shorter journey to the resplendently beautiful Butchart Gardens and, if you like, followed by (ahem) a few
minutes at the nearby Church and State Winery, in the Saanich Penninsula. About an hour there by public bus
from UVic, or 30 minutes by car.
Suggested Outing 3, Saltspring Island (self-organised; a full day, car/bus + ferry combo)
All day
Suggested Outings
Why not take a day to explore and celebrate the funky, laid back, Canadian gulf island lifestyle on Saltspring
Island. Ferry departs regularly from the Schwartz Bay ferry terminal, which is about one hour by bus / 30
minutes by car from UVic. You may decide to stay on forever ....
Suggested Outing 4, Paddling Victoria's Inner Harbour (self-organised)
A shorter time, seeing Victoria's beautiful city centre from the waterways that initially inspired its foundation. A
great choice is the day is sunny and warm. Canoes, kayaks, and paddle boards are readily rented from Ocean
River Adventures and conveniently launched from right behind the store. Very chill.
And more!
Self-organised High Tea at the Empress Hotel, scooter rentals, visit to the Royal BC Museum, darts at Christies
Carriage House, a hangry breakfast at a local diner, whale watching, kayaking, brew pub sampling (at
Spinnaker's, Swans, Moon Under Water, and beyond!), paddle-boarding, a tour of used bookstores,and more
have also been suggested!
Sunday, 12 June 2016 [ELO + DHSI Registration, Workshops]
8:30 to 10:00
ELO Town Hall Meeting
ELO Gallery Exhibit Opens (10:00)
10:30 to Noon
ELO Concurrent Session 4: 4.1 Narratives and Narrativity; 4.2 Historical & Critical Perspectives; 4.3 Emergent
Media; 4.4 Narrative and Poetic Experiences; 4.5 The E-Literary Object; 4.6 Next Narrative
ELO Action Session 3
12:15 to 1:30
Lunch
ELO Artists' Talks
12.30 to 5:00
DHSI Registration (NEW LOCATION: MacLaurin Building, Room A100)
After registration, many will wander to Cadboro Bay and the pub at Smuggler's Cove OR the other direction to
Shelbourne Plaza and Maude Hunter's Pub.
1:00 to 4:00
DHSI 3-hour Workshops
- 47. Crowdsourcing as a Tool for Research and Public Engagement [12 June] (David Strong C114, Classroom)
- 48. Text Analysis with the HathiTrust Research Center [12 June] (Cornett B129, Classroom)
- 49. Dynamics of Explanatory Annotation [12 June] (Clearihue D131, Classroom)
- 50. Dynamic Ontologies for the Humanities [12 June] (Cornett A128, Classroom)
- 51. Digital Preservation Basics for DH [12 June] (Human and Social Development A264, Classroom)
- 52. Understanding Digital Video [12 June] (Human and Social Development A270, Classroom)
- 53. DHSI Knits, or, In The Loop [12 June] (Cornett A229, Classroom)
- 53. DHSI Knits, or, In The Loop [12 June] (Cornett A229, Classroom)
1:30 to 3:00
ELO Concurrent Session 5: 5.1 Subversive Texts; 5.2 Experiments in #NetProv & Participatory Narratives; 5.3
Emergent Media; 5.4 E-Lit Labs; 5.5 Transmedia Publishing; 5.6 Feminist Horizons
3:30 to 5:30
ELO Featured Session, Stories & Their Platforms
7:30 to 9:00
ELO Special Event at Limbic Media Gallery
Monday, 13 June 2016
Your hosts for the week are Alyssa Arbuckle, Ray Siemens and Dan Sondheim.
8:15 to 8:45
DHSI Last-minute Registration (MacLaurin Building, Room A100)
9:00 to 10:00
Welcome, Orientation, and Instructor Overview (MacLaurin A144)
Classes in Session (click for details and locations)
10:15 to Noon
22. [Foundations] Introduction to Computation for Literary Criticism (Clearihue A102, Lab)
23. [Foundations] Web Development / Project Prototyping with Ruby on Rails (Human and Social
Development A270, Classroom)
24. [Foundations] Models for DH at Liberal Arts Colleges (& 4 Yr Institutions) (Hickman 120, Classroom)
25. Wrangling Big Data for DH (Clearihue A015, Lab)
26. Conceptualising and Creating a Digital Edition (McPherson Library A003, Classroom)
27. Stylometry with R: Computer-Assisted Analysis of Literary Texts (Business and Economics 180, Lab)
28. R, Interactive Graphics, and Data Visualization for the Humanities (Human and Social Development A160,
Lab)
29. Online Collaborative Scholarship: Principles and Practices (A CWRCShop) (Human and Social
Development A264, Classroom)
30. Sounds and Digital Humanities (MacLaurin D111, Classroom)
31. Accessibility & Digital Environments (David Strong C108, Classroom)
32. Critical Pedagogy and Digital Praxis in the Humanities (MacLaurin D109, Classroom)
33. Text Mapping as Modelling (Clearihue A311, Classroom)
34. Building a Professional Identity and Skillset in the Digital Humanities (Cornett A128, Classroom)
35. Digital Humanities with a Global Outlook (Cornett A229, Classroom)
36. Physical Computing and Desktop Fabrication (MacLaurin D016, Classroom)
37. XML Applications and Databases for Historical and Literary Research (Clearihue D132, Classroom)
38. Games for Digital Humanists (David Strong C114, Classroom); M PM and Tu PM in (Human and Social
Development A170, Lab)
39. Understanding Topic Modeling (MacLaurin D105, Classroom)
40. Feminist Digital Humanities: Theoretical, Social, and Material Engagements (MacLaurin D107, Classroom)
42. Advanced Criticism and Engagement of Electronic Literature (MacLaurin D115, Classroom)
43. Advanced TEI Concepts / TEI Customisation (MacLaurin D114, Classroom)
12:15 to 1:15
Lunch break / Unconference Coordination Session (MacLaurin A144)
1:30 to 4:00
Classes in Session
Institute Lecture
Laura Estill (Texas A&M): "Digital Futures: Long-term Planning for your Project"
Chair: TBA
(MacLaurin A144)
Abstract: The start of every digital project is as exhilarating as it is overwhelming. But when should you start
4:10 to 5:00
5:00 to 6:00
looking ahead to the end? And does there have to be an end? As Robin Camille Davis reports, 45% of projects
discussed at DH '05 are no longer available online. In this talk, I'd like to offer some best practices for thinking
about your DH project in the long term, from workflow and technologies to people and resources. I will share my
experience taking on a long-standing existing digital project (The World Shakespeare Bibliography Online).
Although thinking about documentation and archiving might strike terror in your hearts, or make your eyelids
droop with boredom, ultimately, having a well-thought-out plan leads to serenity and peace of mind (or at least
leaves you free to worry about other important things, like how you're not flossing enough).
Light Reception
(Felicitas, Student Union Building)
Tuesday, 14 June 2016
9:00 to Noon
Classes in Session
12:15 to 1:15
Lunch break / Unconference
1:30 to 4:00
Classes in Session
DHSI Colloquium Session 4. Special Theme: Building an Inclusive DH Community (MacLaurin A144)
4:15 to 6:00
Chair: TBA
Working in the Digital Humanities - An Exploration of Scholarly Practices for Early-career Academics. Steve
Anderson (U California, Riverside); Matt Bouchard (U Toronto); Andy Keenan (U Toronto); Lee Zickel (Case
Western Reserve U)
DH Internships: Building Digital Humanities Capacity with Care at Emory. Alan G. Pike (Emory U)
ABC's of Gamification: How we Gamified a Social Media Course. Rob Bajko (Ryerson U), Jaigris Hodson
(Royal Roads U)
Undergraduate Contributorship Models with TaDiRAH. Aaron Mauro (Penn State, Behrend)
The Anti-MOOC: An Online Small Seminar Format for Distance Mentoring and Digital Public History Projects.
Cathy Kroll (Sonoma State U)
Wednesday, 15 June 2016
9:00 to Noon
Classes in Session
12:15 to 1:15
Lunch break / Unconference
ILiADS: Institute for Liberal Arts Digital Scholarship information session (location TBA)
1:30 to 4:00
Classes in Session
DHSI Colloquium Session 5 (MacLaurin A144)
4:15 to 6:00
Chair: TBA
Sounds and Digital Humanities. John F. Barber (Washington State U, Vancouver)
Mapping a Digital History of Big Science. Elyse Graham (SUNY, Stony Brook)
Mapping a Global Renaissance with 53,829 Texts. James Lee (Grinnel College)
Quantifying the Language of Othering. Bryor Snefjella (McMaster U)
Digitization and Dissemination of Movable Books Data. Emily Brooks (U Florida)
Diversity in Aesthetic Categories: Using Biblical Translation to Examine Word Sense and Popular Belief. Zach
Bleemer (U California, Berkeley)
Thursday, 16 June 2016
9:00 to Noon
DHSI Classes in Session
12:15 to 1:15
Lunch break / Unconference
[Instructor lunch meeting]
1:30 to 4:00
Classes in Session
DHSI Colloquium Session 6 (MacLaurin A144)
4:15 to 6:00
Chair: TBA
Using DH to Increase Legal Literacy and Agency. Susan Tanner (Carnegie Mellon U)
Mapping German POWs in the Soviet Gulag, 1941-1956. Susan Grunewald (Carnegie Mellon U)
Detecting Text Reuse in Nineteenth-Century Legal Codes of Civil Procedure. Lincoln Mullen (George Mason
U); Kellen Funk (Princeton U)
Collection, Curation, and Collaboration: Representing Canadian Gay Liberationists. Nadine Boulay (Simon
Fraser U); Anderson Tuguinay (Ryerson U); Stefanie Martin (Ryerson U); Seamus Riordan-Short (U British
Columbia, Okanagan); Raymon Sandhu (U British Columbia, Okanagan)
Privacy, Legality, and Feminism: How Do We Build a Feminist Politics into Open Access Data Structures?
Emily Christina Murphy (Queen’s U)
Recreating the Fiction Factory: The James Malcolm Rymer Collection. Rebecca Nesvet (U Wisconsin, Green
Bay)
Digital History and Archiving: Fostering the “Afterlife” and Accessibility of American Civil War Letters. Ashley
Hughes (Texas Christian U)
Friday, 17 June 2016
9:00 to Noon
Classes in Session
12:15 to 1:15
Lunch Reception / Course E-Exhibits (MacLaurin A100)
Awards and Bursaries Recognition
Institute Lecture
James Cummings (Oxford U): “Grass Roots and Ivory Towers: Building communities and inspiring participation
in the Digital Humanities”
Chair: TBA
(MacLaurin A144)
1:30 to 2:45
2:45 to 3:00
Abstract: The various fields and organisations in the big tent of Digital Humanities all approach community
involvement in different ways. Outside traditional academic teaching, much of the work is done by communities
which thrive partly as a result of in-kind commitments and volunteer labour, often donated in evenings and
weekends by those striving to make their particular endeavour a success. By re-examining a number of the DH
communities I have been involved with, I hope to trace some of their successes, failures, risks, and the
necessary fragilities of their development and undertakings. This re-examination will encompass a range of
community projects from entirely grass roots and mostly finance-free endeavours such as the open DH Awards
(http://dhawards.org) and Digital Medievalist (http://digitalmedievalist.org) project through to self-funding breakeven institutional activities such as DHOxSS (http://digital.humanities.ox.ac.uk/dhoxss/) and membership
consortia such as the TEI Consortium (http://www.tei-c.org/). Each of these projects employs different strategies
to ensure that their work relates to their communities and engages the public, with some methods proving more
successful than others in eliciting help or volunteer labour from these communities. By investigating these
methods, I hope to tease out the more successful strategies and to discuss how these might be employed by
those setting up new DH initiatives.
Closing, DHSI in Review (MacLaurin A144)
Contact info:
[email protected] P: 250-472-5401 F: 250-472-5681
Sayward
.
h Rd
Pat Bay Hwy.
83
r.
6
2
UVic
P
X
Direction of Travel
Route Name
Transit Exchange
Park & Ride Lot
(no overnight parking)
Major Stop
Coo
3 7N
Passenger
Ferry to Seattle
Beacon Hill
27 28
3
3
30 33
Memo
75 66 99
Ferry to
Port Angeles
rial
31 32 61 71 72
Fairfield
Plaza
Ross
ourne
7
Rich
da
y
Ba
oro
db
Ca
Foul B
ay
mon
d
ker
or
Centr
al
Oak Bay
8
Winds
1
Oak Bay
Marina
2
OAK
BAY
3
ch
Clover
Point
Cattle
Point
Willows
Beach
2
son
eld
50 70 27 28 30
2A
8
Richard
irfi
May
Bow
7
1
7
Shelb
wood
ay
Fa
Legislature
Legend
tic
ll
Rd.
nich
Saa
rd
Cook
ood
Oak B
k
3
Fort
15 14 11
1
Beacon
Hill Park
Cadboro
Bay
Royal
Jubilee
Hospital
ria
WESTSHORE
28
13
11
14
10
2
3
O
Bay
son
an
go
e
sw
Ogden Point
Breakwater
27
3
3
28
in
2
8
Gyro
Park
15
der
Hen
Lansd
owne
4
22
Haulta
Camosun
College
Lansdowne
11
11
Henderson
Rec. Centre
tev
Es
DND
Work Point
Barracks
8
27
22
ra
Lam
ser
25
Quad
15
Blansha
10
10
4
24
Bay
4
Victo
nd
n
pso
n
ea
Fra
10 14
Hillside
13
UVic
7
12 13 26 39 51 76
ty
ner
Fin
14
ill
15 16 4 14 7 11
s
nro
24
Gore
nzie
Hillside
Centre
VICTORIA
75
uir
Dunsm
12
McKe
rH
da
Ce
Finlayson
72
t
15
d.
Bea
X
UVic
16 39
XR
24
71
ner
Mu
25
8
ad
He
ir
22
quimal
Lya
ll
rd
Go
cla
Mayfair
Shopping
Centre
Skin
Old Es
on
Blair
Sin
s
Esquimalt
Plaza
6
31
70
imalt
ill
25
ok
21
le
erda
Clov
30
8 11
Esqu
rH
31 71 72 75
Co
ral
15 25 26
16 70 26 30
m
utu
ide
26
da
27
12
Feltha
51
Ce
Maple
26
um
lic
Til
Go
rg
e
mi
Dockyard
26
27
Arb
11
Colville
24
16
rns
8
Gordon Head
Comm. Centre
12
University
Heights
Shopping
Centre
Uptown
Bu
r
14
ESQUIMALT
Ad
Esquimalt
Harbour
Saanich
Plaza
Douglas
y.
Hw
nd
Isla
Saanich
Centre
26
50 61
ed
we
flo
aig
t
on
lm
Be
24
25
52 53 54 61
Oc
a
6
51
Silver City
& Pearkes Tillicum
Mall
Arena
ale
ore
McKenzie
Ob
Cr
24
50 25 39 48 51
39
Esquimalt
Lagoon
er
11
X
Colwood
n
rs
Roge
28
12 Ken
m
Uptown
wp
Admirals
Walk
h
rnd
als
Admir
Westshore
Recreation
Centre
Ocea
s
ity
22
As
Fe
16
COLWOOD
52
e
P
Co
P
VIEW
ROYAL
nsid
cken
Helm
75
adr
50
21
28
Lily
Douglas
53
39
31
22N
Bur
25
53
30
8
22
22A
50 61
72
Pat Bay Hwy.
22
53
51
39
71
Qu
53
da
Holla
ana
Carey
14 22 39
sC
70
Interurban
Tra
n
50
22
Vic General
Mt. Douglas
Park
Tyn
gs
Blenkinsop
Hastin
47 61x
SAANICH
Gla
nfo
rd
Thetis
Lake
39
jes
39 83
6A
Ma
8 21 22N
Phelps
ak Dr.
6B
rton
Camosun
32 39
Royal O
Royal
Oak
Shopping
Centre
Chatt
e
Vancouver
Island
Tech Park
Broadmead Village
31
Em
ily C
arr
Ma
N
.
son
kin
Wil
rk
ct
Commonwealth Pool
30
PENINSULA
d
ay R
aB
39
ham
70
dov
Quayle
83
Viadu
Cordova Bay
32
71
75
39 71 72 75 83
rton
Co r
72
Halibu
70x
35
70 6 30 31 32 35
Local Route
20–120 minute service
56
ake D
X
Royal Oak
Frequent Route
15 minute or better service
7am-7pm, Mon-Fri
Elk L
Beaver
Lake
ont
Fernw
Pros
pect
Regional Route
15–60 minute service
with limited stops
Clarem
Foul Bay
Prospect
Lake
Average Frequency
DelM
onte
Lake
Rd.
d.
hR
75
Wes
ley
ic
an
Sa
72
Elk
Lake
Regional Map of Greater Victoria
Fowler
71
aanic
st
We
tS
Wes
Oldfield
Old
70
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
University Systems
home » help centre » internet & telephone » wireless internet
University Systems help centre
Internet and telephone
! Notices & bulletins
Wireless Internet
Connect to UVic: Windows 7 and Vista
NEW UVic wireless configuration utility
The UVic wireless configuration utility will automatically configure the "UVic" wireless network on your
Windows XP SP3, Windows Vista, or Windows 7 computer.
Download now
Note: The UVic wireless configuration utility is still experimental; use this application at your own risk.
UVic is not responsible for any damage caused by the use of the wireless configuration utility. Please
report any problems to the Computer Help Desk.
If the above doesn't work, please follow the manual instructions listed below. After the initial
configuration, you should automatically connect to UVic (the secure wireless network) when you are on
campus.
1. Before you start this procedure, ensure the following:
Your wireless card and its drivers have been installed and you have rebooted your laptop
since the installation.
Your laptop is powered on and booted up.
You are in an area with wireless coverage.
You have a NetLink ID and password.
You are using Windows to manage your wireless connections. If you are using a third-party
application (sometimes network adaptors come with their own applications), you may
experience problems during the configuration process.
2. Temporarily connect to the Internet using UVicStart, an Ethernet port, or your home network.
Download the security certificate by right clicking thawte Primary Root CA and saving the
thawte.cer file to your computer. Once the file is saved to your computer, locate the file, double
click on it, select Install Certificate..., and follow the Certificate Import Wizard instructions.
Exam and Survey Scanning
Posted March 18, 2015
VHS Technology Phase-Out
Posted February 25, 2015
Critical security
vulnerability in Bash
Posted September 25,
2014
Windows 8.1 classroom
training and drop-in
sessions
Posted September 18,
2014
Exchange email and
calendaring system
upgrade
Posted July 14, 2014
" Links of interest
" Subscribe to RSS
" Twitter
Status of our
services
Service
E-mail
Connectivity
WebApps
Storage
Telephone
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
1/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
3. Once you have successfully installed the certificate, open your Start menu and click on Control
Panel.
4. Click on Network and Internet or Network and Sharing Center.
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
2/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
5. Click on Network and Sharing Center.
6. Click on Manage wireless networks, located on the left menu.
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
3/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
7. Click Add.
8. Click Manually create a network profile.
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
4/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
9. Enter the following information:
Network name: UVic (case sensitive).
Security type: select WPA2-Enterprise.
Encryption type: automatically sets to AES.
Security Key/Passphrase: (leave blank).
Ensure both checkboxes are selected (by default, the second box is not). Click Next.
10. Click Change connection settings. For now, ignore the pop-up window in the bottom-right corner.
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
5/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
On the Connection tab, ensure the Connect to a more preferred network if available checkbox
is not checked.
11. Click the Security tab. Ensure the authentication method is PEAP. Then click Settings.
12. Check the box beside thawte Primary Root CA in the list of Trusted Root Certification
Authorities.
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
6/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
If you cannot find the correct certificate listed, please return to step 2 to download the certificate.
At the bottom of the dialogue, ensure that the Authentication Method is Secured password
(EAP-MSCHAP v2). Click Configure.
13. Deselect the checkbox for Automatically use my Windows logon... and click OK.
14. Close the remaining windows. In the bottom-right corner of your screen, you should see a small
window pop-up informing you that Additional information is required to connect to UVic. Click
on it to provide additional information.
15. Enter your personal NetLink ID followed by @uvic.ca in the User name field, and your NetLink ID
password in the Password field. Click OK.
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
7/8
3/30/15
Connect to UVic: Windows 7 and Vista - University of Victoria
You should now be connected to the UVic secure wireless network.
Related support
Related services
How-tos
AirNet wireless coverage
Setups
Connect to eduroam: HTC mobile device
Connect to eduroam: Mac OS X 10.5 or newer
Connect to eduroam: Windows 7
Connect to eduroam: Windows Vista
Connect to eduroam: Windows XP
Connect to UVic: Android version 4 and newer
Connect to UVic: iPhone or iPod Touch
Connect to UVic: OS 10.5 and newer
Connect to UVic: Windows 7 and Vista
Connect to UVic: Windows XP
Connect to UVic wireless
Connect to UVic Wireless: eduroam
Wireless Internet
University of Victoria - 3800 Finnerty Road - Victoria BC V8P 5C2 - Canada - Phone: 1-250-721-7211 - Fax: 1-250-721-7212
www.uvic.ca/systems/support/internettelephone/wireless/default7.php
8/8
3/30/15
Connect to UVic: OS 10.5 and newer - University of Victoria
University Systems
home » help centre » internet & telephone » wireless internet
University Systems help centre
! Notices & bulletins
Internet and telephone
Wireless Internet
Connect to UVic: Mac OS X 10.5 and newer
After the initial configuration, you should automatically connect to UVic (the secure wireless network)
when you are using UVic's wireless network.
1. Before you start this procedure, ensure the following:
Your wireless card and its drivers have been installed and you have rebooted your laptop
since the installation.
Your laptop is powered on and booted up.
You are in an area with wireless coverage.
You have a NetLink ID and password.
2. At the top-right corner of your screen there should be the AirPort icon (a semi-circle). If you do not
see this icon, your AirPort card or AirPort software may not have been installed properly.
3. Click on the AirPort icon (it may be partially darkened) to reveal a menu. Ensure your AirPort is
On.
Exam and Survey Scanning
Posted March 18, 2015
VHS Technology Phase-Out
Posted February 25, 2015
Critical security
vulnerability in Bash
Posted September 25,
2014
Windows 8.1 classroom
training and drop-in
sessions
Posted September 18,
2014
Exchange email and
calendaring system
upgrade
Posted July 14, 2014
" Links of interest
" Subscribe to RSS
" Twitter
4. Scroll down the AirPort menu and select Join Other Network ....
Status of our
services
Service
E-mail
Connectivity
WebApps
Storage
Telephone
5. In the window that opens, enter the following information:
Network Name: UVic (case sensitive)
Security: WPA2-Enterprise
User Name: your NetLink ID
Password: your NetLink ID password
802.1X: Automatic
Click Join.
www.uvic.ca/systems/support/internettelephone/wireless/defaultosx.php
1/3
3/30/15
Connect to UVic: OS 10.5 and newer - University of Victoria
6. If you see a message about Mac OS X wanting to access your Keychain, click Always Allow.
7. A Verify Certificate window will open saying that the certificate is not trusted.
Click Show Certificate.
Check the box that says Always trust "sac1cled050..." (the exact name may vary) and click
Continue.
If you are prompted for your computer password, enter it and click OK.
www.uvic.ca/systems/support/internettelephone/wireless/defaultosx.php
2/3
3/30/15
Connect to UVic: OS 10.5 and newer - University of Victoria
You should now be connected to the UVic secure wireless network. To disconnect from the wireless
network, click on the AirPort icon and click Turn Airport Off. Next time you connect to UVic, you should
not need to enter any additional credentials.
Related support
Related services
How-tos
AirNet wireless coverage
Setups
Connect to eduroam: HTC mobile device
Connect to eduroam: Mac OS X 10.5 or newer
Connect to eduroam: Windows 7
Connect to eduroam: Windows Vista
Connect to eduroam: Windows XP
Connect to UVic: Android version 4 and newer
Connect to UVic: iPhone or iPod Touch
Connect to UVic: OS 10.5 and newer
Connect to UVic: Windows 7 and Vista
Connect to UVic: Windows XP
Connect to UVic wireless
Connect to UVic Wireless: eduroam
Wireless Internet
University of Victoria - 3800 Finnerty Road - Victoria BC V8P 5C2 - Canada - Phone: 1-250-721-7211 - Fax: 1-250-721-7212
www.uvic.ca/systems/support/internettelephone/wireless/defaultosx.php
3/3
3/30/15
Connect to UVic: iPhone or iPod Touch - University of Victoria
University Systems
home » help centre » internet & telephone » wireless internet
University Systems help centre
Internet and telephone
Wireless Internet
Connect to UVic: iPhone or iPod Touch
After the initial configuration, you should automatically connect to UVic (the secure wireless network)
when you are using UVic's wireless network.
1. Before you start this procedure, ensure the following:
Your device is using firmware version 4.0 or higher.
Your device is powered on and booted up.
You are in an area with wireless coverage.
You have a NetLink ID and password.
2. From the Home screen, press the Settings button.
3. Press the Wi-Fi option.
! Notices & bulletins
Exam and Survey Scanning
Posted March 18, 2015
VHS Technology Phase-Out
Posted February 25, 2015
Critical security
vulnerability in Bash
Posted September 25,
2014
Windows 8.1 classroom
training and drop-in
sessions
Posted September 18,
2014
Exchange email and
calendaring system
upgrade
Posted July 14, 2014
" Links of interest
" Subscribe to RSS
" Twitter
Status of our
services
Service
E-mail
Connectivity
4. Under the Choose a Network... heading, select UVic.
WebApps
Storage
Telephone
www.uvic.ca/systems/support/internettelephone/wireless/defaultiphone.php
1/4
3/30/15
Connect to UVic: iPhone or iPod Touch - University of Victoria
5. Enter your personal NetLink ID followed by @uvic.ca in the Username field. Enter your NetLink
ID password in the Password field. Press Join.
6. If prompted, press Accept to verify the thawte Primary Root CA certificate.
www.uvic.ca/systems/support/internettelephone/wireless/defaultiphone.php
2/4
3/30/15
Connect to UVic: iPhone or iPod Touch - University of Victoria
Your device should now be connected to the UVic secure wireless network.
Related support
Related services
How-tos
AirNet wireless coverage
Setups
Connect to eduroam: HTC mobile device
Connect to eduroam: Mac OS X 10.5 or newer
Connect to eduroam: Windows 7
Connect to eduroam: Windows Vista
Connect to eduroam: Windows XP
Connect to UVic: Android version 4 and newer
Connect to UVic: iPhone or iPod Touch
Connect to UVic: OS 10.5 and newer
Connect to UVic: Windows 7 and Vista
Connect to UVic: Windows XP
Connect to UVic wireless
Connect to UVic Wireless: eduroam
Wireless Internet
www.uvic.ca/systems/support/internettelephone/wireless/defaultiphone.php
3/4
3/30/15
Connect to UVic: Android version 4 and newer - University of Victoria
University Systems
home » help centre » internet & telephone » wireless internet
University Systems help centre
! Notices & bulletins
Internet and telephone
Wireless Internet
Connect to UVic: Android version 4 and newer
Please note: Android devices do not fully support Exchange ActiveSync encryption so they are not
recommended standards.
After the initial configuration, you should automatically connect to UVic (the secure wireless network)
when you are using UVic's wireless network. Please note that University Systems only supports devices
running Android version 4 or newer; all other devices are best-effort support only.
1. Before you start this procedure, ensure the following:
Your device is running version 4.0 or higher
Your device is powered on and booted up.
You are in an area with wireless coverage.
You have a NetLink ID and password.
2. Go into Settings.
3. Press Wi-Fi.
Exam and Survey Scanning
Posted March 18, 2015
VHS Technology Phase-Out
Posted February 25, 2015
Critical security
vulnerability in Bash
Posted September 25,
2014
Windows 8.1 classroom
training and drop-in
sessions
Posted September 18,
2014
Exchange email and
calendaring system
upgrade
Posted July 14, 2014
" Links of interest
" Subscribe to RSS
" Twitter
Status of our
services
Service
E-mail
Connectivity
WebApps
Storage
Telephone
www.uvic.ca/systems/support/internettelephone/wireless/uvicandroid.php
1/4
3/30/15
Connect to UVic: Android version 4 and newer - University of Victoria
4. Select UVic
5. In the window that opens, enter your UVic NetLink ID followed by @uvic.ca (e.g.
[email protected]) in the Identity field. Enter the corresponding NetLink ID password and press
Connect.
www.uvic.ca/systems/support/internettelephone/wireless/uvicandroid.php
2/4
3/30/15
Connect to UVic: Android version 4 and newer - University of Victoria
6. Your device should now be connected to the UVic wireless network.
Related support
Related services
www.uvic.ca/systems/support/internettelephone/wireless/uvicandroid.php
3/4
!
Creating LAMP Infrastructure for Digital Humanities Projects - DHSI 2016
Instructor: Jonathan Martin ([email protected] && @songsthatsaved)
Suggested Course Hashtag: #DHLinux
Figure 1. Obligatory XKCD 1 - https://xkcd.com/456/
Cheerful, Disarming Course Introduction
Hi, everyone!
Welcome to the course! In our time together at DHSI, we’re going to learn how to build a
semi-traditional LAMP1 stack. In so doing, we’re going to become hackers. While this
won’t involve spending time with Matthew Lillard, or – perhaps more poignantly – Chris
Hemsworth, it will mean that we will soon have all the tools and knowledge we need to
build a server anywhere, at any time, using freely available tools.
Even better, we’ll learn how to keep that server safe and stable. (We may learn how not
to gloat about this. Maybe not.)
We’ll learn how to customize (hack) that server, and we’ll learn how to build programs
1
As you likely know, LAMP stands for “Linux, Apache, MySQL, and PHP.” Our ‘M’ is MariaDB, a
fully interoperable fork of MySQL.
from scratch to suit a wide range of project needs.2
Below, you will find our schedule, some suggested resources, and some general
reading material. Don’t obsess too much over this last set of material, as we will cover
most of it in class. I’ve provided it as a reference for the work ahead.
So, then, let me just say that I’m thrilled to embark on this journey with all of you, and I
hope you will find the course helpful. Please don’t hesitate to get in touch (during, or out
of, class) with any questions or concerns.
Cheers, Jonathan
2
After all that, we’ll probably have lunch. ;)
Draft Course Schedule (Implementation May Vary)
Monday
Tuesday
Command
Line, Linux
Conventions
Review
(lecture and
hands-on)
Apache
Overview
(lecture),
Setting up
Apache
(hands- on)
Installing,
Updating,
and
Securing
Linux
(lecture and
hands-on)
Configuring
Apache,
Setting up
PHP,
Introduction
to Package
Management
(lecture and
hands-on)
Wednesday
Thursday
Friday
Morning
MariaDB/SQL
Server
Nginx, Mail
Overview
Administration
servers,
(lecture),
Tools (lecture
MAMP,
Setting up
and handsWAMP,
MariaDB
on),
Q&A
(hands-on)
Resources for
(lecture
further study and demo)
(lecture)
Afternoon
Server
Caching,
Installing
Languages,
and Web
frameworks
(lecture and
hands-on)
Hands-on
projects and
practice.
N/A
Useful Resources:
Our Virtual Environment: VirtualBox
Our Linux Distro: Debian
The Command Line Crash Course – Awesome for review purposes.
-
Alternative review at Codeacademy.
ExplainShell.com – Wonderful way to figure out “What the hell this command does...”
Finally, please remember that this course could save your life…
Plausible hypothetical:
Figure 2. Obligatory XKCD 2 - http://xkcd.com/1168/
SECOND EDITION
Linux
Pocket Guide
Daniel J. Barrett
Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo
Linux Pocket Guide
Welcome to Linux! If you’re a new user, this book can serve as
a quick introduction, as well as a guide to common and practical commands. If you have Linux experience, feel free to skip
the introductory material.
What’s in This Book?
This book is a short guide, not a comprehensive reference. We
cover important, useful aspects of Linux so you can work productively. We do not, however, present every single command
and every last option (our apologies if your favorite was omitted), nor delve into detail about operating system internals.
Short, sweet, and essential, that’s our motto.
We focus on commands, those pesky little words you type on
a command line to tell a Linux system what to do. Here’s an
example command that counts lines of text in a file, myfile:
wc -l myfile
We’ll cover the most important Linux commands for the average user, such as ls (list files), grep (search for text in a file),
amarok (play audio files), and df (measure free disk space). We
touch only briefly on graphical windowing environments like
GNOME and KDE, each of which could fill a Pocket Guide by
itself.
1
We’ve organized the material by function to provide a concise
learning path. For example, to help you view the contents of a
file, we introduce all file-viewing commands together: cat for
short text files, less for longer ones, od for binary files, acro
read for PDF files, and so on. Then we explain each command
in turn, briefly presenting its common uses and options.
We assume you have an account on a Linux system and know
how to log in with your username and password. If not, speak
with your system administrator, or if the system is your own,
use the account created when you installed Linux.
What’s Linux?
Linux is a popular, open source operating system that competes with Microsoft Windows and the Apple Macintosh.
There are two ways to work with a Linux system:
• A graphical user interface with windows, icons, and
mouse control.
• A command-line interface, called the shell, for typing and
running commands like the preceding wc.
Windows and Mac OS computers can be operated by command line as well (Windows with its cmd and PowerShell command tools, and OS X with its Terminal application), but most
of their users can survive without typing commands. On Linux,
however, the shell is critical. If you use Linux without the shell,
you are missing out.
What’s a Distro?
Linux is extremely configurable and includes thousands of
programs. As a result, different varieties of Linux have arisen
to serve different needs and tastes. They all share certain core
components but may look different and include different programs and files. Each variety is called a distro (short for “distribution”). Popular distros include Ubuntu Linux, Red Hat
2 | Linux Pocket Guide
Enterprise Linux, Slackware, Mint, and more. This book covers core material that should apply to every distro.
What’s a Command?
A Linux command typically consists of a program name followed by options and arguments, typed within a shell, like this:
$ wc -l myfile
The program name (wc, the “word count” program) refers to a
program somewhere on disk that the shell will locate and run.
Options, which usually begin with a dash, affect the behavior
of the program. In the preceding command, the -l option tells
wc to count lines rather than words. The argument myfile
specifies the file that wc should read and process. The leading
dollar sign ($) is a prompt from the shell, indicating that it is
waiting for your command.
Commands can have multiple options and arguments. Options
may be given individually:
$ wc -l -w myfile
Two individual options
or combined behind a single dash:
$ wc -lw myfile
Same as -l -w
though some programs are quirky and do not recognize combined options. Multiple arguments are also OK:
$ wc -l myfile1 myfile2
Count lines in two files
Options are not standardized. The same option letter (say,
-l) may have different meanings to different programs: in
wc -l it means “lines of text,” but in ls -l it means “longer
output.” In the other direction, two programs might use different options to mean the same thing, such as -q for “run quietly” versus -s for “run silently.”
Likewise, arguments are not standardized, unfortunately. They
usually represent filenames for input or output, but they can
be other things too, like directory names or regular
expressions.
What’s in This Book? | 3
Commands can be more complex and interesting than a single
program with options:
• Commands can run more than one program at a time,
either in sequence (one program after another) or in a
“pipeline” with the output of one command becoming the
input of the next. Linux experts use pipelines all the time.
• The Linux command-line user interface—the shell—has
a programming language built in. So instead of a command saying “run this program,” it might say, “if today is
Tuesday, run this program; otherwise, run another command six times for each file whose name ends in .txt.”
Reading This Book
We’ll describe many Linux commands in this book. Each description begins with a standard heading about the command;
Figure 1 shows one for the ls (list files) command. This heading
demonstrates the general usage in a simple format:
ls [options] [files]
which means you’d type “ls” followed, if you choose, by options and then filenames. You wouldn’t type the square brackets “[” and “]”: they just indicate their contents are optional;
and words in italics mean you have to fill in your own specific
values, like names of actual files. If you see a vertical bar between options or arguments, perhaps grouped by parentheses:
(file | directory)
This indicates choice: you may supply either a filename or directory name as an argument.
The special heading also includes six properties of the command printed in black (supported) or gray (unsupported):
stdin
The command reads from standard input, i.e., your keyboard, by default. See “Input and Output” on page 12.
4 | Linux Pocket Guide
Figure 1. Standard command heading
stdout
The command writes to standard output, i.e., your screen,
by default. See “Input and Output” on page 12.
- file
When given a dash (-) argument in place of an input filename, the command reads from standard input; and likewise, if the dash is supplied as an output filename, the
command writes to standard output. For example, the
following wc command line reads the files file1 and file2,
then standard input, then file3:
$ wc file1 file2 - file3
-- opt
If you supply the command-line option “--” it means “end
of options”: anything appearing later on the command
line is not an option. This is sometimes necessary to operate on a file whose name begins with a dash, which
otherwise would be (mistakenly) treated as an option. For
example, if you have a file named -foo, the command wc
-foo will fail because -foo will be treated as an (invalid)
option. wc -- -foo works. If a command does not support
“--”, you can prepend the current directory path “./” to
the filename so the dash is no longer the first character:
$ wc ./-foo
--help
The option --help makes the command print a help message explaining proper usage, then exit.
--version
The option --version makes the command print its version information and exit.
What’s in This Book? | 5
Shell prompts
Some commands in this book can be run successfully only by
the superuser, a special user with permission to do anything on
the system. In this case, we use a hash mark (#) as the shell
prompt:
# superuser command goes here
Otherwise, we will use the dollar sign prompt, indicating an
ordinary user:
$ ordinary command goes here
Keystrokes
Throughout the book, we use certain symbols to indicate keystrokes. Like many other Linux documents, we use the ^ symbol to mean “press and hold the Control (Ctrl) key,” so for
example, ^D (pronounced “control D”) means “press and hold
the Control key and type D.” We also write ESC to mean “press
the Escape key.” Keys like Enter and the space bar should be
self-explanatory.
Your friend, the echo command
In many of our examples, we’ll print information to the screen
with the echo command, which we’ll formally describe in
“Screen Output” on page 168. echo is one of the simplest
commands: it merely prints its arguments on standard output,
once those arguments have been processed by the shell.
$ echo My dog has fleas
My dog has fleas
$ echo My name is $USER
My name is smith
Shell variable USER
Getting Help
If you need more information than this book provides, there
are several things you can do.
6 | Linux Pocket Guide
Run the man command
The man command displays an online manual page, or
manpage, for a given program. For example, to learn about
listing files with ls, run:
$ man ls
To search for manpages by keyword for a particular topic,
use the -k option followed by the keyword:
$ man -k database
Run the info command
The info command is an extended, hypertext help system
covering many Linux programs.
$ info ls
While info is running, some useful keystrokes are:
• To get help, type h
• To quit, type q
• To page forward and backward, use the space bar and
Backspace keys
• To jump between hyperlinks, press TAB
• To follow a hyperlink, press Enter
If info has no documentation on a given program, it displays the program’s manpage. For a listing of available
documentation, type info by itself. To learn how to navigate the info system, type info info.
Use the --help option (if any)
Many Linux commands respond to the option --help by
printing a short help message. Try:
$ ls --help
If the output is longer than the screen, pipe it into the
less program to display it in pages (press q to quit):
$ ls --help | less
Getting Help | 7
Examine the directory /usr/share/doc
This directory contains supporting documents for many
programs, usually organized by program name and version. For example, files for the text editor emacs, version
23, are likely found (depending on distro) in /usr/share/
doc/emacs23.
GNOME and KDE Help
For help with GNOME or KDE, visit http://www.gnome
.org or http://www.kde.org.
Distro-specific websites
Most Linux distros have an official site that includes documentation, discussion forums for questions and answers, and other resources. Simply enter the distro name
(e.g., “Ubuntu”) into any popular search engine to find its
web site. You can also visit the web site for this book: http:
//shop.oreilly.com/product/0636920023029.do.
Linux help sites
Many web sites answer Linux questions, such as http://
www.linuxquestions.org, http://unix.stackexchange.com,
http://www.linuxhelp.net, and http://www.linuxforums
.org.
Web search
To decipher a specific Linux error message, enter the message into a web search engine, word for word, and you
will likely find helpful results.
Linux: A First View
Linux has four major parts:
The kernel
The low-level operating system, handling files, disks, networking, and other necessities we take for granted. Most
users rarely notice the kernel.
Supplied programs
Thousands of programs for file manipulation, text editing,
mathematics, web browsing, audio, video, computer
8 | Linux Pocket Guide
programming, typesetting, encryption, DVD burning…
you name it.
The shell
A user interface for typing commands, executing them,
and displaying the results. Linux has various shells: the
Bourne shell, Korn shell, C shell, and others. This book
focuses on bash, the Bourne-Again Shell, which is often
the default for user accounts. However, all these shells
have similar basic functions.
X
A graphical system that provides windows, menus, icons,
mouse support, and other familiar GUI elements. More
complex graphical environments are built on X; the most
popular are KDE and GNOME. We’ll discuss a few programs that open X windows to run.
This book focuses on the second and third parts: supplied programs and the shell.
The Graphical Desktop
When you log into a Linux system, you’re likely to be greeted
by a graphical desktop1 like Figure 2, which contains:
• A main menu or taskbar. Depending on your distro and
system settings, this might be at the top, bottom, or side
of the screen.
• Desktop icons representing the computer, a folder representing your home directory for personal files, a trash can,
and more.
• Icons to run applications, such as the Firefox web browser
and the Thunderbird email program.
• Controls for opening and closing windows and running
multiple desktops at once.
1. Unless you’re logging in remotely over the network, in which case you’ll
see just a command prompt, waiting for you to type a command.
Linux: A First View | 9
• A clock and other small, informational icons.
Figure 2. Graphical desktops (CentOS Linux with GNOME, Ubuntu
with KDE). Desktops can look wildly different, depending on your
distro and system settings.
10 | Linux Pocket Guide
Linux systems have several graphical interfaces, the most common being GNOME and KDE. Identify yours by clicking your
system’s equivalent of a main menu or start menu and looking
for the words GNOME, KDE, Kubuntu (KDE on Ubuntu Linux), or similar.
Running a Shell
The icons and menus in GNOME and KDE are, for some users,
the primary way to work with Linux. This is fine for simple
tasks like reading email and browsing the Web. Nevertheless,
the true power of Linux lies beneath this graphical interface,
in the shell.
To get the most out of Linux, take the time to become proficient with the shell. (That’s what this book is all about.) It
might initially be more difficult than icons and menus, but once
you’re used to it, the shell is quite easy to use and very powerful.
To run a shell within GNOME, KDE, or any other graphical
interface for Linux, you need to open a shell window: a window
with a shell running in it. Figure 2 shows two shell windows
with “$” shell prompts, awaiting your commands. Look
through your system menus for an application to do this. Typical menu items are Terminal, xterm, gnome-terminal, konsole,
and uxterm.
Don’t confuse the window program (like konsole) with the
shell running inside it. The window is just a container—
possibly with fancy features of its own—but the shell is what
prompts you for commands and runs them.
If you’re not running a graphical interface—say, you’re logging
in remotely over the network, or directly over an attached
terminal—a shell will run immediately when you log in. No
shell window is required.
This was just a quick introduction. We’ll discuss more details
in “The Shell” on page 22, and cover more powerful constructs in “Programming with Shell Scripts” on page 195.
Linux: A First View | 11
Input and Output
Most Linux commands accept input and produce output. Input can come from files or from standard input, which is usually
your keyboard. Likewise, output is written to files or to standard output, which is usually your shell window or screen. Error messages are treated specially and displayed on standard
error, which also is usually your screen but kept separate from
standard output.2 Later we’ll see how to redirect standard input, output, and error to and from files or pipes. But let’s get
our vocabulary straight. When we say a command “reads,” we
mean from standard input unless we say otherwise. And when
a command “writes” or “prints,” we mean on standard output,
unless we’re talking about computer printers.
Users and Superusers
Linux is a multiuser operating system: multiple people can use
a single Linux computer at the same time. On a given computer, each user is identified by a unique username, like
“smith” or “funkyguy,” and owns a (reasonably) private part
of the system for doing work. There is also a special user named
root—the superuser—who has the privileges to do anything at
all on the system. Ordinary users are restricted: though they
can run most programs, in general they can modify only the
files they own. The superuser, on the other hand, can create,
modify, or delete any file and run any program.
To become the superuser, you needn’t log out and log back in;
just run the su command (see “Becoming the Superuser” on page 138) and provide the superuser password:
$ su -l
Password: *******
#
2. For example, you can capture standard output in a file and still have
standard error messages appear on screen.
12 | Linux Pocket Guide
The superuser prompt (#) indicates that you’re ready to run
superuser commands. Alternatively, run the sudo command (if
your system is configured to use it), which executes a single
command as the superuser, then returns control to the original
user:
$ sudo ls /private/secrets
Password: *******
secretfile1
secretfile2
$
View a protected directory
It worked!
The Filesystem
To make use of any Linux system, you need to be comfortable
with Linux files and directories (a.k.a. folders). In a “windows
and icons” system, the files and directories are obvious on
screen. With a command-line system like the Linux shell, the
same files and directories are still present but are not constantly
visible, so at times you must remember which directory you
are “in” and how it relates to other directories. You’ll use shell
commands like cd and pwd to “move” between directories and
keep track of where you are.
Let’s cover some terminology. As we’ve said, Linux files are
collected into directories. The directories form a hierarchy, or
tree, as in Figure 3: one directory may contain other directories,
called subdirectories, which may themselves contain other files
and subdirectories, and so on, into infinity. The topmost directory is called the root directory and is denoted by a slash (/).3
We refer to files and directories using a “names and slashes”
syntax called a path. For instance, this path:
/one/two/three/four
refers to the root directory /, which contains a directory called
one, which contains a directory two, which contains a directory
3. In Linux, all files and directories descend from the root. This is unlike
Windows or DOS, in which different devices are accessed by drive
letters.
The Filesystem | 13
Figure 3. A Linux filesystem (partial). The root folder is at the top.
The “dan” folder’s full path is /home/dan.
three, which contains a final file or directory, four. If a path
begins with the root directory, it’s called an absolute path, and
if not, it’s a relative path. More on this in a moment.
Whenever you are running a shell, that shell is working “in”
some directory (in an abstract sense). More technically, your
shell has a current working directory, and when you run commands in that shell, they operate relative (there’s that word
again) to the directory. More specifically, if you refer to a relative file path in that shell, it is relative to your current working
directory. For example, if your shell is “in” the directory /one/
two/three, and you run a command that refers to a file myfile,
then the file is really /one/two/three/myfile. Likewise, a relative
path a/b/c would imply the true path /one/two/three/a/b/c.
Two special directories are denoted . (a single period) and ..
(two periods in a row). The former means your current directory, and the latter means your parent directory, one level
above. So if your current directory is /one/two/three, then .
refers to this directory and .. refers to /one/two.
You “move” your shell from one directory to another using the
cd command:
$ cd /one/two/three
More technically, this command changes your shell’s current
working directory to be /one/two/three. This is an absolute
14 | Linux Pocket Guide
change (since the directory begins with “/”); of course you can
make relative moves as well:
$ cd d
$ cd ../mydir
Enter subdirectory d
Go up to my parent, then into directory mydir
File and directory names may contain most characters you expect: capital and lowercase letters,4 numbers, periods, dashes,
underscores, and most symbols (but not “/”, which is reserved
for separating directories). For practical use, however, avoid
spaces, asterisks, parentheses, and other characters that have
special meaning to the shell. Otherwise, you’ll need to quote
or escape these characters all the time. (See “Quoting” on page 29.)
Home Directories
Users’ personal files are often found in /home (for ordinary
users) or /root (for the superuser). Your home directory is typically /home/your-username: /home/smith, /home/jones, etc.
There are several ways to locate or refer to your home directory.
cd
With no arguments, the cd command returns you (i.e., sets
the shell’s working directory) to your home directory.
HOME variable
The environment variable HOME (see “Shell variables” on page 25) contains the name of your home
directory.
$ echo $HOME
/home/smith
The echo command prints its arguments
˜
When used in place of a directory, a lone tilde is expanded
by the shell to the name of your home directory.
$ echo ˜
/home/smith
4. Linux filenames are case-sensitive, so capital and lowercase letters are
not equivalent.
The Filesystem | 15
When followed by a username (as in ~fred), the shell expands this string to be the user’s home directory:
$ cd ˜fred
$ pwd
/home/fred
The “print working directory” command
System Directories
A typical Linux system has tens of thousands of system
directories. These directories contain operating system files,
applications, documentation, and just about everything except personal user files (which typically live in /home).
Unless you’re a system administrator, you’ll rarely visit most
system directories—but with a little knowledge you can understand or guess their purposes. Their names often contain
three parts, which we’ll call the scope, category, and application. (These are not standard terms, but they’ll help you understand things.) For example, the directory /usr/local/share/
emacs, which contains local data for the emacs text editor, has
scope /usr/local (locally installed system files), category share
(program-specific data and documentation), and application
emacs (a text editor), shown in Figure 4. We’ll explain these
three parts, slightly out of order.
Figure 4. Directory scope, category, and application
Directory path part 1: category
A category tells you the types of files found in a directory. For
example, if the category is bin, you can be reasonably assured
that the directory contains programs. Common categories are:
16 | Linux Pocket Guide
Categories for programs
bin
Programs (usually binary files)
sbin
Programs (usually binary files) intended to be run by the superuser
lib
Libraries of code used by programs
libexec
Programs invoked by other programs, not usually by users; think “library
of executable programs”
Categories for documentation
doc
Documentation
info
Documentation files for emacs’s built-in help system
man
Documentation files (manual pages) displayed by the man program; the
files are often compressed, or sprinkled with typesetting commands for
man to interpret
share
Program-specific files, such as examples and installation instructions
Categories for configuration
etc
Configuration files for the system (and other miscellaneous stuff)
init.d
Configuration files for booting Linux
rc.d
Configuration files for booting Linux; also rc1.d, rc2.d, ...
Categories for programming
include
Header files for programming
src
Source code for programs
Categories for web files
cgi-bin
Scripts/programs that run on web pages
html
Web pages
public_html
Web pages, typically in users’ home directories
www
Web pages
Categories for display
fonts
Fonts (surprise!)
X11
X window system files
Categories for hardware
dev
Device files for interfacing with disks and other hardware
The Filesystem | 17
media
Mount points: directories that provide access to disks
mnt
Mount points: directories that provide access to disks
misc
Mount points: directories that provide access to disks
Categories for runtime files
var
Files specific to this computer, created and updated as the computer runs
lock
Lock files, created by programs to say, “I am running”; the existence of a
lock file may prevent another program, or another instance of the same
program, from running or performing an action
log
Log files that track important system events, containing error, warning,
and informational messages
mail
Mailboxes for incoming mail
run
PID files, which contain the IDs of running processes; these files are often
consulted to track or kill particular processes
spool
Files queued or in transit, such as outgoing email, print jobs, and scheduled jobs
tmp
Temporary storage for programs and/or people to use
proc
Operating system state: see “Operating System Directories”
on page 19
Directory path part 2: scope
The scope of a directory path describes, at a high level, the purpose of an entire directory hierarchy. Some common ones are:
/
System files supplied with Linux (pronounced “root”)
/usr
More system files supplied with Linux (pronounced “user”)
/usr/games
Games (surprise!)
/usr/local
System files developed “locally,” either for your organization or your
individual computer
/usr/X11R6
Files pertaining to the X window system
So for a category like lib (libraries), your Linux system might
have directories /lib, /usr/lib, /usr/local/lib, /usr/games/lib,
and /usr/X11R6/lib.
18 | Linux Pocket Guide
There isn’t a clear distinction between / and /usr in practice,
but there is a sense that / is “lower-level” and closer to the
operating system. So /bin contains fundamental programs like
ls and cat, /usr/bin contains a wide variety of applications supplied with your Linux distribution, and /usr/local/bin contains
programs your system administrator chose to install. These are
not hard-and-fast rules but typical cases.
Directory path part 3: application
The application part of a directory path, if present, is usually
the name of a program. After the scope and category (say, /usr/
local/doc), a program may have its own subdirectory (say, /usr/
local/doc/myprogram) containing files it needs.
Operating System Directories
Some directories support the Linux kernel, the lowest-level
part of the Linux operating system.
/boot
Files for booting the system. This is where the kernel lives,
typically named /boot/vmlinuz.
/lost+found
Damaged files that were rescued by a disk recovery tool.
/proc
Describes currently running processes; for advanced
users.
The files in /proc provide views into the running kernel and
have special properties. They always appear to be zero sized,
read-only, and dated now:
$ ls -l /proc/version
-r--r--r-1 root
root
0 Oct
3 22:55 /proc/version
However, their contents magically contain information about
the Linux kernel:
$ cat /proc/version
Linux version 2.6.32-71.el6.i686 ...
The Filesystem | 19
Files in /proc are used mostly by programs, but feel free to explore them. Here are some examples:
/proc/ioports
A list of your computer’s input/output hardware.
/proc/version
The operating system version. The uname command prints the same
information.
/proc/uptime
System uptime, i.e., seconds elapsed since the system was last booted.
Run the uptime command for a more human-readable result.
/proc/nnn
Where nnn is a positive integer, information about the Linux process
with process ID nnn.
/proc/self
Information about the current process you’re running; a symbolic link
to a /proc/nnn file, automatically updated. Try ls -l /proc/
self several times in a row: you’ll see /proc/self changing where it
points.
File Protections
A Linux system may have many users with login accounts. To
maintain privacy and security, most users can access only
some files on the system, not all. This access control is embodied in two questions:
Who has permission?
Every file and directory has an owner who has permission
to do anything with it. Typically the user who created a
file is its owner, but relationships can be more complex.
Additionally, a predefined group of users may have permission to access a file. Groups are defined by the system
administrator and are covered in “Group Management” on page 140.
Finally, a file or directory can be opened to all users with
login accounts on the system. You’ll also see this set of
users called the world or simply other.
What kind of permission is granted?
File owners, groups, and the world may each have permission to read, write (modify), and execute (run) particular files. Permissions also extend to directories, which
20 | Linux Pocket Guide
users may read (access files within the directory), write
(create and delete files within the directory), and execute
(enter the directory with cd).
To see the ownership and permissions of a file, run:
$ ls -l myfile
-rw-r--r-- 1 smith smith
7384 Jan 04 22:40 myfile
To see the ownership and permissions of a directory, run:
$ ls -ld dirname
drwxr-x--- 3 smith smith
4096 Jan 08 15:02 dirname
In the output, the file permissions are the 10 leftmost characters, a string of r (read), w (write), x (execute), other letters, and
dashes. For example:
-rwxr-x---
Here’s what these letters and symbols mean.
Position
Meaning
1
File type: - = file, d = directory, l = symbolic link, p = named pipe,
c = character device, b = block device
2–4
Read, write, and execute permissions for the file’s owner
5–7
Read, write, and execute permissions for the file’s group
8–10
Read, write, and execute permissions for all other users
So our example -rwxr-x--- means a file that can be read, written, and executed by the owner, read and executed by the
group, and not accessed at all by the rest of the world. We
describe ls in more detail in “Basic File Operations” on page 36. To change the owner, group ownership,
or permissions of a file, use the chown, chgrp, and chmod commands, respectively, as described in “File Properties” on page 59.
The Filesystem | 21
The Shell
In order to run commands on a Linux system, you’ll need
somewhere to type them. That “somewhere” is called the
shell, which is Linux’s command-line user interface: you type
a command and press Enter, and the shell runs whatever program (or programs) you’ve requested. (See “Running a
Shell” on page 11 to learn how to open a shell window.)
For example, to see who’s logged in, you could execute this
command in a shell:
$ who
silver
byrnes
barrett
silver
:0
pts/0
pts/1
pts/2
Sep
Sep
Sep
Sep
23
15
22
22
20:44
13:51
21:15
21:18
(The dollar sign is the shell prompt, which means the shell is
ready to run a command.) A single command can also invoke
several programs at the same time, and even connect programs
together so they interact. Here’s a command that redirects the
output of the who program to become the input of the wc program, which counts lines of text in a file; the result is the number of lines in the output of who:
$ who | wc -l
4
telling you how many users are logged in.5 The vertical bar,
called a pipe, makes the connection between who and wc.
A shell is actually a program itself, and Linux has several. We
focus on bash (the Bourne-Again Shell), located in /bin/bash,
which is usually the default in Linux distros.
5. Actually, how many interactive shells those users are running. If a user
has two shells running, like the user silver in our example, he’ll have
two lines of output from who.
22 | Linux Pocket Guide
The Shell Versus Programs
When you run a command, it might invoke a Linux program
(like who), or instead it might be a built-in command, a feature
of the shell itself. You can tell the difference with the type
command:
$ type who
who is /usr/bin/who
$ type cd
cd is a shell builtin
It is helpful to know what the shell provides versus what Linux
does. The next few sections describe features of the shell.
Selected Features of the bash Shell
A shell does much more than simply run commands. It also
has powerful features to make this task easier: wildcards for
matching filenames, a “command history” to recall previous
commands quickly, pipes for making the output of one command become the input of another, variables for storing values
for use by the shell, and more. Take the time to learn these
features, and you will become faster and more productive with
Linux. Let’s skim the surface and introduce you to these useful
tools. (For full documentation, run info bash.)
Wildcards
Wildcards are a shorthand for sets of files with similar names.
For example, a* means all files whose names begin with lowercase “a”. Wildcards are “expanded” by the shell into the actual set of filenames they match. So if you type:
$ ls a*
the shell first expands a* into the filenames that begin with “a”
in your current directory, as if you had typed:
$ ls aardvark adamantium apple
ls never knows you used a wildcard: it sees only the final list
of filenames after the shell expands the wildcard. Importantly,
The Shell | 23
this means every Linux command, regardless of its origin,
works with wildcards and other shell features.
Wildcards never match two characters: a leading period, and
the directory slash (/). These must be given literally, as
in .pro* to match .profile, or /etc/*conf to match all filenames
ending in conf in the /etc directory.
Dot Files
Filenames with a leading period, called dot files, are special in
Linux. When you name a file beginning with a period, it will
not be displayed by some programs:
• ls will omit the file from directory listings, unless you
provide the -a option
• Shell wildcards do not match a leading period
Effectively, dot files are hidden unless you explicitly ask to see
them. As a result, sometimes they are called “hidden files.”
Wildcard Meaning
*
Zero or more consecutive characters
?
Any single character
[set]
Any single character in the given set, most commonly a sequence of
characters, like [aeiouAEIOU] for all vowels, or a range with a dash, like
[A-Z] for all capital letters
[^set]
Any single character not in the given set (as in the earlier example)
[!set]
Same as ^
When using character sets, if you want to include a literal dash
in the set, put it first or last. To include a literal closing square
bracket in the set, put it first. To include a ^ or ! symbol literally, don’t put it first.
24 | Linux Pocket Guide
Brace expansion
Similar to wildcards, expressions with curly braces also expand
to become multiple arguments to a command. The commaseparated expression:
{X,YY,ZZZ}
expands first to X, then YY, and finally ZZZ within a command
line, like this:
$ echo sand{X,YY,ZZZ}wich
sandXwich sandYYwich sandZZZwich
Braces work with any strings, unlike wildcards, which are limited to filenames. The preceding example works regardless of
which files are in the current directory.
Shell variables
You can define variables and their values by assigning them:
$ MYVAR=3
To refer to a value, simply place a dollar sign in front of the
variable name:
$ echo $MYVAR
3
Some variables are standard and commonly defined by your
shell upon login.
Variable
Meaning
DISPLAY
The name of your X window display
HOME
Your home directory, such as /home/smith
LOGNAME
Your login name, such as smith
MAIL
Your incoming mailbox, such as /var/spool/mail/smith
OLDPWD
Your shell’s previous directory, prior to the last cd command
PATH
Your shell search path: directories separated by colons
PWD
Your shell’s current directory
SHELL
The path to your shell, e.g., /bin/bash
The Shell | 25
Variable
Meaning
TERM
The type of your terminal, e.g., xterm or vt100
USER
Your login name
To see a shell’s variables, run:
$ printenv
The scope of the variable (i.e., which programs know about it)
is, by default, the shell in which it’s defined. To make a variable
and its value available to other programs your shell invokes
(i.e., subshells), use the export command:
$ export MYVAR
or the shorthand:
$ export MYVAR=3
Your variable is now called an environment variable, since it’s
available to other programs in your shell’s “environment.” So
in the preceding example, the exported variable MYVAR is available to all programs run by that same shell (including shell
scripts: see “Variables” on page 196).
To make a variable value available to a specific program just
once, prepend variable=value to the command line:
$ printenv HOME
/home/smith
$ HOME=/home/sally printenv HOME
/home/sally
$ printenv HOME
/home/smith
The original value is unaffected
Search path
Programs are scattered all over the Linux filesystem, in directories like /bin and /usr/bin. When you run a program via a shell
command, how does the shell find it? The critical variable
PATH tells the shell where to look. When you type any
command:
$ who
26 | Linux Pocket Guide
the shell has to find the who program by searching through Linux directories. The shell consults the value of PATH, which is
a sequence of directories separated by colons:
$ echo $PATH
/usr/local/bin:/bin:/usr/bin:/home/smith/bin
and looks for the who command in each of these directories. If
it finds who (say, /usr/bin/who), it runs the command. Otherwise, it reports:
bash: who: command not found
To add directories to your shell’s search path temporarily,
modify its PATH variable. For example, to append /usr/sbin to
your shell’s search path:
$ PATH=$PATH:/usr/sbin
$ echo $PATH
/usr/local/bin:/bin:/usr/bin:/home/smith/bin:/usr/sbin
This change affects only the current shell. To make it permanent, modify the PATH variable in your startup file
~/.bash_profile, as explained in “Tailoring Shell Behavior” on page 36. Then log out and log back in.
Aliases
The built-in command alias defines a convenient shorthand
for a longer command, to save typing. For example:
$ alias ll='ls -l'
defines a new command ll that runs ls -l:
$ ll
total 436
-rw-r--r--rwxr-xr-x
...
1 smith
1 smith
3584 Oct 11 14:59 file1
72 Aug 6 23:04 file2
Define aliases in your ~/.bashrc file (see “Tailoring Shell Behavior” on page 36) to be available whenever you log in.6 To
list all your aliases, type alias. If aliases don’t seem powerful
6. Some setups use a separate file, ~/.bash_aliases, for this purpose.
The Shell | 27
enough for you (since they have no parameters or branching),
see “Programming with Shell Scripts” on page 195, run
info bash, and read up on “shell functions.”
Input/output redirection
The shell can redirect standard input, standard output, and
standard error to and from files. In other words, any command
that reads from standard input can have its input come from a
file instead with the shell’s < operator:
$ mycommand < infile
Likewise, any command that writes to standard output can
write to a file instead:
$ mycommand > outfile
$ mycommand >> outfile
Create/overwrite outfile
Append to outfile
A command that writes to standard error can have its output
redirected to a file as well, while standard output still goes to
the screen:
$ mycommand 2> errorfile
To redirect both standard output and standard error to files:
$ mycommand > outfile 2> errorfile
$ mycommand >& outfile
Separate files
Single file
Pipes
You can redirect the standard output of one command to be
the standard input of another, using the shell’s pipe (|) operator. For example:
$ who | sort
sends the output of who into the sort program, printing an alphabetically sorted list of logged-in users. Multiple pipes work
too. Here we sort the output of who again, extract the first column of information (using awk), and display the results one
page at a time (using less):
$ who | sort | awk '{print $1}' | less
28 | Linux Pocket Guide
Combining commands
To invoke several commands in sequence on a single command
line, separate them with semicolons:
$ command1 ; command2 ; command3
To run a sequence of commands as before, but stop execution
if any of them fails, separate them with && (“and”) symbols:
$ command1 && command2 && command3
To run a sequence of commands, stopping execution as soon
as one succeeds, separate them with || (“or”) symbols:
$ command1 || command2 || command3
Quoting
Normally, the shell treats whitespace simply as separating the
words on the command line. If you want a word to contain
whitespace (e.g., a filename with a space in it), surround it with
single or double quotes to make the shell treat it as a unit. Single
quotes treat their contents literally, while double quotes let
shell constructs be evaluated, such as variables:
$ echo 'The variable HOME has value $HOME'
The variable HOME has value $HOME
$ echo "The variable HOME has value $HOME"
The variable HOME has value /home/smith
Backquotes (“backticks”) cause their contents to be evaluated
as a shell command. The contents are then replaced by the
standard output of the command:
$ whoami
Program that prints your username
smith
$ echo My name is `whoami`
My name is smith
Escaping
If a character has special meaning to the shell but you want it
used literally (e.g., * as a literal asterisk rather than a wildcard),
precede the character with the backward slash “\” character.
This is called escaping the special character:
The Shell | 29
$ echo a*
aardvark agnostic apple
$ echo a\*
a*
$ echo "I live in $HOME"
I live in /home/smith
$ echo "I live in \$HOME"
I live in $HOME
As a wildcard, matching “a” filenames
As a literal asterisk
Dollar sign means a variable value
A literal dollar sign
You can also escape control characters (tabs, newlines, ^D,
and so forth) to have them used literally on the command line,
if you precede them with ^V. This is particularly useful for tab
(^I) characters, which the shell would otherwise use for filename completion (see “Filename completion” on page 31).
$ echo "There is a tab between here^V^I and here"
There is a tab between here
and here
Command-line editing
Bash lets you edit the command line you’re working on, using
keystrokes inspired by the text editors emacs and vi (see “File
Creation and Editing” on page 54). To enable command-line
editing with emacs keys, run this command (and place it in
your ~/.bash_profile to make it permanent):
$ set -o emacs
For vi keys:
$ set -o vi
emacs keystroke
vi keystroke (after ESC)
Meaning
^P or up arrow
k
Go to previous command
^N or down arrow
j
Go to next command
^F or right arrow
l
Go forward one character
^B or left arrow
h
Go backward one character
^A
0
Go to beginning of line
^E
$
Go to end of line
^D
x
Delete next character
^U
^U
Erase entire line
30 | Linux Pocket Guide
Command history
You can recall previous commands you’ve run—that is, the
shell’s history—and re-execute them. Some useful
history-related commands are listed below.
Command
Meaning
history
Print your history
history N
Print the most recent N commands in your history
history
-c
Clear (delete) your history
!!
Re-run previous command
!N
Re-run command number N in your history
!-N
Re-run the command you typed N commands ago
!$
Represents the last parameter from the previous command; great for
checking that files are present before removing them:
$ ls a*
acorn.txt
$ rm !$
!*
affidavit
Represents all parameters from the previous command:
$ ls a b c
a
b
c
$ wc !*
103
252
2904 a
12
25
384 b
25473 65510 988215 c
25588 65787 991503 total
Filename completion
Press the TAB key while you are in the middle of typing a filename, and the shell will automatically complete (finish typing)
the filename for you. If several filenames match what you’ve
typed so far, the shell will beep, indicating the match is ambiguous. Immediately press TAB again and the shell will
present the alternatives. Try this:
$ cd /usr/bin
$ ls un<TAB><TAB>
The Shell | 31
The shell will display all files in /usr/bin that begin with un,
such as uniq, units, and unzip. Type a few more characters to
disambiguate your choice and press TAB again.
Shell Job Control
jobs
List your jobs.
&
Run a job in the background.
^Z
Suspend the current (foreground) job.
suspend
Suspend a shell.
fg
Unsuspend a job: bring it into the foreground.
bg
Make a suspended job run in the background.
All Linux shells have job control: the ability to run programs in
the background (multitasking behind the scenes) and foreground (running as the active process at your shell prompt). A
job is simply the shell’s unit of work. When you run a command
interactively, your current shell tracks it as a job. When the
command completes, the associated job disappears. Jobs are
at a higher level than Linux processes; the Linux operating
system knows nothing about them. They are merely constructs
of the shell. Some important vocabulary about job control is:
Foreground job
Running in a shell, occupying the shell prompt so you
cannot run another command
Background job
Running in a shell, but not occupying the shell prompt,
so you can run another command in the same shell
Suspend
To stop a foreground job temporarily
Resume
To cause a suspended job to start running again
32 | Linux Pocket Guide
jobs
The built-in command jobs lists the jobs running in your current
shell.
$ jobs
[1]- Running
[2]+ Stopped
emacs myfile &
su
The integer on the left is the job number, and the plus sign identifies
the default job affected by the fg (foreground) and bg (background)
commands.
&
Placed at the end of a command line, the ampersand causes the
given command to run as a background job.
$ emacs myfile &
[2] 28090
The shell’s response includes the job number (2) and the process
ID of the command (28090).
^Z
Typing ^Z in a shell, while a job is running in the foreground, will
suspend that job. It simply stops running, but its state is
remembered.
$ mybigprogram
^Z
[1]+ Stopped
$
mybigprogram
Now you’re ready to type bg to put the command into the background, or fg to resume it in the foreground.
suspend
The built-in command suspend will suspend the current shell if
possible, as if you’d typed ^Z to the shell itself. For instance, if you’ve
run the su command and want to return to your original shell:
The Shell | 33
$ whoami
smith
$ su -l
Password: *******
# whoami
root
# suspend
[1]+ Stopped
$ whoami
smith
su
bg
bg [%jobnumber]
The built-in command bg sends a suspended job to run in the
background. With no arguments, bg operates on the most recently
suspended job. To specify a particular job (shown by the jobs command), supply the job number preceded by a percent sign:
$ bg %2
Some types of interactive jobs cannot remain in the background—
for instance, if they are waiting for input. If you try, the shell will
suspend the job and display:
[2]+
Stopped
command line here
You can now resume the job (with fg) and continue.
fg
fg [%jobnumber]
The built-in command fg brings a suspended or backgrounded job
into the foreground. With no arguments, it selects a job, usually the
most recently suspended or backgrounded one. To specify a particular job (as shown by the jobs command), supply the job number
preceded by a percent sign:
$ fg %2
34 | Linux Pocket Guide
Killing a Command in Progress
If you’ve launched a command from the shell running in the
foreground, and want to kill it immediately, type ^C. The shell
recognizes ^C as meaning, “terminate the current foreground
command right now.” So if you are displaying a very long file
(say, with the cat command) and want to stop, type ^C:
$ cat bigfile
This is a very long file with many lines. Blah blah blah
blah blah blah blahblahblah ^C
$
To kill a program running in the background, you can bring it
into the foreground with fg and then type ^C, or alternatively,
use the kill command (see “Controlling Processes” on page 121).
Typing ^C is not a friendly way to end a program. If the program
has its own way to exit, use that when possible: see the sidebar
for details.
Surviving a Kill
Killing a foreground program with ^C may leave your shell in
an odd or unresponsive state, perhaps not displaying the keystrokes you type. This happens because the killed program had
no opportunity to clean up after itself. If this happens to you:
1. Press ^J to get a shell prompt. This produces the same
character as the Enter key (a newline) but will work even
if Enter does not.
2. Type the shell command reset (even if the letters don’t
appear while you type) and press ^J again to run this
command. This should bring your shell back to normal.
^C works only with shells. It will likely have no effect if typed
in a window that is not a shell window. Additionally, some
programs are written to “catch” the ^C and ignore it: an example is the text editor emacs.
The Shell | 35
Terminating a Shell
To terminate a shell, either run the exit command or type ^D.7
$ exit
Tailoring Shell Behavior
To configure all your shells to work in a particular way, edit
the files .bash_profile and .bashrc in your home directory.
These files execute each time you log in (~/.bash_profile) or
open a shell (~/.bashrc). They can set variables and aliases, run
programs, print your horoscope, or whatever you like.
These two files are examples of shell scripts: executable files
that contain shell commands. We’ll cover this feature in more
detail in “Programming with Shell Scripts” on page 195.
This concludes our basic overview of Linux and the shell. Now
we turn to Linux commands, listing and describing the most
useful commands for working with files, processes, users, networking, multimedia, and more.
Basic File Operations
ls
List files in a directory.
cp
Copy a file.
mv
Rename (“move”) a file.
rm
Delete (“remove”) a file.
ln
Create links (alternative names) to a file.
One of the first things you’ll need to do on a Linux system is
manipulate files: copying, renaming, deleting, and so forth.
7. Control-D sends an “end of file” signal to any program reading from
standard input. In this case, the program is the shell itself, which
terminates.
36 | Linux Pocket Guide
ls
stdin
stdout
- file
-- opt
--help
--version
ls [options] [files]
The ls command (pronounced as it is spelled, ell ess) lists attributes
of files and directories. You can list files in the current directory:
$ ls
in given directories:
$ ls dir1 dir2 dir3
or individually:
$ ls file1 file2 file3
The most important options are -a, -l, and -d. By default, ls hides
files whose names begin with a dot, as explained in the sidebar
“Dot Files” on page 24. The -a option displays all files.
$ ls
myfile1
myfile2
$ ls -a
.hidden_file
myfile1
myfile2
The -l option produces a long listing:
-rw-r--r--
1 smith users
149 Oct 28
2011 my.data
that includes, from left to right: the file’s permissions (-rw-r--r--),
owner (smith), group (users), size (149 bytes), last modification
date (Oct 28 2011) and name. See “File Protections” on page 20 for
more information on permissions.
The -d option lists information about a directory itself, rather than
descending into the directory to list its files.
$ ls -ld my.dir
drwxr-xr-x
1 smith users
4096 Oct 29
2011 my.dir
Useful options
-a
List all files, including those whose names begin with a dot.
-l
Long listing, including file attributes. Add the -h option (human-readable) to print
file sizes in kilobytes, megabytes, and gigabytes, instead of bytes.
-F
Decorate certain filenames with meaningful symbols, indicating their types. Appends “/” to directories, “*” to executables, “@” to symbolic links, “|” to named
Basic File Operations | 37
pipes, and “=” to sockets. These are just visual indicators for you, not part of the
filenames!
-i
Prepend the inode numbers of the files.
-s
Prepend the size of the file in blocks, useful for sorting files by their size:
-R
If listing a directory, list its contents recursively.
-d
If listing a directory, do not list its contents, just the directory itself.
$ ls -s | sort -n
cp
stdin
stdout
- file
-- opt
--help
--version
cp [options] files (file | directory)
The cp command normally copies a file:
$ cp file file2
or copies multiple files into a directory:
$ cp file1 file2 file3 file4 destination_directory
Using the -a option, you can also recursively copy directories.
Useful options
-p
Copy not only the file contents, but also the file’s permissions, timestamps and, if
you have sufficient permission to do so, its owner and group. (Normally the copies
will be owned by you, timestamped now, with permissions set by applying your
umask to the original permissions.)
-a
Copy a directory hierarchy recursively, preserving all file attributes and links.
-r
Copy a directory hierarchy recursively. This option does not preserve the files’
attributes such as permissions and timestamps. It does preserve symbolic links.
-i
Interactive mode. Ask before overwriting destination files.
-f
Force the copy. If a destination file exists, overwrite it unconditionally.
mv
stdin
stdout
- file
mv [options] source target
The mv (move) command can rename a file:
38 | Linux Pocket Guide
-- opt
--help
--version
$ mv file1 file2
or move files and directories into a destination directory:
$ mv file1 file2 dir3 dir4 destination_directory
Useful options
-i
Interactive mode. Ask before overwriting destination files.
-f
Force the move. If a destination file exists, overwrite it unconditionally.
rm
stdin
stdout
- file
-- opt
--help
--version
rm [options] files | directories
The rm (remove) command can delete files:
$ rm file1 file2 file3
or recursively delete directories:
$ rm -r dir1 dir2
Useful options
-i
Interactive mode. Ask before deleting each file.
-f
Force the deletion, ignoring any errors or warnings.
-r
Recursively remove a directory and its contents. Use with caution, especially if
combined with the -f option, as it can wipe out all your files.
ln
stdin
stdout
- file
-- opt
--help
--version
ln [options] source target
A link is a reference to another file, created by the ln command.
Intuitively, links give the same file multiple names, allowing it to
live in two (or more) locations at once.
There are two kinds of links. A symbolic link (also called a symlink
or soft link) refers to another file by its path, much like a Windows
“shortcut” or a Macintosh “alias.” To create a symbolic link, use
the -s option:
$ ln -s myfile mysoftlink
Basic File Operations | 39
If you delete the original file, the now-dangling link will be invalid,
pointing to a nonexistent file path. A hard link, on the other hand,
is simply a second name for a physical file on disk (in tech talk, it
points to the same inode). If you delete the original file, the link still
works. Figure 5 illustrates the difference. To create a hard link, type:
$ ln myfile myhardlink
Figure 5. Hard link versus symbolic link
Symbolic links can point to files on other disk partitions, since they
are just references to file paths; hard links cannot, since an inode
on one disk has no meaning on another. Symbolic links can also
point to directories, whereas hard links cannot...unless you are the
superuser and use the -d option.
40 | Linux Pocket Guide
Useful options
-s
Make a symbolic link. The default is a hard link.
-i
Interactive mode. Ask before overwriting destination files.
-f
Force the link. If a destination file exists, overwrite it unconditionally.
-d
Create a hard link to a directory (superusers only).
It’s easy to find out where a symbolic link points with either of these
commands:
$ readlink linkname
$ ls -l linkname
Directory Operations
cd
Change your current directory.
pwd
Print the name of your current directory, i.e., “where you are now” in the
filesystem.
basename
Print the final part of a file path.
dirname
Print a file path without its final part.
mkdir
Create (make) a directory.
rmdir
Delete (remove) an empty directory.
rm -r
Delete a nonempty directory and its contents.
We discussed the directory structure of Linux in “The Filesystem” on page 13. Now we’ll cover commands that create,
modify, delete, and manipulate directories within that
structure.
cd
stdin
stdout
- file
-- opt
--help
--version
cd [directory]
The cd (change directory) command sets your current working
directory:
$ cd /usr/games
Directory Operations | 41
With no directory supplied, cd defaults to your home directory:
$ cd
pwd
stdin
stdout
- file
-- opt
--help
--version
pwd
The pwd command prints the absolute path of your current working
directory:
$ pwd
/users/smith/mydir
basename
stdin
stdout
- file
-- opt
--help
--version
basename path [suffix]
The basename command prints the final component in a file path:
$ basename /users/smith/finances/money.txt
money.txt
If you provide an optional suffix, it gets stripped from the result:
$ basename /users/smith/finances/money.txt .txt
money
dirname
stdin
stdout
- file
-- opt
--help
--version
dirname path
The dirname command prints a file path with its final component
removed:
$ dirname /users/smith/mydir
/users/smith
dirname does not change your current working directory. It simply
manipulates a string, just like basename does.
42 | Linux Pocket Guide
mkdir
stdin
stdout
- file
-- opt
--help
--version
mkdir [options] directories
mkdir creates one or more directories:
$ mkdir directory1 directory2 directory3
Useful options
-p
Given a directory path (not just a simple directory name), create
any necessary parent directories automatically:
mkdir -p /one/two/three creates /one and /one/two if
they don’t already exist, then /one/two/three.
-m mode
Create the directory with the given permissions:
$ mkdir -m 0755 mydir
By default, your shell’s umask controls the permissions. See the
chmod command in “File Properties” on page 59, and “File
Protections” on page 20.
rmdir
stdin
stdout
- file
-- opt
--help
--version
rmdir [options] directories
The rmdir (remove directory) command deletes one or more empty
directories you name:
$ rmdir /tmp/junk
Useful options
-p
If you supply a directory path (not just a simple directory name), delete not only
the given directory, but the specified parent directories automatically, all of which
must be empty. So rmdir -p /one/two/three will delete not only /one/
two/three, but also /one/two and /one.
To delete a nonempty directory and its contents, use (carefully)
rm -r directory. Use rm -ri to delete interactively, or rm -rf to
annihilate without any error messages or confirmation.
Directory Operations | 43
File Viewing
cat
View files in their entirety.
less
View text files one page at a time.
head
View the first lines of a text file.
tail
View the last lines of a text file.
nl
View text files with their lines numbered.
strings
Display text that’s embedded in a binary file.
od
View data in octal (or other formats).
xxd
View data in hexadecimal.
acroread
View PDF files.
gv
View PostScript or PDF files.
xdvi
View TeX DVI files.
In Linux, you’ll encounter various types of files to view: plain
text, PostScript, binary data, and more. Here we’ll explain how
to view them. Note that commands for viewing graphics files
are covered in “Graphics and Screensavers” on page 181, and
video files in “Video” on page 188.
cat
stdin
stdout
- file
-- opt
--help
--version
cat [options] [files]
The simplest viewer is cat, which just prints its files to standard
output, concatenating them (hence the name). Large files will likely
scroll off screen, so consider using less if you plan to read the output. That being said, cat is particularly useful for sending a set of
files into a shell pipeline:
$ cat * | wc
cat can also manipulate its output in small ways, optionally dis-
playing nonprinting characters, prepending line numbers (though
nl is more powerful for this purpose), and eliminating whitespace.
44 | Linux Pocket Guide
Useful options
-T
Print tabs as ^I.
-E
Print newlines as $.
-v
Print other nonprinting characters in a human-readable format.
-n
Prepend line numbers to every line.
-b
Prepend line numbers to nonblank lines.
-s
Squeeze each sequence of blank lines into a single blank line.
less
stdin
stdout8
- file
-- opt
--help
--version
less [options] [files]
Use less to view text one “page” at a time (i.e., one window or
screenful at a time). It’s great for text files, or as the final command
in a shell pipeline with lengthy output.
$ command1 | command2 | command3 | command4 | less
While running less, type h for a help message describing all its features. Here are some useful keystrokes for paging through files.
Keystroke
Meaning
h, H
View a help page.
Space bar, f, ^V, ^F
Move forward one screenful.
Enter
Move forward one line.
b, ^B, ESC-v
Move backward one screenful.
/
Enter search mode. Follow it with a regular expression and
press Enter, and less will look for the first line matching
it.
?
Same as /, but it searches backward in the file.
n
Repeat your most recent search forward.
N
Repeat your most recent search backward.
8. Although technically less can be plugged into the middle of a pipeline,
or its output redirected to a file, there isn’t much point to doing this.
File Viewing | 45
Keystroke
Meaning
v
Edit the current file with your default text editor (the value
of environment variable VISUAL, or if not defined,
EDITOR, or if not defined, vi).
<
Jump to beginning of file.
>
Jump to end of file.
:n
Jump to next file.
:p
Jump to previous file.
less has a mind-boggling number of features; we’re presenting only
the most common. (For instance, less will display the contents of
a compressed Zip file: try less myfile.zip.) The manpage is rec-
ommended reading.
Useful options
-c
Clear the screen before displaying the next page. This avoids scrolling and may be
more comfortable on the eyes.
-m
Print a more verbose prompt, displaying the percentage of the file displayed so far.
-N
Display line numbers.
-r
Display control characters literally; normally less converts them to a
human-readable format.
-s
Squeeze multiple, adjacent blank lines into a single blank line.
-S
Truncate long lines to the width of the screen, instead of wrapping.
head
stdin
stdout
- file
-- opt
--help
--version
head [options] [files]
The head command prints the first 10 lines of a file: great for previewing the contents.
$ head myfile
$ head * | less
46 | Linux Pocket Guide
Preview all files in the current directory
It’s also good for previewing the first few lines of output from a
pipeline:
$ grep 'E' very-big-file | head
Useful options
-N
Print the first N lines instead of 10.
-n N
Print the first N lines instead of 10.
-c N
Print the first N bytes of the file.
-q
Quiet mode: when processing more than one file, don’t print a banner above
each file. Normally, head prints a banner containing the filename.
tail
stdin
stdout
- file
-- opt
--help
--version
tail [options] [files]
The tail command prints the last 10 lines of a file, and does other
tricks as well.
$ tail myfile
The ultra-useful -f option causes tail to watch a file actively while
another program is writing to it, displaying new lines as they are
written to the file. This is invaluable for watching log files in active
use:
$ tail -f /var/log/messages
Useful options
-N
Print the last N lines of the file instead of 10.
-n N
Print the last N lines of the file instead of 10.
-n +N
Print all lines except the first N.
-c N
Print the last N bytes of the file.
-f
Keep the file open, and whenever lines are appended to the file, print them.
This is extremely useful. Add the --retry option if the file doesn’t exist yet,
but you want to wait for it to exist.
-q
Quiet mode: when processing more than one file, don’t print a banner above
each file. Normally tail prints a banner containing the filename.
File Viewing | 47
nl
stdin
stdout
- file
-- opt
--help
--version
nl [options] [files]
nl copies its files to standard output, prepending line numbers.
$ nl myfile
1 Once upon a time, there was
2 a little operating system named
3 Linux, which everybody loved.
It’s more flexible than cat with its -n and -b options, providing an
almost bizarre amount of control over the numbering. nl can be
used in two ways: on ordinary text files, and on specially markedup text files with predefined headers and footers.
Useful options
-b [a|t|n|p R ] Prepend numbers to all lines (a), nonblank lines (t), no lines (n),
or only lines that contain regular expression R. (Default=a)
-v N
Begin numbering with integer N. (Default=1)
-i N
Increment the number by N for each line, so for example, you could
use odd numbers only (-i2) or even numbers only (-v2 -i2).
(Default=1)
-n [ln|rn|rz]
Format numbers as left-justified (ln), right-justified (rn), or rightjustified with leading zeroes (rz). (Default=ln)
-w N
Force the width of the number to be N columns. (Default=6)
-s S
Insert string S between the line number and the text. (Default=TAB)
Additionally, nl has the wacky ability to divide text files into virtual
pages, each with a header, body, and footer with different numbering schemes. For this to work, however, you must insert nl-specific
delimiter strings into the file, such as \:\:\: (start of header),
\:\: (start of body), and \: (start of footer). Each must appear on
a line by itself. Then you can use additional options (see the manpage) to affect line numbering in the headers and footers of your
decorated file.
48 | Linux Pocket Guide
strings
stdin
stdout
- file
-- opt
--help
--version
strings [options] [files]
Binary files, such as executable programs and object files, usually
contain some readable text. The strings program extracts that text
and displays it on standard output. You can discover version information, authors’ names, and other useful tidbits with strings.
$ strings /usr/bin/who
David MacKenzie
Copyright %s %d Free Software Foundation, Inc.
Report %s bugs to %s
...
Combine strings and grep to make your exploring more efficient.
Here we look for email addresses:
$ strings /usr/bin/who | grep '@'
[email protected]
Useful options
-n length
Display only strings with length greater than length (the default is 4).
od
stdin
stdout
- file
-- opt
--help
--version
od [options] [files]
When you want to view a binary file, consider od (Octal Dump) for
the job. It copies one or more files to standard output, displaying
their data in ASCII, octal, decimal, hexadecimal, or floating point,
in various sizes (byte, short, long). For example, this command:
$ od -w8 /usr/bin/who
0000000 042577 043114 000401 000001
0000010 000000 000000 000000 000000
0000020 000002 000003 000001 000000
...
displays the bytes in binary file /usr/bin/who in octal, eight bytes per
line. The column on the left contains the file offset of each row,
again in octal.
File Viewing | 49
If your binary file also contains text, consider the -tc option, which
displays character data. For example, binary executables like who
contain the string “ELF” at the beginning:
$ od -tc -w8 /usr/bin/who | head -3
0000000 177
E
L
F 001 001 001
0000010 \0 \0 \0 \0 \0 \0 \0
0000020 002 \0 003 \0 001 \0 \0
\0
\0
\0
Useful options
-N B
Display only the first B bytes of each file, specified in decimal,
hexadecimal (by prepending 0x or 0X), 512-byte blocks (by
appending b), kilobytes (by appending k), or megabytes (by appending m). (Default displays the entire file.)
-j B
Begin the output at byte B +1 of each file; acceptable formats are
the same as for the -N option. (Default=0)
-w [ B ]
Display B bytes per line; acceptable formats are the same as in the
-N option. Using -w by itself is equivalent to -w32. (Default=16)
-s [ B ]
Group each row of bytes into sequences of B bytes, separated by
whitespace; acceptable formats are the same as in the -N option.
Using -s by itself is equivalent to -s3. (Default=2)
-A (d|o|x|n)
Display file offsets in the leftmost column, in decimal (d), octal
(o), hexadecimal (h), or not at all (n). (Default=o)
-t (a|c)[z]
Display output in a character format, with nonalphanumeric characters printed as escape sequences (c) or by name (a). For z, see
below.
-t (d|o|u|x)
[SIZE[z]]
Display output in an integer format, including octal (o), signed
decimal (d), unsigned decimal (u), hexadecimal (x). (For binary
output, use xxd instead.) SIZE represents the number of bytes
per integer; it can be a positive integer or any of the values C, S, I,
or L, which stand for the size of a char, short, int, or long datatype,
respectively. For z, see below.
-t f[SIZE[z]]
Display output in floating point. SIZE represents the number of
bytes per integer; it can be a positive integer or any of the values
F, D, or L, which stand for the size of a float, double, or long double
datatype, respectively. For z, see below. If -t is omitted, the
default is -to2.
50 | Linux Pocket Guide
Appending z to the -t option prints a new column on the right-hand
side of the output, displaying the printable characters on each line,
much like the default output of xxd.
xxd
stdin
stdout
- file
-- opt
--help
--version
xxd [options] [files]
Similar to od, xxd produces a hexadecimal or binary dump of a file
in several different formats. It can also do the reverse, converting
from its hex dump format back into the original data. For example,
here’s a hex dump of binary file /usr/bin/who:
$ xxd /usr/bin/who
0000000: 7f45 4c46
0000010: 0200 0300
0000020: 6824 0000
0000030: 1900 1800
...
0101
0100
0000
0600
0100
0000
0000
0000
0000
a08c
3400
3400
...
...
...
...
0000
0000
2800
0408
.ELF............
............4...
h$......4. ...(.
........4...4...
The left column indicates the file offset of the row, the next eight
columns contain the data, and the final column displays the printable characters in the row, if any.
By default, xxd outputs three columns: file offsets, the data in hex,
and the data as text (printable characters only).
Useful options
-l N
Display only the first N bytes. (Default displays the entire file,)
-s N
Skip the first N bytes of the file.
-s -N
Begin N bytes from the end of the file. (There is also a +N syntax for more
advanced skipping through standard input; see the manpage.)
-c N
Display N bytes per row. (Default=16)
-g N
Group each row of bytes into sequences of N bytes, separated by whitespace,
like od -s. (Default=2)
-b
Display the output in binary instead of hexadecimal.
-u
Display the output in uppercase hexadecimal instead of lowercase.
-p
Display the output as a plain hexdump, 60 contiguous bytes per line.
File Viewing | 51
Part 2 – Learning The Shell
Part 2 – Learning The Shell
9
2 – What Is The Shell?
2 – What Is The Shell?
When we speak of the command line, we are really referring to the shell. The shell is a
program that takes keyboard commands and passes them to the operating system to carry
out. Almost all Linux distributions supply a shell program from the GNU Project called
bash. The name “bash” is an acronym for “Bourne Again SHell”, a reference to the fact
bash is an enhanced replacement for sh, the original Unix shell program written by
Steve Bourne.
Terminal Emulators
When using a graphical user interface, we need another program called a terminal
emulator to interact with the shell. If we look through our desktop menus, we will
probably find one. KDE uses konsole and GNOME uses gnome-terminal, though
it's likely called simply “terminal” on our menu. There are a number of other terminal
emulators available for Linux, but they all basically do the same thing; give us access to
the shell. You will probably develop a preference for one or another based on the number
of bells and whistles it has.
Your First Keystrokes
So let's get started. Launch the terminal emulator! Once it comes up, we should see
something like this:
[[email protected] ~]$
This is called a shell prompt and it will appear whenever the shell is ready to accept
input. While it may vary in appearance somewhat depending on the distribution, it will
usually include your [email protected], followed by the current working
directory (more about that in a little bit) and a dollar sign.
If the last character of the prompt is a pound sign (“#”) rather than a dollar sign, the
terminal session has superuser privileges. This means either we are logged in as the root
user or we selected a terminal emulator that provides superuser (administrative)
10
Your First Keystrokes
privileges.
Assuming that things are good so far, let's try some typing. Type some gibberish at the
prompt like so:
[[email protected] ~]$ kaekfjaeifj
Since this command makes no sense, the shell will tell us so and give us another chance:
bash: kaekfjaeifj: command not found
[[email protected] ~]$
Command History
If we press the up-arrow key, we will see that the previous command “kaekfjaeifj”
reappears after the prompt. This is called command history. Most Linux distributions
remember the last five hundred commands by default. Press the down-arrow key and the
previous command disappears.
Cursor Movement
Recall the previous command with the up-arrow key again. Now try the left and rightarrow keys. See how we can position the cursor anywhere on the command line? This
makes editing commands easy.
A Few Words About Mice And Focus
While the shell is all about the keyboard, you can also use a mouse with your
terminal emulator. There is a mechanism built into the X Window System (the
underlying engine that makes the GUI go) that supports a quick copy and paste
technique. If you highlight some text by holding down the left mouse button and
dragging the mouse over it (or double clicking on a word), it is copied into a
buffer maintained by X. Pressing the middle mouse button will cause the text to
be pasted at the cursor location. Try it.
Note: Don't be tempted to use Ctrl-c and Ctrl-v to perform copy and paste
inside a terminal window. They don't work. These control codes have different
meanings to the shell and were assigned many years before Microsoft Windows.
11
2 – What Is The Shell?
Your graphical desktop environment (most likely KDE or GNOME), in an effort
to behave like Windows, probably has its focus policy set to “click to focus.”
This means for a window to get focus (become active) you need to click on it.
This is contrary to the traditional X behavior of “focus follows mouse” which
means that a window gets focus by just passing the mouse over it. The window
will not come to the foreground until you click on it but it will be able to receive
input. Setting the focus policy to “focus follows mouse” will make the copy and
paste technique even more useful. Give it a try. I think if you give it a chance
you will prefer it. You will find this setting in the configuration program for your
window manager.
Try Some Simple Commands
Now that we have learned to type, let's try a few simple commands. The first one is
date. This command displays the current time and date.
[[email protected] ~]$ date
Thu Oct 25 13:51:54 EDT 2007
A related command is cal which, by default, displays a calendar of the current month.
[[email protected] ~]$ cal
October 2007
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31
To see the current amount of free space on your disk drives, type df:
[[email protected] ~]$ df
Filesystem
/dev/sda2
/dev/sda5
/dev/sda1
tmpfs
12
1K-blocks
15115452
59631908
147764
256856
Used Available Use% Mounted on
5012392
9949716 34% /
26545424 30008432 47% /home
17370
122765 13% /boot
0
256856
0% /dev/shm
Try Some Simple Commands
Likewise, to display the amount of free memory, type the free command.
[[email protected] ~]$ free
total
used
Mem:
513712
503976
-/+ buffers/cache: 375748
Swap: 1052248
104712
free
9736
137964
947536
shared
0
buffers
5312
cached
122916
Ending A Terminal Session
We can end a terminal session by either closing the terminal emulator window, or by
entering the exit command at the shell prompt:
[[email protected] ~]$ exit
The Console Behind The Curtain
Even if we have no terminal emulator running, several terminal sessions continue
to run behind the graphical desktop. Called virtual terminals or virtual consoles,
these sessions can be accessed on most Linux distributions by pressing CtrlAlt-F1 through Ctrl-Alt-F6 on most systems. When a session is accessed,
it presents a login prompt into which we can enter our user name and password.
To switch from one virtual console to another, press Alt and F1-F6. To return to
the graphical desktop, press Alt-F7.
Further Reading
●
To learn more about Steve Bourne, father of the Bourne Shell, see this Wikipedia
article:
http://en.wikipedia.org/wiki/Steve_Bourne
●
Here is an article about the concept of shells in computing:
http://en.wikipedia.org/wiki/Shell_(computing)
13
3 – Navigation
3 – Navigation
The first thing we need to learn to do (besides just typing) is how to navigate the file
system on our Linux system. In this chapter we will introduce the following commands:
●
pwd - Print name of current working directory
●
cd - Change directory
●
ls - List directory contents
Understanding The File System Tree
Like Windows, a Unix-like operating system such as Linux organizes its files in what is
called a hierarchical directory structure. This means that they are organized in a tree-like
pattern of directories (sometimes called folders in other systems), which may contain
files and other directories. The first directory in the file system is called the root
directory. The root directory contains files and subdirectories, which contain more files
and subdirectories and so on and so on.
Note that unlike Windows, which has a separate file system tree for each storage device,
Unix-like systems such as Linux always have a single file system tree, regardless of how
many drives or storage devices are attached to the computer. Storage devices are
attached (or more correctly, mounted) at various points on the tree according to the
whims of the system administrator, the person (or persons) responsible for the
maintenance of the system.
The Current Working Directory
Most of us are probably familiar with a graphical file manager which represents the file
system tree as in Figure 1. Notice that the tree is usually shown upended, that is, with the
root at the top and the various branches descending below.
However, the command line has no pictures, so to navigate the file system tree we need
to think of it in a different way.
14
The Current Working Directory
Imagine that the file system is a maze shaped like an upside-down tree and we are able to
Figure 1: File system tree as shown by a
graphical file manager
stand in the middle of it. At any given time, we are inside a single directory and we can
see the files contained in the directory and the pathway to the directory above us (called
the parent directory) and any subdirectories below us. The directory we are standing in is
called the current working directory. To display the current working directory, we use the
pwd (print working directory) command.
[[email protected] ~]$ pwd
/home/me
When we first log in to our system (or start a terminal emulator session) our current
working directory is set to our home directory. Each user account is given its own home
directory and when operating as a regular user, the home directory is the only place the
user is allowed to write files.
Listing The Contents Of A Directory
To list the files and directories in the current working directory, we use the ls command.
[[email protected] ~]$ ls
Desktop Documents Music
Pictures
Public
Templates
Videos
15
3 – Navigation
Actually, we can use the ls command to list the contents of any directory, not just the
current working directory, and there are many other fun things it can do as well. We'll
spend more time with ls in the next chapter.
Changing The Current Working Directory
To change your working directory (where we are standing in our tree-shaped maze) we
use the cd command. To do this, type cd followed by the pathname of the desired
working directory. A pathname is the route we take along the branches of the tree to get to
the directory we want. Pathnames can be specified in one of two different ways; as
absolute pathnames or as relative pathnames. Let's deal with absolute pathnames first.
Absolute Pathnames
An absolute pathname begins with the root directory and follows the tree branch by
branch until the path to the desired directory or file is completed. For example, there is a
directory on your system in which most of your system's programs are installed. The
pathname of the directory is /usr/bin. This means from the root directory (represented
by the leading slash in the pathname) there is a directory called "usr" which contains a
directory called "bin".
[[email protected] ~]$ cd /usr/bin
[[email protected] bin]$ pwd
/usr/bin
[[email protected] bin]$ ls
...Listing of many, many files ...
Now we can see that we have changed the current working directory to /usr/bin and
that it is full of files. Notice how the shell prompt has changed? As a convenience, it is
usually set up to automatically display the name of the working directory.
Relative Pathnames
Where an absolute pathname starts from the root directory and leads to its destination, a
relative pathname starts from the working directory. To do this, it uses a couple of special
symbols to represent relative positions in the file system tree. These special symbols are
"." (dot) and ".." (dot dot).
The "." symbol refers to the working directory and the ".." symbol refers to the working
directory's parent directory. Here is how it works. Let's change the working directory to /
16
Changing The Current Working Directory
usr/bin again:
[[email protected] ~]$ cd /usr/bin
[[email protected] bin]$ pwd
/usr/bin
Okay, now let's say that we wanted to change the working directory to the parent of
/usr/bin which is /usr. We could do that two different ways. Either with an absolute
pathname:
[[email protected] bin]$ cd /usr
[[email protected] usr]$ pwd
/usr
Or, with a relative pathname:
[[email protected] bin]$ cd ..
[[email protected] usr]$ pwd
/usr
Two different methods with identical results. Which one should we use? The one that
requires the least typing!
Likewise, we can change the working directory from /usr to /usr/bin in two
different ways. Either using an absolute pathname:
[[email protected] usr]$ cd /usr/bin
[[email protected] bin]$ pwd
/usr/bin
Or, with a relative pathname:
[[email protected] usr]$ cd ./bin
[[email protected] bin]$ pwd
/usr/bin
Now, there is something important that I must point out here. In almost all cases, you can
17
3 – Navigation
omit the "./". It is implied. Typing:
[[email protected] usr]$ cd bin
does the same thing. In general, if you do not specify a pathname to something, the
working directory will be assumed.
Some Helpful Shortcuts
In table 3-1 we see some useful ways the current working directory can be quickly
changed.
Table 3- 1: cd Shortcuts
Shortcut
cd
Result
Changes the working directory to your home directory.
cd -
Changes the working directory to the previous working directory.
cd ~user_name
Changes the working directory to the home directory of
user_name. For example, cd ~bob will change the directory to
the home directory of user “bob.”
Important Facts About Filenames
1. Filenames that begin with a period character are hidden. This only means that
ls will not list them unless you say ls -a. When your account was created,
several hidden files were placed in your home directory to configure things
for your account. Later on we will take a closer look at some of these files to
see how you can customize your environment. In addition, some applications
place their configuration and settings files in your home directory as hidden
files.
2. Filenames and commands in Linux, like Unix, are case sensitive. The
filenames “File1” and “file1” refer to different files.
3. Linux has no concept of a “file extension” like some other operating systems.
You may name files any way you like. The contents and/or purpose of a file is
18
Changing The Current Working Directory
determined by other means. Although Unix-like operating system don’t use
file extensions to determine the contents/purpose of files, some application
programs do.
4. Though Linux supports long filenames which may contain embedded spaces
and punctuation characters, limit the punctuation characters in the names of
files you create to period, dash, and underscore. Most importantly, do not
embed spaces in filenames. If you want to represent spaces between words in
a filename, use underscore characters. You will thank yourself later.
19
4 – Exploring The System
4 – Exploring The System
Now that we know how to move around the file system, it's time for a guided tour of our
Linux system. Before we start however, we’re going to learn some more commands that
will be useful along the way:
●
ls – List directory contents
●
file – Determine file type
●
less – View file contents
More Fun With ls
The ls command is probably the most used command, and for good reason. With it, we
can see directory contents and determine a variety of important file and directory
attributes. As we have seen, we can simply type ls to see a list of files and
subdirectories contained in the current working directory:
[[email protected] ~]$ ls
Desktop Documents Music
Pictures
Public
Templates
Videos
Besides the current working directory, we can specify the directory to list, like so:
[email protected] ~]$ ls /usr
bin games
kerberos libexec
etc include lib
local
sbin
share
src
tmp
Or even specify multiple directories. In this example we will list both the user's home
directory (symbolized by the “~” character) and the /usr directory:
[[email protected] ~]$ ls ~ /usr
/home/me:
20
More Fun With ls
Desktop
Documents
/usr:
bin games
etc include
Music
kerberos
lib
Pictures
libexec
local
Public
sbin
share
Templates
Videos
src
tmp
We can also change the format of the output to reveal more detail:
[[email protected]
total 56
drwxrwxr-x 2
drwxrwxr-x 2
drwxrwxr-x 2
drwxrwxr-x 2
drwxrwxr-x 2
drwxrwxr-x 2
drwxrwxr-x 2
~]$ ls -l
me
me
me
me
me
me
me
me
me
me
me
me
me
me
4096
4096
4096
4096
4096
4096
4096
2007-10-26
2007-10-26
2007-10-26
2007-10-26
2007-10-26
2007-10-26
2007-10-26
17:20
17:20
17:20
17:20
17:20
17:20
17:20
Desktop
Documents
Music
Pictures
Public
Templates
Videos
By adding “-l” to the command, we changed the output to the long format.
Options And Arguments
This brings us to a very important point about how most commands work. Commands
are often followed by one or more options that modify their behavior, and further, by one
or more arguments, the items upon which the command acts. So most commands look
kind of like this:
command -options arguments
Most commands use options consisting of a single character preceded by a dash, for
example, “-l”, but many commands, including those from the GNU Project, also support
long options, consisting of a word preceded by two dashes. Also, many commands allow
multiple short options to be strung together. In this example, the ls command is given
two options, the “l” option to produce long format output, and the “t” option to sort the
result by the file's modification time.
[[email protected] ~]$ ls -lt
21
4 – Exploring The System
We'll add the long option “--reverse” to reverse the order of the sort:
[[email protected] ~]$ ls -lt --reverse
The ls command has a large number of possible options. The most common are listed in
the Table 4-1.
Table 4- 1: Common ls Options
Option
-a
Long Option
--all
Description
-d
--directory
Ordinarily, if a directory is specified, ls will
list the contents of the directory, not the
directory itself. Use this option in
conjunction with the -l option to see details
about the directory rather than its contents.
-F
--classify
This option will append an indicator character
to the end of each listed name. For example,
a “/” if the name is a directory.
-h
--human-readable
In long format listings, display file sizes in
human readable format rather than in bytes.
-l
-r
List all files, even those with names that begin
with a period, which are normally not listed
(i.e., hidden).
Display results in long format.
--reverse
Display the results in reverse order.
Normally, ls displays its results in ascending
alphabetical order.
-S
Sort results by file size.
-t
Sort by modification time.
A Longer Look At Long Format
As we saw before, the “-l” option causes ls to display its results in long format. This
format contains a great deal of useful information. Here is the Examples directory
from an Ubuntu system:
22
More Fun With ls
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--
1
1
1
1
1
1
1
1
1
1
1
root
root
root
root
root
root
root
root
root
root
root
root 3576296 2007-04-03 11:05 Experience ubuntu.ogg
root 1186219 2007-04-03 11:05 kubuntu-leaflet.png
root
47584 2007-04-03 11:05 logo-Edubuntu.png
root
44355 2007-04-03 11:05 logo-Kubuntu.png
root
34391 2007-04-03 11:05 logo-Ubuntu.png
root
32059 2007-04-03 11:05 oo-cd-cover.odf
root 159744 2007-04-03 11:05 oo-derivatives.doc
root
27837 2007-04-03 11:05 oo-maxwell.odt
root
98816 2007-04-03 11:05 oo-trig.xls
root 453764 2007-04-03 11:05 oo-welcome.odt
root 358374 2007-04-03 11:05 ubuntu Sax.ogg
Let's look at the different fields from one of the files and examine their meanings:
Table 4-2: ls Long Listing Fields
Field
-rw-r--r--
Meaning
1
File's number of hard links. See the discussion of links
later in this chapter.
root
The user name of the file's owner.
root
The name of the group which owns the file.
32059
Size of the file in bytes.
2007-04-03 11:05
Date and time of the file's last modification.
oo-cd-cover.odf
Name of the file.
Access rights to the file. The first character indicates the
type of file. Among the different types, a leading dash
means a regular file, while a “d” indicates a directory.
The next three characters are the access rights for the
file's owner, the next three are for members of the file's
group, and the final three are for everyone else. The full
meaning of this is discussed in Chapter 10 – Permissions.
Determining A File's Type With file
As we explore the system it will be useful to know what files contain. To do this we will
use the file command to determine a file's type. As we discussed earlier, filenames in
Linux are not required to reflect a file's contents. While a filename like “picture.jpg”
would normally be expected to contain a JPEG compressed image, it is not required to in
Linux. We can invoke the file command this way:
23
4 – Exploring The System
file filename
When invoked, the file command will print a brief description of the file's contents.
For example:
[[email protected] ~]$ file picture.jpg
picture.jpg: JPEG image data, JFIF standard 1.01
There are many kinds of files. In fact, one of the common ideas in Unix-like operating
systems such as Linux is that “everything is a file.” As we proceed with our lessons, we
will see just how true that statement is.
While many of the files on your system are familiar, for example MP3 and JPEG, there
are many kinds that are a little less obvious and a few that are quite strange.
Viewing File Contents With less
The less command is a program to view text files. Throughout our Linux system, there
are many files that contain human-readable text. The less program provides a
convenient way to examine them.
What Is “Text”?
There are many ways to represent information on a computer. All methods
involve defining a relationship between the information and some numbers that
will be used to represent it. Computers, after all, only understand numbers and all
data is converted to numeric representation.
Some of these representation systems are very complex (such as compressed
video files), while others are rather simple. One of the earliest and simplest is
called ASCII text. ASCII (pronounced "As-Key") is short for American Standard
Code for Information Interchange. This is a simple encoding scheme that was first
used on Teletype machines to map keyboard characters to numbers.
Text is a simple one-to-one mapping of characters to numbers. It is very compact.
Fifty characters of text translates to fifty bytes of data. It is important to
understand that text only contains a simple mapping of characters to numbers. It
is not the same as a word processor document such as one created by Microsoft
Word or OpenOffice.org Writer. Those files, in contrast to simple ASCII text,
24
Viewing File Contents With less
contain many non-text elements that are used to describe its structure and
formatting. Plain ASCII text files contain only the characters themselves and a
few rudimentary control codes like tabs, carriage returns and line feeds.
Throughout a Linux system, many files are stored in text format and there are
many Linux tools that work with text files. Even Windows recognizes the
importance of this format. The well-known NOTEPAD.EXE program is an editor
for plain ASCII text files.
Why would we want to examine text files? Because many of the files that contain system
settings (called configuration files) are stored in this format, and being able to read them
gives us insight about how the system works. In addition, many of the actual programs
that the system uses (called scripts) are stored in this format. In later chapters, we will
learn how to edit text files in order to modify systems settings and write our own scripts,
but for now we will just look at their contents.
The less command is used like this:
less filename
Once started, the less program allows you to scroll forward and backward through a
text file. For example, to examine the file that defines all the system's user accounts,
enter the following command:
[[email protected] ~]$ less /etc/passwd
Once the less program starts, we may view the contents of the file. If the file is longer
than one page, we can scroll up and down. To exit less, press the “q” key.
The table below lists the most common keyboard commands used by less.
Table 4-3: less Commands
Command
Action
Page Up or b
Scroll back one page
Page Down or space
Scroll forward one page
Up Arrow
Scroll up one line
25
4 – Exploring The System
Down Arrow
Scroll down one line
G
Move to the end of the text file
1G or g
Move to the beginning of the text file
/characters
Search forward to the next occurrence of characters
n
Search for the next occurrence of the previous search
h
Display help screen
q
Quit less
Less Is More
The less program was designed as an improved replacement of an earlier Unix
program called more. The name “less” is a play on the phrase “less is more”—a
motto of modernist architects and designers.
less falls into the class of programs called “pagers,” programs that allow the
easy viewing of long text documents in a page by page manner. Whereas the
more program could only page forward, the less program allows paging both
forward and backward and has many other features as well.
A Guided Tour
The file system layout on your Linux system is much like that found on other Unix-like
systems. The design is actually specified in a published standard called the Linux
Filesystem Hierarchy Standard. Not all Linux distributions conform to the standard
exactly but most come pretty close.
Next, we are going to wander around the file system ourselves to see what makes our
Linux system tick. This will give you a chance to practice your navigation skills. One of
the things we will discover is that many of the interesting files are in plain humanreadable text. As we go about our tour, try the following:
1. cd into a given directory
2. List the directory contents with ls -l
3. If you see an interesting file, determine its contents with file
4. If it looks like it might be text, try viewing it with less
26
A Guided Tour
Remember the copy and paste trick! If you are using a mouse, you can double
click on a filename to copy it and middle click to paste it into commands.
As we wander around, don't be afraid to look at stuff. Regular users are largely
prohibited from messing things up. That's the system administrators job! If a command
complains about something, just move on to something else. Spend some time looking
around. The system is ours to explore. Remember, in Linux, there are no secrets!
Table 4-4 lists just a few of the directories we can explore. Feel free to try more!
Table 4-4: Directories Found On Linux Systems
Directory
Comments
/
The root directory. Where everything begins.
/bin
Contains binaries (programs) that must be present for the
system to boot and run.
/boot
Contains the Linux kernel, initial RAM disk image (for
drivers needed at boot time), and the boot loader.
Interesting files:
● /boot/grub/grub.conf or menu.lst, which
are used to configure the boot loader.
● /boot/vmlinuz, the linux kernel
/dev
This is a special directory which contains device nodes.
“Everything is a file” also applies to devices. Here is where
the kernel maintains a list of all the devices it understands.
/etc
The /etc directory contains all of the system-wide
configuration files. It also contains a collection of shell
scripts which start each of the system services at boot time.
Everything in this directory should be readable text.
Interesting files: While everything in /etc is interesting,
here are some of my all-time favorites:
● /etc/crontab, a file that defines when
automated jobs will run.
● /etc/fstab, a table of storage devices and their
associated mount points.
● /etc/passwd, a list of the user accounts.
27
4 – Exploring The System
Directory
Comments
/home
In normal configurations, each user is given a directory in
/home. Ordinary users can only write files in their home
directories. This limitation protects the system from errant
user activity.
/lib
Contains shared library files used by the core system
programs. These are similar to DLLs in Windows.
/lost+found
Each formatted partition or device using a Linux file system,
such as ext3, will have this directory. It is used in the case
of a partial recovery from a file system corruption event.
Unless something really bad has happened to your system,
this directory will remain empty.
/media
On modern Linux systems the /media directory will
contain the mount points for removable media such USB
drives, CD-ROMs, etc. that are mounted automatically at
insertion.
/mnt
On older Linux systems, the /mnt directory contains mount
points for removable devices that have been mounted
manually.
/opt
The /opt directory is used to install “optional” software.
This is mainly used to hold commercial software products
that may be installed on your system.
/proc
The /proc directory is special. It's not a real file system in
the sense of files stored on your hard drive. Rather, it is a
virtual file system maintained by the Linux kernel. The
“files” it contains are peepholes into the kernel itself. The
files are readable and will give you a picture of how the
kernel sees your computer.
/root
This is the home directory for the root account.
/sbin
This directory contains “system” binaries. These are
programs that perform vital system tasks that are generally
reserved for the superuser.
/tmp
The /tmp directory is intended for storage of temporary,
transient files created by various programs. Some
configurations cause this directory to be emptied each time
the system is rebooted.
28
A Guided Tour
Directory
Comments
/usr
The /usr directory tree is likely the largest one on a Linux
system. It contains all the programs and support files used
by regular users.
/usr/bin
/usr/bin contains the executable programs installed by
your Linux distribution. It is not uncommon for this
directory to hold thousands of programs.
/usr/lib
The shared libraries for the programs in /usr/bin.
/usr/local
The /usr/local tree is where programs that are not
included with your distribution but are intended for systemwide use are installed. Programs compiled from source code
are normally installed in /usr/local/bin. On a newly
installed Linux system, this tree exists, but it will be empty
until the system administrator puts something in it.
/usr/sbin
Contains more system administration programs.
/usr/share
/usr/share contains all the shared data used by
programs in /usr/bin. This includes things like default
configuration files, icons, screen backgrounds, sound files,
etc.
/usr/share/doc
Most packages installed on the system will include some
kind of documentation. In /usr/share/doc, we will
find documentation files organized by package.
/var
With the exception of /tmp and /home, the directories we
have looked at so far remain relatively static, that is, their
contents don't change. The /var directory tree is where
data that is likely to change is stored. Various databases,
spool files, user mail, etc. are located here.
/var/log
/var/log contains log files, records of various system
activity. These are very important and should be monitored
from time to time. The most useful one is
/var/log/messages. Note that for security reasons on
some systems, you must be the superuser to view log files .
Symbolic Links
As we look around, we are likely to see a directory listing with an entry like this:
29
4 – Exploring The System
lrwxrwxrwx 1 root root
11 2007-08-11 07:34 libc.so.6 -> libc-2.6.so
Notice how the first letter of the listing is “l” and the entry seems to have two filenames?
This is a special kind of a file called a symbolic link (also known as a soft link or
symlink.) In most Unix-like systems it is possible to have a file referenced by multiple
names. While the value of this may not be obvious, it is really a useful feature.
Picture this scenario: a program requires the use of a shared resource of some kind
contained in a file named “foo,” but “foo” has frequent version changes. It would be
good to include the version number in the filename so the administrator or other
interested party could see what version of “foo” is installed. This presents a problem. If
we change the name of the shared resource, we have to track down every program that
might use it and change it to look for a new resource name every time a new version of
the resource is installed. That doesn't sound like fun at all.
Here is where symbolic links save the day. Let's say we install version 2.6 of “foo,”
which has the filename “foo-2.6” and then create a symbolic link simply called “foo” that
points to “foo-2.6.” This means that when a program opens the file “foo”, it is actually
opening the file “foo-2.6”. Now everybody is happy. The programs that rely on “foo”
can find it and we can still see what actual version is installed. When it is time to
upgrade to “foo-2.7,” we just add the file to our system, delete the symbolic link “foo”
and create a new one that points to the new version. Not only does this solve the problem
of the version upgrade, but it also allows us to keep both versions on our machine.
Imagine that “foo-2.7” has a bug (damn those developers!) and we need to revert to the
old version. Again, we just delete the symbolic link pointing to the new version and
create a new symbolic link pointing to the old version.
The directory listing above (from the /lib directory of a Fedora system) shows a
symbolic link called “libc.so.6” that points to a shared library file called “libc-2.6.so.”
This means that programs looking for “libc.so.6” will actually get the file “libc-2.6.so.”
We will learn how to create symbolic links in the next chapter.
Hard Links
While we are on the subject of links, we need to mention that there is a second type of
link called a hard link. Hard links also allow files to have multiple names, but they do it
in a different way. We’ll talk more about the differences between symbolic and hard
links in the next chapter.
Further Reading
●
30
The full version of the Linux Filesystem Hierarchy Standard can be found here:
http://www.pathname.com/fhs/
5 – Manipulating Files And Directories
5 – Manipulating Files And Directories
At this point, we are ready for some real work! This chapter will introduce the following
commands:
●
cp – Copy files and directories
●
mv – Move/rename files and directories
●
mkdir – Create directories
●
rm – Remove files and directories
●
ln – Create hard and symbolic links
These five commands are among the most frequently used Linux commands. They are
used for manipulating both files and directories.
Now, to be frank, some of the tasks performed by these commands are more easily done
with a graphical file manager. With a file manager, we can drag and drop a file from one
directory to another, cut and paste files, delete files, etc. So why use these old command
line programs?
The answer is power and flexibility. While it is easy to perform simple file manipulations
with a graphical file manager, complicated tasks can be easier with the command line
programs. For example, how could we copy all the HTML files from one directory to
another, but only copy files that do not exist in the destination directory or are newer than
the versions in the destination directory? Pretty hard with with a file manager. Pretty easy
with the command line:
cp -u *.html destination
Wildcards
Before we begin using our commands, we need to talk about a shell feature that makes
these commands so powerful. Since the shell uses filenames so much, it provides special
characters to help you rapidly specify groups of filenames. These special characters are
31
5 – Manipulating Files And Directories
called wildcards. Using wildcards (which is also known as globbing) allow you to select
filenames based on patterns of characters. The table below lists the wildcards and what
they select:
Table 5-1: Wildcards
Wildcard
Meaning
*
Matches any characters
?
Matches any single character
[characters]
Matches any character that is a member of the set characters
[!characters]
Matches any character that is not a member of the set
characters
[[:class:]]
Matches any character that is a member of the specified
class
Table 5-2 lists the most commonly used character classes:
Table 5-2: Commonly Used Character Classes
Character Class
Meaning
[:alnum:]
Matches any alphanumeric character
[:alpha:]
Matches any alphabetic character
[:digit:]
Matches any numeral
[:lower:]
Matches any lowercase letter
[:upper:]
Matches any uppercase letter
Using wildcards makes it possible to construct very sophisticated selection criteria for
filenames. Here are some examples of patterns and what they match:
Table 5-3: Wildcard Examples
Pattern
Matches
*
All files
g*
Any file beginning with “g”
b*.txt
Any file beginning with “b” followed by
any characters and ending with “.txt”
32
Wildcards
Data???
Any file beginning with “Data” followed
by exactly three characters
[abc]*
Any file beginning with either an “a”, a
“b”, or a “c”
BACKUP.[0-9][0-9][0-9]
Any file beginning with “BACKUP.”
followed by exactly three numerals
[[:upper:]]*
Any file beginning with an uppercase letter
[![:digit:]]*
Any file not beginning with a numeral
*[[:lower:]123]
Any file ending with a lowercase letter or
the numerals “1”, “2”, or “3”
Wildcards can be used with any command that accepts filenames as arguments, but we’ll
talk more about that in Chapter 8.
Character Ranges
If you are coming from another Unix-like environment or have been reading
some other books on this subject, you may have encountered the [A-Z] or the
[a-z] character range notations. These are traditional Unix notations and
worked in older versions of Linux as well. They can still work, but you have to
be very careful with them because they will not produce the expected results
unless properly configured. For now, you should avoid using them and use
character classes instead.
Wildcards Work In The GUI Too
Wildcards are especially valuable not only because they are used so frequently on
the command line, but are also supported by some graphical file managers.
●
In Nautilus (the file manager for GNOME), you can select files using the
Edit/Select Pattern menu item. Just enter a file selection pattern with
wildcards and the files in the currently viewed directory will be highlighted
for selection.
●
In Dolphin and Konqueror (the file managers for KDE), you can enter
wildcards directly on the location bar. For example, if you want to see all the
files starting with a lowercase “u” in the /usr/bin directory, type “/usr/bin/u*”
into the location bar and it will display the result.
33
5 – Manipulating Files And Directories
Many ideas originally found in the command line interface make their way into
the graphical interface, too. It is one of the many things that make the Linux
desktop so powerful.
mkdir – Create Directories
The mkdir command is used to create directories. It works like this:
mkdir directory...
A note on notation: When three periods follow an argument in the description of a
command (as above), it means that the argument can be repeated, thus:
mkdir dir1
would create a single directory named “dir1”, while
mkdir dir1 dir2 dir3
would create three directories named “dir1”, “dir2”, and “dir3”.
cp – Copy Files And Directories
The cp command copies files or directories. It can be used two different ways:
cp item1 item2
to copy the single file or directory “item1” to file or directory “item2” and:
cp item... directory
to copy multiple items (either files or directories) into a directory.
34
cp – Copy Files And Directories
Useful Options And Examples
Here are some of the commonly used options (the short option and the equivalent long
option) for cp:
Table 5-4: cp Options
Option
Meaning
-a, --archive
Copy the files and directories and all of their attributes,
including ownerships and permissions. Normally,
copies take on the default attributes of the user
performing the copy.
-i, --interactive
Before overwriting an existing file, prompt the user for
confirmation. If this option is not specified, cp will
silently overwrite files.
-r, --recursive
Recursively copy directories and their contents. This
option (or the -a option) is required when copying
directories.
-u, --update
When copying files from one directory to another, only
copy files that either don't exist, or are newer than the
existing corresponding files, in the destination
directory.
-v, --verbose
Display informative messages as the copy is
performed.
Table 5-5: cp Examples
Command
Results
cp file1 file2
Copy file1 to file2. If file2 exists, it is overwritten
with the contents of file1. If file2 does not exist, it
is created.
cp -i file1 file2
Same as above, except that if file2 exists, the user is
prompted before it is overwritten.
cp file1 file2 dir1
Copy file1 and file2 into directory dir1. dir1 must
already exist.
cp dir1/* dir2
Using a wildcard, all the files in dir1 are copied
into dir2. dir2 must already exist.
35
5 – Manipulating Files And Directories
cp -r dir1 dir2
Copy the contents of directory dir1 to directory
dir2. If directory dir2 does not exist, it is created
and, after the copy, will contain the same contents
as directory dir1.
If directory dir2 does exist, then directory dir1 (and
its contents) will be copied into dir2.
mv – Move And Rename Files
The mv command performs both file moving and file renaming, depending on how it is
used. In either case, the original filename no longer exists after the operation. mv is used
in much the same way as cp:
mv item1 item2
to move or rename file or directory “item1” to “item2” or:
mv item... directory
to move one or more items from one directory to another.
Useful Options And Examples
mv shares many of the same options as cp:
Table 5-6: mv Options
Option
-i, --interactive
Meaning
-u, --update
When moving files from one directory to another, only
move files that either don't exist, or are newer than the
existing corresponding files in the destination
directory.
-v, --verbose
Display informative messages as the move is
36
Before overwriting an existing file, prompt the user for
confirmation. If this option is not specified, mv will
silently overwrite files.
mv – Move And Rename Files
performed.
Table 5-7: mv Examples
Command
Results
mv file1 file2
Move file1 to file2. If file2 exists, it is overwritten
with the contents of file1. If file2 does not exist, it
is created. In either case, file1 ceases to exist.
mv -i file1 file2
Same as above, except that if file2 exists, the user is
prompted before it is overwritten.
mv file1 file2 dir1
Move file1 and file2 into directory dir1. dir1 must
already exist.
mv dir1 dir2
If directory dir2 does not exist, create directory
dir2 and move the contents of directory dir1 into
dir2 and delete directory dir1.
If directory dir2 does exist, move directory dir1
(and its contents) into directory dir2.
rm – Remove Files And Directories
The rm command is used to remove (delete) files and directories:
rm item...
where “item” is one or more files or directories.
Useful Options And Examples
Here are some of the common options for rm:
Table 5-8: rm Options
Option
Meaning
-i, --interactive
Before deleting an existing file, prompt the user for
confirmation. If this option is not specified, rm will
silently delete files.
37
5 – Manipulating Files And Directories
-r, --recursive
Recursively delete directories. This means that if a
directory being deleted has subdirectories, delete them
too. To delete a directory, this option must be
specified.
-f, --force
Ignore nonexistent files and do not prompt. This
overrides the --interactive option.
-v, --verbose
Display informative messages as the deletion is
performed.
Table 5-9: rm Examples
Command
Results
rm file1
Delete file1 silently.
rm -i file1
Same as above, except that the user is prompted for
confirmation before the deletion is performed.
rm -r file1 dir1
Delete file1 and dir1 and its contents.
rm -rf file1 dir1
Same as above, except that if either file1 or dir1 do
not exist, rm will continue silently.
Be Careful With rm!
Unix-like operating systems such as Linux do not have an undelete command.
Once you delete something with rm, it's gone. Linux assumes you're smart and
you know what you're doing.
Be particularly careful with wildcards. Consider this classic example. Let's say
you want to delete just the HTML files in a directory. To do this, you type:
rm *.html
which is correct, but if you accidentally place a space between the “*” and the
“.html” like so:
rm * .html
the rm command will delete all the files in the directory and then complain that
there is no file called “.html”.
38
rm – Remove Files And Directories
Here is a useful tip. Whenever you use wildcards with rm (besides carefully
checking your typing!), test the wildcard first with ls. This will let you see the
files that will be deleted. Then press the up arrow key to recall the command and
replace the ls with rm.
ln – Create Links
The ln command is used to create either hard or symbolic links. It is used in one of two
ways:
ln file link
to create a hard link, and:
ln -s item link
to create a symbolic link where “item” is either a file or a directory.
Hard Links
Hard links are the original Unix way of creating links, compared to symbolic links, which
are more modern. By default, every file has a single hard link that gives the file its name.
When we create a hard link, we create an additional directory entry for a file. Hard links
have two important limitations:
1. A hard link cannot reference a file outside its own file system. This means a link
may not reference a file that is not on the same disk partition as the link itself.
2. A hard link may not reference a directory.
A hard link is indistinguishable from the file itself. Unlike a symbolic link, when you list
a directory containing a hard link you will see no special indication of the link. When a
hard link is deleted, the link is removed but the contents of the file itself continue to exist
(that is, its space is not deallocated) until all links to the file are deleted.
It is important to be aware of hard links because you might encounter them from time to
time, but modern practice prefers symbolic links, which we will cover next.
39
5 – Manipulating Files And Directories
Symbolic Links
Symbolic links were created to overcome the limitations of hard links. Symbolic links
work by creating a special type of file that contains a text pointer to the referenced file or
directory. In this regard, they operate in much the same way as a Windows shortcut
though of course, they predate the Windows feature by many years ;-)
A file pointed to by a symbolic link, and the symbolic link itself are largely
indistinguishable from one another. For example, if you write some something to the
symbolic link, the referenced file is also written to. However when you delete a symbolic
link, only the link is deleted, not the file itself. If the file is deleted before the symbolic
link, the link will continue to exist, but will point to nothing. In this case, the link is said
to be broken. In many implementations, the ls command will display broken links in a
distinguishing color, such as red, to reveal their presence.
The concept of links can seem very confusing, but hang in there. We're going to try all
this stuff and it will, hopefully, become clear.
Let's Build A Playground
Since we are going to do some real file manipulation, let's build a safe place to “play”
with our file manipulation commands. First we need a directory to work in. We'll create
one in our home directory and call it “playground.”
Creating Directories
The mkdir command is used to create a directory. To create our playground directory
we will first make sure we are in our home directory and will then create the new
directory:
[[email protected] ~]$ cd
[[email protected] ~]$ mkdir playground
To make our playground a little more interesting, let's create a couple of directories inside
it called “dir1” and “dir2”. To do this, we will change our current working directory to
playground and execute another mkdir:
[[email protected] ~]$ cd playground
[[email protected] playground]$ mkdir dir1 dir2
Notice that the mkdir command will accept multiple arguments allowing us to create
40
Let's Build A Playground
both directories with a single command.
Copying Files
Next, let's get some data into our playground. We'll do this by copying a file. Using the
cp command, we'll copy the passwd file from the /etc directory to the current
working directory:
[[email protected] playground]$ cp /etc/passwd .
Notice how we used the shorthand for the current working directory, the single trailing
period. So now if we perform an ls, we will see our file:
[[email protected]
total 12
drwxrwxr-x 2
drwxrwxr-x 2
-rw-r--r-- 1
playground]$ ls -l
me
me
me
me 4096 2008-01-10 16:40 dir1
me 4096 2008-01-10 16:40 dir2
me 1650 2008-01-10 16:07 passwd
Now, just for fun, let's repeat the copy using the “-v” option (verbose) to see what it does:
[[email protected] playground]$ cp -v /etc/passwd .
`/etc/passwd' -> `./passwd'
The cp command performed the copy again, but this time displayed a concise message
indicating what operation it was performing. Notice that cp overwrote the first copy
without any warning. Again this is a case of cp assuming that you know what you’re are
doing. To get a warning, we'll include the “-i” (interactive) option:
[[email protected] playground]$ cp -i /etc/passwd .
cp: overwrite `./passwd'?
Responding to the prompt by entering a “y” will cause the file to be overwritten, any
other character (for example, “n”) will cause cp to leave the file alone.
41
5 – Manipulating Files And Directories
Moving And Renaming Files
Now, the name “passwd” doesn't seem very playful and this is a playground, so let's
change it to something else:
[[email protected] playground]$ mv passwd fun
Let's pass the fun around a little by moving our renamed file to each of the directories and
back again:
[[email protected] playground]$ mv fun dir1
to move it first to directory dir1, then:
[[email protected] playground]$ mv dir1/fun dir2
to move it from dir1 to dir2, then:
[[email protected] playground]$ mv dir2/fun .
to finally bringing it back to the current working directory. Next, let's see the effect of mv
on directories. First we will move our data file into dir1 again:
[[email protected] playground]$ mv fun dir1
then move dir1 into dir2 and confirm it with ls:
[[email protected]
[[email protected]
total 4
drwxrwxr-x 2
[[email protected]
total 4
-rw-r--r-- 1
42
playground]$ mv dir1 dir2
playground]$ ls -l dir2
me me
4096 2008-01-11 06:06 dir1
playground]$ ls -l dir2/dir1
me
me
1650 2008-01-10 16:33 fun
Let's Build A Playground
Note that since dir2 already existed, mv moved dir1 into dir2. If dir2 had not
existed, mv would have renamed dir1 to dir2. Lastly, let's put everything back:
[[email protected] playground]$ mv dir2/dir1 .
[[email protected] playground]$ mv dir1/fun .
Creating Hard Links
Now we'll try some links. First the hard links. We’ll create some links to our data file
like so:
[[email protected] playground]$ ln fun fun-hard
[[email protected] playground]$ ln fun dir1/fun-hard
[[email protected] playground]$ ln fun dir2/fun-hard
So now we have four instances of the file “fun”. Let's take a look our playground
directory:
[[email protected]
total 16
drwxrwxr-x 2
drwxrwxr-x 2
-rw-r--r-- 4
-rw-r--r-- 4
playground]$ ls -l
me
me
me
me
me
me
me
me
4096
4096
1650
1650
2008-01-14
2008-01-14
2008-01-10
2008-01-10
16:17
16:17
16:33
16:33
dir1
dir2
fun
fun-hard
One thing you notice is that the second field in the listing for fun and fun-hard both
contain a “4” which is the number of hard links that now exist for the file. You'll
remember that a file will aways have at least one because the file's name is created by a
link. So, how do we know that fun and fun-hard are, in fact, the same file? In this
case, ls is not very helpful. While we can see that fun and fun-hard are both the
same size (field 5), our listing provides no way to be sure. To solve this problem, we're
going to have to dig a little deeper.
When thinking about hard links, it is helpful to imagine that files are made up of two
parts: the data part containing the file's contents and the name part which holds the file's
name. When we create hard links, we are actually creating additional name parts that all
refer to the same data part. The system assigns a chain of disk blocks to what is called an
inode, which is then associated with the name part. Each hard link therefore refers to a
specific inode containing the file's contents.
43
5 – Manipulating Files And Directories
The ls command has a way to reveal this information. It is invoked with the “-i” option:
[[email protected] playground]$ ls -li
total 16
12353539 drwxrwxr-x 2 me
me
4096
12353540 drwxrwxr-x 2 me
me
4096
12353538 -rw-r--r-- 4 me
me
1650
12353538 -rw-r--r-- 4 me
me
1650
2008-01-14
2008-01-14
2008-01-10
2008-01-10
16:17
16:17
16:33
16:33
dir1
dir2
fun
fun-hard
In this version of the listing, the first field is the inode number and, as we can see, both
fun and fun-hard share the same inode number, which confirms they are the same
file.
Creating Symbolic Links
Symbolic links were created to overcome the two disadvantages of hard links: hard links
cannot span physical devices and hard links cannot reference directories, only files.
Symbolic links are a special type of file that contains a text pointer to the target file or
directory.
Creating symbolic links is similar to creating hard links:
[[email protected] playground]$ ln -s fun fun-sym
[[email protected] playground]$ ln -s ../fun dir1/fun-sym
[[email protected] playground]$ ln -s ../fun dir2/fun-sym
The first example is pretty straightforward, we simply add the “-s” option to create a
symbolic link rather than a hard link. But what about the next two? Remember, when we
create a symbolic link, we are creating a text description of where the target file is
relative to the symbolic link. It's easier to see if we look at the ls output:
[[email protected] playground]$ ls -l dir1
total 4
-rw-r--r-- 4 me
me
1650 2008-01-10 16:33 fun-hard
lrwxrwxrwx 1 me
me
6 2008-01-15 15:17 fun-sym -> ../fun
The listing for fun-sym in dir1 shows that is it a symbolic link by the leading “l” in
the first field and that it points to “../fun”, which is correct. Relative to the location of
fun-sym, fun is in the directory above it. Notice too, that the length of the symbolic
link file is 6, the number of characters in the string “../fun” rather than the length of the
44
Let's Build A Playground
file to which it is pointing.
When creating symbolic links, you can either use absolute pathnames:
ln -s /home/me/playground/fun dir1/fun-sym
or relative pathnames, as we did in our earlier example. Using relative pathnames is
more desirable because it allows a directory containing symbolic links to be renamed
and/or moved without breaking the links.
In addition to regular files, symbolic links can also reference directories:
[[email protected]
[[email protected]
total 16
drwxrwxr-x 2
lrwxrwxrwx 1
drwxrwxr-x 2
-rw-r--r-- 4
-rw-r--r-- 4
lrwxrwxrwx 1
playground]$ ln -s dir1 dir1-sym
playground]$ ls -l
me
me
me
me
me
me
me
me
me
me
me
me
4096
4
4096
1650
1650
3
2008-01-15
2008-01-16
2008-01-15
2008-01-10
2008-01-10
2008-01-15
15:17
14:45
15:17
16:33
16:33
15:15
dir1
dir1-sym -> dir1
dir2
fun
fun-hard
fun-sym -> fun
Removing Files And Directories
As we covered earlier, the rm command is used to delete files and directories. We are
going to use it to clean up our playground a little bit. First, let's delete one of our hard
links:
[[email protected]
[[email protected]
total 12
drwxrwxr-x 2
lrwxrwxrwx 1
drwxrwxr-x 2
-rw-r--r-- 3
lrwxrwxrwx 1
playground]$ rm fun-hard
playground]$ ls -l
me
me
me
me
me
me
me
me
me
me
4096
4
4096
1650
3
2008-01-15
2008-01-16
2008-01-15
2008-01-10
2008-01-15
15:17
14:45
15:17
16:33
15:15
dir1
dir1-sym -> dir1
dir2
fun
fun-sym -> fun
That worked as expected. The file fun-hard is gone and the link count shown for fun
is reduced from four to three, as indicated in the second field of the directory listing.
Next, we'll delete the file fun, and just for enjoyment, we'll include the “-i” option to
show what that does:
45
5 – Manipulating Files And Directories
[[email protected] playground]$ rm -i fun
rm: remove regular file `fun'?
Enter “y” at the prompt and the file is deleted. But let's look at the output of ls now.
Noticed what happened to fun-sym? Since it's a symbolic link pointing to a nownonexistent file, the link is broken:
[[email protected]
total 8
drwxrwxr-x 2
lrwxrwxrwx 1
drwxrwxr-x 2
lrwxrwxrwx 1
playground]$ ls -l
me
me
me
me
me
me
me
me
4096
4
4096
3
2008-01-15
2008-01-16
2008-01-15
2008-01-15
15:17
14:45
15:17
15:15
dir1
dir1-sym -> dir1
dir2
fun-sym -> fun
Most Linux distributions configure ls to display broken links. On a Fedora box, broken
links are displayed in blinking red text! The presence of a broken link is not, in and of
itself dangerous but it is rather messy. If we try to use a broken link we will see this:
[[email protected] playground]$ less fun-sym
fun-sym: No such file or directory
Let's clean up a little. We'll delete the symbolic links:
[[email protected]
[[email protected]
total 8
drwxrwxr-x 2
drwxrwxr-x 2
playground]$ rm fun-sym dir1-sym
playground]$ ls -l
me
me
me
me
4096 2008-01-15 15:17 dir1
4096 2008-01-15 15:17 dir2
One thing to remember about symbolic links is that most file operations are carried out
on the link's target, not the link itself. rm is an exception. When you delete a link, it is
the link that is deleted, not the target.
Finally, we will remove our playground. To do this, we will return to our home directory
and use rm with the recursive option (-r) to delete playground and all of its contents,
including its subdirectories:
[[email protected] playground]$ cd
46
Let's Build A Playground
[[email protected] ~]$ rm -r playground
Creating Symlinks With The GUI
The file managers in both GNOME and KDE provide an easy and automatic
method of creating symbolic links. With GNOME, holding the Ctrl+Shift keys
while dragging a file will create a link rather than copying (or moving) the file.
In KDE, a small menu appears whenever a file is dropped, offering a choice of
copying, moving, or linking the file.
Summing Up
We've covered a lot of ground here and it will take a while to fully sink in. Perform the
playground exercise over and over until it makes sense. It is important to get a good
understanding of basic file manipulation commands and wildcards. Feel free to expand
on the playground exercise by adding more files and directories, using wildcards to
specify files for various operations. The concept of links is a little confusing at first, but
take the time to learn how they work. They can be a real lifesaver.
47
6 – Working With Commands
6 – Working With Commands
Up to this point, we have seen a series of mysterious commands, each with its own
mysterious options and arguments. In this chapter, we will attempt to remove some of
that mystery and even create some of our own commands. The commands introduced in
this chapter are:
●
type – Indicate how a command name is interpreted
●
which – Display which executable program will be executed
●
man – Display a command's manual page
●
apropos – Display a list of appropriate commands
●
info – Display a command's info entry
●
whatis – Display a very brief description of a command
●
alias – Create an alias for a command
What Exactly Are Commands?
A command can be one of four different things:
1. An executable program like all those files we saw in /usr/bin. Within this
category, programs can be compiled binaries such as programs written in C and
C++, or programs written in scripting languages such as the shell, perl, python,
ruby, etc.
2. A command built into the shell itself. bash supports a number of commands
internally called shell builtins. The cd command, for example, is a shell builtin.
3. A shell function. These are miniature shell scripts incorporated into the
environment. We will cover configuring the environment and writing shell
functions in later chapters, but for now, just be aware that they exist.
4. An alias. Commands that we can define ourselves, built from other commands.
48
Identifying Commands
Identifying Commands
It is often useful to know exactly which of the four kinds of commands is being used and
Linux provides a couple of ways to find out.
type – Display A Command's Type
The type command is a shell builtin that displays the kind of command the shell will
execute, given a particular command name. It works like this:
type command
where “command” is the name of the command you want to examine. Here are some
examples:
[[email protected] ~]$ type type
type is a shell builtin
[[email protected] ~]$ type ls
ls is aliased to `ls --color=tty'
[[email protected] ~]$ type cp
cp is /bin/cp
Here we see the results for three different commands. Notice that the one for ls (taken
from a Fedora system) and how the ls command is actually an alias for the ls command
with the “-- color=tty” option added. Now we know why the output from ls is displayed
in color!
which – Display An Executable's Location
Sometimes there is more than one version of an executable program installed on a
system. While this is not very common on desktop systems, it's not unusual on large
servers. To determine the exact location of a given executable, the which command is
used:
[[email protected] ~]$ which ls
/bin/ls
which only works for executable programs, not builtins nor aliases that are substitutes
for actual executable programs. When we try to use which on a shell builtin, for
49
6 – Working With Commands
example, cd, we either get no response or an error message:
[[email protected] ~]$ which cd
/usr/bin/which: no cd in
(/opt/jre1.6.0_03/bin:/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/opt/jre1
.6.0_03/bin:/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:/home/me/bin
)
which is a fancy way of saying “command not found.”
Getting A Command's Documentation
With this knowledge of what a command is, we can now search for the documentation
available for each kind of command.
help – Get Help For Shell Builtins
bash has a built-in help facility available for each of the shell builtins. To use it, type
“help” followed by the name of the shell builtin. For example:
[[email protected] ~]$ help cd
cd: cd [-L|-P] [dir]
Change the current directory to DIR. The variable $HOME is the
default DIR. The variable CDPATH defines the search path for the
directory containing DIR. Alternative directory names in CDPATH are
separated by a colon (:). A null directory name is the same as the
current directory, i.e. `.'. If DIR begins with a slash (/), then
CDPATH is not used. If the directory is not found, and the shell
option `cdable_vars' is set, then try the word as a variable name.
If that variable has a value, then cd to the value of that variable.
The -P option says to use the physical directory structure instead of
following symbolic links; the -L option forces symbolic links to be
followed.
A note on notation: When square brackets appear in the description of a command's
syntax, they indicate optional items. A vertical bar character indicates mutually exclusive
items. In the case of the cd command above:
cd [-L|-P] [dir]
This notation says that the command cd may be followed optionally by either a “-L” or a
“-P” and further, optionally followed by the argument “dir”.
While the output of help for the cd commands is concise and accurate, it is by no
50
Getting A Command's Documentation
means tutorial and as we can see, it also seems to mention a lot of things we haven't
talked about yet! Don't worry. We'll get there.
--help – Display Usage Information
Many executable programs support a “--help” option that displays a description of the
command's supported syntax and options. For example:
[[email protected] ~]$ mkdir --help
Usage: mkdir [OPTION] DIRECTORY...
Create the DIRECTORY(ies), if they do not already exist.
-Z, --context=CONTEXT (SELinux) set security context to CONTEXT
Mandatory arguments to long options are mandatory for short options
too.
-m, --mode=MODE
set file mode (as in chmod), not a=rwx – umask
-p, --parents
no error if existing, make parent directories as
needed
-v, --verbose
print a message for each created directory
--help
display this help and exit
--version
output version information and exit
Report bugs to <[email protected]>.
Some programs don't support the “--help” option, but try it anyway. Often it results in an
error message that will reveal the same usage information.
man – Display A Program's Manual Page
Most executable programs intended for command line use provide a formal piece of
documentation called a manual or man page. A special paging program called man is
used to view them. It is used like this:
man program
where “program” is the name of the command to view.
Man pages vary somewhat in format but generally contain a title, a synopsis of the
command's syntax, a description of the command's purpose, and a listing and description
of each of the command's options. Man pages, however, do not usually include
examples, and are intended as a reference, not a tutorial. As an example, let's try viewing
51
6 – Working With Commands
the man page for the ls command:
[[email protected] ~]$ man ls
On most Linux systems, man uses less to display the manual page, so all of the familiar
less commands work while displaying the page.
The “manual” that man displays is broken into sections and not only covers user
commands but also system administration commands, programming interfaces, file
formats and more. The table below describes the layout of the manual:
Table 6-1: Man Page Organization
Section
Contents
1
User commands
2
Programming interfaces kernel system calls
3
Programming interfaces to the C library
4
Special files such as device nodes and drivers
5
File formats
6
Games and amusements such as screen savers
7
Miscellaneous
8
System administration commands
Sometimes we need to look in a specific section of the manual to find what we are
looking for. This is particularly true if we are looking for a file format that is also the
name of a command. Without specifying a section number, we will always get the first
instance of a match, probably in section 1. To specify a section number, we use man like
this:
man section search_term
For example:
[[email protected] ~]$ man 5 passwd
52
Getting A Command's Documentation
This will display the man page describing the file format of the /etc/passwd file.
apropos – Display Appropriate Commands
It is also possible to search the list of man pages for possible matches based on a search
term. It's very crude but sometimes helpful. Here is an example of a search for man
pages using the search term “floppy”:
[[email protected] ~]$ apropos floppy
create_floppy_devices (8) - udev callout to create all possible
floppy device based on the CMOS type
fdformat
(8) - Low-level formats a floppy disk
floppy
(8) - format floppy disks
gfloppy
(1) - a simple floppy formatter for the GNOME
mbadblocks
(1) - tests a floppy disk, and marks the bad
blocks in the FAT
mformat
(1) - add an MSDOS filesystem to a low-level
formatted floppy disk
The first field in each line of output is the name of the man page, the second field shows
the section. Note that the man command with the “-k” option performs the exact same
function as apropos.
whatis – Display A Very Brief Description Of A Command
The whatis program displays the name and a one line description of a man page
matching a specified keyword:
[[email protected] ~]$ whatis ls
ls
(1) - list directory contents
The Most Brutal Man Page Of Them All
As we have seen, the manual pages supplied with Linux and other Unix-like
systems are intended as reference documentation and not as tutorials. Many man
pages are hard to read, but I think that the grand prize for difficulty has got to go
to the man page for bash. As I was doing my research for this book, I gave it
careful review to ensure that I was covering most of its topics. When printed, it's
53
6 – Working With Commands
over eighty pages long and extremely dense, and its structure makes absolutely no
sense to a new user.
On the other hand, it is very accurate and concise, as well as being extremely
complete. So check it out if you dare and look forward to the day when you can
read it and it all makes sense.
info – Display A Program's Info Entry
The GNU Project provides an alternative to man pages for their programs, called “info.”
Info pages are displayed with a reader program named, appropriately enough, info.
Info pages are hyperlinked much like web pages. Here is a sample:
File: coreutils.info,
Up: Directory listing
Node: ls invocation,
Next: dir invocation,
10.1 `ls': List directory contents
==================================
The `ls' program lists information about files (of any type,
including directories). Options and file arguments can be intermixed
arbitrarily, as usual.
For non-option command-line arguments that are directories, by
default `ls' lists the contents of directories, not recursively, and
omitting files with names beginning with `.'. For other non-option
arguments, by default `ls' lists just the filename. If no non-option
argument is specified, `ls' operates on the current directory, acting
as if it had been invoked with a single argument of `.'.
By default, the output is sorted alphabetically, according to the
--zz-Info: (coreutils.info.gz)ls invocation, 63 lines --Top----------
The info program reads info files, which are tree structured into individual nodes, each
containing a single topic. Info files contain hyperlinks that can move you from node to
node. A hyperlink can be identified by its leading asterisk, and is activated by placing the
cursor upon it and pressing the enter key.
54
Getting A Command's Documentation
To invoke info, type “info” followed optionally by the name of a program. Below is a
table of commands used to control the reader while displaying an info page:
Table 6-2: info Commands
Command
Action
?
Display command help
PgUp or Backspace
Display previous page
PgDn or Space
Display next page
n
Next - Display the next node
p
Previous - Display the previous node
u
Up - Display the parent node of the currently displayed
node, usually a menu.
Enter
Follow the hyperlink at the cursor location
q
Quit
Most of the command line programs we have discussed so far are part of the GNU
Project's “coreutils” package, so typing:
[[email protected] ~]$ info coreutils
will display a menu page with hyperlinks to each program contained in the coreutils
package.
README And Other Program Documentation Files
Many software packages installed on your system have documentation files residing in
the /usr/share/doc directory. Most of these are stored in plain text format and can
be viewed with less. Some of the files are in HTML format and can be viewed with a
web browser. We may encounter some files ending with a “.gz” extension. This
indicates that they have been compressed with the gzip compression program. The gzip
package includes a special version of less called zless that will display the contents
of gzip-compressed text files.
55
6 – Working With Commands
Creating Your Own Commands With alias
Now for our very first experience with programming! We will create a command of our
own using the alias command. But before we start, we need to reveal a small
command line trick. It's possible to put more than one command on a line by separating
each command with a semicolon character. It works like this:
command1; command2; command3...
Here's the example we will use:
[[email protected] ~]$ cd /usr; ls; cd bin games
kerberos lib64
local
etc include lib
libexec sbin
/home/me
[me[email protected] ~]$
share
src
tmp
As we can see, we have combined three commands on one line. First we change
directory to /usr then list the directory and finally return to the original directory (by
using 'cd -') so we end up where we started. Now let's turn this sequence into a new
command using alias. The first thing we have to do is dream up a name for our new
command. Let's try “test”. Before we do that, it would be a good idea to find out if the
name “test” is already being used. To find out, we can use the type command again:
[[email protected] ~]$ type test
test is a shell builtin
Oops! The name “test” is already taken. Let's try “foo”:
[[email protected] ~]$ type foo
bash: type: foo: not found
Great! “foo” is not taken. So let's create our alias:
[[email protected] ~]$ alias foo='cd /usr; ls; cd -'
56
Creating Your Own Commands With alias
Notice the structure of this command:
alias name='string'
After the command “alias” we give alias a name followed immediately (no whitespace
allowed) by an equals sign, followed immediately by a quoted string containing the
meaning to be assigned to the name. After we define our alias, it can be used anywhere
the shell would expect a command. Let's try it:
[[email protected] ~]$ foo
bin games
kerberos
etc include lib
/home/me
[[email protected] ~]$
lib64
libexec
local
sbin
share
src
tmp
We can also use the type command again to see our alias:
[[email protected] ~]$ type foo
foo is aliased to `cd /usr; ls ; cd -'
To remove an alias, the unalias command is used, like so:
[[email protected] ~]$ unalias foo
[[email protected] ~]$ type foo
bash: type: foo: not found
While we purposefully avoided naming our alias with an existing command name, it is
not uncommon to do so. This is often done to apply a commonly desired option to each
invocation of a common command. For instance, we saw earlier how the ls command is
often aliased to add color support:
[[email protected] ~]$ type ls
ls is aliased to `ls --color=tty'
To see all the aliases defined in the environment, use the alias command without
arguments. Here are some of the aliases defined by default on a Fedora system. Try and
57
6 – Working With Commands
figure out what they all do:
[[email protected]
alias l.='ls
alias ll='ls
alias ls='ls
~]$ alias
-d .* --color=tty'
-l --color=tty'
--color=tty'
There is one tiny problem with defining aliases on the command line. They vanish when
your shell session ends. In a later chapter, we will see how to add our own aliases to the
files that establish the environment each time we log on, but for now, enjoy the fact that
we have taken our first, albeit tiny, step into the world of shell programming!
Revisiting Old Friends
Now that we have learned how to find the documentation for commands, go and look up
the documentation for all the commands we have encountered so far. Study what
additional options are available and try them out!
Further Reading
There are many online sources of documentation for Linux and the command line. Here
are some of the best:
58
●
The Bash Reference Manual is a reference guide to the bash shell. It’s still a
reference work but contains examples and is easier to read than the bash man
page.
http://www.gnu.org/software/bash/manual/bashref.html
●
The Bash FAQ contains answers to frequently asked questions regarding bash.
This list is aimed at intermediate to advanced users, but contains a lot of good
information.
http://mywiki.wooledge.org/BashFAQ
●
The GNU Project provides extensive documentation for its programs, which form
the core of the Linux command line experience. You can see a complete list here:
http://www.gnu.org/manual/manual.html
●
Wikipedia has an interesting article on man pages:
http://en.wikipedia.org/wiki/Man_page
7 – Redirection
7 – Redirection
In this lesson we are going to unleash what may be the coolest feature of the command
line. It's called I/O redirection. The “I/O” stands for input/output and with this facility
you can redirect the input and output of commands to and from files, as well as connect
multiple commands together into powerful command pipelines. To show off this facility,
we will introduce the following commands:
●
cat - Concatenate files
●
sort - Sort lines of text
●
uniq - Report or omit repeated lines
●
grep - Print lines matching a pattern
●
wc - Print newline, word, and byte counts for each file
●
head - Output the first part of a file
●
tail - Output the last part of a file
●
tee - Read from standard input and write to standard output and files
Standard Input, Output, And Error
Many of the programs that we have used so far produce output of some kind. This output
often consists of two types. First, we have the program's results; that is, the data the
program is designed to produce, and second, we have status and error messages that tell
us how the program is getting along. If we look at a command like ls, we can see that it
displays its results and its error messages on the screen.
Keeping with the Unix theme of “everything is a file,” programs such as ls actually send
their results to a special file called standard output (often expressed as stdout) and their
status messages to another file called standard error (stderr). By default, both standard
output and standard error are linked to the screen and not saved into a disk file.
In addition, many programs take input from a facility called standard input (stdin) which
is, by default, attached to the keyboard.
59
7 – Redirection
I/O redirection allows us to change where output goes and where input comes from.
Normally, output goes to the screen and input comes from the keyboard, but with I/O
redirection, we can change that.
Redirecting Standard Output
I/O redirection allows us to redefine where standard output goes. To redirect standard
output to another file besides the screen, we use the “>” redirection operator followed by
the name of the file. Why would we want to do this? It's often useful to store the output
of a command in a file. For example, we could tell the shell to send the output of the ls
command to the file ls-output.txt instead of the screen:
[[email protected] ~]$ ls -l /usr/bin > ls-output.txt
Here, we created a long listing of the /usr/bin directory and sent the results to the file
ls-output.txt. Let's examine the redirected output of the command:
[[email protected] ~]$ ls -l ls-output.txt
-rw-rw-r-- 1 me
me
167878 2008-02-01 15:07 ls-output.txt
Good; a nice, large, text file. If we look at the file with less, we will see that the file
ls-output.txt does indeed contain the results from our ls command:
[[email protected] ~]$ less ls-output.txt
Now, let's repeat our redirection test, but this time with a twist. We'll change the name of
the directory to one that does not exist:
[[email protected] ~]$ ls -l /bin/usr > ls-output.txt
ls: cannot access /bin/usr: No such file or directory
We received an error message. This makes sense since we specified the non-existent
directory /bin/usr, but why was the error message displayed on the screen rather than
being redirected to the file ls-output.txt? The answer is that the ls program does
not send its error messages to standard output. Instead, like most well-written Unix
programs, it sends its error messages to standard error. Since we only redirected standard
output and not standard error, the error message was still sent to the screen. We'll see
60
Redirecting Standard Output
how to redirect standard error in just a minute, but first, let's look at what happened to our
output file:
[[email protected] ~]$ ls -l ls-output.txt
-rw-rw-r-- 1 me
me
0 2008-02-01 15:08 ls-output.txt
The file now has zero length! This is because, when we redirect output with the “>”
redirection operator, the destination file is always rewritten from the beginning. Since
our ls command generated no results and only an error message, the redirection
operation started to rewrite the file and then stopped because of the error, resulting in its
truncation. In fact, if we ever need to actually truncate a file (or create a new, empty file)
we can use a trick like this:
[[email protected] ~]$ > ls-output.txt
Simply using the redirection operator with no command preceding it will truncate an
existing file or create a new, empty file.
So, how can we append redirected output to a file instead of overwriting the file from the
beginning? For that, we use the “>>” redirection operator, like so:
[[email protected] ~]$ ls -l /usr/bin >> ls-output.txt
Using the “>>” operator will result in the output being appended to the file. If the file
does not already exist, it is created just as though the “>” operator had been used. Let's
put it to the test:
[[email protected]
[[email protected]
[[email protected]
[[email protected]
-rw-rw-r-- 1
~]$
~]$
~]$
~]$
me
ls -l
ls -l
ls -l
ls -l
me
/usr/bin >> ls-output.txt
/usr/bin >> ls-output.txt
/usr/bin >> ls-output.txt
ls-output.txt
503634 2008-02-01 15:45 ls-output.txt
We repeated the command three times resulting in an output file three times as large.
Redirecting Standard Error
Redirecting standard error lacks the ease of a dedicated redirection operator. To redirect
61
7 – Redirection
standard error we must refer to its file descriptor. A program can produce output on any
of several numbered file streams. While we have referred to the first three of these file
streams as standard input, output and error, the shell references them internally as file
descriptors zero, one and two, respectively. The shell provides a notation for redirecting
files using the file descriptor number. Since standard error is the same as file descriptor
number two, we can redirect standard error with this notation:
[[email protected] ~]$ ls -l /bin/usr 2> ls-error.txt
The file descriptor “2” is placed immediately before the redirection operator to perform
the redirection of standard error to the file ls-error.txt.
Redirecting Standard Output And Standard Error To One File
There are cases in which we may wish to capture all of the output of a command to a
single file. To do this, we must redirect both standard output and standard error at the
same time. There are two ways to do this. First, the traditional way, which works with
old versions of the shell:
[[email protected] ~]$ ls -l /bin/usr > ls-output.txt 2>&1
Using this method, we perform two redirections. First we redirect standard output to the
file ls-output.txt and then we redirect file descriptor two (standard error) to file
descriptor one (standard output) using the notation 2>&1.
Notice that the order of the redirections is significant. The redirection of
standard error must always occur after redirecting standard output or it doesn't
work. In the example above,
>ls-output.txt 2>&1
redirects standard error to the file ls-output.txt, but if the order is changed to
2>&1 >ls-output.txt
standard error is directed to the screen.
Recent versions of bash provide a second, more streamlined method for performing this
62
Redirecting Standard Error
combined redirection:
[[email protected] ~]$ ls -l /bin/usr &> ls-output.txt
In this example, we use the single notation &> to redirect both standard output and
standard error to the file ls-output.txt.
Disposing Of Unwanted Output
Sometimes “silence is golden,” and we don't want output from a command, we just want
to throw it away. This applies particularly to error and status messages. The system
provides a way to do this by redirecting output to a special file called “/dev/null”. This
file is a system device called a bit bucket which accepts input and does nothing with it.
To suppress error messages from a command, we do this:
[[email protected] ~]$ ls -l /bin/usr 2> /dev/null
/dev/null In Unix Culture
The bit bucket is an ancient Unix concept and due to its universality, has appeared
in many parts of Unix culture. When someone says he/she is sending your
comments to /dev/null, now you know what it means. For more examples,
see the Wikipedia article on “/dev/null”.
Redirecting Standard Input
Up to now, we haven't encountered any commands that make use of standard input
(actually we have, but we’ll reveal that surprise a little bit later), so we need to introduce
one.
cat – Concatenate Files
The cat command reads one or more files and copies them to standard output like so:
cat [file...]
63
7 – Redirection
In most cases, you can think of cat as being analogous to the TYPE command in DOS.
You can use it to display files without paging, for example:
[[email protected] ~]$ cat ls-output.txt
will display the contents of the file ls-output.txt. cat is often used to display
short text files. Since cat can accept more than one file as an argument, it can also be
used to join files together. Say we have downloaded a large file that has been split into
multiple parts (multimedia files are often split this way on USENET), and we want to
join them back together. If the files were named:
movie.mpeg.001
movie.mpeg.002 ... movie.mpeg.099
we could join them back together with this command:
cat movie.mpeg.0* > movie.mpeg
Since wildcards always expand in sorted order, the arguments will be arranged in the
correct order.
This is all well and good, but what does this have to do with standard input? Nothing yet,
but let's try something else. What happens if we type “cat” with no arguments:
[[email protected] ~]$ cat
Nothing happens, it just sits there like it's hung. It may seem that way, but it's really
doing exactly what it's supposed to.
If cat is not given any arguments, it reads from standard input and since standard input
is, by default, attached to the keyboard, it's waiting for us to type something! Try this:
[[email protected] ~]$ cat
The quick brown fox jumped over the lazy dog.
Next, type a Ctrl-d (i.e., hold down the Ctrl key and press “d”) to tell cat that it has
reached end of file (EOF) on standard input:
64
Redirecting Standard Input
[[email protected] ~]$ cat
The quick brown fox jumped over the lazy dog.
The quick brown fox jumped over the lazy dog.
In the absence of filename arguments, cat copies standard input to standard output, so
we see our line of text repeated. We can use this behavior to create short text files. Let's
say that we wanted to create a file called “lazy_dog.txt” containing the text in our
example. We would do this:
[[email protected] ~]$ cat > lazy_dog.txt
The quick brown fox jumped over the lazy dog.
Type the command followed by the text we want in to place in the file. Remember to
type Ctrl-d at the end. Using the command line, we have implemented the world's
dumbest word processor! To see our results, we can use cat to copy the file to stdout
again:
[[email protected] ~]$ cat lazy_dog.txt
The quick brown fox jumped over the lazy dog.
Now that we know how cat accepts standard input, in addition to filename arguments,
let's try redirecting standard input:
[[email protected] ~]$ cat < lazy_dog.txt
The quick brown fox jumped over the lazy dog.
Using the “<” redirection operator, we change the source of standard input from the
keyboard to the file lazy_dog.txt. We see that the result is the same as passing a
single filename argument. This is not particularly useful compared to passing a filename
argument, but it serves to demonstrate using a file as a source of standard input. Other
commands make better use of standard input, as we shall soon see.
Before we move on, check out the man page for cat, as it has several interesting options.
Pipelines
The ability of commands to read data from standard input and send to standard output is
utilized by a shell feature called pipelines. Using the pipe operator “|” (vertical bar), the
65
7 – Redirection
standard output of one command can be piped into the standard input of another:
command1 | command2
To fully demonstrate this, we are going to need some commands. Remember how we
said there was one we already knew that accepts standard input? It's less. We can use
less to display, page-by-page, the output of any command that sends its results to
standard output:
[[email protected] ~]$ ls -l /usr/bin | less
This is extremely handy! Using this technique, we can conveniently examine the output
of any command that produces standard output.
Filters
Pipelines are often used to perform complex operations on data. It is possible to put
several commands together into a pipeline. Frequently, the commands used this way are
referred to as filters. Filters take input, change it somehow and then output it. The first
one we will try is sort. Imagine we wanted to make a combined list of all of the
executable programs in /bin and /usr/bin, put them in sorted order and view it:
[[email protected] ~]$ ls /bin /usr/bin | sort | less
Since we specified two directories (/bin and /usr/bin), the output of ls would have
consisted of two sorted lists, one for each directory. By including sort in our pipeline,
we changed the data to produce a single, sorted list.
uniq - Report Or Omit Repeated Lines
The uniq command is often used in conjunction with sort. uniq accepts a sorted list
of data from either standard input or a single filename argument (see the uniq man page
for details) and, by default, removes any duplicates from the list. So, to make sure our
list has no duplicates (that is, any programs of the same name that appear in both the
/bin and /usr/bin directories) we will add uniq to our pipeline:
66
Pipelines
[[email protected] ~]$ ls /bin /usr/bin | sort | uniq | less
In this example, we use uniq to remove any duplicates from the output of the sort
command. If we want to see the list of duplicates instead, we add the “-d” option to
uniq like so:
[[email protected] ~]$ ls /bin /usr/bin | sort | uniq -d | less
wc – Print Line, Word, And Byte Counts
The wc (word count) command is used to display the number of lines, words, and bytes
contained in files. For example:
[[email protected] ~]$ wc ls-output.txt
7902 64566 503634 ls-output.txt
In this case it prints out three numbers: lines, words, and bytes contained in lsoutput.txt. Like our previous commands, if executed without command line
arguments, wc accepts standard input. The “-l” option limits its output to only report
lines. Adding it to a pipeline is a handy way to count things. To see the number of
programs we have in our sorted list, we can do this:
[[email protected] ~]$ ls /bin /usr/bin | sort | uniq | wc -l
2728
grep – Print Lines Matching A Pattern
grep is a powerful program used to find text patterns within files. It's used like this:
grep pattern [file...]
When grep encounters a “pattern” in the file, it prints out the lines containing it. The
patterns that grep can match can be very complex, but for now we will concentrate on
simple text matches. We'll cover the advanced patterns, called regular expressions in a
67
7 – Redirection
later chapter.
Let's say we want to find all the files in our list of programs that had the word “zip”
embedded in the name. Such a search might give us an idea of some of the programs on
our system that had something to do with file compression. We would do this:
[[email protected] ~]$ ls /bin /usr/bin | sort | uniq | grep zip
bunzip2
bzip2
gunzip
gzip
unzip
zip
zipcloak
zipgrep
zipinfo
zipnote
zipsplit
There are a couple of handy options for grep: “-i” which causes grep to ignore case
when performing the search (normally searches are case sensitive) and “-v” which tells
grep to only print lines that do not match the pattern.
head / tail – Print First / Last Part Of Files
Sometimes you don't want all of the output from a command. You may only want the
first few lines or the last few lines. The head command prints the first ten lines of a file
and the tail command prints the last ten lines. By default, both commands print ten
lines of text, but this can be adjusted with the “-n” option:
[[email protected]
total 343496
-rwxr-xr-x 1
-rwxr-xr-x 1
-rwxr-xr-x 1
-rwxr-xr-x 1
[[email protected]
-rwxr-xr-x 1
-rwxr-xr-x 1
-rw-r--r-- 1
-rw-r--r-- 1
lrwxrwxrwx 1
~]$ head -n 5 ls-output.txt
root root
31316 2007-12-05
root root
8240 2007-12-09
root root
111276 2007-11-26
root root
25368 2006-10-06
~]$ tail -n 5 ls-output.txt
root root
5234 2007-06-27
root root
691 2005-09-10
root root
930 2007-11-01
root root
930 2007-11-01
root root
6 2008-01-31
These can be used in pipelines as well:
68
08:58
13:39
14:27
20:16
[
411toppm
a2p
a52dec
10:56
04:21
12:23
12:23
05:22
znew
zonetab2pot.py
zonetab2pot.pyc
zonetab2pot.pyo
zsoelim -> soelim
Pipelines
[[email protected] ~]$ ls /usr/bin | tail -n 5
znew
zonetab2pot.py
zonetab2pot.pyc
zonetab2pot.pyo
zsoelim
tail has an option which allows you to view files in real-time. This is useful for
watching the progress of log files as they are being written. In the following example, we
will look at the messages file in /var/log. Superuser privileges are required to do
this on some Linux distributions, since the /var/log/messages file may contain
security information:
[[email protected] ~]$ tail -f /var/log/messages
Feb 8 13:40:05 twin4 dhclient: DHCPACK from 192.168.1.1
Feb 8 13:40:05 twin4 dhclient: bound to 192.168.1.4 -- renewal in
1652 seconds.
Feb 8 13:55:32 twin4 mountd[3953]: /var/NFSv4/musicbox exported to
both 192.168.1.0/24 and twin7.localdomain in
192.168.1.0/24,twin7.localdomain
Feb 8 14:07:37 twin4 dhclient: DHCPREQUEST on eth0 to 192.168.1.1
port 67
Feb 8 14:07:37 twin4 dhclient: DHCPACK from 192.168.1.1
Feb 8 14:07:37 twin4 dhclient: bound to 192.168.1.4 -- renewal in
1771 seconds.
Feb 8 14:09:56 twin4 smartd[3468]: Device: /dev/hda, SMART
Prefailure Attribute: 8 Seek_Time_Performance changed from 237 to 236
Feb 8 14:10:37 twin4 mountd[3953]: /var/NFSv4/musicbox exported to
both 192.168.1.0/24 and twin7.localdomain in
192.168.1.0/24,twin7.localdomain
Feb 8 14:25:07 twin4 sshd(pam_unix)[29234]: session opened for user
me by (uid=0)
Feb 8 14:25:36 twin4 su(pam_unix)[29279]: session opened for user
root by me(uid=500)
Using the “-f” option, tail continues to monitor the file and when new lines are
appended, they immediately appear on the display. This continues until you type Ctrlc.
tee – Read From Stdin And Output To Stdout And Files
In keeping with our plumbing metaphor, Linux provides a command called tee which
creates a “tee” fitting on our pipe. The tee program reads standard input and copies it to
both standard output (allowing the data to continue down the pipeline) and to one or more
69
7 – Redirection
files. This is useful for capturing a pipeline's contents at an intermediate stage of
processing. Here we repeat one of our earlier examples, this time including tee to
capture the entire directory listing to the file ls.txt before grep filters the pipeline's
contents:
[m[email protected] ~]$ ls /usr/bin | tee ls.txt | grep zip
bunzip2
bzip2
gunzip
gzip
unzip
zip
zipcloak
zipgrep
zipinfo
zipnote
zipsplit
Summing Up
As always, check out the documentation of each of the commands we have covered in
this chapter. We have only seen their most basic usage. They all have a number of
interesting options. As we gain Linux experience, we will see that the redirection feature
of the command line is extremely useful for solving specialized problems. There are
many commands that make use of standard input and output, and almost all command
line programs use standard error to display their informative messages.
Linux Is About Imagination
When I am asked to explain the difference between Windows and Linux, I often
use a toy analogy.
Windows is like a Game Boy. You go to the store and buy one all shiny new in
the box. You take it home, turn it on and play with it. Pretty graphics, cute
sounds. After a while though, you get tired of the game that came with it so you
go back to the store and buy another one. This cycle repeats over and over.
Finally, you go back to the store and say to the person behind the counter, “I want
a game that does this!” only to be told that no such game exists because there is
no “market demand” for it. Then you say, “But I only need to change this one
thing!” The person behind the counter says you can't change it. The games are
70
Summing Up
all sealed up in their cartridges. You discover that your toy is limited to the games
that others have decided that you need and no more.
Linux, on the other hand, is like the world's largest Erector Set. You open it up
and it's just a huge collection of parts. A lot of steel struts, screws, nuts, gears,
pulleys, motors, and a few suggestions on what to build. So you start to play with
it. You build one of the suggestions and then another. After a while you discover
that you have your own ideas of what to make. You don't ever have to go back to
the store, as you already have everything you need. The Erector Set takes on the
shape of your imagination. It does what you want.
Your choice of toys is, of course, a personal thing, so which toy would you find
more satisfying?
71

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project