Programming | UNIX-Linux » Dr. Nicholas C. Maliszewskyj - Introduction to UNIX

Source: http://www.doksinet Introduction to UNIX Instructor: Dr. Nicholas C Maliszewskyj (nickm@nistgov) Textbook: Learning the UNIX Operating System (Peek, Todino, & Strang) Resources: Running Linux (Matt Welsh & Lar Kaufman) Essential System Administration (Æleen Frisch) The Linux Documentation Project (http://www.tldporg) 1. 2. 3. 4. 5. 6. 7. Introduction a. Understand the history and applications of b. Log in to an existing account and change passwords c. UNIX online help d. Additional information and resources for UNIX The UNIX filesystem a. File tree structure b. System and user directories c. Navigating the file system (cd, ls) File pathnames a. Valid and recommended filenames b. Hidden files – definition and use c. Absolute and relative pathnames d. Creating and deleting directories (mkdir, rmdir) File manipulation a. Moving, copying, and renaming files (mv, cp) b. Deleting files (rm) c. Copying and removing subtrees d. Create and use symbolic links File

information and content a. Examine text files using common utilities (head, tail) b. Paging through text files (more, less) c. Determine file types using the file command. File access permissions a. UNIX file security model b. Access categories and permission modes c. Displaying file permissions d. Changing file permissions (chmod) e. Default file permissions (umask) Shell variables a. 8. 9. 10. 11. 12. 13. 14. The shell is a programming environment b. Setting and displaying variables c. Process, local, and exportable variables d. Exporting variables to child processes Shell command line interpolation a. Order and sequence of command line components b. Wildcard characters, quoting expressions Input and Output (redirection) a. STDIN, STDOUT, and STDERR b. Redirection of input and output Pipelines a. Using pipes to connect processes via I/O b. Building filters using pipelines c. Common filter commands: tee, sort, grep Editors a. Using the vi editor b. Using the emacs editor c.

Other editors (jed, jot, nedit) Customizing your login session a. System and user configuration files b. Display and set terminal characteristics (stty, reset) Printing a. lpr, lprq, lprm, lpc b. Printing files c. Query the print queue d. Cancelling a print request e. Print filters (mpage, nenscript) Advanced file utilities a. Finding files – find/locate b. Search for text in files – grep c. Compression – compress, zip, gzip, bzip2 d. Archiving – tar and cpio Source: http://www.doksinet Introduction History 1969 – MULTICS Project (DARPA) 1975 – UNIX Version 6 (AT&T) (Ken Thompson & Dennis Ritchie) 1977 – Berkeley Systems Division/Berkeley Standard Distribution (BSD) 1984 – UNIX System V (AT&T) 1991 – Linux Version 0.03 (Linus Torvalds (named for “Linus’ MINIX”)) 1993 – Linux Version 1.0 (first production kernel) 1996 – Linux Version 2.0 (loadable kernel modules) 1999 – Linux Version 2.2 (symmetric multiprocessing) 2001 – Linux Version 2.4

(USB, virtual Framebuffers, devFS) Today all UNIXes are considered to have some ratio of BSD/SysV character. Distributions Historically, computer vendors have packaged their own distributions of the operating system tailored to run on their hardware. • Proprietary: HP-UX, IRIX, UNICOS, Solaris, Tru64 UNIX • Open Source: FreeBSD, Linux (Red Hat, Fedora, Mandrake, Debian, Slackware, SUSE, etc.) What is Linux? In class, we will be concentrating on the Linux variant of UNIX. Linux is a cross platform operating system that runs on a great deal of hardware, including computers using Intel, Compaq Alpha, MIPS, Motorola, among other processors. It has many attractive features and exceptional performance. It was one of the first fully POSIXcompliant operating systems by design It is also unique in that it is a free operating system: • You can obtain and use it free of any charges. • The kernel and applications are distributed in source form. Applications of Linux • • • • •

• • • File and Print Server Database server Web Server Firewall Engineering Workstation Scientific Data Acquisition Industrial Automation and Control Scientific and business data processing Source: http://www.doksinet Documentation Books • • • • • Learning the UNIX Operating System (Peek, Todino, Strang) Learning Red Hat Linux (McCarty) Running Linux (Welsh & Kaufman) Linux in a Nutshell (Welsh) Essential System Administration (Frisch) Websites • • • • • Linux Documentation Project http://www.linuxdocorg Linux Gazette http://www.linuxgazettecom Linux Today http://www.linuxtodaycom Linux Journal http://www.ljcom O’Reilly & Associates http://www.oreillycom Text files • HOWTO documents – describe how to perform specific tasks Online help • • • man – display system manual pages apropos – look for commands whose descriptions match keywords (man –k) info – display system “info” pages – an alternative, more extensive set

of documentation Certifications RHCE (Red Hat Certified Engineer) LPIC (Linux Professional Institute Certificate) CompTIA Linux+ Accounts Individuals using a UNIX computer will access it via their own personal accounts. Your account has a number of properties: • Username – the name you use to log in • User ID (UID) – the number by which the operating system knows you • Group ID (GID) – your default group membership • Password – your (encrypted) password • Home directory – your default working directory • Shell – the program you use to interact with the operating system Capabilities are determined by user/group membership The superuser (root) account (UID = 0) has privileges to do anything Source: http://www.doksinet To change account information: chfn [[-f fullname] [-o office] [-p phone]] username Passwords Passwords are stored on the computer in encrypted form, either in (/etc/passwd or /etc/shadow). Use the passwd command to change your account password

Change your password periodically Don’t write your password down Don’t share your password with others Good passwords: Are easy to remember, but hard for others to guess Have at least six characters Mix numbers, letters, and punctuation Use unusual capitalization Basic Commands Basic Form: /path/command [[-switches] optional args] args Exercises 1. 2. 3. 4. Log into your account and change your password Read the “chfn” man page to learn its usage. Change your account information using the “chfn” command Log out (logout) and log in again to verify that your password was changed. Source: http://www.doksinet The UNIX Filesystem A filesystem is the means by which data are organized on a computer’s disk. On all UNIXes, files and directories are arranged hierarchically in a directory tree. The root directory, which is written as a forward slash (/) is the topmost directory of the hierarchy. Unlike Windows and MSDOS, which have separate hierarchies for each disk, UNIX fits

all of its disks (local or networked) into a single hierarchy under the root directory. / /bin /etc /home /home/student /tmp /usr /usr/bin /usr/lib /var /usr/local /usr/local/bin Navigating a UNIX filesystem 1. The current working directory can be represented as a single dot (“”), while its parent can be represented as two dots (“.”) 2. The default working directory for your account is called your home directory 3. Print the current working directory using pwd 4. Change directories using the command cd 5. Change directories and add to the directory stack using pushd 6. Change directories to the one at the top of the directory stack using popd 7. Create a directory using mkdir 8. Delete an (empty) directory using rmdir 9. List directory contents using ls Exercises 1. 2. 3. 4. List the contents of your home directory using ls List the contents of your home directory using ls –a. What’s different? Change to your parent directory Change back to your home directory

Source: http://www.doksinet File Pathnames Unlike primitive operating systems (e.g, Windows), UNIX does not insist on requiring that a file adhere to specific naming conventions. Even so, because certain characters have special meaning to the shell, there are characters that should be avoided because they present difficulties when it comes time to manipulate them. These characters include: !@$&*;(){}[]/’”<>? Files whose names begin with a single dot are “hidden” and will not be listed by ls. These files are typically used to store state or configuration information. The path to a file can be either absolute or relative. Absolute pathnames indicate a file’s location with respect to the root directory: /usr/bin/man Relative pathnames indicate a file’s location with respect to the current working directory: .//bin/man bin/man The shell provides a number of wildcard characters to generate filenames for ease in locating and manipulating files: Wildcard Meaning * Zero

or more characters ? One character [] Lists of single characters {} Lists of multiple characters ~ Home directory To “protect” expressions against expansion by the shell, enclose them in single quotes. To “protect” single characters again expansion/interpretation, use a backslash to “escape” them (e.g, to get a literal “*”, type ). Source: http://www.doksinet File Manipulation Commands for manipulating regular directories and files cp mv ln rm Copy files Move/rename files or directories Create/reset links Delete files or directories (there is NO “undo” button) Links A link is a mechanism that allows several filenames to refer to a single file on a disk. There are two types of links: hard links and symbolic links. A hard link associates two files with the same inode (or disk location). A symbolic link is a pointer file that points to another pathname in the system. To create a symbolic link: ln –sf existingfile linkfile Examples of symbolic links can be found

in the runlevel directories (/etc/rc?.d) Using commands recursively It is possible to apply some of the above commands recursively to affect entire branches of the directory hierarchy: ls –lR directory List all details of the children of directory cp –R source target Recursively copy all of source to target rm –rf target Delete target and all its children Exercises 1. 2. 3. 4. 5. 6. Create a file using touch Verify it has been created using ls Create a copy of that file Create a symbolic link to that file Rename the file using mv Delete the original file and list the link. What do you see? Source: http://www.doksinet File information and content To print file contents: cat file - print all of file head file - print first few lines of file tail file - print last few lines of file more file - scroll through file less file - scroll through file + extra functionality Key Command SPACE Display next page RETURN Display next line Search forward for word /word Search backward for

word ?word q Quit less diff file1 file2 – compare two files and print the lines that differ The assumption made when using the above commands is that the files upon which they are used are text. If you wish to print the text within a binary file, use the strings command. UNIX files are all created equally. Their names have (to first order) nothing to do with their contents. The file command compare the contents of a file against a collection of signatures in a database to determine what type a file may be: file filename Source: http://www.doksinet File access permissions File permissions File permissions uniquely dictate types of access. The following is output from ls –l: { { { -rwxrwxr-- 1 nickm Links nickm Owner Group File type (-,d,l,b,c) Owner Permissions Group Permissions Other Permissions 1000000 Size Feb 10 11:07 vscaler.c Month Day Time Filename Permissions come in triplets: read, write, and execute. Special Access Modes t: “sticky” bit

(context-dependent) s: SETUID/SETGID (run with effective UID/GID) Minimum Access cd /home/chavez ls /home/chavez/*.c ls –s /home/chavez/*.c cat runme cat >> runme runme runme rm runme On File Itself N/A None None r w x (if binary) rx (if script) None On Directory x r rx x x x x wx Changing Access Modes Change access modes using the “chmod” command: chmod access-string file-list The access string is of the form “ugo+rwx” To give group members read access: chmod g+r files To make files opaque to others: chmod o-rwx files The access string may also be specific as a numeric mask (read=4, write=2, execute=1): A “read-only” file has permission 444 A file readable, writeable, and executable has permission 777 Your shell’s umask will set the default file permissions. The umask is the complement of the permission mask. Only the owner of a file/directory or the superuser can change its permissions. Changing ownership To change group ownership: chgrp newgrp files To

change user ownership: chown newuser files Source: http://www.doksinet Shell variables The shell is the program that interprets and executes your commands. All UNIX shells are case-sensitive. The original Bourne shell is sh Other, more modern shells are csh, tcsh, ksh, zsh, and bash. To change your default shell use the chsh command The shell is also a programming environment supporting control structures, expression evaluation, and variables. To set a variable in the shell type an expression like variable=value You may then access the value associate with the variable by placing a dollar sign before the variable name: echo $variable Environment variables are a special case of shell variables in that they provide a mechanism for the shell to communicate configuration information to a application. Examples of important environment variables are listed below: HOME PATH EDITOR TERM PS1 Account home directory Search path for commands Default text editor Default terminal identification

Primary prompt string To set an environment variable (and make available for shell AND child processes) export VARIABLE=value in bash setenv VARIABLE value in sh, csh, tcsh To examine the current values of environment variables: echo $VARIABLE printenv [VARIABLE] env [VARIABLE] Source: http://www.doksinet Shell command line interpolation Acknowledging the fact that most users will be using a shell for submitting commands rather than programming, most shells offer facilities that hope to improve the operating environment of the user through command line editing, variable interpolation, and command history. Command Line Editing Many modern shells support command line editing. The keystrokes for editing are the same as those used in emacs: Action Keystrokes Forward character C-f or Right cursor Backward character C-b or Left cursor C-a Beginning of line C-e End of line C-d Delete character C-k Delete to end of line Additionally, pressing the “tab” key will cause the shell to try

to complete a command for you. Command history Many shells also support command histories (the number of saved commands is adjustable). To access the history: Scroll through previous commands using arrow keys Run a previous command by typing !command Substitute previous commands by typing ^old^new Search through command history by typing ctrl-r expression Source: http://www.doksinet Input and Output (redirection) Every program recognizes a group of standard (open) files: STDIN: standard input (file descriptor = 0; default=keyboard) STDOUT: standard output (file descriptor = 1; default=screen) STDERR: standard error (file descriptor = 2; default=screen) STDIN 0 1 PROGRAM 2 STDOUT STDERR Redirection Input program < input Take input from file Output program > output program >> output Write output to file Append output to file program 2> output program > output 2>&1 program &> output Write STDERR to file Write STDERR & STDOUT Write STDERR

& STDOUT Error Source: http://www.doksinet Pipelines In UNIX shells it is a common practice to string small efficient programs together to perform more complicated tasks. This is done by sending the output from one program into the standard input of another. The STDERR channel is used to communicate warnings or errors to the user outside of the pipeline. STDIN STDOUT Program 1 STDERR STDIN STDOUT Program 2 STDERR STDIN STDOUT Program 3 STDERR On the command line, the general form of a pipeline is: [!]command1 args | command2 args | command3 args Pipelines are typically used to take the copious output of one program (which may sometimes just be the cat command listing the contents of a file) and to filter it so that only the interesting bits are shown. Common commands used in pipelines are shown below: tee – splits input to STDOUT + file cat something | tee file sort – sorts input alphabetically or numerically sort [opts] [+POS1 [-POS2]] [file] -n Sort numerically

-b Ignore leading blanks +POS1 [-POS2] Specify field to use as sorting key -k POS1[,POS2] Alternate syntax for above --help Print help message grep – searches input for string or expression & prints matches grep [options] expression [file] -i Ignore case -v Print lines that don’t match expression -e pattern Use pattern as pattern -n Prefix matching line number to output uniq – print unique lines xargs – builds and executes commands from standard input xargs [options] command [initial-arguments] -p Prompt user before invoking command -s max-chars Use at most max-chars characters per line -t Display constructed command before executing ps –ef | grep “[d]oom” | awk ‘{print $2}’ | xargs renice +10 find /usr/src –name ‘*.c’ | xargs /usr/local/bin/prettify {} Source: http://www.doksinet Editors There are two editors found all over UNIX: vi (Visual Editor) and emacs. The choice of which to use as a primary editor can be a religious one. As a general rule,

however, you can find vi on nearly every UNIX platform. Action Command mode Edit mode (insert) Open file Save file Exit Delete character Delete line Help Move cursor up Move cursor down Move cursor left Move cursor right Search forward Search backward Replace string Cut Paste Undo vi M i :e filename :w :q! x dd :help k j h l /pattern/ ?pattern? :s/old/new/ yy p u emacs C-x C-x C-x C-d C-k C-h C-p C-n C-b C-f C-s C-r M-% C-k C-y C- C-f filename ret C-s C-c pattern ret pattern ret old ret new ret M = escape key C = control key Note: If you are connecting to linux/UNIX from Win9x/NT, you may have to set your TERM environment variable to “vt100” in order for the text editors to work properly. Other editors include: ed, pico, jed, joe, nedit, jot, and others Select your default editor by setting the EDITOR environment variable: export EDITOR=/usr/bin/emacs Text files in UNIX are much like text files in any other operating system, differing primarily in the end-of-line character.

The end of a line is represented in Windows by a carriage return + linefeed (“ ”), and in UNIX by a single linefeed (“ ”). Source: http://www.doksinet Customizing your login session Aliases Aliases are redefinitions of commands, usually in shortened form: alias newcmd=”oldcmd options” alias clobber=”rm –f” alias rm=”rm –I” To print the current value of an alias: alias newcmd Configuration files The changes that you make to your environment from the command prompt are not persistent. That is, they will be lost when you log out unless you make changes to the configuration files that are read when you log in. /etc/profile ~/.bash profile ~/.bashrc ~/.bash logout ~/.Xclients ~/.inputrc System wide initialization file, executed for login shells Personal initialization file, executed for login shells Individual per-interactive-shell startup file The individual login session cleanup file, executed when a login shell exits X windows startup file Individual

“readline” initialization file Source: http://www.doksinet Printing Basic printing commands To spool files to a designated printer: lpr [-Pqueue] filename To print the output from a command: command |lpr To examine the state of a print queue: lpq [-Pqueue] To remove a job from a print queue: lprm [-Pqueue] To get status for all queues: lpc status Print queues are defined in /etc/printcap. Special printing The enscript command converts text files to postscript and spools its output to a printer enscript [options] [filename ] -2 Print two columns of text -G Print in “gaudy” mode: title and date at top -r Print in landscape mode (rotate 90 degrees) -p file Send output to file (if file=’-‘, then print to STDOUT) -Pqueue Print to designated queue Multiple printed pages can be fit on a single page thanks to mpage. mpage [options] [filename] -2 Print two normal pages per sheet (also 1,4,8) -Ffont Specify font to use (default is courier) -l Print in landscape mode (default is

portrait) -Pqueue Print to designated queue It is possible to use both of the above tools in a pipeline: enscript –2Gr –p – file.txt | mpage –2 | lpr –Pqueue Source: http://www.doksinet Advanced file utilities Finding files The find command is used to search all mounted filesystems for the name or partial name of a file or directory. The basic syntax is shown below: find starting-dir matching-criteria-and-actions -atime n -mtime n -newer file -size n[bckw] -type c -name name -perm p -user u -group g -nouser File last accessed n days ago File last modified n days ago File newer than file File is exactly n*512 bytes long File is of type c File name matches name pattern File has permissions p File owned by user u File owner by group g File owner not listed in /etc/passwd File types: block device character device directory named pipe regular file b c d p f The locate command can also be used to find files matching certain patterns: locate pattern Locate runs faster than

find because it compares the input pattern against a database which is periodically rebuilt. Source: http://www.doksinet Archiving data & compression The tar command was originally a program intended to write files and directories to a tape drive, but is now a generally useful tool for creating portable archives. Create an archive: tar cvf archive.tar files List archive contents: tar tvf archive.tar Extract files from archive: tar xvf archive.tar The tar command does not implicitly compress the files in the archive, but can be made to do so with the addition of the z switch: tar cvfz archive.tgz files As an aside, tar can be used to efficiently copy entire directory trees, while at the same time preserving file ownership and permissions: cd /usr/user2 tar –cf - -C /usr/user1 olddir | tar –xvpf – There are a few utilities available to compress data (listed in order of efficiency): Archive compress zip gzip bzip2 Unarchive uncompress unzip gunzip bunzip2 Extension .Z .zip

.gz .bz2 Exercises 1. Look for all of the files in /usr/local greater than 1 megabyte in size 2. Replicate an entire directory tree, preserving permissions in the transfer 3. Try all of the various compression schemes on logs/access log in your account. Which is the most efficient scheme?