Basic Linux Resources (V2.1)

February 27, 2020 16 minute read

Last Update

15APR21 - Added filefrag command

16JAN23 - Added Sequential Numbering

20JUN24 - Added useful forensic commands

Introduction

These resources Have been provided for a Linux course I have had to instruct. It contains a command cheat sheet & posts that will help basic analysts be able use and research how to use Linux.

Bash keyboard commands

The mouse won’t work, use the left/right arrow keys to move around the line.

When the cursor is where you want it in the line, typing inserts text – i.e. it doesn’t always over-type what’s already there.

Back to table of contents

Moving Cursor

Ctrl + A or Home Moves the cursor to the Start of a line.

Ctrl + E or End Moves the cursor to the End of a line.

ESC + B Moves to the Beginning of the previous or current word.

Ctrl + K Deletes form the current cursor position to the end of the line.

Ctrl + U Deletes from the start of the line to the current cursor position.

Ctrl + W Deletes the WORD before the cursor.

ALT + B Moves Back one word at a time

ALT + F Moves Forward one word at a time

ALT + C capitalises letter where cursor is and moves to end of word

Back to table of contents

Exiting a running command or accessing another command window

Ctrl + C Kills the current Process

Ctrl + Alt + F1-6 Changes terminals (Up to 6)

Ctrl + Z Suspends the current process (Can be bought back by typing fg)

Back to table of contents

Search for text in a file (less, more)

/pattern Searches forward n for next N for previous

Back to table of contents

Linux Commands

Navigating Linux

Command	Description
`pwd`	Where am I
`whoami`	who am I
`id`	list my user and groups
`ls –al`	What is in the location
`cd <Directory Name>`	change directory
`cd ~`	change to home directory
`cd ..`	change up a directory
`cd -`	go back to where you were
`w`	Who is logged on and what are the doing
`who`	Who is logged onto and what is there session
`cat /etc/release`	System information
`uname -r`	System information
`man <command>`	Traditional UNIX era help system
`info <command>`	GNU projects helps system, typically more comprehensive, not installed by default
`apropos`	search all man pages for a given keyword alias of `man -k <searchterm>`
`man <cat number> <command>`	view a man page other than the default command man-page

Back to table of contents

Command Line Tricks

Command	Description
`History`	lists command history
`!<number from history>`	Executes that command from history
`!<string>`	Repeat last command that started with
`!?<string>`	Repeat last command that has sub-string
`!!:s/foo/bar/`	Repeat last command substituting foo for bar
`grep`	allows you to search the output for a pattern, i.e. cat /var/log/auth.log \| grep

Back to table of contents

Finding a file

Command	Description
`sudo updatedb`	Update the locate db
`locate <filename>`	locate some filename
`find <directory>` or `. <filename>`	find a file in a directory or where I am with a name

Back to table of contents

Reading a file

Command	Description
`Cat <filename>`	List the contents of a file
`less <filename>`	Browse through a file
`head <filename>`	shows first 10 lines of file, can be changed with –n
`tail <filename>`	shows last 10 lines of file, can be changed with –n
`tail –f <filename>`	Display the last 10 lines and follow a files as it grows (Very useful for debugging)
`file <filename>`	Displays what kind of file it is

Back to table of contents

Editing a file

Command	Description
`touch <filename>`	make an empty file
`echo "hello world" > file.txt`	create or overwrite a file with one line, saying hello world
`echo "Hello world 2" >> file.txt`	append a new line to a file
`vim <filename>`	advanced text editor for file name
`nano <filename>`	basic text editor for file name

Back to table of contents

Moving a file

Command	Description
`cp <Source> <destination>`	copies a file, use –r to copy directories
`mv <source> <destination>`	moves a file or renames a file, use –r for directories
`scp <source> <destination>`	Secure copy over ssh for example scp file.txt user@server:/tmp/
`rsync –a <source> <Destination>`	very useful for coping LARGE files and directories, has options for retaining permissions
`wget http://www.webserver.com/file.txt`	downloads a text file from webserver.com
`git clone https://www.github.com/angry-bender/UbuntuLive.git`	Downloads a git project

Back to table of contents

Deleting a file

Command	Description
`rm <filename>`	removes a filename
`rm –r <directoryname>`	removes a directory and all sub directories
`rm –rf <directoryname>`	forces removal of a directory

Back to table of contents

Changing Permissions on a file

permissions usergroupother

Command	Description
`chown <user>:<group>`	Changes ownership on a file
`chmod 754 <filename>`	Changes the permissions on a file Number Sequence User, Group, Others 4 = Read,2 = Write,1 = Execute, 0 = No Permission,(4+2+1=7)
`chmod u=rwx,g=rx,o=r`	Changes the permissions on a file u=user,g=group,o=others r=read,w=write,x=execute
`chgrp <group> <filename>`	change the group owner of a file or folder

Back to table of contents

Comparing a file

Command	Description
`diff –ur <filename1> <filename2>`	Compares the difference between two files, without –ur it will just tell us if there is a difference

Back to table of contents

History

Command	Description
`History`	lists command history
`History 5`	Lists the 5 previous commands
`!<number from history>`	Executes that command from history
ctrl-r	press to search the history, continue pressing to scroll one at a time through search results

Back to table of contents

System Utilities

Command	Description
`fdisk –l`	Lists disks and partitions
`df`	Lists the mount points for inserted disks
`ps –aux` (Old way) `ps -elf` (New Way)	Lists the running processes
`ps -lfu`	lists processes started by a user
`ps -f --ppid 1`	lists processes started on boot by systemd
`top`	Lists processes in order
`kill <pid>`	Kill a Running process
`systemctl -l --type service --all`	Lists all the running Services
`systemctl <start / stop / restart / status> <service>`	stop\|start\|restart or get status of a service
`systemctl list-unit-files --state=enabled`	list startup services
`journalctl –r`	View the system log from the most recent entry, to start from the oldest entry use no –r (can be the same as cat /var/log/syslog
`journalctl -u <servicename>`	View logs about a particular service
`journalctl -k`	View only kernel messages
`journalctl -f`	follow messages as they appear
`sudo mount <device> <directory>`	mount a hard drive or USB
`systemctl <disable / enable> <service>`	enable or disable a system service on startup
`systemctl <reboot / poweroff /emergency / default>`	Reboots, power’s off, enters into single user recovery mode, restores normal operation
`lsblk`	lists basic hard drive information
`lspci`	lists all pci hardware connected to the computer
`tar -xzf <filename>`	extract tar.gz
`tar -zcvf <tarname>.tar.gz <source-folder-name>`	make a tar.gz file
`zcat secure*.gz`	cats out gz log files
`ln -s <location of link> <location of file>`	make a softlink to a file

Back to table of contents

Networking

The new systemd way (Manual Configuration)

Note: This will not persist on a reboot

Command	Description	Order
`ip addr`	List your IP interfaces or addresses	1
`sudo ip link set <interfacename> up`	Set the interface to be up	2
`sudo dhclient`	get a DHCP address	3
`ip addr add 192.168.0.3/24` dev	Enter a static IP address	3
`ip addr`	verify your configuration	4

Back to table of contents

The new legacy way (Manual Configuration)

Command	Description	Order
`ifconfig`	List your interfaces or addresses	1
`ifconfig <interfacename> up`	Set the interface to be up	2
`ifconfig <interfacename> 0.0.0.0 0.0.0.0 && dhclient`	get a DHCP address	3
`ifconfig <interfacename> 192.168.0.3/24 netmask 255.255.255.0`	Enter a static IP address	3
`ifconfig`	verify your configuration	4

Back to table of contents

Creating a profile for Systemd-networkd (networkd)

Creating a profile for ubuntu

In edit, review https://netplan.io/examples/

Other network utilities & Troubleshooting tools

Command	Description
`dig` / `host` / `nslookup`	DNS Lookup utility
`nslookup`	Simple DNS record lookup
`nmtui` / `nmcli`	network manager terminal user interface
`whois`	domain record lookup
`vi /etc/resolv`	edit dns servers (Manual Override)
`ip route`	view routes and default gateway
`ip neighbour`	equivalent to windows arp command
`ip link`	Interface status and MAC Addresses
`ip addr`	Show IP addresses
`ip route <add / del> <net / CIDR> via <IP> dev <int>`	Add or remove static route
`ip route add default via <IP>`	Add Default route
`ss`	Netstat equivalent (`-tu` minimum info, `-put` process info, `-4put` show ipv4 only, `4punt` show ports as numbers

Back to table of contents

Ubuntu Package Management

Command	Description
`apt install <packagename>`	Installs a package
`apt remove <packagename>`	Removes a package
`apt purge <packagename>`	Removes a package and all its configuration
`apt update`	Updates the package lists
`apt upgrade`	Upgrades packages
`apt autoremove`	Removes unneeded packages after updates
`apt list --installed`	List installed packages
`dpkg –i <packagename>`	install a package from a <.deb> file
`dpkg –r <packagename>`	remove a package installed by <.deb> file
`dpkg --list`	list all installed packages
`fuser -k /var/lib/apt/lists/lock`	Tries to kill the process accessing the package lock file gracefully
`lsof -t /var/lib/apt/lists/lock \| xargs kill -9`	Tries to kill the process accessing the package lock file ungracefully

Back to table of contents

User Management

Command	Description
`useradd -m -d /home/<username> -c "Full name" <username>`	adds a user and creates their home directory
`usermod -aG <Groupname> <username>`	Add user to a group
`userdel -r -f <username>`	forces removal of user and home directory
`groupadd <groupname>`	adds a group
`groupdel <groupname>`	deletes a group
`vigr`	edits groups
`passwd <username>`	change a users password
`cat /etc/group`	List groups on system
`cat /etc/passwd`	List users on system
`echo <Username>\|<Password>\| chpassword`	Allows passwords to be scripted in bash for new users
`su - <username>`	switch to a new user and use their shell environment

Back to table of contents

Working with Sequential Files in Bash

Say you have a series of text files you’d like to manipulate that are named 00001.txt, 00010.txt and 00018.txt, you can use the following logic to test and manipulate those files

  for n in {00000..50}; do
    file="$($pwd)/${n}.txt"
    # Test if the file exists
    if test -f "$file"; then
       # Here is where you can manipulate those files
       echo "found file" > $file
    fi
  done

Back to table of contents

Basic Forensics Commands

File commands

Command	Description
`filefrag -v <filename>`	Gives you the offset of where a file is physically on disk
`ls -alh --time=birth --full-time`	Can be handy to show full birth times of a file
`ls -alh --full-time`	Can be handy to show the full modified time of a file
`ls -alhi`	Shows inode number of a file
`stat`	full timestamps of a given file
`find -type f -name "<targetname>" -exec md5sum {} \;`	Find a specific file name and then exec a file hash or another command

Set a forensics alias…. to you know, forensicate

open your ~/.bashrc

add the following line to the end

bash

alias forensicate="sort | uniq -c | sort -nr

then, after using an awk, or any method of reducing data down to a single character, pipe it to forensicate, and get your quick wins

forensicate

Back to table of contents

Grep

Introduction & Description

Grep is a Linux users “Swiss Army knife” of searching for strings or patterns, think of it like your find button in notepad, but on steroids. It works by scanning each line of text in a steam or file, searching it for your pattern, then printing it to the terminal.

grepcomic

Working example

Using this example, we might only want to limit our results to POST requests (Where the client has upload something to the web server). This is useful, as it enables us to quickly search a file, for where a malicious script might have been uploaded from our client.

In this case, we have reduced the analyst’s work, by reducing a file of over 1000 lines to 58, a much easier bit of data to sort through.

grep

Note - WSL USERS : If you are using Windows Subshell for Linux (WSL) you must edit any input file (Files you want to use with -f) within WSL (Use Nano or VIM). Using windows notepad will not work, due to the way it encodes.

Command	Description
`grep “boo” <filename>`	Standard GREP
`grep –n “boo” <filename>`	Prints a line number where the value is present
`grep –vn “boo” <filename>`	opposite of the expression you’ve entered
`grep –c “boo” <filename>`	prints how many times is present
`grep –l “boo” *`	prints filenames that contain boo
`grep –i “BOO” <filename>`	ignores case
`grep –x “boo” <filename>`	Exact Matches
`grep –A2 “aara” <filename>`	Prints extra lines for Context
`grep –E “e$” <filename>`	Use a Regular expression
`grep -rail -f <sourcefile>`	Recursively list every file that contains a value from a source file
`grep -raih -f <sourcefile>`	Recursively list every file and string that contains a value from a source file
`grep -raiH -f <sourcefile>`	Recursively list string that contains a value from a source file
`egrep ‘<pattern1>\|<pattern2>’ <filename>`	enhanced grep to search for two patterns simultaneously

Back to table of contents

Sed

Introduction & Description

Sed, is a command that is used to search and replace a string from a stream or file, without having to open it in a text editor. Think of it like the function in Microsoft word, where you have used replace to fix every instance of a typo you might have done. Similar to grep, it works by searching each line in a file for your pattern, and replaces it with something you have typed.

sedcomic

Working example

Within forensics, I have normally used this command to replace whitespace, or spaces with comma’s so we can convert it into a .csv file, to read in excel. Using this as an example.

sed

Command	Description
`sed -e ‘s/<search>/<replace>/’ <filename>`	Searches for and replaces the first instance with on each line
`sed -e ‘s/<search>/<replace>/2’ <filename>`	Searches for and replaces the second instance with on each line
`sed -e ‘1s/<search>/<replace>/1’ <filename>`	Searches for and replaces on a single line
`sed -e ‘1,4s/<search>/<replace>/1’ <filename>`	Searches for and replaces on the first 4 lines
`sed -e ‘s/<search>/<replace>/g’ <filename>`	Replaces every instance of for with Globally
`sed -i -e ‘s/<search>/<replace>/g’ <filename>`	edits and existing file and replaces the string
`sed -e ‘s/[0-9]*/(&)/’ <filename>`	Replaces ([0-9]) for a number of any length (*) and inserts a Bracket around them (&)
`sed -e ‘/s/\/etc/\/etc\/ssh\/sshd_conf/’`	Replaces /etc/ with /etc/ssh/sshd.conf

Back to table of contents

Awk

Introduction & Description

Awk itself is a command that uses its own language to conduct processing and scanning on text, with a number of pre-defined patterns. These patterns scan a input file, or steam and splits each line into a set of fields, that we can manipulate or sort through.

Within a forensics context I have personally not seen these features used. Mainly for Forensics, I have seen it used to sort columns.

awkcomic

Working example

Using this example, we can see a lot of data present, that might be hard to read. Or, might need to be piped into another command like a GREP. To make analysis easier, we can reduce the file to just the relevant columns that we need. Do do this, you will need to count each of the columns by hand, and use the corresponding number in the $<number> within the print argument. Each column should be separated with a ,. See the below example

awk

Command	Description
`awk [OPTIONS] filename`	Pattern scanning and text processing
`awk -f <awkfile> filename`	Allows the use of an awk file for processing
`awk ‘{print $3}’`	prints all the 3’rd arguments in a file

Extra tutorials

Awk tutorial
How to Use tr, sed, and aspell: Linux Terminal 201 - HakTip 170

Back to table of contents

Regex Reference

A nice way to test your regex before using it

https://regexr.com/

https://regex101.com/

Email Address’s regex resources - Apparently, not as easy as it sounds

https://emailregex.com/

IP Address regex

Not 100% valid, as will match all numbers outside of 255-999, but good for a quick win

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" file.txt

The valid way of doing it

grep -E -o "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)" file.txt

Credit https://www.shellhacks.com/regex-find-ip-addresses-file-grep/

Symbol	Description
`.`	replaces any character
`^`	matches start of string
`$`	matches end of string
`*`	matches up zero or more times the preceding character
`\`	Represent special characters
`()`	Groups regular expressions
`?`	Matches up exactly one character
`\|`	One OR another character / string I.E boot\|boots
`[abcde]`	Match any of the listed characters
`[a-e]`	Match any character in the range
`[!abcde]`	match any character that’s not listed
`[!a-e]`	match and character not in the range
`{linux, shell}`	match any word in the given options

Back to table of contents

Other useful tools

Command	Description
`tr a-z`	Replaces any lowercase letters with uppercase letters
`tr -s t`	removes any duplicates of the letter t
`cut`	Remove sections from each line of files
`paste`	Inserts Sections into each line of files
`join`	Join files together
`sort`	Sort a file by a five order
`uniq`	Report or filter out repeated lines in a file
`aspell check <filename>`	spell checks a file interactively

Back to table of contents

Linux Resources

Package Management

Tarball Example

ftp://ftp.lfs-matrix.net/pub/lfs/lfs-packages/9.1

Using Tarballs

https://www.tldp.org/HOWTO/Software-Building-HOWTO-3.html

Linux Distributions

https://en.wikipedia.org/wiki/Comparison_of_Linux_distributions

https://upload.wikimedia.org/wikipedia/commons/1/1b/Linux_Distribution_Timeline.svg

Linux File systems

https://en.wikipedia.org/wiki/Category:File_systems_supported_by_the_Linux_kernel

Filesystem Hierarchy Standard

http://www.pathname.com/fhs

https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

Binary Directory Standards

http://lists.busybox.net/pipermail/busybox/2010-December/074114.html

Back to table of contents

Programs & Services

Elf Diagram

ELF Diagram

Process Signals

https://linux.die.net/man/7/signal

https://www.bogotobogo.com/Linux/linux_process_and_signals.php

Sysvinit

https://www.oreilly.com/library/view/essential-system-administration/0596003439/ch04s02.html

https://wiki.gentoo.org/wiki/Sysvinit

if you want to play with a sysvinit system

http://ftp.iij.ad.jp/pub/linux/centos-vault/6.0/isos/x86_64/CentOS-6.0-x86_64-LiveDVD.iso

Sysvinit vs system d (Nice comparison graph)

https://access.redhat.com/articles/1189123

Init System and Daemons (Boot Process)

https://opensource.com/article/17/2/linux-boot-and-startup

https://www.golinuxcloud.com/beginners-guide-systemd-tutorial-linux/

Understanding the Linux Boot Process - CompTIA Linux+, LPIC-1

Common Linux Processes / Daemons

https://www.techrepublic.com/article/linux-101-a-comprehensive-list-of-available-linux-services/

https://meterpreter.org/common-linux-services-list/

Back to table of contents

Bash Usage

User Management

Home Directory Specifications

https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

Passwd file specifications

http://www.linfo.org/etc_passwd.html

Back to table of contents

Text Editors

Vim Cheat sheet for programmers

vim cheatsheet

Vim Plugins / Features

http://codyveal.com/posts/vim-killer-features-part-1-text-objects/

https://vimawesome.com

https://www.reddit.com/r/vimporn/

Back to table of contents

Bash Scripting

How To Setup Bash IDE

You can see my post at Full Featured IDE For Bash Scripting

Internal Variables

you can use printenv or for shell scripting the following may be useful https://www.tldp.org/LDP/abs/html/internalvariables.html

I also have found the following variables quire useful, and have used them a lot

variable	explanation
` $? `	Used for capturing the last processes exit code
`$EUID`	Captures the user id that started the script, useful for checking sudo UID 0
`$SUDO_USER`	Captures the username that started a bash script
`$PWD`	present working directory, saves using `$(pwd)`