Essential commands

`apropos list` => find existing commands
`CMD + A` => move to start
`CMD + E` => move to end
`CMD + U` => delete to the start of the line
`CMD + K` => delete to the end of the line
`CTRL - R` => search in command history
`CTRL - C` => cancel the command

Linux File System explored

`/bin` => binary files
`/boot` => boot loader and kernel
`/dev` => device files
`/etc` => configuration files
`/home` => user home directory
`/lib` => library files
`/media` => temporarily mounted storage
`/opt` => optional installed software
`/proc` => kernel presses
`/root` => root user home folder
`/run` => information about running processes
`/src` => files being served by device like NFS
`/sys` => information about system hardware
`/tmp` => temporary data
`/usr` => another location for software

Working with file System

`ls -R path` => recursively print content on the directory.
`ls -l` => show list with info.
`mkdir -p` =>  create whole path.

Creating link

`ln -s filename link name` => creating symlink.
`ln  filename link name` => creating hard link (same size).
`file` => command show file for hard link and  symbolic link for symlinks.

Finding files

`find . -name apple` => search in current directory
`find . -name *apple*` => searching with glob
`find . -size -10M` => find file less than 10 MB
`find . -size +10M` => find file more than 10 MB
`find . -name *apple* -d` => searching for directories
`find . -name *apple* -f` => searching for files
`find . -name *apple* -l` => searching for links

Outputs and redirects

• Standard input(stdin) 0
• Standard output(stdout) 1
• Standard error(stderr) 2
• Redirect output: ls 1> output.txt or ls 1> output.txt
• Redirect error ls 2> output.txt
• Append to file: echo 'some text' >> file.txt

Comparing files

`diff -u file1 file2` => gives output for git
`cmp file1 file2` => compare binary files and returns first difference
`cmp -l file1 file2` => compare binary files and returns list of differences
`hexdump` => return hexadecimal representation
`stat file1` => possible to compare file meta

Archiving

`tar -cvf folder => c` => (create an archive), v(verbose), f(send to the file), a(figure out what type of compression to use)
`tar -xf` => archive
`tar -tf archive` => shows what inside archive
`zip -R archive.name folder` => zip a folder
`unzip archive.name` => unzip a folder

Modifying text

awk - extract text from a file according to a rule.

sed - stream editor.

Find and replace for all fixes:

find apps/admin/src -type f -exec sed -i ''  "s/instanceOf(Map)/instanceOf(Object)/g" {} \;

Security and administrations

Permissions

- `chmod` => change permission mode
- `chown, chgrp` => change file owner and group

vim /etc/sudoers => find all users who can do sudo

Installing Software

`apt update` => update index of software
`apt upgrade` => install updagrade
`apt search` => search for packages
`apt remove` => remove the software

Remove access

• Remote terminal access possible using SSH (secure shell)
• For it we need OpenSSH server
• OpenSSH client already installed for ubuntu to connect to server
• "who" shows who is connected

Transfer file

SFTP:

• SSH file transfer protocol
• FileZilla and other software is available
• Work like FTP
• Only for files

`sftp root@ip` => Connect using the sftp protocol

From sftp console:

`get filename` => loads file
`put filename` => upload file

SCP:

• Secure copy protocol
• Scp source destination
• Need to know file path

`user@host:path-to-file` => Remote component

Operation of Running System

Booting and starting up

`shutdown -r now` => shutdown the machine
`shutdown -h +5` => in 5 mins
`shutdown -c` => cancel shutdown

Booting the System

• Firmware loads bootloader (from MBR or image) GRUB (Grand Unified Bootloader)
• Bootloader loads kernel and initial file system
• Kernel mount root file system
• Kernel starts systemd (process control daemon)
• System boots to selected target level

Working with the Bootloader

• GRUB is usually configured during setup
• If not you can install it with grub-install /dev/sdx
• Ctrl + C get to the GRUB shell
• Default configuration /etc/default/grub, after changing we need to run sudo update-grub
• Grub configuration /etc/grub.d

Also possible to use targets for system:

`systemctl isolate reboot.target` => reboot using targets
`systemctl set-default rescue.target` => restart machine in rescue mode
`systemctl set-default graphical.target` => back to original mode

Startup Process

• Initialization system manages all the processes
• You can manage systemd with systemctl

`systemctl` => list all services
`systemctl name` => show info about service
`systemctl disable name` => disable service, not to start at boot
`systemctl start|stop name` => start|stop service

Update kernel

Download kernel from www.kernel.ubuntu.com.

`uname -r` => print kernel version
`dpkg -i name of the kernel` => update karnel

Change kernel parameters

`ls /proc/sys`
`sysctl -a` => shows all parameters we can set
`sysctl value` => read kernel value
`cat /proc/sys/net/ipv4/ip_froward` => to check the config value
`sysctl -w net.ipv4.ip_forward=1` => set kernel value temporary
Possible to add it to the `/etc/sysctl.conf` file so it’s not removed after reboot

Updating software

APT (advanced package tool) is a front end for apt-get, apt-cache etc.

dpkg stands for Debian package.

Package manager searches repositories and manages software.

`apt list —upgradeable` => check all packages with updates
`apt upgrade` => update all package
`apt-mark hold a package-name` => hold package version
`apt-mark untold package-name` => un hold package version
`apt search` => searches the description and name of packages
`apt install —download-only` => only download package
`ls /var/cache/apt/archives`
`dpkg -i` => install software without internet
`apt remove` or `dpkg -r` => removes package
`apt dist-upgrade — install` => update system

Installing from source

`apt install build-essential` => loads package for building packages from source code (C, C++).

Build from source:

`./configure` => sh script to create a Makefile
`make` => uses Makefile to build software
`make install` => install the package

Understanding sharing library

`ldd /path to script` => show dependencies for the command
`ldconfig -p` => show all libraries

Installation usually update the cache:

`export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/file`
add to `/etc/ld.so.conf`
run `ldconfig` to update

AppArmor

• Provides mandatory access control
• Uses policies to specify what resources, file path, etc, a program or process can use

apparmor_status => to check status for all programs
ls -l /etc/apparmor.d => checks what’s running on the system and what permissions are there

In each file we see

• Allowed packages imported
• Allowed services
• Allowed files
• Allowed write

Working with processes

`ps -ef | grep "process name"` => show processes related to the name
`ps process name` => show additional information
`kill pid` => kill process

Exploring bottlenecks

• Using top processor load: 1.00 = 1 core full capacity
• 2.00 with 1 core => processor is being asked to do 2 times more job then it could
• I/O wait: time spent waiting for disk read/write
• Cache: small, fast memory reserved to help disk I/O
• Network: congestion that can be problematic
• Swap: disk space reserved to make more room in RAM

Exploring system hardware

• To test memory, reboot PC and go to the GRUB, there you can find util program called "memtest86+", possible to check cache and RAM
• To check file system integrity we can use "sudo tun2fs -l /dev/sda1"
• To check disk we can use "smartctl", sudo apt install smartmontools, after installation "sudo smartctl -a /dev/sda1"
• Speed of reading from a disk "sudo hdparm -tT /dev/sda1"
• To check processor information "lscpu"
• All the hardware system knows about "lshw"

Upgrading hardware

`lsmode` => show all available drivers
`modinfo name` =>  show detail information about the module

Explore system log files

`cd /var/log` => contain all logs for the system
`less /var/log/syslog` => system logs
`dmessage -H` => Kernel log

Monitor security and audit the system

`cat /var/log/auth.log` => show all commands ran with sudo
`cat /etc/password` => show all users
`cat /etc/group` => show all groups
`last` => last login to the system and reboots

System usage reporting

`df -h` => how much space left
`free  -h` => info about memory
`du -hd1 / ` => disk usage, one level deep
`cat /proc/cpuinfo` => info about core

Generating combined report for storing in logs:

free -h | awk ’NR==2{ print $2" total" }’
free -h | awk ’NR==2{ print $4" available" }’
df -h /dev/sda1 | awk NR==2{ print $3" used" }’
df -h /dev/sda1 | awk NR==2{ print $3" available" }’

Scheduling tasks

cron => long history; suitable for machines that are always on.

anacron => newer; suitable for machines that sometimes sleep.

To show all cron tasks "cat /etc/crontab".

To show all anacron tasks "cat /etc/anacrontab".

Configuration management

Tools like Chef, Ansible, Puppet, Salt make changes across many systems at once Some of tools are using agent which is run on the machines and checks updates, other query the server itself without running and agent Use infrastructure as code IaC apprroach

Consider a disaster recovery plan

Backups are critical

• Backup your data and configuration
• Test your backups
• Consider how your backups are stored
• Keep an offline copy in the different location

Off-site Hosting

• Consider mirroring your infrastructure off-site
• Virtual or physical machines
• Business has to decide which level of preparedness they need to build

Tags

• technology