Blog

Part -1 — Linux for front-end developers, windows programmers, and the mere mortals

 101 total views,  1 views today

Just enough Linux to make you a full stack developer. Notes from my learning journal.

Photo by Markus Spiske on Unsplash

CAUTION: Linux OS is case sensitive

NOTE: If you see a # in front of any commands, I have put it there as a comment. Do not put it when you try out the commands/scripts.

Installation

As there are various flavors of Linux distribution, please visit the respective website for installation details.

For windows users, the easiest setup is to use WSL (Windows Subsystems for Linux) which is already available with the latest version of windows 10.

File System

Everything in a Linux system is a file. Some common Linux File System Directories are listed below.

Linux File System Directories

📁/bin: Where Linux core commands reside like ls, mv.
 📁/boot: Where boot loader and boot files are located.
 📁/dev: Where all physical drives are mounted like USBs DVDs.
 📁/etc: Contains configurations for the installed packages.
 📁/home: Where every user will have a personal folder to put his folders with his name like /home/rajesh.
 📁/lib: Where the libraries of the installed packages located since libraries shared among all packages
 📁/media: Here is the external devices like DVDs and USB sticks that are mounted and you can access their files from here.
 📁/mnt: Where you mount other things Network locations and some distros you may find your mounted USB or DVD.
 📁/opt: Some optional packages are located here and this is managed by the package manager.
 📁/proc: Because everything on Linux is a file, this folder for processes running on the system, and you can access them and see much info about the current processes.
 📁/root: The home folder for the root user.
 📁/sbin: Like /bin, but binaries here are for root user only.
 📁/tmp: Contains the temporary files.
 📁/usr: Where the utilities and files shared between users on Linux.
 📁/var: Contains system logs and other variable data.

Reference: https://likegeeks.com/linux-file-system/

What is Shell?

The shell is the command interpreter in an operating system such as Unix or GNU/Linux. It is a program that executes other programs.

It provides a computer user an interface to the Unix/GNU Linux system so that the user can run different commands or utilities/tools with some input data

Common shells available (Note that each shell has its merits and demerits). I am listing here the commonly available shell for reference.

  1. The Bourne Shell (sh) — (Year 1977)
  2. The C shell (csh) — (Year 1978)
  3. The Korn SHell (ksh) — (Year 1983)
  4. The Bourne-Again Shell (bash) — (Year 1989) -> The commonly used shell
  5. The Z shell (Zsh) — (Year 1990) — Commonly used and also default in macOS

NOTE: Ubuntu, CentOS uses default bash shell.

What is “Terminal?”

It’s a program called a terminal emulator. This is a program that opens a window and lets you interact with the shell. There are a bunch of different terminal emulators you can use.

Most Linux distributions supply several, such as: gnome-terminal, konsole, xterm, rxvt, kvt, nxterm, and eterm.

For e.g. Ubuntu uses Gnome Terminal by default.

Command Line

The below listed are the most commonly used commands. Please note, all the commands demonstrated below are executed withing the Linux terminal window.

Date related commands

date # shows the current date
cal  # shows calendar of the current month

Download files

wget <url>  # when giving the url don't specify the <> brackets

Let’s download some sample data to work with. Execute the below commands to download some population data that is openly available.

wget -O population.csv "https://population.un.org/wpp/Download/Files/1_Indicators%20(Standard)/CSV_FILES/WPP2019_TotalPopulationBySex.csv"

The above command will download the data and save it in a file named “population.csv”. This is quite a large file with about 200 hundred thousand lines.

NOTE: If you don’t specify -O <filename> wget will save it by the default fielname. ‘O’ is uppercased.

To see the contents of the file type in the below command

cat population.csv

head

As you observe the above file is quite huge, if you want to see only a few sample records, use the head command. The head command will list the first 10 lines.

head population.csv

tail

Similarly, the tail command displays the last 10 lines from the file.

tail population.csv

NOTE: The head and tail command takes some options which allow you to specify the number of lines required, bytes, etc.

For eg. To display the first 5 lines use the below head command with the -n parameter.

head -n 5 population.csv

The tail command additionally takes an option +, to start displaying from the specific line until the end of the file. (This feature is not available in the head command)

For eg.

tail +25 population.csv

Additionally, the tail command also takes a parameter -f to follow/monitor the file.

tail -f population.csv

The above command displays the last 10 lines and the terminal goes into a waiting mode. Now if some other application is adding more data to the file, the terminal will automatically display the newly added lines.

NOTE: To cancel the command press CTRL + C

TIP: tail -f is very useful for monitoring log files.

Unzip zip files

unzip <filename>

Create Directory

mkdir <directoryname>

Go to Home directory

cd ~

Check Current Directory

pwd  # should shows "home/rajesh" for me

Create multiple nested directories in one command

# NOTE: -p is required.  it will first create directory d1
# then within d1 it will create directory d2 and then finally
# it will create directory d3 within d2
mkdir -p d1/d2/d3

Go to d3 directory

cd d1/d2/d3

Go to the parent directory (Assume you are in d3)

cd ..   #  You will goto d2
cd .. # You will goto d1
cd .. # You will goto home directory (as thats' where we started from)

OR (assuming you are in d3 folder) You can use this one command as well

cd ../../../

Go to last working directory from where we moved

cd -

Listing Directories

ls  # short listing of all files and directories

List the contents of the specific directory

ls <directoryname>

Listing based on pattern (Here assuming d1 is accessible from your current folder)

# list all the filename with .csv extension within the d1 directory
ls d1/*.csv

Listing Directory Recursively

ls -R d1

Long Listing

ls -l  # will show long listing with more details

The output of the long list is shown below.

Observe the highlighted section.

The drwxrwxrwx sections are for permission settings. The ‘d’ indicates that the entry is for a directory.

r →read permission
w →write permission
x →execute permission (execute is for scripts and for a directory it means you can enter a directory (i.e. cd into it and to access any of its file)

We will cover more about permissions in details.

Change Directory

cd <directoryname>

Clear Screen

clear

Create a file

touch hello.txt

Show the content of the file

cat hello.txt # the file is empty so nothing is shown

Getting Help

whatis wc

It outputs

wc (1) — print newline, word, and byte counts for each file

man command (Get very detailed help)

man wc

To quite the help window type ‘q’

Get a history of all previously executed commands

history

The output should resemble something like below (but your commands listed could be different)

1 pwd
 2 node -v
 3 pwd
 4 sudo apt update && sudo apt upgrade
 5 node -v
 6 git status
 7 python
 8 vi
 9 exit
 10 pwd
 11 f:
 12 e:
 13 cd e:/
 14 drive
 15 ls
 16 pwd
 17 cd .
 18 ls
 19 pwd
 20 md lab
 21 mkdir lab
 22 ls
 23 history

NOTE: There is a number against each command

We can execute any of the above commands from history using the bang/exclamation! operator

For eg.

!19     # will execute the command pwd

Ctrl + R

Reverse interactive search. (You don’t need to type in the + character). Type in the command and press enter to execute it

Up Arrow

Within the terminal, if you press up-arrow, it will move to the previously executed command.

Count words in a file

wc [filename]

  • It is used to find out number of lines, word count, byte and characters count in the files specified in the file arguments.
  • By default it displays four-columnar output.
wc population.csv  # counts words and prints it on the standard output

Output

Pipe | (redirecting)

We use the pipe | symbol for redirecting the output.

cat population.csv | wc

Take the output of cat and send it as input to wc command.

Output

grep (find/search patterns)

grep [OPTIONS] pattern [files...]

To find the records for India with the populatin.csv file execute the below command

grep India population.csv

NOTE: It will bring in all the records, including where the word India is part of other word, like “Indian”.

To exactly match the word India we can use the -w flag

grep -w "India" population.csv

By default grep does a case-sensitive match. To ignore case sensitivity use -i.

NOTE: You can use regular expression with grep. It will be covered in the subsequent part of the tutorial.

Using them all together

Let’s say we ant to find the text “Lower” or “lower”, within all the records that contain the text “India”.

cat population.csv | grep -wi "India" | grep -wi "Lower"

NOTE: Mastering grep will make you a linux ninja.

So head over to →man grep for more details.

Make a copy of a file

cp <source> <destination>

Output: A new file with the name <source> will be created with the content of the <destination>. The destination can be a directory as well. If it’s a directory the source is copied to the destination directory.

For e.g.

cp population.csv pop-backup.csv

Output: The content of population.csv will be copied to a new file pop-backup.csv

NOTE: If pop-backup.csv already exist, it will silently overwrite it.

Copy all files from the current folder to the new folder

cp *.* <new-folder>

NOTE: The new folder should already be present.

cp *.* backup

Assuming there is a backup folder with the current directory the above command will copy all files to the backup directory.

Recursively copy all files from a directory to another directory

cp -r <source-directory> <target-directory>

If the <target-directory> does not exist, cp will automatically create it.

For example

cp -r *.* backup3

The above command will copy all files and directory from the current folder to a new folder called backup3. If backup3 already exists it will be overwritten.

Renaming/Move a file

To rename a file use the below command

mv <old-file> <new-file-name>

Example

mv pop-backup.csv population_backup.csv

Output: After executing the above command the pop-backup.csv will be renamed to population_backup.csv. Also, pop-backup.csv will no longer be available.

Removing files and directories

rm command is used to remove files.

rm <filename>

rmdir is used to remove a directory (if the directory is not empty).

rmdir <directory-name>

To remove a directory which is not empty use the below command

rm -rf backup3

The above command will remove all the files in backup3 directory as well as the directory itself.

Know the amount of disk space available

df command displays the amount of disk space available on the file system containing each file name argument. If no filename is passed as an argument with df command then it shows the space available on all currently mounted file systems.

For example to know the disk space on the file for population.csv execute the below command

df population.csv

Observe the output

The bytes are quite huge and to display in human readable form use the -h option.

df -h population.csv

So, -h prints the output in human-readable form.

To know the size of all mounted devices execute the below command

df -h

There are lot of options that goes with df. So

man df

to learn more about df and other commands.

Know the amount of diskspace used

du command, short for disk usage, is used to estimate file space usage. The du command can be used to track the files and directories which are consuming an excessive amount of space on the hard disk drive.

du

Show the disk usage of all the folders in the current directory.

du /home/rajesh

Output

Shows the disk usage of all the folders within the rajesh directory. You can use the -h option for human readable form.

To get the summary information only use the -s option

du -sh /home/rajesh

To get summary information of all files /folders use the below command

du -sh * /home/rajesh

Users and Groups

Users are accounts that can be used to login to a system. Each user is identified by a unique identification number or UID by the system. All the information of users in a system are stored in 
/etc/passwd file. The hashed passwords for users are stored in 
/etc/shadow file.

Users can be divided into two categories on the basis of the level of access:

  1. Superuser/root/administrator: Access to all the files on the system.
  2. Normal users: Limited access.

Let’s create some more users in the system.

Create New User (default home directory)

By default when you use useradd to create a new user it creates the new users home directory under /home directory with username. This you can verify in the below figure showing content of the /etc/passwd file.

useradd user1

If you get the above error, it means you don’t have enough permission to do the above operation. To fix this execute the above command as a superuser by using sudo.

sudo useradd user1

Once you create a user, a new entry is added to the /etc/passwd file. You can verify it by executing the below command.

tail /etc/passwd

The format of the entry in the passwd is as below.

[username]:[password]:[UID]:[GID]:[GECOS]:[home_dir]:[shell_path]
  1. Username field
  2. Password field — An `x` in this field denotes that the encrypted password is stored in the /etc/shadow file.
  3. The user ID number (UID)
  4. User’s group ID number (GID)
  5. Additional information field such as the full name of the user or comment (GECOS)
  6. The absolute path of the user’s home directory
  7. Login shell of the user

What happens when you create a new user?

When a new user is created, by default system takes the following actions:

  • Assigns UID to the user.
  • Creates a home directory /home/.
  • Sets the default shell of the user to be /bin/sh.
  • Creates a private user group, named after the username itself.
  • Contents of /etc/skel are copied to the home directory of the new user.
  • .bashrc.bash_profile and .bash_logout are copied to the home directory of new user.These files provide environment variables for this user’s session.

By default the newly created user will be in a locked state.

Setting a password for the newly created user.

sudo passwd user1

It will ask to enter and confirm a new password.

NOTE: Using sudo here as I don’t have access by default for changing the password.

Login as the newly created user

Execute the ‘su’ command

su user1   ↲

And enter the password and if successful you will be logged in as user1.

Create a User with different Home directory

By default ‘useradd’ command creates a user’s home directory under /home directory with username.

This can be changed by using ‘-d’ option along with the location of the new home directory.

For example, to create a new user ‘user2’ with a home directory ‘/projects’ execute the below command.

$ sudo useradd -d /projects user2

Let’s give it a password

$ sudo passwd user1

Verify the output with

$ getent passwd

You can see that the user2 is assigned /projects as the home folder.

Login as user2

su user2

Change to home directory

cd ~

Do a pwd

pwd

The output should be /projects

Create a User with different Home directory

In some cases we may want to create users without assigning a home directory. In this case when the user logs in the home directory will be the root.

When such user uses su command to change login, its login directory will be the previous user’s home directory.

To create user without a home directory use the -M switch.

$ useradd -M user3

Create a User with Account Expiry date

By default when we create a user with ‘useradd’ command, the user account never expires, i.e. the expiry date is set to 0.

Let’s create a user with an expiry date. A very common security need. We have to use the ‘-e’ switch/parameter to set a new expiry date.

$ useradd -e 2019-12-31 tempuser

Executing the above command creates a user with the username ‘tempuser’ which is set to expire on 31st Dec 2019.

Now post 31st Dec 2019 if tempuser tries to log in to the system, the access will be denied.

Change / See user’s password aging/expiry information

use the ‘chage’ (change age) command.

$ chage -l username

Output

📖Exercise: Learn more about ‘chage’ command.

Modify user attributes

usermod command can be used to modify user attributes.

Set User Account Expiry Date

Let’s set change the account expiry date of user2.

First, lets check the current expirty date

$ chage -l user2

As you can see, the account is set to never expire.

$ sudo usermod -e 2019-12-01 user2

Let’s check the status again

$ chage -l user2

The expiry date is successfully set.

NOTE: The date format for the command is yyyy-mm-dd

Change User Primary Group

Let’s change the user’s primary group. By default when a new user is created a it belongs a group with the same name as the username.

For e.g. let’s check the group name of user2. We can use the id command for the same.

$ id user2

The output

So the above output shows that user2 belongs to the group user2.

Let’s change the group to ‘developers’.

$ usermod -g developers user2

If the “developers” group is not already created, we will get the error as

So, let’s create the group first.The groupadd command can be used to create user groups.

$ sudo groupadd developers

To see all the user groups in the system execute the below command.

$ cat /etc/group

Now execute the usermod command again.

$ sudo usermod -g developers user2

Let’s confirm the change using the id command

id user2

The output

So, this successfully changes the user’s default group.

Assigning supplementary groups

Supplementary groups are additional groups that can be assigned to user.

To add “user2” to the “testers” group leaving the other group assignments as is, execute the below command. (Assuming “testers” group is already created)

If not then create the group first as shown below

$ sudo groupadd testers   # if not already created
$ usermod -a -G testers user2 # NOTE: upper case G
  • a ->add the user to the supplementary group(s). Use only with the -G option
  • -G -> a list of supplementary groups which the user is also a member of.

Check whether a user exists in the Linux system

Let’s check whether user2 exist in the system.

$ getent passwd | grep user2
OR
$ getent passwd user2

Check how many user accounts we have on our system

$ getent passwrd | wc -l

Get logged in username

$ whoami

List all users in the system

$ getent passwd

Permission in Linux (Work in Progress)

Linux is secured by default. Let’s do an ls long listing and take a look at the permission.ls -l # will show long listing with more details

The output of the long list is shown below.

Observe the highlighted section.

The drwxrwxrwx sections are for permission settings. The ‘d’ indicates that the entry is for a directory. (Other than the letter ‘d’, there can also be an ‘l’, indicating symbolic link (we will talk about it in later part of the article).

r →read permission
w →write permission
x →execute permission (execute is for scripts and for a directory it means you can enter a directory (i.e. cd into it and to access any of its files)

Let’s take a detailed look at the ls -l listing.

  1. The first character will always be either a ‘-‘, which means it’s a file, or a ‘d’, which means it’s a directory.
  2. The next nine characters (rw-r–r–) show the security; we’ll talk about them later.
  3. The next column shows the owner of the file. In this case, it is me, my userID is “rajesh”.
  4. The next column shows the group owner of the file, in this case, the default name of the group is rajesh.
  5. The next column shows the size of the file in bytes.
  6. The next column shows the date and time the file was last modified.
  7. And the final column gives the filename.

Grokking the security permission

Let’s break down the 9 characters into an easy representation.

Read, write, execute and –

  • The ‘r’ means you can “read” the file’s contents.
  • -The ‘w’ means you can “write”, or modify the file’s contents.
  • The ‘x’ means you can “execute” the file. This permission is given only if the file is a program or a directory.
  • If any of the “rwx” characters are replaced by a ‘-‘, then that permission has been revoked.

User, group and others

  • user — The user permissions apply only to the owner of the file or directory. They will not impact the actions of other users.
  • group — The group permissions apply only to the group that has been assigned to the file or directory. They will not affect the actions of other users.
  • others — The others permissions apply to all other users on the system. This is the permission group that you want to watch the most.

Reading the security permissions

Let’s undersand a bit more about how to read the security permissions.

Consider that the user’s permissions for the above file, posts. The first character is -meaning its not a directory.

The first three character is “rw-”. This means that the owner of the file (“rajesh”, i.e. me) can “read” the contents of the file and “write” it (modify its contents). I cannot execute it because it is not a program or the execute permission is not yet assigned.

If “r-x” is the second set of 3 characters it means that the members of the group named “rajesh” can only read and execute the files.

The final three characters reprsents the permissions allowed to anyone who has a UserID on this Linux system. Let’s say we have the permission (“r–“). This means anyone in our Linux system can read, but they cannot modify the contents of the files or execute them.

Changing security permissions

chmod is the command used to change the security permissons. It stands for “change mode”.

TIP: Good time to header over to $ man chmod

Steps to change permissions.

  1. The first argument to chmod is ‘u’,’g’,’o’ respectively for user, group and others. A combination of them can also be used.
  2. After this use a ‘+’ for adding or ‘-’ for removing and ‘=’ for assigning a permission.
  3. Then specify the permission, as r, w, or x (read, write, execute). Here too we can use a combination of rwx.
  4. One or more permissions can be modified by using a commas
  5. And finally the name of the file whose permission is being changed.

Example

To give “execute” permission to the “owner” for file “install.sh”, you would execute the below command.

Let’s create a simple shell script. A script is a collection of code which can be executed.

$ touch install.sh

Let’s put some content into the file. You can use the “vi” or “nano” editor to edit the file. Put the below content into the

echo "This is a demo script"

Now, check the permission of the file by typing

$ ls -l

Take a look. Even the owner (u), in this case ‘rajesh’ doesn’t have execute permission.

Let’s try to execute the file as shown below.

$ ./install.sh

You get the permission denied error.

Now let’s give execute permission to the user.

chmod u+x install.sh

Now, let’s try to execute the script as shown below again.

$ ./install.sh

Output: Success

CAUTION: Do not put spaces in file names or folder names. You can separate long file/folder name with a hyphen — or an underscore _

How useful was this post?

Click on a heart to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Leave a Reply

Your email address will not be published. Required fields are marked *