Introduction to shell script

Absrtact: briefly introduce the knowledge related to shell script, including variables, strings, arrays, operations, functions, script debugging, etc.

1, shell script

shell is an application program of Linux system. It starts, pauses, stops the program or controls the computer by accepting the commands entered by the user, so as to connect the user with the Linux system.

Shell script is an executable file composed of shell commands, which is used to complete specific functions. The script runs through the interpreter, which is relatively slow. The most important thing in the script is the use and combination of shell commands and some language features to complete the desired functions.

Comments: divided into single line and multi line comments

# Sing line comment
: << EOF
 First line comment
 Comment on the second line
 The first N Line comment
 Note: there are many multi line annotation methods, which is relatively safe.
EOF
# Note that there is a space after ":.

2, shell variable

1. Variable type

Local variables: variables are defined in scripts or commands and are only valid in the current shell. Other shell programs cannot access local variables.
Environment variables: variables that can be accessed by all programs. The normal operation of some programs depends on environment variables. Environment variables can be defined in the script.
Shell variable: a special variable set by the shell program. Shell variables include environment variables and local variables.

2. Variable operation

Common variable assignment: name = "any text" ("=" with no space on both sides)
Local variable assignment: local name = "any text"
Use variables: echo $name or echo ${name} (curly braces are recommended)
Variable re assignment: name = "new text"
Read only variable: name = "only_read" - > readonly name
Delete variable: unset name; (read only variables cannot be deleted)

3. String variable

1) Single quote variable: VAR = 'var string'

   Output as is. There cannot be separate single quotation marks and escape characters in single quotation marks.

2) Double quotation marks: var = "your name is ${name}", where escape characters are available

3) String splicing

   name="this is"" my name"; name="this is my name"; name="this" is "my name" equivalent
   name='this is'' my nam'; name='this is my name'; name='this' is 'my name' equivalent

4) String length

   stay ${}Used in“#"Get length name="test";echo ${#name};  # The output is 4

5) Substring extraction

   1:4 Intercept 4 characters from the second
   ::4 Intercept 4 characters from the first one
   name="this is my name";
   echo ${name:1:4} # Output is i
   echo ${name::4} # Output this

4. Array

bash only supports one-dimensional arrays, not multi-dimensional arrays

1) Array operation

Define array: array_name=(li wang xiang zhang) 
Define array elements individually: array_para[0]="w"; array_para[3]="s" 
Assignment array elements: array_name[0]="zhao";
Get array elements:
    array_name[0]="li"
    array_name[3]="zhang"
    echo ${array_name[0]} # Output "li"
    echo ${array_name[1]} # Output ''
    echo ${array_name[3]} # Output "zhang"
    echo ${array_name[@]} # Output "li zhang" output all elements of the array
 Number of elements: ${#array_name[@]} or ${#array_name}
Single element length: ${#array_name[1]}

2) Parameter transfer

Get parameter value:
    $0 :  Fixed, representing the file name of the execution
    $1 :  Represents the first parameter passed in
    $n :  Represents the incoming third party n Parameters

    \$#: number of parameters
    \$*:  Displays all parameters passed to the script in a single string.
        as"$*"use「"」Include the situation in"$1 $2 ... $n"Output all parameters in the form of.
    \$@: And $*Same, but use it in quotation marks and return each parameter in quotation marks.
    \$\$: The current process number of the script run
    \$!: Of the last process running in the background ID
    \$?:  Displays the exit status of the last command. 0 indicates no error and other values indicate an error.
    \$* And $@ Same point: all parameters are referenced.
    \$* And $@ Difference: only in double quotation marks.
        Suppose you write three parameters 1 when the script runs, 2, 3, be " * " Equivalent to "1 2 3"(Pass a parameter),
        and "@" Equivalent to "1" "2" "3"(Three parameters were passed).

5. Operator

arithmetic operation

+ ,-,*,\ :  The multiplication sign must be preceded by\Multiplication can only be performed after escape
val=`expr 2 + 2` (use linux command expr (auxiliary operation)
val=$[2+2] (4 Spaces are not necessary, different from conditional judgment)
val=$((2+2))

Relational operators: support numbers and strings with numeric values
Suppose variable a is 10 and variable b is 20.

-eq : Check whether the two numbers are equal and return if they are equal true.  [ $a -eq $b ] return false. 
-ne:  Check whether the two numbers are not equal and return if they are not equal true.  [ $a -ne $b ] return true. 
-gt:  Check whether the number on the left is greater than that on the right. If so, return true.  [ $a -gt $b ] return false. 
-lt :  Check whether the number on the left is less than that on the right. If so, return true.  [ $a -lt $b ] return true. 
-ge:  Check whether the number on the left is greater than or equal to that on the right. If so, return true.  [ $a -ge $b ] return false. 
-le :  Check whether the number on the left is less than or equal to that on the right. If so, return true.  [ $a -le $b ] return true.

String operator
Common string operators are listed below, assuming that variable a is "abc" and variable b is "efg":.

 = : Check whether two strings are equal and return true.  [ $a = $b ] return false. 
 != : Check whether two strings are equal. If not, return true.  [ $a != $b ] return true. 
 -z : Check whether the string length is 0, and return 0 true.  [ -z $a ] return false. 
 -n : Check whether the string length is 0. If not, return true.  [ -n "$a" ] return true. 
 $ : Check whether the string is empty. If not, return true.  [ $a ] return true.

Boolean operator
Common Boolean operators are listed below, assuming that variable a is 10 and variable b is 20.

 ! : Non operation, the expression is true Then return false，Otherwise return true.  [ ! false ] return true. 
 -o : Or operation, one expression is true Then return true.  [ $a -lt 20 -o $b -gt 100 ] return true. 
 -a : And operation, both expressions are true Just return true.  [ $a -lt 20 -a $b -gt 100 ] return false.

Logical operator
Suppose variable a is 10 and variable b is 20.

 && : logic AND [[ $a -lt 100 && $b -gt 100 ]] return false
 || : Logical OR [[ $a -lt 100 || $b -gt 100 ]] return true

File operator

 -b file : Whether the file is a block device file, if yes, return true.  [ -b $file ] return false. 
 -c file : Whether the file is a character device file. If so, return true.  [ -c $file ] return false. 
 -d file : If yes, the directory is returned true.  [ -d $file ] return false. 
 -f file : Whether the file is a normal file (neither a directory nor a device file). If so, return true. [ -f $file ] return true. 
 -g file : Is the file set SGID Bit, if yes, return true.  [ -g $file ] return false. 
 -k file : Whether the file has an adhesive bit set(Sticky Bit)，If yes, return true. [ -k $file ] return false. 
 -p file : Whether the file is a named pipeline. If so, return true.  [ -p $file ] return false. 
 -u file : Is the file set SUID Bit, if yes, return true.  [ -u $file ] return false. 
 -r file : Whether the file is readable, if yes, return true.  [ -r $file ] return true. 
 -w file : Whether the file is writable. If so, return true.  [ -w $file ] return true. 
 -x file : Whether the file is executable, if yes, return true.  [ -x $file ] return true. 
 -s file : Whether the file is empty (whether the file size is greater than 0), if not, return true.  [ -s $file ] return true. 
 -e file : Detect whether files (including directories) exist. If so, return true.  [ -e $file ] return true.

3, Scripting and execution

1. Command replacement

Command replacement is similar to variable replacement. It is used to reorganize the command line. First complete the command line in quotation marks, then replace the result, and then reorganize the new command line.

Execute command:

`ls /etc` :  Backquote (all) unix (supported by all systems)
$(ls /etc) :  $+() (part unix (not supported by the system)

Note: when multiple nesting is used, it is used in the for file in \s /etc \ or for file in $(ls /etc) loop from inside to outside.

Gets the directory where the script file is located

dirname \$0` Gets the directory where the script file is located.
path=$(cd `dirname $0`;pwd) :  Get the current directory of the script and execute it cd Command to reach the directory, use pwd Get the path and assign it to path variable

2. Calculation and judgment

Arithmetic operation:

$[ ] : add , subtract , multiply and divide,You don't have to add spaces
$(( )) : Addition, subtraction, multiplication and division, etc,You don't have to add spaces

Logical judgment:

[ ] :  Spaces must be added beside brackets and operators (can be used, not recommended)
[[ ]]: Spaces must be added beside brackets and operators (recommended for string validation)
(()) :  Spaces must be added beside brackets and operators (recommended for numeric verification)
[[]] and (()) namely[ ]An enhanced version of for mathematical comparison expressions and string expressions.

Note: using [[...]] conditional judgment structure instead of [...], can prevent many logic errors in the script. For example, & &, |, < and > operators can normally exist in the [[]] conditional judgment structure, but if they appear in the [] structure, an error will be reported. If you can directly use if [[$a! = 1 & & $a! = 2]], if double brackets are not applicable, it is if [$a - Ne 1] & & [$a! = 2] or if [$a - Ne 1 - a $a! = 2].

[[ ]] Add pattern matching special effects in;

(()) there is no need to escape the size in the expression. In addition to the standard mathematical operators, the following symbols can be added:

Symbol	Description
val++	Post increase
val--	Post subtraction
++val	First increase
--val	First decrease
!	Logical negation
~	Bit inversion
**	exponentiation
<<	Left displacement
>>	Right displacement
&	Bit Boolean sum
\|	Bit Boolean or
&&	Logical sum
\|\|	Logical or

3. Output

echo: it is only used for string output, not as portable as printf. Printf is recommended
printf: line breaks will not be added automatically like echo, but can be added manually \ n without curly braces and separated by spaces

Format: printf format string [arguments...] where (format string: format control string, arguments: parameter list)
Case: printf "% - 10s% - 8s% - 4.2f \ n" Guo Jingnan 66.1234 #% s% C% d% F are format substitutes
%d: The position parameter corresponding to Decimal must be a Decimal integer
%s: The position parameter corresponding to String must be String or character type
%c: The position parameter corresponding to Char must be string or character type
%f: The position parameter corresponding to Float must be numeric
%-10s: refers to the width of 10 characters (- indicates left alignment and none indicates right alignment). If it is insufficient, it will be filled with spaces, and all contents will be displayed if it exceeds.
%-4.2f: refers to the format as decimal, with a width of 4 characters, of which. 2 refers to retaining 2 decimal places.

Escape character:

    \a : Warning character, usually ASCII of BEL character
    \b : back off
    \c : Do not display any ending newline characters in the output results (only in %b (valid under indicator control), and any characters left in the parameter, any subsequent parameters, and any characters left in the format string are ignored
    \f : Page change( formfeed)
    \n : Line feed
    \r : Carriage return( Carriage return)
    \t : Horizontal tab
    \v : vertical tab 
    \ : A literal backslash character
    \ddd : A character representing a 1 to 3 digit octal value, valid only in a format string
    \0ddd : A character representing an octal value of 1 to 3 digits

4. Process control

Unlike Java, PHP and other languages, the shell script process control cannot be empty, that is, there are no statements in the braces of if or else

if else

ifif condition then command1 command2 ... commandN fi
if elseif condition then command1 command2 ... commandN else command fi
if else-if elseif condition1 then command1 elif condition2 then command2 else commandN fi

for

for var in item1 item2 ... itemN do command1 command2 ... commandN done

while

while condition while condition do command done
while Infinite loop while : do command done

until
The until loop executes a series of commands until the condition is true.
until loop and while loop are handled in the opposite way.

until condition do command done

case
shell case statements are multiple choice statements. A case statement can be used to match a value with a pattern. If the match is successful, the corresponding command will be executed.
A case needs an esac (i.e. the reverse of the case) as the closing mark. Each case branch uses a right parenthesis and two semicolons to represent a break, where " Instead of jumping out of the loop, it is no longer to match the following pattern.

The format of the case statement is as follows:

case value in Mode 1) command1 command2 ... commandN ;; Mode 2) command1 command2 ... commandN ;; esac

Jump out of loop

break : Jump out of total cycle
continue: Jump out of the current cycle and continue to the next cycle

Define function
It can be defined by function fun() or directly fun() without any parameters.

[ function ] funname [()]
{
    action;
    [return int;]
}

Function definition [function] funname() {action; [return int;]}
Parameter transfer
- Calling function: fun_name 2 3 4
- Used in the function: the same as the shell function, $n $# $* $? Or add {} funwithparam() {echo "the first parameter is $1!" Echo "the second parameter is $2!" Echo "the tenth parameter is $10!" Echo "the tenth parameter is !" Echo "the eleventh parameter is !" Echo "there are $# parameters in total!" Echo "output all parameters $*!" as a string} funWithParam 1 2 3 4 5 6 7 8 9 34 73 echo $? # Judge whether the execution is successful
Function return value
- The word return may or may not exist
- Return can only be return [0-255]. The status returned here can be executed as a function through $? This is the return value
- If return is not added, the execution status of the last statement is the return value of the function execution status by default. If the last statement is executed successfully, $? 0, otherwise not 0
Use the return value of the function (the script fragment of the official startup service of the Janusgraph database)
- The number returned by return is only the return value of the function execution status, that is, the next $? Gets the value of the
- For bin = \ ABS similar to the following_ Path ` ` statement, which obtains a string composed of all echo and printf outputs in the function body
- ```
 abs_path() { SOURCE="${BASH_SOURCE[0]}" while [ -h "$SOURCE" ]; do DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" SOURCE="$(readlink "$SOURCE")" [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" done echo "test" echo "$( cd -P "$( dirname "$SOURCE" )" && pwd )" # The two echo outputs of this function will be combined into a string as the following BIN value}
```
- BIN=abs_ The return value of the path # bin assignment function. If there is no return, all echo and printf outputs in the function will be combined into a string and passed into bin
- ```
path=${BIN}/nodetool # Can be used directly
```

Input / output redirection
Typically, each Unix/Linux command runs with three files open:

Standard input file (stdin): the file descriptor of stdin is 0, and Unix programs read data from stdin by default.
Standard output file (stdout): the file descriptor of stdout is 1. Unix programs output data to stdout by default.
Standard error file (stderr): the file descriptor of stderr is 2, and Unix program will write error information to stderr stream.
By default, command > file redirects stdout to file, and command < file redirects stdin to file.
If you want to execute a command without displaying the results on the screen, redirect the output to / dev/null

input redirection

bash.sh < file :  Redirect script input to file，from file Provide parameters

Output redirection

 bash.sh > file :  Redirect the output data of the script to file In, overwrite data
 bash.sh >> file :  Redirect the output data of the script to file In, append data
 command >> file 2>&1 :  take stdout and stderr Redirect to after merge file

Read external input

Command:`read arg` (The script reads the external input and assigns it to the variable)

Stop executing the current script, wait for external input, assign the input to the arg variable, and then continue executing the script

File reference
After referencing other files, you can use their variables and functions in two ways:

1. **.** file_path\file_name
2. **source** file_path\file_name####

Color identification

printf "\033[32m SUCCESS: yay \033[0m\n";
printf "\033[33m WARNING: hmm \033[0m\n";
printf "\033[31m ERROR: fubar \033[0m\n";

Output result:

SUCCESS: yay
WARNING: hmm
ERROR: fubar

Long sentence wrap
In order to avoid a statement being too long in the shell, you can use "\" to wrap a line, but when the script is executed, it is still executed as a statement on a line, which is different from the direct line wrapping of enter.

Note: add a space before \ and wrap without space after \ directly.

4, Example of script running

1. shell operation mysql

Let's take mysql login as an example to see how to use shell script to operate the database and import data.

/mysql/mysql/bin/mysql \ -h test_host -P 000 \ -u test_user -ptest_password \ -e"use test_database; source data_faile; " # -e stands for executing sql statements

-u user name
-p User password
-h The server ip address
-D Connected database
-N Do not output column information
-B use tab Key instead of separator
-e Executive SQL sentence
 Exit script command: exit

Different error codes are used when exiting the script, which can be used to judge what error has occurred.

In most shell scripts, exit 0 indicates successful execution and exit 1 indicates an error.
One to one mapping of errors and error codes is helpful for script debugging.

Command: set -e or set +e

set -e Indicates that starting from the current position, it will be triggered if any error occurs exit. contrary, set +e Means to continue executing the script regardless of any errors.

If the script is stateful (each subsequent step depends on the previous step), use set -e to exit the script immediately when there is an error in the script.
If all commands are required to be executed (which is rarely the case), use set +e.

2. shell script debugging

Check for syntax errors -n:

bash -n script_name.sh

Use the following command to execute and debug the Shell script -x:

bash -x script_name.sh

Debug count_odd_number.sh procedure:

    #!/usr/bin.env bash
    # Used to calculate the sum of odd numbers in the array
    # @author San Zhang 
    # @time 2022/02/11
    sum=0
    for num in 1 2 3 4;do 
    re=${num}%2
    if (( ${re} == 1 ));then
    sum=$[${sum}+${num}]
    fi
    done
    echo ${sum}

First check for syntax errors:
```
 bash -n count_odd_number.sh        
```
If there is no output, there is no error. Start the actual debugging:
```
bash -x count_odd_number.sh
```
The commissioning results are as follows:
1. + sum=0
2. + for num in 1 2 3 4
3. + re=1%2
4. + (( 1%2 == 1 ))
5. + sum=1
6. + for num in 1 2 3 4
7. + re=2%2
8. + (( 2%2 == 1 ))
9. + for num in 1 2 3 4
10. + re=3%2
11. + (( 3%2 == 1 ))
12. + sum=4
13. + for num in 1 2 3 4
14. + re=4%2
15. + (( 4%2 == 1 ))
16. + echo 4
17. 4
The above shows each step of the program execution. The debugging effect can be achieved by observing whether the steps of the program execution meet the expectations.
With + indicates the output of the Shell debugger, and without + indicates the output of the program.

3. es (ElasticSearch) official startup service script

#!/usr/bin/env bash
# CONTROLLING STARTUP:
#
# This script relies on a few environment variables to determine startup
# behavior, those variables are:
#
# ES_PATH_CONF -- Path to config directory
# ES_JAVA_OPTS -- External Java Opts on top of the defaults set
#
# Optionally, exact memory values can be set using the `ES_JAVA_OPTS`. Note that
# the Xms and Xmx lines in the JVM options file must be commented out. Example
# values are "512m", and "10g".
#
# ES_JAVA_OPTS="-Xms8g -Xmx8g" ./bin/elasticsearch
source "`dirname "$0"`"/elasticsearch-env
parse_jvm_options() {
if [ -f "$1" ]; then
echo "`grep "^-" "$1" | tr '\n' ' '`"
fi
}
ES_JVM_OPTIONS="$ES_PATH_CONF"/jvm.options
ES_JAVA_OPTS="`parse_jvm_options "$ES_JVM_OPTIONS"` $ES_JAVA_OPTS"
# manual parsing to find out, if process should be detached
if ! echo $* | grep -E '(^-d |-d$| -d |--daemonize$|--daemonize )' > /dev/null; then
exec \
"$JAVA" \
$ES_JAVA_OPTS \
-Des.path.home="$ES_HOME" \
-Des.path.conf="$ES_PATH_CONF" \
-cp "$ES_CLASSPATH" \
org.elasticsearch.bootstrap.Elasticsearch \
"$@"
else
exec \
"$JAVA" \
$ES_JAVA_OPTS \
-Des.path.home="$ES_HOME" \
-Des.path.conf="$ES_PATH_CONF" \
-cp "$ES_CLASSPATH" \
org.elasticsearch.bootstrap.Elasticsearch \
"$@" \
<&- &
retval=$?
pid=$!
[ $retval -eq 0 ] || exit $retval
if [ ! -z "$ES_STARTUP_SLEEP_TIME" ]; then
sleep $ES_STARTUP_SLEEP_TIME
fi
if ! ps -p $pid > /dev/null ; then
exit 1
fi
exit 0
fi
exit $?

be careful

Some of the above text has been changed for ease of reading. The code is not checked. If there is a problem, it needs to be debugged or found by yourself.

Keywords: Linux bash

Added by Dragoonus on Fri, 11 Feb 2022 23:08:19 +0200

Programming VIP

Introduction to shell script

Introduction to shell script

1, shell script

2, shell variable

1. Variable type

2. Variable operation

3. String variable

4. Array

5. Operator

3, Scripting and execution

1. Command replacement

2. Calculation and judgment

3. Output

4. Process control

4, Example of script running

1. shell operation mysql

2. shell script debugging

3. es (ElasticSearch) official startup service script

be careful

Popular Keywords