Hello, everyone. I'm Xiaoyan youcong noodles. Recently, I always use MongoDB database, which is really convenient. When you deploy the program, you always use database migration and backup. Today we will summarize briefly. Welcome to share and share (code word is not easy, I hope you can identify the source). Please correct it if there is something wrong. I hope you will pay attention to my WeChat official account, "I can not remember Mr. and miss miss". There is not only the technology of "can't remember", but also the feeling of "can't forget". Thank you very much ^^

Operating system: Window 10 enterprise edition 2015

MongoDB version: MongoDB windows x86_ 64-5.0.5

There are two main types of backup and restore in MongoDB data. One is mongodump and mongorestore for the library, and the other is mongoexport and mongoimport for the tables in the library. Let's pay attention to the following:

1. mongodump and mongorestore backup and restore databases

(1) mongodump backup database basic command

> mongodump -h IP address --port Port number -u user name -p password -d Database name -o File storage path 

Among them, some parameters can be adjusted selectively, as follows:

1. If the user name and password are not set, - u and - p contents can be removed.
2. If you are exporting a native database, -h content can be removed.
3. If it is the default port, - port content can be removed.
4. If you want to export all databases, -d contents can be removed.  

Reference example:

a) Export all data

C:\mongodb\Server\5.0\bin>mongodump -h 127.0.0.1 -o C:\data
2022-01-21T15:25:34.638+0800    writing admin.system.version to C:\data\admin\system.version.bson
2022-01-21T15:25:34.656+0800    done dumping admin.system.version (1 document)
2022-01-21T15:25:34.656+0800    writing test_db.test_data to C:\data\test_db\test_data.bson
2022-01-21T15:25:35.076+0800    done dumping test_db.test_data (100044 documents)
...

b) Export the data of the specified table

C:\mongodb\Server\5.0\bin>mongodump -h 192.168.1.1 -d test_db -o C:\data
2022-01-21T15:25:34.656+0800    writing test_db.test_data to C:\data\test_db\test_data.bson
2022-01-21T15:25:35.076+0800    done dumping test_db.test_data (100044 documents)
...

(2) mongorestore restore database basic commands

> mongorestore -h IP address --port Port number -u user name -p password -d Database name --drop File storage path

Reference example:

a) Import all data

C:\mongodb\Server\5.0\bin>mongorestore -h 127.0.0.1 -o C:\data
2022-01-21T15:25:34.638+0800    writing admin.system.version to C:\data\admin\system.version.bson
2022-01-21T15:25:34.656+0800    done dumping admin.system.version (1 document)
2022-01-21T15:25:34.656+0800    writing test_db.test_data to C:\data\test_db\test_data.bson
2022-01-21T15:25:35.076+0800    done dumping test_db.test_data (100044 documents)
...

b) Import data from the specified table

C:\mongodb\Server\5.0\bin>mongodump -h 192.168.1.1 -d test_db -o C:\data
2022-01-21T15:25:34.656+0800    writing test_db.test_data to C:\data\test_db\test_data.bson
2022-01-21T15:25:35.076+0800    done dumping test_db.test_data (100044 documents)
...

2. mongoexport and mongoimport back up and restore tables or fields

(1) mongoexport export table or field

mongoexport -h IP address --port Port number -u user name -p password -d Database name -c Table name -f field -q Conditional export --csv -o Export file name  

Among them, some parameters can be adjusted selectively, as follows:

1. -f: Export refers to fields separated by English commas. For example: - F key1, key2 and Key3 export key1, key2 and Key3 respectively.
2. -q: To export conditions, you can write a statement to export data according to the conditions, for example: - Q '{"key1": "name"}', that is, export the data with key1 as name.
3. --csv: export files in csv format. You can not write them if you don't need them.

Reference example:

a) Export whole table data to txt file

C:\mongodb\Server\5.0\bin>mongoexport -h 127.0.0.1 -d test_db -c test_table -o C:\data\test_table.txt
2022-03-01T18:27:47.555+0800    connected to: mongodb://127.0.0.1/
2022-03-01T18:27:47.587+0800    exported 1 record

b) Export some field contents to csv file by conditions

C:\mongodb\Server\5.0\bin>mongoexport -h 127.0.0.1 -d test_db -c test_table -q "{\"name\":\"admin\"}" -f name,email,age --csv -o C:\data\test_table.csv
2022-03-01T19:21:41.774+0800    connected to: mongodb://127.0.0.1/
2022-03-01T19:21:41.793+0800    exported 1 record

Note that there was a pit here. I would like to emphasize that you should pay attention to:

When writing conditional statements, the official examples are "- q query filter, as a JSON string, e.g., '{x:{$gt:1}}'". After consulting a lot of materials, I should use English single quotation marks to wrap the standard json string. Later, when I tested under the window, I reported an error, and the error contents are as follows:

C:\mongodb\Server\5.0\bin>mongoexport -h 127.0.0.1 -d test_db -c test_table -q '{"name":"admin"}' -f name,email,age -o C:\data\test_table.json
2022-03-01T19:28:41.678+0800    query '[39 123 117 115 101 114 110 97 109 101 58 97 100 109 105 110 125 39]' is not valid JSON: json: cannot unmarshal string into Go value of type map[string]interface {}
2022-03-01T19:28:41.696+0800    try 'mongoexport --help' for more information

Many articles were queried. Some said that single quotation marks wrapped in double quotation marks under linux and double quotation marks wrapped in single quotation marks under window. After modification, they still reported an error. Later, they found an article accidentally( Original blog address )It is mentioned that "starting from MongoDB 4.2, the query must adopt the extended JSON v2 format (loose or standard / strict mode), including enclosing the field name and operator in quotation marks". The MongoDB version in this article is 5.0.5, so the correct format is:

-q "{\"money\":{\"$gt\":1000}}"

In addition, if this mode is inconvenient to write, it is recommended to use -- queryFile to import files, so there is no difference between escape characters.

C:\mongodb\Server\5.0\bin>mongoexport -h 127.0.0.1 -d test_db -c test_table --queryFile C:\query.txt -f name,email,age -o C:\data\test_table.json
2022-03-01T19:21:41.774+0800    connected to: mongodb://127.0.0.1/
2022-03-01T19:21:41.793+0800    exported 1 record

query.txt reads as follows:

{
    "name":"admin"
}

(2) mongoimport import table or field

# Import non csv files
mongoimport -h IP address --port Port number -u user name -p password -d Database name -c Table name --upsert --file File path
# Import some fields
mongoimport -h IP address --port Port number -u user name -p password -d Database name -c Table name --upsertFields field --file File path
# Import CSV file
mongoimport -h IP address --port Port number -u user name -p password -d Database name -c Table name --type type --headerline --file File path

Similarly, some parameters need to be explained as follows:

1. --upsert: insert or update existing data. If it is not added, the duplicate data will report an error.
2. --Type: the file format to import. csv needs to specify the format as: -- type csv.
3. -Header line: indicates that the first row is the column name and does not need to be imported.
4. --File: the path of the file to be imported.

Reference example:

a) Import table data

C:\mongodb\Server\5.0\bin>mongoimport -h 127.0.0.1 -d test_db -c test_table --upsert --file C:\data\test_table.txt
2022-03-02T17:59:19.256+0800    connected to: mongodb://127.0.0.1/
2022-03-02T17:59:19.284+0800    1 document(s) imported successfully. 0 document(s) failed to import.

b) Import some field data

C:\mongodb\Server\5.0\bin>mongoimport -h 127.0.0.1 -d test_db -c test_table --upsertFields name,age --file C:\data\test_table.txt
2022-03-02T17:59:19.256+0800    connected to: mongodb://127.0.0.1/
2022-03-02T17:59:19.284+0800    1 document(s) imported successfully. 0 document(s) failed to import.

c) Import CSV data

C:\mongodb\Server\5.0\bin>mongoimport -h 127.0.0.1 -d test_db -c test_table --type csv --headerline --file C:\data\test_table.csv
2022-03-02T17:59:19.256+0800    connected to: mongodb://127.0.0.1/
2022-03-02T17:59:19.284+0800    1 document(s) imported successfully. 0 document(s) failed to import.

Well, the above are some common methods of bulk import and export of MongoDB data. In fact, there are still many parameters and methods that need to be studied slowly. They are not completely written here. A full version will be issued later. Please look forward to it.

"When drunk, I don't know the sky is in the water. The boat is full of clear dreams and the river of stars is pressed." Hahaha, this article was finished on a slightly drunk afternoon. It was getting late. I happened to see a picture unexpectedly. The sea and sky were quiet and deep blue, the spots in the lake were dotted, and a sail boat seemed to ripple in the Milky way. It was so wonderful. Mind this poem, dedicated to everyone who is full of hope and still struggling. May your dream be as attractive and wonderful as a fairy tale as the scene described in this poem. I am a small eye excellent porridge noodles. Welcome to share and correct the mistakes. Thank you very much.